I

The frame-systems are linked, in turn, by an

information retrieval network. When a proposed frame
I
cannot be made to fit reality -- when we cannot find terminal
assignments that suitably match its terminal marker conditions
Here is the essence of the frame theory: When one
encounters a new situation (or makes a substantial change tn
-- this network provides a replac:ement frame. These inter-
frame structures make possible other ways to represent
I
one's view of a problem), one selects from memory a knowledge about facts, analogies, and other information useful
structure called a frame. This is a remembered framework to
be adapted to fit reality by changing details as necessary.
A frame is a data-structure for representing a
in understanding.
Once a frame is proposed to represent a situation,
a matching process tries to assign values to each frame's
I
terminals, consistent with the markers at each place. The
stereotyped situation like being in a certain kind of living
matching process is partly controlled by information
room or going to a child's birthday party. Attached to each
frame are several kinds of information. Some of this
information is about how to use the frame. Some is about
associated with the frame (which includes information about
how to deal with surprises) and partly by knowledge about
I
the system's current goals. There are important uses for the
what one can expect to happen next. Some is about what to
do if these expectations are not confirmed.
We can think of a frame as a network of nodes
and relations. The "top levels" of a frame ere fixed, and
information, obtained when a matching process fails; it can be
used to select an alternative frame that better suits the
situation.
I
represent things that are always true about the supposed
situation. The lower levels have many terminals -- "slots"
that must be filled by specific instances or data. Each
terminal can specify conditions its assignments must meet.
LOCAL ~1~1~~LOS~L T1~EOK~E~FO~ Vt~I01~ I
When we enter a room we seem to see the entire
(The assignments themselves are usually smaller "sub-
frames.") Simple conditions are specified by markers that
might require a terminal assignment to be a person, an object
of sufficient value, or a pointer to a sub-frame of a certain
scene at a glance. But seeing is really an extended process.
It takes time to fill in details, collect evidence, make
conjectures, test, deduce, and interpret in ways that depend
I
on our knowledge, expectations and goals. Wrong first
type. More complex conditions can specify relations among
the things assigned to several terminals.
Collections of related frames are linked together
impressions have to be revised. Nevertheless, all this
proceeds so quickly and smoothly that it seems to demand a
special explanation.
I
into frame-systems. The effects of important actions ere
Would parallel processing help? This is a more
mirrored by transformations between the framesof a system.
These are used t o make certain kinds of calculations
economical, to represent changes of emphasis and attention,
technical question than it might seem. At the level of
detecting elementary visual features, texture elements,
stereoscopic and motion-parallax cues, it is obvious that
I
and to account for the effectiveness of "imagery."
parallel processing might be useful. At the level of IFouping
For visual scene analysis, the different frames of a
system describe the scene from different viewpoints, and the
transformations between one frame and another represent
features into objects, it is harder to see exactly how to use
parallelism, but one can at least conceive of the aggregation
of connected "nuclei" (Guzman TR-228), or the application of
I
the effects of moving from place to place. For non-visual
boundary line constraint semantics (Waltz T R - 2 7 1 ) ,
kinds of frames, the differences between the frames of a
system can represent actions, cause-effect relations, o r
changes in conceptual viewpoint. Different frames of a
performed in a special parallel network.
At "higher" levels of cognitive processing,
however, one suspects fundamental limitations in the
I
system share th._~esame terminals; this is the critical point
usefulness of parsllelism. Many "integral" schemes were
that makes it possible to coordinate information gathered
from different viewpoints.
Much of the phenomenological power of the theory
proposed in the literature on "pattern recognition" for
parallel operations on pictorial material --perceptrons, I
hinges on the inclusion of expectations and other kinds of integral transforms, skeletonizers, and so forth. These
mathematically and computationally interesting schemes might
presumptions. A frame's terminals are normally already filled
with "default" assignments. T h u s ' ~ frame may contain a
great many details whose supposition is not specifically
quite possibly serve as ingredients of perceptual processing
theories. But as ingredients only] Bas!celly, "integral"
I
warranted by the situation. These have many uses in methods work only on*isolated figures in two dimensions.
representing general information, most likely cases,
techniques for by-passing "logic," and ways to make useful
They fail disastrously in coping with complicated, three-
dimensional scenery.
The new, more successful symbolic theories use
I
generalizations. hypothesis formation and confirmation methods that seem, on
The default assignments are attached loosely to
their terminals, so that they can be easily displaced by new
items that fit better the current situation. They thus can
the surface at least, more inherently serial. It is hard t_oo
solve any very complicated problem without L,ivinl¢ essentially
full attention s at different times, to different sub-problems.
I
serve also as =variables" or as special cases for "reasoning Fortunately, however, beyond the brute idea of doing many
by example," or as "textbook cases," and often make the use
of logical quantifiers unnecessary.
things in parallel, one c'an imagine a more serial process that
deals with large, complex, symbolic structures as units! Thia
opens a new theoretical "niche" for .performing a rapid
I
selection of large substructures; in this niche our theory
hopes to find the secret of speed, both in vision and in
ordinary thinking. I
I
I
~EEtl16 i~ C'JB£

I Inthe tradition of Guzman and Winston, we assume

that the result of looking at a cube is a structure something
like that in figure 1 . . T h e substructures "A" and "B"
represent details or decorations on two faces of the cube.
I Whenwe move to the right, face "A" disappears from view,
while the new face decorated with "C" is now seer If we
had to analyse the scene from the start, we would have to

I (l) lose the knowledge about "A,"

(2) recompute B, and
(3) compute the description of "C.*
cube
~XI\

I ~ , - - ~ left-above-~.~(~ -

I I 1 I parallelogram
~....~L~ ,~ ,"--- i 0 > etc.
A B
I But since we know we moved to the right, we can save "B"

I by assigning it also to the "left face" termlnII of a second

cube-frame. To save "A (just in cased we connect it also
to an extra, invisible face-terminal of the niw cube-scheme
as in figure Z

i /~invisible

,. d
I A E B C
i If later we,move back to the left,

we can reconstruct the first scene without any perceptual

computation at all:
! .
just restore the top-level pointers to the first cube-frame.
We now need a place to store ~ " ; we Can eK:ldyet another
invisible face to the right in the first cube-framel See filNre
I 3. We could extend this to represent further excursions

" M°ve-Right ~ C~ ft~ VKE 3

transformation ~ / l ~

I % / --"left-vertical parallelogram"
ht-vertical parallelogram"

__
square" (in space)
"II P.._hl I A pl

I tO5"

I
 I
around the object. This would lead to a more comprehensive B
frame system, in which each frame represents a different
"perspective" of a cube. In figure 4 there ere three frames

~RIGH_~T~RIGHT_ ~ ~ SpatialFrames Ii

PictorialFrames
I
Rel
ccoamamtioonnM
-
t
e a
r
mrkn
eastructure
i rslin I
B _ left - C)
nrepresent
iant m odreim
invar-
left FIC~U~E ~.
properties. ensional)
(e.g. thre e -
corresponding to 45-degree MOVE-RIGHT and MOVE-LEFT
actions. If we pursue this analysis, the resulting system can
I
become very large; more complex objects need even more
different projections. It is not obvious either that all of them
are normally necessary or that just one of each variety is
adequate. It all depends.
( l ) A frame, once evoked on the basis of partial evidence
or expectat!on, would first direct a test to confirm
its own appropriateness , using knowledge about
I
It is not proposed that this kind of complicated recently noticed features, loci, relations, and
structure is recreated every time one examines an object. It
is imagined instead that a great collection of frame systems is
stored in permanent memory, and one of them is evoked
plausible Sub-frames. The current goal list is used
to decide which terminals and conditions must be
made to match reality.
I
when evidence and expectation make it plausible that the
scene in view will fit It. How are they acquired? We
propose that if a chosen frame does not fit well enough, end
if no better one is easily found, and if the matter Is important
(2) Next it would request information needed to assign
values to those terminals that cannot retain their
default assignments. For example, it might request
I
enough, then an adaptation of the best one so far discovered a description of face "C," if this terminal is
will be constructed and remembered for future use. currently unassigned, but only if it is not marked
"invisible." Such assignments must avee with the
current markers at the terminal. Thus, face "C"
I
Each frame has terminals for attaching pointers to
substructures. Different frames can share the same terminal, might already~have markers for such constraints or
which can thus correspond to the same physical feature as
seen in different views. This permits us to represent~ in a
expectations as:
I
single place, view-independent information gathered st
different times and places. This is important also in non- , Right-middle visual field.
visual applications. , Must be assigned.
, Should be visible; if not, consider
moving right.
I
. , Should be a cube-face sub-frame.
The matching.process which decides whether a
proposed frame is suitable is controlled partly by one's
current goals and partly by information attached to the frame!
* Share left vertical b o u n d a r y
terminal with face "B."
* If failure, consider box-lying-on-side
I
the frames carry terminal markers and other constraints,
frame.
while the goals are used to decide which of these constraints
are currently relevantL Generally, the matching process could
have these components:
* Same backip'ound color as face "B."
I
(3) Finally, if informed about a transformation (e.g., an
impending motion) it would transfer control to the
appropriate other frame of that system I
Within the details of the control scheme are opportunities to
embed many kinds of knowledge. When a terminol-essilpdng
attempt fails, the resulting error message can be used to
i
propose a second-guess alternative. Later it is shown how
memory can be organized into a "Similarity Network" as
proposed in Winston's thesis (TR-23|). I
lOI=
I
 The simplest sort of room-frame candidate is like
I@VI@I@}l~YIII]3OhIC? the inside of a box. Following our cube-model, the room-
frame might have the top-level structure shown in figure 5.
Can one really believe that a person's appreciation
o f three-dimensional structure can be so fragmentary and
atomic as to be representable in terms of the relations
~ ~ aa
, ceiling /c
between parts of two-dimensional views? Let us separate,
at once, the two issues: is imagery symbolic? and is it based
on two-dimensional fragments? The first problem is one of I
degree; sure!y everyone would agree that at some level
vision is essentially symbolic. The quarrel would be between left wall g centerwall lh right wall

certain naive conceptions on one side -- in which one accepts

seeing either as picture-like or as evoking imaginary solids - -
against the confrontation of such experimental results of
!
i Piaget (1956) and others in which many limitations that one
might fear wou!d result from symbolicrepresentations ere
•

fl oor
"'•

shown actually to exist!

As for our second question:
I the issue of two-vs, three-dimensions evaporates at the
FIGUR*" 5"
One has to •assign to the frame's terminals the
symbolic evel. things that are seen. If the room is familiar, some are already
assigned. If no expectations are recorded already, the first
I The very concept of dimension becomes inappropriate. Each
type of symbolic representation of an object serves some
priority might be Iocalcing the principal geometric landmarks.

goals well and others poorly. If we attach the relation labels

To fill in LEFT WALL one might first try to find edges "a" end
I left-of~ right-o_~f and abo~'e between parts of the structure,
Say, as markers on pairs of terminals, certain manipulations •
will work out smoothly; for example, some properties of
"d" and then the associated corners "ag" and "gd." Edge "g,"
for example, is usually easy to find because it should
these relations are "invariant" if we rotate the cube while intersect any eye-level horizontal scan from left to right.

I keeping the same face on the table. Most objects have

"permanent" tops and bottoms. But if we turn the cube on
its side such predictions become harder to make; people
Eventually, "ag," "gb," and "ba" must not be too inconsistent
with one another - - because they are the same physical
vertex.
• have great difficulty keeping track of the faces of a six- However the process is directed, there are some

I colored cube if one makes them roll it around in their mind.

If one uses instead more "intrinsic" relations like
next-to and opposite-t_o,o then turning the object on its side
generally useful knowledge-based tactics. It is probably
easier to find edge "e'~ than any other edge, because if we
have just entered a normal rectangular room, then we may
disturbs the "image" much less. In Winston's thesis we see expect that

I how systematic replacements (e.g., of *left" for "behind," end

"right" for "in-front-of") can deal with the effect of spatial
rotation.
* Edge "e" is a horizontal line.
* R is below eye level.
* It defines a floor-wall texture

I boundary.

Given an expectation about the size of • room, w e can

Visual experience seems continuous. One reason
is that we move continuously. A deeper explanation is that estimate the elevation of "e,= and vice verse. In outdoor

I our "expectations". usually interact smoothly with our

perceptions. Suppose you were to leave a room, close the
scenes, "e" is the horizon and on flat round we can expect
to see it at eye-level. If we fail quickly to locate and essiin
this horizon, we must consider rejecting the proposed frame:
door, turn to reopen it, and find an entirely different room.
either the room is not normal or there is a large obstruction.
I You would be shocked, The sense of change would be hardly
less striking if the world suddenly changed before your eyes.
A naive theory of phenomenological continuity is that we see
The room-analysis strategy might try n e x t to
establish some other landmarks. Given "e,= we next look for
its left and right corners, and then for the verticals rising
so quickly that our image changes as fast as'does the scene.
from them. Once such gross geometrical landmarks ere
I There is an alternative theory: the changes in one's frame-
structure representation proceed at their own pace; the
system prefers to make small changes whenever possible;
located, we can guess the room's general shape end size.
This might lead to selecting a new frame better matched to
that shape and size, with additional markers confirming the
and the•illusion of continuity is due to the persistence of
choice and completing the structure with further details.
I assignments to terminals common to th__eedifferent view-
frames. Thus, continuity depends on the confirmation of
expectations which in turn depends on rapid access to
remembered knowledge about the visual world.

I Just before you enter a room, you usually know

enough to "expect" a room rather than, say, • landscape. You
can •usually tell just by the character of the door. And you
can often select in advance a frame for the new room. Very

I often, one expects a certain particular room. Then many

assignments are already filled in,

Io'/
I
 =,,uu,u transform in the same way a,J does its wall. If a
"center-rectangle" is drawn on a left wall it will appear to
If the new room is unfamiliar, no pre-assembled
frame can supply fine details; more scene-analysis is needed.
Even so, the complexity of the work can be reduced, given
project out because one makes the default assumption that
any such quadrilateral is actually a rectangle hence must lie
in a plane that would so project. In figure 7A, both
I
suitable subframes for constructing hypotheses about
substructures in the scene. How useful these will be
depends both on their inherent adequacy and on the quality
of the expectation process that selects which one to use
quadrilaterals could "look like" rectangh.s, but the one to the
right does not match the markers for a "left rectangle"
subframe (these require, e.g.,, that the left side be longer
I
than the right side). That rectangle is therefore represented
next. One can say a lot even about an unfamiliar roont Most
rooms are like boxes, and they can be categorized into types"
kitchen, hall, living room, theater, and so on. One knows
by a center-rectangle frame, and seems to project out as
though parallel to the center wall.
!
dozens of kinds of rooms and hundreds of particular rooms; Thus we must not simply assign the .label
one no doubt has them structured into some sort of similarity "rectangle" to a quadrilateral but to a particular frame of a
network for effective access. This will be discussed later. rectangle-system. When we move, we expect"w--hatever
A typical room-frame has three or four visible space-transformation is applied to the top-level system will
walls, each perhaps of a different "kind." One knows many be applied also to its subsystems as suggested in figure 7B.
kinds of walls: walls with windows, shelves, pictures, and Similarly the sequence of elliptical projections of a
fireplaces. Each kind of room has its own kinds of walls. A circle contains congruent pairs that are visually ambiguous as
typical wall might have a 3 x 3 array of region-terminals shown in figure 8. But because wall objects usually lie flat,
(left-center-right) x (top-middle-bottom) so that wall-objects we assume that an ellipse on a left wall is a left-ellipse,
Can be assigned qualitat!ve locations. One would further expect it to .transform the same way as the left wall, and are
want to locate objects relative to geometric inter-relations in •
surprised if the prediction is not confirmed.

I,
order to represent such facts as "Y is a little above the
center of the line between X and Z."
In three dimensions, the location of a visual feature
of a subframe is ambiguous, given only eye direction. A
feature in the middle of the visual field could belong either to
a Center Front Wall object or to a High Middle Floor object;
these attach to different subframes. The decision could
depend on reasoned evidence for support, on more directly
visual distance information derived from stereo disparity or
motion-parallax, or on plausibility information derived from
other frames: a clock would be plausible only on the wall-
frame while a person is almost certainly standing on the floor.
Given a box-shaped room, lateral motions induce
orderly changes in the quadrilateral shapes of the walls as i n
figure 6. A picture-frame rectangle, lying flat against a wall,

MOVE RIGHT

FIC~URE" Io

I
I
!
I * ~ . ~ . _ I. 7A

|
right-side r

(A) "colorless green ideas sleep furiously"

is treated very differently than the non-sentence

! O()O
(B) "furiously sleep ideas green colorless"

! and suggests that because both are "equally nonsensical,"

what is involved in the recognition of sentences must be
quite different from what is involved in the appreciation ef
! meanings.
There is no doubt that there are processes

I
I00 0 001
F~u~E 9
especially concerned with grammar. Since the meaning of an
utterance is "encoded" as much in the positional and
structural relations between the words as in the word
DEF~ILT ~ ] 6 R ~ I ~ R T choices themselves, there must be processes concerned with
analysing those relations in the course of building the
While both Seeing and Imagining r e s u l t in structures that will more directly represent the meaning.

I assignments to frame terminals, Imagination leaves us •wider

choices of detail and variety of such assignments. Frames
are probably n e v e r stored in long-term memory with
What makes the words of (A) more effective and predictable
than (B)in producing such a structure -- putting aside the
question of whether that structure should be called semantic
unassigned [erminal values. Instead, what really happens is or syntactic -- is that the word-order relations in (A) exploit

I that frames are stored with weakly-bound d e f a u l t

assignments at every terminalt These manifest themselves as
often-useful but sometimes counter-productive stereotypes.
the (grammatical) convention and rules people usually use to
induce others to make assignments to terminals of structures.
This is entirely consistent with grammar theories. A
Thus in the sentence "John kicked the ball," you generative grammar.would be a summary description of the

I probably cannot think of a purely abstract ball, but must

imagine characteristics of a vaguely particular ball; it
probably has a certain default size, default color, default
exterior appearance of those frame rules - - or their
associated processes - - while the o p e r a t o r s of
transformational grammars seem similar enough to some of
weight. Perhaps it is a descendant of one you first owned or our frame transformations.

I were injured by. Perhaps it resembles your latest one. In

any case your image lacks the sharpness of presence
We certainly cannot assume that "logical"
meaninglessness has a precise psychological counterpart.
Sentence (A) can certainly generate an image! The dominant
because the processes that inspect and operate upon the
weakly-bound default features are very likely to change, frame is perhaps .that of someone sleeping; the default

I adapt, or detach them. •

WOBI)B, BE~'ITE~CE~ ~l:ll) II~ERI:IIRfi~

system assigns a particular bed, and in it lies a mummy-like
shape-frame with a translucent green color property. In this
frame there is a terminal for the character of the sleep - -
restless, perhaps - - and "furiously" seems somewhat
I The concepts of frame and default assignment
seem helpful in discussing the phenomenology of "meaning."
inappropriate at that terminal, perhaps because the terminal
does not like to accept anything so "intentional" for a sleeper.
Chomsky (1957) points out that such a sentence as "Idea" is even more disturbing , because one expects e
person, or at least something animate. One senses frustrated
I procedures trying to resolve these tensions and conflicts
more properly, here or there, into the sleeping framework
that has been evoked:

I Io9

i
 I
Utterance (B) does not get nearly so far because
no subframeaccepts any substantial fragment. As a result no
FJC--URE q
I
larger frame finds anything to match its terminals, hence
finally, no top level "meaning" or "sentence" frame can
organize the utterance as either meaningful or grammatical.
By combining this "soft" theory with gradations of assignment
I
tolerances, one could develop systems that degrade properly
for sentences with "poor" grammar rather than none~ if the
smaller fragments -- phrases and sub-clauses,-- satisfy
subframes well enough, an image adequate for certain kinds
I
of comprehension could be constructed anyway, even though
some parts of the top level structure are not entirely
satisfied, Thus, we arrive at a qualitative theory of
"grammatical:"
E I
if the top levels are satisfied but some lower terminals are
not we have a meaningless sentence; if the top is weak but
the bottom solid, we can have an ungrammatical but
~-~ .~ S ~ I
meaningful utterance.

I)t~COBI~E next-to~ ~ - - _ _ ~ ~ i
Linguistic activity involves larger structures than
can be described in terms of sentential ip'ammar, and these
larger structures further blur the distinctness of the wntex-
Butan adequate definition would need a good deal more.
What about the fact that the order of things being
transported by water currents is not ordinarily changed?. A
I
semantic dichotomy. Consider the following fable, as told by
logician might try to deduce this from a suitably intricate set
W. Chafe ( ! 972): of '"local" axioms, together with appropriate "induction"
axioms. I propose instead to represent this knowledge in a
structure that automatically translocates Spatial descriptions
l
There was once a Wolf who saw a Lamb from the terminals of one frame to those of another frame of
drinking at a river and wanted an
excuse to eat it. For that purpose,
even though he himself was upstream,
the same system. While this might be considered to be a
form of logic, it uses some of the same mechanisms designed
for spatial thinking.
I
he accused the Lamb of stirring up the In many instances we would handle a change over
water and keeping him from drinkln¢.. time, or a cause-effect relation, in the same way as we deal
with a change in position. Thus, the concept r!ver-flow could
evoke a frame-system structure something like the following,
I
To understand this, one must realize that the Wolf is lyingt where S], $2, and S3are abstract slices of the flowing river
To understand the key conjunctive "even though" one must
realize that contamination never flows upstream. This in turn
requires us to understand (among other things) the word
shown in figure 9.
There are many more nuances to fill in. What is
"stirring up" and why would it keep the wolf from drinking?
i
"upstream" itself. Within a declarative, predicate-based
"logical" system, one might try to formalize "upstream" by
some formula like:
One might normally assign default floating objects to the S's,
but here $3 interacts with "stirring up" to yield something
that "drink" does not find acceptable. Was it "deduced" that
i
stirring river-water means that $3 in the first frame should
have "mud" assigned to it; or is this simply the default
[A upstream B]
AND
[Event T, Stream muddy at A]
assignment for stirred water?
Almost any event, action, change, flow of material,
I
or even flow of information can be represented to a first
=>
[Exists
=

[Event U, Stream muddy at B]]

approximation by a.two-frame generalized event. The
frame-system can have slots for agents, tools, side-effects,
preconditions, generalized trajectories, just as in the "trans"
I
verbs of "case grammar" theories, but we have the additional
AND [Later U, T] flexibility of representing changes explicitly. To see if one.
has understood an event or action, one can try to build an
appropriate instantiated frame-Pair.
I
I
t~O
l
I
!
! However, in representing changes by simple
We first hear that Jane is invited to Jack's
"before-after" frame-pairs, we can expect to pay a price.
Pointing to a pair is not the same as describing their Birthday Party. Without the party scenario, or at least an
invitation scenario, the second line seems rather mysterious:
t differences. This makes it less convenient to do planning or
abstract reasoning; there is no explicit place to attach
information about the transformation. As a second
She wondered if he would like a kite.
approximation, we could label pairs of nodes that point to

I corresponding terminals, obtaining a structure like the

"comparison-notes" in Winston (TR-23]), or we might place
at the top of the frame-system information describing the To explain one's rapid comprehension ofthis, we make a
differences more abstractly. Something of this sort will be somewhat radical proposal:

I needed eventually.

~CE~510~
to represent explicitly, in the frame for a scenario structure,
pointers to a collection of the most serious problems and
questions commonly associated with it.

I We condense and conventionalize, in language and

thought, complex situations and sequences into compact
words and symbols. Some words can perhaps be "defined" in
In fact we shall consider the idea that the frame terminals are
exactly those questions.
elegant, simple structures, but only a small part of the

I meaning of "trade" is captured by: Thus, for the birthday party:

Y must get P for X . . . . . . . . Choose P!

first frame second frame X must like P . . . . . . . . - . . . . Will X likeP?

I A has × B has Y B has X • A has Y

Buy P -:- . . . . . . . . . . . . - - - Where to buy P?
Get money to buy P . . . . Where to get money?
(Sub-questions of the "present" frame?)
Y must dress up What should Y wear?
I Trading normally occurs in a social context of law, trust and
convention. Unless we also represent these other facts, most
Certainly these are one's first concerns when one is invited
trade transactions will be almost meaningless. It is usually

t essential to know that each party usually wants both things

but has to compromise. It is a happy but unusual
circumstance in which each trader is glad to get rid of what
he has. To represent trading strategies, one could insert the
to a partY.Th e 'reader is free to wonder •whether this solution
is acceptable. The question "Will X like P?" certainly matches
"She wondered if he would like a kite?" and correctly assigns
the kite to P. But is:our world regular enough that such
I basic maneuvers right into the above frame-pair scenario: in
order for A to make B want X more (or want Y less) we
expect him to select one of the familiar tactics:
question sets could be pro-compiled to make this mechanism
often work smoothly? The answer is mixed. We do indeed
• expect many such questions; we surely do not expect all o!
them. But surely "expertise" consists partly in not having to
I Offer more for Y.
Explain why X is so good.
Create favorable side-effect of B having
realize, a._bbi nitio, what are the outstanding problems and
interactions insituations. Notice, for example, that there is
no default assignment for the Present in our party-scenario

I, I~sparage'the competition.
Make B think C wants X.
These only scratch, the surface. Trades usually occur within a
fr-ame. This mandates attention to that assignment problem
and prepares us for a Possible thematic concern. In any case,
we probably need a more active mechanism for understanding
"wondered" which can apply the information currently in the
| • scenario tied together by more than a simple chain of events
each linked to the next. No single such scenario will do;
when a clue about trading appears it is essential to guess
frame to produce an expectation of what Jane will think
about.
which of the different available scenarios is most tikely to be
useful. The key words and ideas of a discourse evoke substantial

I Charniak's thesis (TR-266) studies questions about

transactions that seem easy for people to comprehend yet
obviously need rich default structures. We find in
thematic or scenario structures, drawn from memory with rich
default assumptions.
In any event, the individual statements of a
elementary school reading books such stories aS: •discourse lead to temporary representations - - which seem

I Jane was invited to Jack's Birthday Party.

to correspond to what contemporary linguists call "deep
structures" - - which are then quickly rearranged or
consumed in elaborating the growing scenario representation.
She wonderedif he would like a kite. In order of "scale," among the ingredients of such a structure

I She went to her room and shook her piggy

bank.
It made no sound.
there might be these kinds of levels:

l
i
 I
EXCU~E~
Winston's thesis (TR-23]) proposes a way to
construct a retrieval system that cart represent classes but
!
has additional flexibility. His retrieval pointers can be made
We can think of a frame as describing an "ideal."
If an ideal does not match reality because it is "basically"
wrong, it must be replaced.
to represent goal requirements and action effects as well as
class memberships. !
What does it mean to expect a chair? Typically,
But it is in the nature of'ideals.that they are really elegant
simplifications; their attractiveness derives from their
simplicity, but their real power depends upon additional
knowledge about interactions between them! Accordingly we
four legs, some assortment of rungs, a level seat,
an upper back. One expects also certain relations
between these "parts." The legs must be below
I
the seat, the back above. The legs must be
need not abandon an ideal because of a failure to instantiate
it, provided one can explain the discrepancy in terms of such
an interaction. Here are some examples in which such an
"excuse" can save a failing match:
supported by the floo'r. The seat must be
horizontal, the back vertical, and so forth.
Now suppose that this description does not match;
i
the vision system finds four legs, a level plane, but
OCCLUSION: A table, in a certain view, should have four legs,
but a chair might occlude one of them One can look
for things like T-joints and shadows to support such an
no back. The "difference" between what we
expect and what we see is "too few backs." This
suggests not a chair, but a table or a bench.
t
excuse.

FUNCTIONAL VARIANT: A chair-leg is usually a stick,

Winston proposes pointers from each description
in memory to other descriptions, with each pointer labelled
by a difference marker. Complaints about mismatch are
t
geometrically; but more important, it is functionally a
matched to the difference pointers leaving the frame and thus

I
support. Therefore, a strong center post, with an
may propose a better candidate frame. Winston calls t h e
adequate base plate, should be an a c c e p t a b l e resulting structure a Similarity Network.
replacement for all the legs. Many objects are multiple
purpose and need functional rather than physical
descriptions.
Is a Similarity Network practical? At first sight,
there might seem to be a danger of unconstrained growth of
memory. If there are N frames, and K kinds of differences,
|
BROKEN: A visually missing component could be explained as
then there could be as many as K*N*N interframe pointers.
in fact physically missing, or it could be broken. One might fear that:
Reality has a variety of ways to frustrate ideals.

PARASITIC CONTEXTS: An object that is just like a chair,

(]) If N is large, say 10, then N*N is very large - -
of the order of 10 - - which might be
t
except in size, could be (and probably is) a toy chair.
The complaint "too small" could often be so interpreted
in contexts with other things too small, children playing,
impractical, at least for human memory.

(2) There might be so many pointers for a given

|
peculiarly large "grain," and so forth.
difference and a given frame that the
In most of those examples, the kinds of knowledge to make
the repair -- and thus salvage the current frame - - are
system will not be selective enough to be
useful. |
"general" enough usually to be attached to the thematic
(3) K itself might be very large if the system is
context of a superior frame. sensitive to many different kinds of issues.

But, according to contemporary opinions (admittedly, not very

i
conclusive) about the rate of storage into human long-term
In moving about a familiar house, we already know
a dependable structure for "information retrieval" of room
frames. When we move through Door D, in Room X, we
memory there are probably not enough seconds in a lifetime
to cause a saturation problem. |
So the real problem, paradoxically, is that there
expect to enter Room Y (assuming D I.s not the Exit). We will be too few connections! One cannot expect to have
could represent this as an action transformation of the
simplest kind, consisting of pointers between pairs of room
frames of a particular house system.
When the house is not familiar, a "!ogical" strategy
enough time to fill out the network to saturation Given two
frames that should be linked by a difference, we cannot count
on that pointer being there; the problem may not have
I
occurred before. However, in the next section we see how
might be to move up a level of classification: when you
leave one room, you may not know which room you are
entering, but you usually know that it is some room. Thus,
to partially escape this p~oblem.
I
one can partially evade lack of specific information by dealing
with classes -- and one has to use some form of abstraction
or generalization to escape the dilemma of Bartlett's
commander.
I
It?...
|
!
 Surface Syntactic Frames --- Mainly verb and noun

I structures..
Prepositional and word-order indicator
conventions.
~TCTIIR~
When replacing a frame, we do not want to start

! Surface Semantic Frames - - - A c t i o n - c e n t e r e d

• meanings of words.
all over again. How can we remember what was already
"seen?" We consider here only the case in which the system
has no specific knowledge about what to do and must resort
Qualifiers and r e l a t i o n s c o n c e r n i n g to some "general" strate~. No completely general method
participants, instruments, trajectories and

I strategies, goals, consequences and side-

effects.
can be very good, but if we could find a new frame that
shares enough terminals with the old frame, then some of the
common assignments can be retained, and we will probably do
better than chance.
Thematic Frames - - - Scenarios concerned with

t topics, activities, portraits, setting.

Outstanding •problems and s t r a t e g i e s
commonly connected with topics.
The problem can be formulated as follows: let E
be the cost of losing a certain already assigned terminal and
let F be the cost of being unable to assign some other
terminal. If E is worse than F, then any new frame should

I Narrative Frames - - - Skeleton forms for typical

s t o r i e s , explanations, and arguments.
Conventions about loci, protagonists, plot
retain the old subframe. Thus, given any sort of priority
ordering on the terminals, a typical request for a new frame
should include:
forms, development, etc., designed to help a
i listener construct.a new, i n a t a n t i e t e d
Thematic Frame in his own mind.
(1) Find a frame with as many terminals in common
with [a,b,..,z] as possible, where we list
high priority terminals already assigned in

I 5EO~EgTg TO ~EtIIOST
We can now imagine the memory system as driven
the old frame.

But the frame being replaced is usually already a subframe of

I by two complementary needs.

On one side are items demanding to be properly represented

some other frame and must satisfy the markers of it._ss
attachment terminal, lest the entire structure be lost. This
suggests another form of memory request, looking upward
by being embedded into larger frames; on the other side are rather than downward:

I incompletely-filled frames demanding terminal assignments.

The rest of the system will try to placate these lobbyists, • (2) Find or build a frame that has properties
but not so much in accord with "general principles" as in
[a,b,...,z]

I accord with special knowledge and conditions imposed by the

currently active goals.
When a •frame encounters trouble - - when an If we emphasize d i f f e r e n c e s r a t h e r than a b s o l u t e
important condition cannot be satisfied -- something must be specifications, we can merge (2) and (1):
I done. We envision the following major kinds of accomodation
to trouble.
(3) Find a frame that is like the old frame except
for certain differences [a,b~..,z] between
I MATCHING: When nothing more specific is found, we can
attempt to use some "basic" associative memory
mechanism. This will succeed by itself only in
them.

relatively simple situations, but should play •

t supporting role in the other tactics.

EXCUSE: An apparent misfit can often be excused or

One can imagine a parallel-search or hash-coded •memory to
handle (1) and (2) if the terminals or properties are simple
atomic symbols. (There must be some such mechanism, in any
case, to support a production-based program or some sort of
explained, A "chair" that meets all other conditions but

I is much too small could be a "toy."

ADVICE: The frame contains explicit knowledge about what

pattern matcher.) Unfortunately, there are so many ways to
do this that it implies no specific design requirements.
Although (1) and'(2) are formally special cases of
(3), they are different in •practice because complicated cases
to do about the trouble. Below, we describe an
I extensive, learned "Similarity Network" in which to
embed such knowledge.
of (3) require knowledge about differences. In fact (3) is too
general to be useful as" stated, and we will later propose to
depend on specific, learned, knowledge about differences
between pairs of frames rather than on broad, general
SUMMARY: If a frame cannot be completed or replaced, one
I must give it up..But first one must construct a well-
formulated complaint o r summary to help whatever
process next becomes responsible for reassigning the
principles.
It should be emphasized again that we must not
e x p e c t magic. For difficult, n o v e l problems a new
representation structure will have to be constructed, and th|8

I
subframes left in limbo. will require application of both general and special
knowledge.

!
 I
CI,~I~TES~. CI, R~E~. 6R]~ R 6EO61~RPI~IC
~IR~IhOG¥
.Suppose your car battery ruins down. You believe
I
To make the Similarity Network act more "complete," consider that there is an electricity shortage and blame the generator.
the following analogy. In a city, any person should be able to
visit any other; but we do not build a special road between
The generator can be represented as a mechanical
system: the rotor has a pulley wheel ,driven by a belt from
the engine. Is the belt tight enough? Is it even there? The
I
each pair of houses; we place a group of houses on e
output, seen mechanically, is a cable to the b a t t e r y or
"block." We do not connect roads between each pair of
blocks; but have them share streets. We do not connect ~
each town to every other; but construct main routes,
whatever. Is. it intact? Are the bolts tight? Are the brushes
pressing on the commutator?
Seen electrically, the generator is described
t
connecting the centers.of larger groups. Within such an
differently. The rotor is seen as a flux-linking coil, rather
organization, each member has direct links to some other
individuals at his own "level," mainly to nearby, highly similar
ones; but each individual has also at least a few links to
than as a rotating device. The brushes and commutator are
seen.as electrical switches. The output is current along a
pairof conductors leading from the brushes through control
I
"distinguished" members of higher level groups. The result is
that there is usually a rather short sequence between any
two individuals, if one can but find it.
A t each level, .the aggregates usually have
circuits to the battery.
The differences between the two frames are
substantial. The entire mechanical chassis of the car plays
I
distinguished loci or capitols. These serve as elements for the simple role, in the electrical frame, of one of the battery
clustering at the next level of aggregation. There is no non-
stop airplane service between New Haven and Sen Jose
because it is more efficient overall to share the "trunk" route
connections. The diagnostician has to use b o t h
representations. A failure of current to flow often means
that an intended conductor is not acting like one. For this
t
between New York and San Francisco, which are the capitols case, the basic transformation between the frames depends
at that level of aggregation.
The non-random convergences and divergences of
the similarity pointers, for each difference ~ thus tend to
on the fact that electrical continuity is in general equivalent
to firm mechanical attachment. Therefore, any conduction
disparity revealed by electrical measurements should make us
look for a corresponding disparity in the mechanical frame. In
1
structure our conceptual .world around

' (l). the aggregation into d-clusters

(2) the selection of d_-capitols
fact, since "repair ~ in this universe is synonymous with
"mechanical repair," the diagnosis must end in the mechanical
frame. Eventually, we might ocate a defective mechanical
I
junction and discover a loose connection, corrosion, wear, or
Note that it is perfectly all right to have several capitols in a
clusterj so that there need be no one attribute common to
them all. The "crisscross resemblances" of Wittgenstein are
whatever.
One cannot expect to have e frame exactly right
for any problem or expect always to be able to invent one.
t
then consequences of the local connections in our similarity But we do have a good deal to work with, and it is important
network, which are surely adequate to explain how we can
feel as though we know what is a chair or a game - - yet
cannot always define it in a "logical" way as an element in
to remember the contribution of one's culture in assessing
'the complexity of problems people seem t o solve. Th e
experienced mechanic need not routinely invent~ he already
I
some class-hierarchy or by any other kind of compact, formal,
declarative rule. The apparent coherence of the conceptual
aggregates need not reflect explicit definitions, but can
has engine representations in terms of ignition, lubrication,
cooling, timing, fuel .mixing, transmission, compression, and so
forth. Cooling, for example, is already subdivided into fluid
!
emerge from the success-directed sharpening of the circulation, air flow, thermostasis, etc. Most "ordinary"
difference-describing processes.
The selection of capitols corresponds to selecting
stereotypes or typical elements whose default assignments
problems are presumably solved by systematic use of the
analogies provided by the transformations between pairs of
these structures. The huge network of knowledge, acquired
!
are unusually useful. There are many forms of chairs, for from school, books, apprenticeship, or whatever is interlinked
example, and one should Choose carefully the chair-
description frames that are to be the major capitols of chair-
land. These are used for rapid matching and assigning
by difference and relevancy pointers. No doubt the culture
imparts a good deal of this structure by its conventional use
of the same words in explanations of different views of e
!
priorities to the various differences. The lower priority subject.
features of the clustercenter then serve either as default
properties of the chair types or, if more realism is required,
as dispatch pointers to the local chair villages end towns.
,i
Difference pointers could be "functional" as well as
Over the past decade, it has become widely
r e c o g n i z e d how important are the d e t a i l s of t h e
representation of a "problem space"; but it was not so well
!
geometric. Thus, after rejecting a first try at "chair" one
recognized that descriptions can be useful to a program, as
might try the functional idea of "something one can sit on" to
explain an unconventional form. This requires • deeper
analysis in terms of forces and strengths. Of course, that
well as to the person writing the program Perhaps progress
was actually retarded by ingenious schemes to avoid explicit
manipulation of descriptions. Especially in "theorem-proving"
!
analysis would fail to capture toy chairs, or chairs of such
ornamental delicacy that their actual use would be
unthinkable. These would be better handled by the method
of excuses, in which one would bypass the usual geometrical
and in "game-playing" the dominant paradigm of the pest
might be schematized so:
!
or functional explanations in favor of responding to contexts
involving art or play.
!
I
The central goal of
I a Theory of
Problem Solving i s
I. Make a crude first attempt by the first order
met~hod of simply p u t t i n g t o g e t h e r
procedures that separately achieve the
to find systematic
individual goals.
ways to reduce
I the extent of the
Search through the
2. If something goes wrong, try to characterize one
of the d e f e c t s as a s p e c i f i c (and
• undesirable) kind of interaction between two
Problem Space.
procedures.
I Sometimes a simple problem is indeed solved by trying a
3. Apply a "debugging technique" that, according to a
record in memory, is good at repairing that
specific kind of interaction.
sequence of "methods" until one is found to work. Some
4. Summarize the experience, to add to the
harder problems are solved by a sequence of local
"debugging techniques library" in memory.
improvements, by "hill-climbing" within the problem space.
But even when this solves a particular problem, it tells us
little about the problem-space; hence yielding no improved
These might seem simple-minded, but if the new problem is
1 future competence. The best-developed technology of
Heuristic Search is that of game-playing using tree-pruning,
plausible-move generation, and terminal-evaluation methods.
not too radically different.from the old ones, then they have a
good,chance to work, especially if one picks out the right
first-order approximations. If the new problem is radically
But even those systems that use hierarchies of symbolic
different, one should not expect any learning theory to work
I goals do not improve their understanding or refine their
understanding or refine their representations. But there is a
more mature and powerful paradigm:
well. Without a structured cognitive map - - without the
"near misses" of Winston, or a cultural supply of good training
sequences of problems -- we should not expect radically

i The primary purpose in problem solving

should be better to understand the
new paradigms to appear magically whenever we need them.

gO~E F~EhEV~RT F~E~Dm~

| problem.space, to find representations

within which the problems are easier to
solve. The purpose of search is to get
Abelson, R. P. "The Structure of •Belief Systems." Computer
Models of Thought an._ddLanguage. Ed. R. Schank
information for this reformulation, not -- and K. Colby. San Francisco: W. H. Freeman,
as is usually assumed - - to find
t solutions; orce the space is adequately
understood, solutions to problems will
more easily be found.
1973.

Bartlett, F. C. Remembering. Cambridge: Cambridge

University Press, | 967.

I The value of an intellectual experiment should be assessed

along the dimension of success ~ partial success - failure, or
Berlin, I. T.h.e Hedgehog and the Fox. New York: New
American Library, 1957.
in terms of "improving the situation" or "reducing a
I difference." An application of a "method," or a
reconfiguration of a representation can be valuable if it leads
Celce-Murcia, M. Paradigms for Sentence Recognition. Los
Angeles~ Univ. of California, Dept. of Linguistics,
197Z
to a way to improve the strategy of subsequent trials.
Earlier formulations of the role of heuristic search strategies
i did not emphasize these possibilities, although they are
implicit in discussions of "planning."
Chafe, W. First Tech. Report, Contrastive Semantics ProjecL
Berkeley: Univ. of California, Oept. of Linguistics,
1972.
Papert (1972; see also Minsky 1972) is correct in
! believing that the ability to diagnose and modify one's own
procedures is a collection of specific and important "skills."
Debugging, a fundamentally important component of
Chomsky, N. "Syntactic Structures." (Ori~nally published as
"Strukturen der Syntax") Janua Linguarum Studia
Memoriae, ! 82 (! 957).

l intelligence, has its own special techniques and procedures.

Every normal person is pretty good at them. or otherwise he
would not have learned to see and talk! Goldstein (AIM-305)
and Sussman (TR-297) have designed systems which build
Fillmore, C. J. "The Case for Case." Universals in Linguistic
Theory. Ed. Bach and Harms. Chicago: Holt,
Rinehart and Winston, i 968.
new procedures to satisfy multiple requirements by such
Freeman, P. and A. Newell. "A Model for Functional
I elementary but powerful techniques as:
Reasoning in Design." Proc. Second. Intl. Conf.
o_~nArtificial Intelligence. London: Sept. 1971.

I Gombrich, E H. Ar_!tand Illusion, A_Study in th.__ee~ of

Pictorial Representatio.n. Princeton: Princeton
University Press, t969.

Hogarth, W: The Analys)s of Beauty. Oxford: Oxford

t University Press, 1955.

I
 I
Sandewall, E. "Representing Natural Language Information in
!
Huffman, D. A. "Impossible Objects as Nonsense Sentences."
Machine Intelligence 6. Ed. D. Michie and B. Predicate Calculus." Machine!ntelligence 6. Ed. D.
Meltzer. Edinburgh: Edinburgh University Press,
] 972.
Michie and B~ Meltzer. Edinburgh: Edinburgh
University Press, 1972. t
Schank, R. "Conceptual Dependency: A Theory of Natural
Koffka, K. Principles of Gestalt Psychology. New York:
Harcourt, Brace and World, ] 963. Language Understanding." Cognitive Psychology
(]972), 552-63l. see also Schank, R. and K. Colby,
|
Kuhn, T. The Structure Of Scientific Revolutions. 2nd ed. Computer Models of Thought and Language. San
Chicago: University of Chicago Press, 1970. Francisco: W. I-1.Freeman, ] 973..

Lavoisier, A. Elements of Chemistry. Chicago: Regnery,

] 949.
Simmons, R. F. "Semantic Networks: Their Computation and
Use for Understanding English Sentences."
I
Computer Models of Thought and Language. Ed. R.
Levin, J. A, Network Representation and Rotation of Letters.
Publication of the Dept. of Psychology, University
of California, La Jolla, 1973.
Schank and K. Colby. San Francisco: W. H.
Freeman,. 19'73. I
•Underwood, S. A. and C. L. Gates, Visual Learning and
Minsky, M. "Form and Content in Computer Science." 1970
ACM Turing Lecture. Journal of the~ ACMp 17, No.
2 (April 1970), 197-215.
Recognition by Computer, TR-_]22~3 Publications of
Elect. Res. Center, University of Texas, April,
1972.
'I
Minsky, M. and S. Papert. Perceptrons. Cambridge: M.I.T.
Press, 1969.
Wertheimer, M. Productive Thinking. Evanston, IlL: Harper &
Row, 1959. i
Moore, J. and A. Newell. "How can MERLIN Understand?" Wilks, Y. "Preference Semantics/' Memo AIM-206,
• Knowledge and Cognition. Ed. J. Gregg. Potomac,
Md.: Lawrence Erlbaum Associates, 1973.
Publications of Stanford Artificial Intelligence
Laboratory, Stanford University, July, 1973. tt
Wilks, Y. "An Artificial Intelligence Approach to Machine
Newelli A. Productions Systems: Models .of Control
Structures, Visual Information Processing. New
York: Academic Press, 1973.
Translation." Computer Models of Thought and
Language. Ed. R. Schank and K. Colby. San
Francisco: W. H. Freeman, ! 973.
i
Newell, A. "Artificial Intelligence and the Concept of Mind."
Computer Models of Thought and Language. Ed. R.
Schank and K. Colby. San Francisco: W. H;
,I
Freeman, 1973.

Newell, A. and H. A. Simon. Human Problem Solving.

Englewood-Cliffs, N.J.: Prentice-Hall, 1972.
I
Norman, D. "Memory, Knowledge and the Answering of
Questio.ns." Loyola Symposium on Cognitive
Psychology,Chicago, 1972.
I
Papert, S. "Teaching Children to be Mathematicians vs.
Teaching about Mathematics." Int. J. Matk Edu¢.
Sc.~i. Technol., 3_(1972), 249-262.
t
Piaget, J. Si_~xPsychological Studies. Ed. D. Elklnd. New
York: Vintage, 1968. I
Piaget, J. and B. Inhelder. The Child's Conception of Space.
New York: The Humanities Press, | 956.

Pylyshyn, Z.W. "What the Mind's Eye Tells the Mind's BraiR"
I
Psychological Bulletin. 80 (1973), 1-24.

Roberts, L. G. Machine Perception of Three Dimensional

Solids, Optical a..nd Optoelectric Information,
I
Processing. Cambridge: M.I.T.Press, 1965.
t
1