464

Opinion

TRENDS in Cognitive Sciences Vol.6 No.11 November 2002

Words or Rules cannot exploit the regularity in exceptions

Reply to Pinker and Ullman
Pinker and Ullman [1] succinctly restate their position that the English past tense is governed by two competing mechanisms, identified as words and rules, and taken as examples of distinct procedural and declarative systems. Their mechanisms work separately, so that only one or the other is responsible for yielding a particular past tense form. To produce the past tense of keep, words and rules race to generate a response; as the correct past tense of keep is not regular, it must be generated by the lexical mechanism. For this reason, we think of their approach as the words or rules theory. Our approach is different. An integrated connectionist network maps from the stems of all verbs to their past-tense forms, using a single network of units and connections. For example, in the original Rumelhart and McClelland model [2], the same units and connections that produce regular past tenses from regular stems also process the irregulars, so the network has an inherent tendency to do the same thing to the exceptions that it does to regulars namely, copy the features of the stem to the past-tense form and add /d/, /t/ or /^d/ depending on the final consonant. To produce kept instead of keeped (note that both end with unvoiced /t/) all that is required is to adjust the activations of the output units representing the vowel, something that the network will have learned to do on the basis of experience with keep and its neighbors creep, leap, sleep, sweep and weep. The network uses the same connection-based knowledge that allows it to perform the regular mapping, and also taps into specific connections activated by the particular properties of keep to produce the vowel adjustment. A core difference between these approaches is that one exploits the regularity in the exceptions what we call quasi-regularity and the other does not. Quasi-regularity is the tendency for an exception to exhibit aspects of the regular pattern [3]. If there were only a few quasi-regular items, one might treat them as accidents, but in fact nearly all exceptional past-tenses in English are quasi-regular to some extent. To demonstrate this, we will review the different types (for other taxonomies, see [4,5]).
http://tics.trends.com

James L. McClelland Center for the Neural Basis of Cognition and Dept of Psychology, Carnegie Mellon University, 115 Mellon Institute, 4400 Fifth Avenue, Pittsburgh, PA 15213, USA. e-mail: [email protected] Karalyn Patterson Medical Research Council, Cognition and Brain Sciences Unit, 15 Chaucer Road, Cambridge, UK CB2 2EF . e-mail: karalyn.patterson@ mrc-cbu.cam.ac.uk

(1) Two very frequent verbs, have and make, delete a consonant and add the regular /d/ to what remains, forming had and made. (2) The -eep words listed above and others, including say, do, tell, sell, hear, flee and shoe, form the past tense by adding regular /d/ or /t/ and making a vowel adjustment, producing kept, said, did, told, etc. (3) Twenty-eight verbs, like cut and hit, have past tenses identical to their stems; all end in /d/ or /t/, as regular past tenses do. (4) Another set of verbs ending in /d/ or /t/, including bleed, breed, feed, lead, read, speed, hide, ride, slide and fight, adjust the vowel to create /d/- or /t/-final bled, slid, fought, etc. Several sets of verbs (waning in some dialects) use unvoiced /t/ instead of /d/, usually after /l/ or /n/: (5) One such set, including dwell, smell, spell, spill, burn and learn, would be completely regular except for the de-voicing of the inflection, producing past forms like spelt and burnt. (6) Another group, including mean, dream, deal, feel and kneel, adjust the vowel and add /t/, yielding meant, dealt, etc. (7) A third set, including build, bend, lend, rend, send and spend, replace stem-final /d/ with /t/ to make built, sent, etc. (8) Yet another set bring, catch, seek, teach and think adjust the vowel to /aw/ and replace the final consonant cluster with /t/, creating brought, caught, etc. Overall, 59% of the 181 English exceptions listed by Pinker and Prince [5] have past tenses ending in /d/ or /t/, and fall into one of classes (1)(8). (9) Nearly all of the remaining verbs are also quasi-regular, in that the consonants of the stem are preserved. Instead of adding /d/ or /t/, the past tense is formed by making a vowel change, as in sing-sang, rise-rose and fly-flew. There are only two suppletive verb roots in English, be and go, with derivatives forgo and undergo, where the past-tense form is completely different from the present tense. As noted above, the PinkerUllman theory provides no mechanism for exploiting the aspects of the regular past tense that are so prevalent among exceptions. Pinker did adopt the idea that the lexical system has connectionist-like properties [6]. This provided a way to account for clusters among the exceptions and for creative formation of novel forms consistent with such clusters. This was a step in the right direction, but did not go far enough. Because past tenses of exceptions in this account are formed by the lexical system alone, the theory still fails to explain why many of the exceptions share properties with regular past-tense forms and offers no way to exploit the regular mapping in forming past tenses of these exceptions. By contrast, connectionist models inherently capture the regularity in the exceptions because the exceptions are processed by the same network

1364-6613/02/$ see front matter 2002 Elsevier Science Ltd. All rights reserved. PII: S1364-6613(02)02012-0

Opinion

TRENDS in Cognitive Sciences Vol.6 No.11 November 2002

465

that processes the regulars. As already noted for keep-kept, items that are quasi-regular can make partial use of the same connections that are used in forming exceptions. All nine of the types noted above, encompassing 177 out of 181 forms, exploit to some degree the connection weights that produce regular items. Only the suppletive items fail to make any use of the connections that produce the regular past tense [7]. The past tense of English is just one domain that exhibits quasi-regularity. In English spellingsound mapping, virtually every exception has some degree of regularity; pint, aisle, hymn and champagne all
References 1 Pinker, S. and Ullman, M. (2002) The past and future of the past tense. Trends Cogn. Sci. 6, 456463 2 Rumelhart, D.E. and McClelland, J.L. (1986) On learning the past tenses of English verbs. In Parallel Distributed Processing (Vol 2): Psychological and Biological Models (McClelland J.L. and Rumelhart D.E., eds), pp. 216271, MIT Press 3 Plaut, D.C. et al. (1996) Understanding normal and impaired word reading: computational principles in quasi-regular domains. Psychol. Rev. 103, 56115 4 Bybee, J.L. and Slobin, D.L. (1982) Rules and schemas in the development and

partially adhere to regular correspondences. Quasi-regularity exists in richly inflected languages like Spanish, and in derivational as well as inflectional morphology [8,9]. It is found in language units beyond the word level [10,11] and, beyond language, it characterizes real-world objects, which have properties shared with other related objects as well as some unique properties [12]. Given these observations, the plausible candidate mechanisms of human linguistic and conceptual processes are those that can exploit quasi-regularity. Single-system connectionist models have this property; the Words or Rules theory does not.
9 Burzio, L. (2002) Missing players: phonology and the past tense debate. Lingua 112, 157199 10 Harris, C.L. (1994) Coarse coding and the lexicon. In Continuity in Linguistic Semantics (Fuchs, C. and Victorri, B., eds), John Benjamins 11 McClelland, J. L. (1992) Can connectionist models discover the structure of natural language? In Minds, Brains and Computers (Morelli, R. et al., eds), pp. 168189, Ablex Publishing 12 Rogers, T.T. and McClelland, J.L. Semantic Cognition: A Parallel Distributed Processing Approach. MIT Press (in press)

6 7

use of the English past pense. Language 58, 265289 Pinker, S. and Prince, A. (1988) On language and connectionism: analysis of a parallel distributed processing model of language acquisition. Cognition 28, 73193 Pinker, S. (1991) Rules of language. Science 253, 530535 Plunkett, K, and Marchman, V. A. (1991) U-shaped learning and frequency effects in a multi-layered perceptron: implications for child language acquisition. Cognition 38, 43102 Bybee, J.L. (1985) Morphology: A Study of the Relation Between Meaning and Form. John Benjamins

Rules or connections in past-tense inflections: what does the evidence rule out?
James L. McClelland and Karalyn Patterson
Pinker and colleagues propose two mechanisms a rule system and a lexical memory to form past tenses and other inflections. They predict that childrens acquisition of the regular inflection is sudden; that the regular inflection applies uniformly regardless of phonological, semantic or other factors; and that the rule system is separably vulnerable to disruption. A connectionist account makes the opposite predictions. Pinker has taken existing evidence as support for his theory, but the review of the evidence presented here contradicts this assessment. Instead, it supports all three connectionist predictions: gradual acquisition of the past tense inflection; graded sensitivity to phonological and semantic content; and a single, integrated mechanism for regular and irregular forms, dependent jointly on phonology and semantics.

One view of language, originating with Chomsky [1,2], championed by Fodor and
http://tics.trends.com

Pylyshyn [3] and widely pursued by Pinker [47], holds that abstract symbolic rules play a central role in human language processing. This claim is part of a broader view that human cognitive mechanisms are symbolic, modular, innate and domain-specific [4]. An alternative view, from Rumelhart and McClelland [8] (see Box 1), challenges the need for the use of rules. This view arises within the Parallel Distributed Processing (PDP) or connectionist framework [9], in which cognitive processes are seen as graded, probabilistic, interactive, context-sensitive and domain-general. Acquisition of language and other abilities occurs via gradual adjustment of the connections among simple processing units. Characterizations of performance as rule-governed are viewed as approximate descriptions of patterns of language use; no actual rules operate in the processing of language. These perspectives apply to many aspects of language, and, as Pinker and Ullman suggest [10], to many other domains as well, but here we focus on inflectional morphology, especially the English past tense. The idea of a past tense rule arose from noting that young children sometimes regularize irregular verbs, producing for example, goed or felled [11], and from the finding that children (and adults) typically produce regular forms for nonce (novel) words in a past-tense elicitation task [12]. Given a picture of a man said to be ricking and a request to complete Yesterday he ___,

1364-6613/02/$ see front matter 2002 Elsevier Science Ltd. All rights reserved. PII: S1364-6613(02)01993-9