Handbook

of
Operant Behavior
Edited by

WERNER K. HONIG
Dalhousie University

J. E. R. STADDON
Duke University

Prentice-Hall, Inc., Englewood Cliffs, New Jersey

Library of Congress Cataloging in Publication Data
Main entry under title:

Handbook of Operant Behavior.

(The Prentice-Hall century psychology series)

Includes bibliographies and index.

1. Operant behavior. I. Honig, Werner K. II. Staddon,
J. E. R.
BF319.5.06H36 , 152.3’224 76-26034
ISBN 0-13-380535-2

The Prentice-Hall

CENTURY PSYCHOLOGY SERIES

(g) 1977 by Prentice-Hall, Inc., Englewood Cliffs, New Jersey

All rights reserved. No part of this book

may be reproduced in any form
or by any means without permission
in writing from the publisher.

Printed in the United States o£ America

10 98765432

Prentice-Hall International, Inc., London

Prentice-Hall of Austria Pty. Limited, Sydney
Prentice-Hall of Canada, Ltd., Toronto
Prentice-Hall of India Private Limited, New Delhi
Prentice-Hall of Japan, Inc., Tokyo
Prentice-Hall of Southeast Asia Pte. Ltd., Singapore
Whitehall Books Limited, Wellington, New Zealand
 Contents

ACKNOWLEDGEMENTS, ix Behavior, 23
References, 24

CONTRIBUTORS, xi

2
INTRODUCTION, 1
THE OPERANT REVISITED, 28
Werner K. Honig and J. E. R. Staddon
George Collier, Edward Hirsch, and Robin Kanarek

References, 6
Introduction, 28
Free Feeding, 34
1 Availability, 36
Caloric Regulation and Choice of Dietary Items, 43
LEVELS OF INTEGRATION OF THE OPERANT, 7 Other Environmental Constraints, 44
Response Strength, 44
Philip Teitelbaum
Food Economy, 46
Conclusions, 49
Introduction, 7 References, 50
Historical Background, 8
The Operant as a Criterion for Motivation, 12
Puzzling Operants, 13
Similar Puzzles in Motivated Behavior, 14
3
Recovery from Lateral Hypothalamic Lesions, 16 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR, 53
Parallel Between Recovery and Development in the
Lateral Hypothalamic Syndrome, 17 Barry Schwartz and Elkan Gamzu
Stages of Recovery and Development of the Human
Grasp, 19 Introduction, 53
Transformation of Sensory Control over an Approach Autoshaping and Automaintenance, 54
Response, 19 The Role of Stimulus-Reinforcer Relations in the
Stages of Recovery and Development of Learned Control of Behavior Maintained by Response-
Behavior, 20 Reinforcer Relations, 71
Stages of Encephalization of the Operant, 20 Conclusion, 91
Summary and Conclusions: Levels of Operant References, 92

v
vi Contents

Comparisons of the Effects of Drugs on Performances

4
Maintained by Different Consequences, 188
THE NATURE OF REINFORCING STIMULI, 98 Drug Injections as Consequent Events Maintaining
Behavior, 192
Philip Dunham
Response-Produced Electric Shocks as Consequent
Events Maintaining Behavior, 193
A Historical Perspective, 98 Conclusions, 197
Premack’s Reinforcement Theory, 101 References, 198
Biological Constraints on Reinforcement, 112
Concluding Comments, 122
References, 123

8
5 SCHEDULES OF REINFORCEMENT:
THE CONTROLLING VARIABLES, 201
SCHEDULE-INDUCED BEHAVIOR, 125
Michael Zeiler
J. E. R. Staddon

Introduction, 201
Introduction, 123 Types of Schedules, 202
Behavior Induced by Periodic Food, 126 Types of Controlling Relations: Variables and
Temporal and Sequential Structure of Induced Effects, 203
Activities, HO Variables Determining Response Frequency, 206
Concluding Gommcnts* H8 Response Patterning! The Temporal Organization of
References* 118 Behavior, 213
Sequences and Units, 221
Summary and Concluding Remarks, 228
References, 230

6
THERMOREGULATORY BEHAVIOR, 153

Evelyn Satinoff and Robert Hendersen

9
IntradusPon, 133 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE
Se paration of Operant and Respondent T^ffip^rSttur^
FORMULATION OF THE LAW OF EFFECT, 233
Regulation, 134 Peter de Villiers
Using Behavior to Assess Regulation, 136
Thermoregulation and the Concept of Set Point, 160
Introduction, 233
Thermal Preference, 162
Operant Contingencies in Thermal Homeostasis, 164 Concurrent Schedules, 234
The Operant as a M«5asur£ of Set Poiftt After Dmg
The Matching Relation m Concurrent VI Schedules—
Administration, 163 Reinforcement Frequency, 233
References, 169 Maximizing or Matching, 243
Time Matching as the Fundamental Matching
Process, 246
The Generality of the Matching Relation, 248
Absolute Rates of Responding and a Quantitative
7 Law of Effect, 237
Alternative Theories of Response Strength, 263
DETERMINANTS OF REINFORCEMENT Application of Herrnstein’s Equations to Other
AND PUNISHMENT, 174 Schedules, 265
An Alternative Theory of Matching and Behavioral
W. H, Morse and R, T. Kelleher
Contrast, 272
Discussion, 275
Reproducible Behavioral Processes, 174 Conclusion, 278
The Continuity of Behavior in Time (Shaping), 177 Appendix A, 278
Disparate Effects of Consequent Events, 178 Appendix B, 279
Ongoing Behavior, 180 Appendix C, 279
Characteristics of Responses, 183 Appendix D, 281
Adventitious Reinforcement and Punishment: Appendix E, 281
Importance of History, 184 Appendix F, 282
Criteria for Comparing Consequent Events, 186 References, 282
 Contents vii

10 14
CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS, 288 BY-PRODUCTS OF AVERSIVE CONTROL, 415

Lewis Gollub R. R. Hutchinson

Introduction, 415
Introduction, 288
Methods, 416
Chained Schedules of Reinforcement, 289
Behavior Caused by Aversive Stimulation, 418
Schedules of Brief Stimulus Presentation, 299
Behavior Caused by Aversive Stimuli in Escape
Concluding Remarks, 308
Paradigms, 425
References, 309
Behavior Caused by Aversive Stimuli in Punishment
Paradigms, 427
References, 430

11
CONDITIONED REINFORCEMENT: CHOICE AND
INFORMATION, 313
15
Edmund Fantino
STIMULUS CONTROL AND INHIBITORY PROCESSES, 432

Introduction, 313 Mark Rilling

Observing Responses and Conditioned
Reinforcement, 318 Overdew, 432
Choice and Conditioned Reinforcement, 326 The Definition and Measurement of Stimulus
Conclusions, 336 Control, 433
References, 337 Some Determinants of Generalization Gradients, 436
Influence of Discrimination Training on the
Generalization Gradient, 439
Determinants of the Peak Shift and Inhibitory
12 Stimulus Control, 453
Errorless Learning Reconsidered, 464
CONDITIONED SUPPRESSION AND THE EFFECTS OF
Summary, 475
CLASSICAL CONDITIONING ON OPERANT BEHAVIOR, 340 References, 476
Derek Blackman

Introduction, 340
The Estes-Skinner Procedure and the Measurement 16
of its Effects, 341
STIMULUS CONTROL: ATTENTIONAL FACTORS, 481
Investigations of Classical Conditioning Parameters,
342 N. J. Mackintosh
Investigations of Operant Conditioning Parameters,
344
Introduction, 481
Measurement of Conditioned Suppression, 348
Conditions Affecting the Establishment of Stimulus
Some Interpretations of Conditioned Suppression,
Control, 483
351
Experimental Procedures: Nondifferential
A Brief Review of Some Other Classical-Operant
Reinforcement and Discrimination Training, 488
Interactions, 358
Discussion, 505
Conclusion, 360
References, 510
References, 361

13 17
NEGATIVE REINFORCEMENT AND AVOIDANCE, 364 ANIMAL PSYCHOPHYSICS, 514

Philip N. Hineline Donald Blough and Patricia Blough

Introduction, 364 Introduction, 514

Two Illustrative Experiments, 365 Measuring Sensory Thresholds, 515
Negative Reinforcement Without Added Cues, 367 Supraliminal Stimuli, 525
Negative Reinforcement With Added Cues, 381 Signal Detection Theory in Animal Psychophysics,
Considerations Regarding Initial Acquisition, 406 532
References, 410 References, 537
viii Contents

18 21
OPERANT BEHAVIORAL PHARMACOLOGY, 540 PROCEDURES FOR THE ACQUISITION OF SYNTAX, 619

Travis Thompson and John J. Boren George Robinson

Introduction, 540 References, 621

Principles of Drug Action, 543
Analyzing Behavioral Mechanisms of Drug Action,
551
Traditional Problems Formulated Within an Operant 22
Framework, 560 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE, 628
Future of Behavioral Pharmacology, 562
References, 566 Evalyn Segal

Toward a Coherent Psychology of Language, 628

Competence: Performance "Structure: Function, 628
19 Chomsky’s Standard Theory of Transformational-
GINTRAk RiiNFQRCEMENL A bridsE BETWEEN IRAIN Generative Grammar, 629
fuNCtioN ANd BiHAViOR, 179 The Psychological Reality ol Transformational-
Generative Grammar, 632
Gordon Mogenson and Jan Cioe
Skinner’s Functional Theory of Verbal Behavior,
633
Introduction) 550 Comments on Skinner’s Functional Theory, 635
Methodological Considerations, 552 Functionalism vs. Mentalism, 640
Central Reinforcement Compared to Conventional The Complementarity of Functional and Cognitive
Reinforcement, 55 4 Theory, 642
Some Implications of Comparing Central and Paraphrase, the Problematic Listener, and
Conventional Reinforcers, 580 Mentalese, 649
The Nature of Central Reinforcement, 581 More on the Complementarity of Functional and
Summary, 588 Cognitive Theories, 651
Appendix A, 589 References, 652
References, 590

AUTHOR INDEX, 655

20
THE EXPERIMENTAL PRODUCTION OE ALTERED SUBJECT INDEX, 660
PHYSIOLOGICAL STATES; CONCURRENT AND CONTINGENT
BEHAYIORAL MODELS, 5?b

Joseph Brady and Alan Harris

Introduction, 506
Concurrent Models, 596
Recent Developments, 598
Contingent Models, 606
References, 611
 Acknowledgements

The authors who have contributed to this book de¬ cil of Canada for grant APT-102 to WKH, and to the
serve most of the credit for its content, and as editors U.S. National Science Foundation for grant BMS71-
we want to express our gratitude to them. All of them 01402 to JERS, who gratefully acknowledges support
have worked hard, and cooperated with us. Some have from both NSF and Duke University for a sabbatical
had to await the publication of manuscripts that they year at Oxford University when some of the work that
completed all too long ago; others were called upon went into this book was done.
to prepare chapters late in the game, and had to work Much of the tedious labor has been done by the
under pressure of time. It is to their credit that we following people: Janet Lord handled an enormous
have had little cause for argument or complaint in amount of correspondence, typed up many pages of
preparation of these pages. comments, and organized the complex procedure of
The authors have themselves received help and sup¬ obtaining permissions. Edna Bissette coped with ill-
port from colleagues and granting agencies. These typed manuscripts mailed from overseas, as well as
persons and sources are mentioned in their individual mumbled audio tapes of editorial comment. Janice
chapters. Frank is responsible for the monumental effort that
As editors, we have also been encouraged by our went into the subject index, and also helped with
colleagues, students, and wives in our work on this editorial work. Catherine Moore prepared the author
project. They have been tolerant while we turned our index with its many hundreds of entries. This book
attention to the many detailed considerations that go could not have been published without the unstinting
into the preparation of a book of this size. In particu¬ contribution of their time and effort.
lar, we want to acknowledge the support of Kenneth Many journals, societies, and authors have granted
MacCorquodale, the consulting editor who worked permission for the use of illustrations in this book.
with us, encouraged us, and reviewed each chapter. These are acknowledged in the captions of the figures.
Both of us have benefitted from the support of re¬
search grants that have provided the sort of research W.K.H.
staff that made our own time available for editorial J.E.R.S.
work. Thanks are due to the National Research Coun¬

ix
 Contributors

Derek E. Blackman (Chapter 12), Ph.D. The Queen’s in general comparative problems and in problems of
University of Belfast, is Professor and Head of De¬ stimulus control in animals.
partment of Psychology at University College, Cardiff,
Wales. He is the author of Operant Conditioning: An John J. Boren (Chapter 18), Ph.D. Columbia Uni¬
Experimental Analysis of Behaviour (1974), and he or¬ versity, is Professor of Psychology at The American
ganized the Experimental Analysis of Behavior Group University. He has researched and published on a
in the U.K. for five years. Dr. Blackman has published number of issues in the experimental analysis of be¬
many research papers in British and American jour¬ havior, behavioral pharmacology, and applications of
nals. His current interests center on the effects of behavioral principles to human affairs. He has been
drugs on operant behavior, and he is editing a book the president of the Behavioral Pharmacology Society,
with Dr. D. J. Sanger on behavioral pharmacology. the editor of the Journal of the Experimental Analysis
of Behavior, and the president of the Division of
Donald S. Blough (Chapter 17), Ph.D. Harvard Psychopharmacology of the American Psychological
University, is Professor of Psychology at Brown Uni¬ Association. His current interests include the variables
versity. He is interested in the methodology of animal that determine behavior, the repeated acquisition of
psychophysics and animal information processing, and behavior, the effects of acute and chronic administra¬
more generally in operant conditioning and stimulus tion of cocaine and other stimulant drugs, and the
control; he has published extensively in these areas. applications of behavioral principles to clinical
problems.
Patricia M. Blough (Chapter 17), M. S. Tufts Uni¬
versity, is Research Associate in Psychology at Brown Joseph V. Brady (Chapter 20), Ph.D. University of
University. Her combined interests in the areas of Chicago, is Professor of Behavioral Biology at The
vision and animal psychophysics have led to a number Johns Hopkins University School of Medicine. He
of publications concerning the electrophysiology and has published extensively in the areas of behavioral
psychophysics of pigeon vision. She is also interested physiology and behavioral pharmacology, and served
xii Contributors

on the editorial boards of several professional Society for the Experimental Analysis of Behavior
journals including the Journal of the Experimental (1972-1980) and has been visiting professor at the
Analysis of Behavior, the Journal of Comparative and Primate Research Institute, Kyoto University (1974).
Physiological Psychology, and Behavior Research His principal research interests are choice, condi¬
Methods and Instrumentation. Professor Brady’s tioned reinforcement, and self-control. He is currently
principal scientific contributions have been in the writing a text on learning and motivation, with an
area of somatic and physiological affects of behavioral emphasis on comparative psychology (with Cheryl
stress, Logan).

Jan D. D. Cioe (Chapter 19), M.A. University of Elkan Gamzu (Chapter 3), Ph.D. University of Penn¬
Western Ontario, is at present a Lecturer in Psychol¬ sylvania, is presently a senior scientist at Hoffmann-
ogy at the College of New Caledonia, Prince George, LaRoche Inc. Hi§ research interests include psycho¬
British Columbia, Canada. He was a winner of a num¬ pharmacology, taste aversion learning, autoshaping,
ber of academic awards and prizes in his undergrad¬ and biological constraints on learning.
uate and graduate years and his proposed Ph.D.
thesis is concerned with brain self-stimulation and
Lewis R, Gqlliib (Chapter 10), Ph,D, Harvard Uni¬
ingestive behaviors.
versity, is Professor of Psychology at the University of
Maryland. He has published numerous papers in be=
George Collier (Chapter 2), Fh,B; Indiana Univer¬
havioral and pharmacological journals, and has
sity, is a Professor of Psychology at Rutgers Uni¬
served on the board of editors of the Journal of the
versity, IBs research interest is the evolutionary origin
Experimental Analysis of Behavior, His mam inter¬
of feeding behavior. He is testing ecological models in
ests have been in the experimental analysis of condi¬
laboratory simulations of ecological niches.
tioned reinforcement, behavioral pharmacology, and
effective teaching,
Peter A. be Villiers (Chapter 9)? Ph.D. Harvard Uni¬
versity, is Assistant Professor of Psychology at Harvard
Alan H. Harris (Chapter 20), Ph.D. Columbia Uni¬
University. He has published several research papers
versity, is an Associate Professor in the Department of
in the areas of operant conditioning, language acqui¬
Psychiatry and Behavioral Sciences at The Johns
sition, and psycholinguistics, He is joint author, with
Hopkins University School of Medicine, and the
his wife Jill, of Early Language (forthcoming), an
Assistant Director of the Division of Behavioral Biol¬
introduction to first language acquisition. His current
ogy, His major research interests are in the areas of
research includes quantitative approaches to the law
conditioning and learning, and he is currently a prin¬
of effect with special reference to aversive condition¬
cipal investigator for the National Heart and Lung
ing, and early grammatical and semantic development
Institute studying the circulatory effects of operant
in children.
cardiovascular conditioning m primates.

PillEIS* J, D^niiam (Chapter 4), Ph.D. University of

Missouri, is Associate Professor of Psychology at Bal- Robert W. Hendersen (Chapter 6), Ph.D. University
liousic University in Halifax, Nova Scotia. In addi= of Pennsylvania, is an Assistant Professor m the Psy¬
lion to journal articles and reviews, he is author of chology Department of the University of Illinois at
the textbook Experimental Psychology• A process ap¬ Champaign-Urbana. His major research interests are
proach (Harper and Row, in press). H is main research aversive learning and acquired motivation.
interests are m the areas of animal learning and moti¬

vation, Philip N. Hineline (Chapter IS), Ph.D. Harvard

University, is Associate Professor of Psychology at
Edmund Fantino (Chapter 11), Ph.D. Harvard Uni¬ Temple University. His research has dealt mainly
versity, is Professor of Psychology and a member of with the aversive control of behavior, attempting to
the Neurosciences Group at the University of Cali¬ isolate the several meanings of avoidance, and to
fornia, San Diego. He is author of Introduction to break the constraints of traditional avoidance proce¬
Contemporary Psychology (with George S. Reynolds, dures. Additional work has been concentrated on an
1975) and of articles appearing in professional analysis of “nuisance phenomena” of transience in
journals. He has served as Associate Editor of the aversively conditioned behavior. In addition, he main¬
Journal of the Experimental Analysis of Behavior tains an active interest in experimental approaches to
(1971-1974), is on the Board of Directors of the teaching the psychology of music.
Contributors xiii

Edward Hirsch (Chapter 2), Ph.D. Rutgers Univer¬ University, is Professor of Physiology and Psychology
sity, is an Assistant Professor at Mount Holyoke Col¬ at the University of Western Ontario, London,
lege. His research interests are the nutritional, phys¬ Canada. He has published over 100 research papers
iological, and environmental bases of obesity. and review articles and was Editor of the Canadian
Journal of Psychology from 1969-1974. He is presently
Werner K. Honig (co-editor of the book, co-author a Regional Editor of Physiology and Behavior and
of the Introduction), Ph.D. Duke University, is Pro¬ the Canadian Journal of Psychology. Professor
fessor of Psychology at Dalhousie University. He has Mogenson’s research is concerned with the role of the
published a number of articles and chapters on stim¬ nervous and endocrine systems in the control of
ulus control in animals, and has participated in motivated and emotional behavior.
several conferences. He spent a year working on the
relations between behavior in natural settings and W. H. Morse (Chapter 7), Ph.D. Harvard University,
instrumental learning as a Guggenheim Fellow at is Associate Professor of Psychobiology in the Depart¬
Madingley, England (1970-71). His principal edited ment of Psychiatry at Harvard Medical School. His
book is the 1966 version of Operant Behavior, pub¬ principal work has been in the field of behavioral
lished by Appleton-Century-Crofts. He has also co¬ pharmacology.
edited smaller books on associative learning and on
animal memory. His current interests are in cognitive Mark Rilling (Chapter 15), Ph.D. University of

aspects of animal learning, particularly “working Texas, is Professor of Psychology at Michigan State
memory” in animals, and the associative significance University. His published research papers have ap¬
of stimuli controlling behavior. peared in the Journal of the Experimental Analysis
of Behavior, Learning and Motivation, Animal Learn¬
Ronald R. Hutchinson (Chapter 14), Ph.D. Yale Uni¬ ing and Behavior, and Science. His main research
versity, is President and Director of Research at the interests are stimulus control and inhibition.
Foundation for Behavioral Research. Dr. Hutchin¬
son’s major research interests include the effects of George M. Robinson (Chapter 21), Ph.D. University
aversive stimulation, environmental causes of aggres¬ of Chicago, is Assistant Professor of Psychology at
sion in man and animals, and effects of drugs on emo¬ Duke University. His research efforts and publica¬
tional behavior. tions are in the areas of cognitive psychology and
psycholinguistics. He is coauthor of The Organization
Robin Beth Kanarek (Chapter 2), Ph.D. Rutgers of Language (with }. M. Moulton, forthcoming).
University, is a Research Fellow at the Harvard
School of Public Health. Her research interests are Evelyn Satinoff (Chapter 6), Ph.D. University of

the nutritional, physiological, and environmental Pennsylvania, is Professor of Psychology and Phys¬
bases of feeding behavior. iology at the University of Illinois at Urbana-
Champaign. Her major research interests are in the
R. T. Kelleher (Chapter 7), Ph.D. New York Univer¬ fields of motivation and biological regulation, and
sity, is Professor of Psychobiology at the New England she has published numerous experimental articles
Regional Primate Research Center, Department of and several chapters in these areas. Her current con¬
Psychiatry, Harvard Medical School. His principal cern is with the neuroanatomical and pharmacological
work has been in the field of behavioral pharma¬ bases of thermoregulation.
cology.
Evalyn F. Segal (Chapter 22), Ph.D. University of
N. J. Mackintosh (Chapter 16), is Professor of Psy¬ Minnesota, is Professor of Psychology at San Diego
chology at the University of Sussex. He is the author State University. She is an Associate Editor of Learn¬
of The Psychology of Animal Learning (1974) and ing and Motivation and serves on the editorial boards
(with N. S. Sutherland) of Mechanisms of Animal of the Journal of the Experimental Analysis of Be¬
Discrimination Learning. His research interests in¬ havior, Behaviorism, and Contemporary Psychology.
clude the study of discrimination learning and stim¬ Her research publications include a chapter on induc¬
ulus control, selective association in conditioning, and tion and the provenance of operants and experi¬
comparative psychology. mental reports on schedule-induced polydipsia, tim¬
ing behavior, and conditioned reinforcement. She was
Gordon J. Mogenson (Chapter 19), Ph.D. McGill for a time Director of the Institute for Child and
XIV
Contributors

Family Development at the University of North Principles. His numerous articles have been con¬
Carolina at Greensboro. She recently completed a cerned mainly with the aberrations in food and water
textbook on the principles of operant behavior and intake produced by damage to the hypothalamaus.
the operant analysis of language, which are also her His current major research interest is in the stages of
current research interests. recovery of motivated eating and drinking and their
implications for normal behavior.
J. E. R. Staddon (Chapter 5), Ph.D. Harvard Univer¬
sity, is Professor of Psychology at Duke University. He T. Thompson (Chapter 18), Ph.D. University of
has also taught at the University of Toronto and Minnesota, is Professor of Psychiatry and Psychology
recently spent a sabbatical year with the Animal Be¬ at the University of Minnesota in Minneapolis. He
havior Research Group at Oxford University. He has has published many original research articles, is co¬
clone research on learning and adaptive behavior in editor of Advai'ices m Behavioral Pharmacology and
animals, philosophical aspects of psychology, and is a Regional Editor for Pharmacologyf Biochemistry
neurobiology. He is interested in the relations be¬ and Behavior. Among his books are Behavioral
tween learning mechanics and evolutionary biology, Pharmacology and Reinforcement Schedules and
Multioperant Analysis. HU main research interests
Barry Schwartz (Chapter 3), Ph D. University of are in behavioral pharmacology and interactions
Pennsylvania^ is AssociaProfgggor of Psychology at among environmental and pharmacological variables
Swarthmorc College. His major research interests are in treatment of retarded and psychotic patients.
in the area of biological constraints on learning, par¬
ticular ly the phenomena of autoshaping and be¬ Michael D. Zeiler (Chapter 8), Ph.D. New School
havioral contrast. He has just completed a text on for Social Research, is Professor of Psychology at
learning titled The Control of Behavior. Emory University. He has published a number of re*
search papers in professional journals and authored
Philip Teitelbaum (Chapter 1), Ph.D. The Johns several chapters. He is presently the Editor of the
Hopkins University, is Professor of Psychology at the Journal of th<? Experimental Analysis of Behavior.
!!fliV@r§ity of Illinois at Champaign-Urban^, jg th©
His major research interest is in the experimental and
author Qt Physiological Psychology; fundamental theoretical analysis of schedules of reinforcement.
 Introduction

Werner K. Honig
and
J. E. R. Staddon

Ten years and more have passed since the publica¬ and in press with Prentice-Hall. The other is entitled
tion o£ Operant Behavior, which was the first effort Social and Instrumental Processes: Foundations and
to provide a reasonably comprehensive account of Applications of Behavioral Analysis. It is edited by
those areas of thought and research in psychology T. A. Brigham and A. C. Catania, and will be pub¬
which were influenced substantially by operant meth¬ lished by Irvington.
ods. The time has come for a reassessment of several of Our own book provides a mixture of experimental
those areas, for a description of other topics involving and theoretical material which reflects the current
operant methods or bearing upon them, and for a status of operant behavior. No chapter is “strictly
conceptual examination of the fundamental principles experimental” in the sense that it fails to raise con¬
of the Experimental Analysis of Behavior and its ceptual and theoretical issues, or concentrates entirely
relationship to other parts of experimental psychol¬ upon methodology. Only a few chapters—those on
ogy. The present Handbook cannot pretend to accom¬ language—are largely theoretical, although empirical
plish all of these aims, or even to do justice to any, studies do, of course, provide some of the material for
but it provides much relevant empirical material, and discussion. Perhaps we would be wisest to let the
many discussions which are both incisive and enlight¬ chapters follow without further comment, but after
ening. giving them many hours of scrutiny, we succumb to
Certain aspects of operant behavior were deliber¬ the temptation of providing the reader with a few
ately excluded from the outset. The enormous in¬ general impressions.
crease in the use of operant methods for both funda¬ First, it is becoming quite clear that operant
mental and applied research makes it impossible to methods and principles are becoming increasingly in¬
cover these major areas in one volume. This book is tegrated in general experimental psychology. At first,
devoted entirely to topics in experimental psychology. the operant movement (if such it should be called)
We have welcomed the planning and publication of was quite isolated, largely due to negative reactions
two companion books: One is the Handbook of Ap¬ from its critics and enemies, who were put off by
plied Operant Behavior, edited by Harold Leitenberg, Skinner’s radical behaviorism, by the artificial and

1
 2 INTRODUCTION

restricted environment of the “Skinner box”, by the photobeam. This effect, which can be accomplished
lack of concern with theoretical issues, or by the ap¬ in any way the animal chooses, is termed the response.
parent threat to traditional freedoms and values posed If it changes the environment in a way that has moti¬
by the prospective control of human behavior through vating consequences, giving access to food or water,
operant methods. Furthermore, operant methods facil¬ or escaping electric shock, the animal will generally
itated new research strategies with little regard for learn to make the response more frequently. This
traditional principles of experimental design. The in¬ change defines the consequence as a reinforcer. The
tensive study of individual subjects across a variety prescribed relation between responding and reinforce¬
of treatments was indeed promoted as a kind of model ment is a response-reinforcer contingency. The fre¬
of experimental method (Sidman, 1960). But the isola¬ quency of the response will be strongly affected by
tion felt by workers in the area of operant behavior stimuli that signal availability or unavailability of the
was partly self-imposed. One does not gain a sense reinforcer (discriminative stimuli).
of compromise from Skinner’s writings. The impres¬ The rule or rules prescribing the relations among
sion conveyed is that the study of operant behavior stimuli, responses, and reinforcers is a reinforcement
should not be contaminated by attention to tradi¬ schedule. Schedules are of interest both in their own
tional problems, methods, and theoretical issues. The right, and as useful “contrivances”, in Jenkins’ phrase,
movement was named The Experimental Analysis of that can be used to tease apart the mechanisms that
Behavior, suggesting that the operant method pro¬ underlie learned behavior, Much of this book has as
vides the only valid and constructive approach to the its experimental basis the very extensive work on
systematic study of behavior. schedules that has taken place in the past fifteen
But the interest in operant behavior has not de¬ years, One approach is to treat reinforcement sched¬
clined, and the u§e of operant methods is certainly ules as an opportunity for a sort of experimental
no longer restricted to Skinner’s students and associ¬ ecology, as a way to set up a novel set of relations
ates. The advantages of these methods, reviewed in between an animal’s behavior and its consequences,
the preface to the previous volume {Honig, 1966), are and then to observe how the subject copes with this
§o clear that many psychologists used them to study new situation. By studying a range of situations a
problems outside the original purview of the Experi¬ taxonomy may be derived and general principles in¬
mental Analysis of Behavior. As interest in the con¬ duced m Baconian fashion. Although much data of
struction of and debate over grand theoretical systems this sort has been gathered and is reviewed by
declined, experimental psychologist^ concentrated on Zeiler in this book, Skinner’s strictures against theoriz¬
*he explanation of more limited aspects of behavior, ing and ‘botanizing” have discouraged both syste¬
and these could be studied systematically through matic exploration of non-schedule variables, such as
operant methods in a tractable experimental setting species differences and type of reinforcer, and persis¬
that provided greater flexibility than its reputation tent attempts to make theoretical inferences. Zeiler
had suggested. Younger psychologist^ no longer ab¬ describes schedule control based on the delivery of
sorbed by debates among “schools” of psychology, felt primary rcinforccrs in Chapter 8, while Gollub re¬
less inclined to exclude operant behavior from their views the parallel role of conditioned reinforcers in
scope of interest, or, conversely, to restrict their atten¬ Chapter 10. These writers analyse in detail the con¬
tion to the limited range of problems addressed by trolling variables, involving both response-contingent
Skinner. While the ‘ passing of the pressing of the and non-contingent delivery of stimuli; their treat¬
bar” never did come about, Skinner’s methods and ments verge on theoretical accounts of the temporal
principles have not dominated experimental method¬ patterning of behavior.
ology, nor have they supplanted all other means by Schedules in the broader sense are also used as
which orderly data can be obtained. The operant is analytic devices. A particular set of relations between
still a very viable unit, as demonstrated in the many responding, stimuli, and the reinforcer is used for the
pages of this text, but it can no longer be so clearly study of particular empirical or theoretical questions.
separated from other modes of behavior. The relation The elegant demonstrations of autoshaping (Brown
of operants to the latter has come under close exami¬ and Jenkins, 1968; Williams and Williams, 1969)
nation over the last ten years. Some of the conclusions demonstrated that key-pecking in the pigeon can be
are worth reviewing. generated and maintained through “classical” con¬
Operant behavior is studied by arranging for the tingencies. Likewise, Reynolds’ (1961) experiments
animal to affect its environment in some way—by on behavioral contrast are among many other ex¬
pressing a lever, pecking a lighted key, or breaking a amples of the power of operant techniques to reveal
Werner K. Honig and J. E. R. Sfaddon 3

properties of learned behavior that are indirectly arated from species-specific patterns, greater flexibility
determined by effective schedules. Chapter 3 by may be observed. Responses which, let us say, control
Schwartz and Gamzu brings together these lines of the duration of stimuli differentially correlated with
work. When it is shown that schedules controlling reward, may be more tractable, and thus can be more
instrumental behavior generate and maintain classi¬ “arbitrary” than those that procure the reward itself.
cally conditioned responses, the whole relationship When consummatory behavior is preceded (and “pre¬
between these two classes of behavior needs to be re¬ dicted”) by a signal it often emerges as a classically
examined. Teitelbaum undertakes this task in Chapter conditioned, or “autoshaped”, response to that signal.
1; he shows how physiological techniques are used to The significance of autoshaping for the area of oper¬
identify the motivational substrates that underly in¬ ant behavior is threefold. First, the status of the arbi¬
strumental behavior in the intact animal. trary operant was reduced when it was discovered that
While the use of operant techniques continues to such cherished instrumental behaviors as the pigeon’s
prosper, the conceptual framework that has grown key peck could readily be conditioned through clas¬
up around them has begun to show signs of strain in sical means. Second, it suggested that both the form
recent years. The terms “response”, “reinforcer”, and and the quantity of operant behavior could be in¬
“stimulus” imply classes of events that are similar fluenced through classical (stimulus) contingencies, as
in their essential properties and can be combined in a current analysis of contrast effects suggests. Schwartz
arbitrary ways. One reinforcer is, if not the same and Gamzu trace this relationship in their chapter.
as another, at least not qualitatively different. All re¬ Third, the “instinctive drift” which underlies the
sponses (at least all operant, as opposed to respondent, misbehavior of organisms can be explained through
responses) are more or less equally reinforcible by all the operation of classical conditioning principles; An
reinforcers and can with equal facility come under the instrumental response reliably precedes the consum¬
control of any stimulus. This story is a familiar one matory behavior occasioned by the presentation of the
and recent discussions of “constraints on learning” reinforcer. Thus, depending on the schedule, the in¬
(Hinde and Stevenson-Hinde, 1973) have made its strumental response is a more or less reliable predictor
imperfections well known. The important point with of the consummatory response. In accordance with
respect to constraints is that the selection of one par¬ classical conditioning principles the instrumental re¬
ticular stimulus, response, or reinforcer may well limit sponse may therefore come to act as a conditioned
the selection of others that will be effective in con¬ stimulus, eliciting components of the unconditioned
junction with it. Some stimuli may control behavior response. To the extent that instrumental and un¬
more readily in avoidance paradigms than in con¬ conditioned responses are incompatible, interference
junction with positive reinforcement, and the con¬ may result and instrumental responding may be sup¬
verse can also be demonstrated. This sort of constraint pressed, as the Brelands found. On the other hand,
need not invalidate, although it may extend, princi¬ if the instrumental response is judiciously chosen to
ples obtained with the use of the most appropriate be compatible with the consummatory response, facili¬
experimental elements. In most situations the func¬ tation will be the rule. However, the classically condi¬
tional elements “stimulus”, “response”, and “rein¬ tioned nature of the response can be revealed by
forcer” can be identified and the relations among special scheduling arrangements such as the Williams’s
them are more or less what we have learned to expect omission procedure. The chapters by Schwartz and
from studies of bar pressing or key pecking. So while Gamzu, Dunham, and Staddon deal with these mat¬
the “arbitrary response” is no longer with us, the ters.
concept of reinforcement contingency has, if any¬ A less direct, but equally fruitful, approach to the
thing, gained in scope. relationship between instrumental and consummatory
Instrumental responses are closely related to the responses is provided by the presentation of “free”
species-specific consummatory behavior which is con¬ reinforcers on a temporally defined schedule. Work of
tingent upon them, and this relationship underlies this nature indicates that terminal behaviors approxi¬
some of the constraints just mentioned. In Chapter mating the consummatory response occur shortly be¬
4, Dunham reviews some of the “misbehaviors of fore the presentation of the reinforcer, while other
organisms.” These observations have provided us with interim behaviors occur when the likelihood of rein¬
a general principle, namely that instrumental re¬ forcement is low. This method is but one example of
sponses often approximate consummatory behaviors, a significant change in operant experiments, namely
and may be easiest to teach when they do so. Further¬ the simultaneous observation and recording of various
more, it is likely that if instrumental behavior is sep¬ responses in addition to the instrumental behavior.
4 INTRODUCTION

Such observation, facilitated by closed-circuit televi¬ This approach is presented in an even more radical
sion, is, of course, a change from the traditional em¬ form in Chapter 2 by Collier, Hirsch, and Kanarek,
phasis upon a single “externalized” response, but it who describe a situation where animals live in the
has enormously enriched and broadened the analysis experimental setting and can gain all of their required
of behavior in a controlled environment. It also pro¬ food (or water) in the form of unrestricted meals.
vides a bridge to the etliological study of animal behav¬ When the meal, rather than the pellet, becomes the
ior, a field whose avowed interests sometimes appear unit of reward, it supports a very large amount of
very different from those of operant psychology (al¬ behavior, in spite of the absence of the deprivation
though the reality of the difference is often arguable), condition commonly thought necessary for the per¬
but whose methods are quite similar. Thus, while the formance of an instrumental repertoire. These find¬
pressing of the bar has not passed, it has been supple¬ ings may pose some real problems for reinforcement
mented by other concurrent observations. Staddon theory, while at the same time they support the “rele¬
reviews this work in Chapter 5, and Hutchinson in vance” of the Experimental Analysis of Behavior to
Chapter 14 describes related experimental results in human affairs, since the environment that Collier et
situations involving electric shock. al. are working with is rather naturalistic, and pro¬
Just as the concept of the response has undergone vides an apt parallel for much of the human condi-
a searching analysis which is reflected in this book, the dition. Aside from its contributions toward the
process of reinforcement has also been re-evaluated, theoretical analysis of reinforcement, this research
and in several very different ways. Premack’s theory broadens our concept of instrumental behavior as an
(Premack, 1965), which was being developed while activity rather than a response. It can occur in
Operant Behavior was being written, has left its mark. “bouts”, as do the consummatory behaviors contin¬
The Experimental Analysis of Behavior is well suited gent upon it. Such a view permits a conceptual re-
to the notion that reinforcers have no absolute quali¬ evaluation of instrumental behavior as an activity that
ties, but are functionally defined, and situationally is chosen, from among others, for a proportion of the
determined. The development of these ideas is traced available time.
by Dunham in Chapter 4. Morse and Kelleher, in The theoretical analysis of reinforcement has in
Chapter 7, take a yet more radical view, suggesting this book also been extended far beyond the parallel
that reinforcement and punishment are often the efforts in 1966, especially with respect to quantifica¬
outcome of particular scheduling contingencies, and tion. Where behavior is related in an orderly fashion
their functional analysis is not necessarily bound up to other, controlled aspects of the environment, mathe¬
with the presumed noxious or appetitive qualities. matical analysis becomes fruitful. De Villiers devotes
Their careful argument and analysis cannot be sum¬ Chapter 9 to an examination of quantitative versions
marized in a few words, but one of their contributions of the Law of Effect, largely through a review of ex¬
should be pointed out; Once and for all, they separate periments on choice. His approach resembles Pre-
the presumed appetitive or aversive qualities of re¬ mack’s and, in a related field, Helson’s adaptation-level
sponse-contingent stimuli from the identification of theory, in being relativistic. While debate continues
such stimuli as reinforcers or punishers in terms of on the best form of theory, the notion that levels of
their effects in maintaining patterns of instrumental instrumental responding are determined by the con¬
responding. text of reinforcement, by relative rather than absolute
The process of reinforcement is analyzed in this reinforcement rates, is clearly here to stay. De Villiers
book in two other, quite different ways. In Chapter well illustrates the trend towards integration of oper¬
6, Satinoffi and Hender§eti describe the maintenance ant with general experimental psychology because he
of an internal state, namely temperature, via instru¬ reanalyzes results from standard discrete-trial situ¬
mental behavior. This leads quite naturally into feed¬ ations in accordance with his quantitative formula¬
back theory. Here reinforcing effects are best regarded tions. To this end, he considers running in an alley
not in terms of some presumed strengthening effect (for example), as an extended quantifiable response,
but as adjustments to deviations from an internal “set which can be represented in such a way to make it
point”. Operant behavior is but one of several mech¬ amenable to an analysis originally based on concur¬
anisms, physiological as well as behavioral, that help rent operants. Conditioned reinforcement is also sub¬
maintain the stability of the milieu interne. It is inter¬ jected to a mathematical treatment in Chapter 11 by
esting that behavioral thermoregulatory mechanisms Fantino, as another illustration of the quantitative
appear to be phylogenetically older than the physio¬ trend in the theory of operant behavior.
logical regulatory processes that supplement behavior Not only do we find new and very different treat¬
in so-called warm-blooded animals. ments of the concepts of response and reinforcement,
 Werner K. Honig and J. E. R. Staddon 5

but the role played by theoretical analysis itself seems these areas we see that again, a given treatment or a
quite to have changed in the last ten years, no doubt given behavioral effect may play more than one func¬
due in part to the reanalysis of the basic terms and tional role in the ultimate patterning of responses.
concepts. The chapters on stimulus control and aver¬ Thus, electrical brain stimulation can act as a power¬
sive control, in addition to those cited here already, ful reinforcer, as Mogenson and Cioe show in Chapter
attest to this. Concepts such as inhibition and atten¬ 19, but it can also elicit patterns of behavior closely
tion no longer require justification or defense, but related to the reinforcing effect. Furthermore, a func¬
rather a searching analysis of their determining char¬ tional analysis of the reinforcing process involved with
acteristics, as Rilling and Mackintosh show in Chap¬ such stimuli reveals that they do not act very differ¬
ters 15 and 16 respectively. In his careful review of ently from “standard” reinforcers, once parameters
the role of errors in the attainment of discrimina¬ such as immediacy of delivery are controlled for. With
tions, Rilling concludes that the correlation between the behavioral effects of drugs we see a converse set of
discriminative stimuli and rewards, rather than the relationships. Drugs are not limited to their tradi¬
emission of unreinforced responses, determines the tional actions as depressants, stimulants, and the like.
inhibitory properties of a stimulus. Again, a relation¬ They also can act as powerful reinforcers (or punish¬
ship between stimuli, rather than between a response ers), as those who are concerned with applied prob¬
and a stimulus, seems to govern processes that in turn lems can well testify. Thompson and Boren treat
control instrumental performance. Mackintosh uses behavioral pharmacology in Chapter 18. The effects of
the slope of the generalization gradient as an index of stress on biochemical and other physiological processes
attention to an extent that could hardly have been are reviewed by Brady and Harris in Chapter 20, and
anticipated a decade ago. His theoretical discussion here again we have evidence of the dual role of such
takes into account the role of repeated instrumental “internal effects”. They may reflect external treat¬
responses as stimuli which in their own right may ments which control behavior, but if they can be made
share stimulus control with other events explicitly accessible to the subject by being “externalized” as
programmed by the experimenter. While Blough and feedback stimuli, they can participate in the control
Blough, in Chapter 17, are less concerned with the¬ of behavior as discriminative and reinforcing stimuli.
oretical questions in the study of animal psycho¬ An analysis of language derived from the study of
physics, they give an account of signal detection theory operant behavior in animals was proposed by Skinner
as one method for the analysis of instrumental be¬ in his 1957 tour de force, Verbal Behavior. This work
haviors used to assess the perception of, and the dis¬ has excited much subsequent controversy but little by
crimination between, stimuli. way of empirical test. Noam Chomsky (1959) in a
Other theoretical treatments, particularly in the famous critique roundly condemned the work as
area of aversive control, reflect a more purely behav¬ empirically ill-founded, quantitatively implausible,
ioristic orientation. In Chapter 12 Blackman provides and little more than a restatement of the familiar in
a much needed analysis of the importance of the neologistic terms. Chapter 21 by Robinson deals with
operant baseline in its interaction with other funda¬ one aspect of this debate. He shows how a purely
mental processes when conditioned suppression is ob¬ associationistic model can lead to the development
tained. This topic was not included in the predecessor of a language structure. Hence the existence of struc¬
of this volume. Hineline elaborates on avoidance in ture in language does not require either that its basis
Chapter 13; he continues an analytic orientation to¬ is innate or that people learn “rules77 in the conven¬
ward free-operant avoidance that was already in pro¬ tional sense. It is perhaps helpful to be reminded that
gress a decade ago. It is interesting how this area has learning by association implies only that things be¬
changed from an emphasis on the methods which will come joined to other things through experience, a
produce free-operant avoidance to experiments that “mental chemistry” in Mill’s phrase, and not that the
analyze the variables and processes that maintain such conjoined entities are of necessity stimuli and overt
behavior. The current theoretical context emphasizes responses. In Chapter 22, Segal provides a clear and
the organism’s evaluation, as it were, of the correla¬ concise summary of the meat of Verbal Behavior, a
tions between responding and the absence of aversive book often cited but, we suspect, less often read. She
events. stresses the parallels between Skinner’s views on lan¬
Operant methods have also become ubiquitous in guage and contemporary structural approaches. She
the study of electrical, chemical, and physiological provides a framework for the conciliation of a conflict
determinants of behavior. Aside from some general originally generated by these approaches, a conflict
discussion in Teitelbaum’s chapter, three other chap¬ which was for a long time viewed as typical of the
ters are specifically concerned with these problems. In separation of operant analysis from more traditional
6 INTRODUCTION

forms of theorizing. Her achievement is perhaps sym¬ REFERENCES

bolic of a more general rapprochement that will, in
our opinion, gain strength over at least the next few Brown, P. L., & Jenkins, H. M. Auto-shaping of the pi¬
years. geon’s key-peck. Journal of the Experimental Analysis of
Behavior, 1968, 11, 1-8.
In many ways, then, research based on operant
Chomsky, N. Review of Verbal Behavior. By B. F. Skinner.
behavior is becoming more closely integrated into Language, 1959, 35, 26-58.
general experimental psychology. Theoretical ques¬ Hinde, R. A., & Stevenson-Hinde, J. (Eds.), Constraints on

tions are asked of the manner in which different forms learning. New York: Academic Press, 1973.
of operant behavior are generated and maintained. Honig, W. K. Introductory remarks. In W. K. Honig (Ed.),
Operant behavior: Areas of research and application.
Yet this behavior is itself used in turn to obtain
New York: Appleton-Century-Crofts, 1966. Pp. 1-11.
answers to theoretical questions of all kinds. Our brief Premack, D. Reinforcement theory. In D. Levine (Ed.),
overview of this book has stressed conceptual and Nebraska symposium on motivation, 1965. Nebraska:
theoretical developments relevant to operant behavior University of Nebraska Press, 1965. Pp. 123-180.
over the last ten years. We have said little about Reynolds, G. S. Behavioral contrast. Journal of the Exper¬
imental Analysis of Behavior, 1961, 4, 57-71.
operant methodology itself, which has changed but
Sidman, M. Tactics of scientific research. New York: Basic
little, Operant methods continue to be used widely as Books, 1960.
tools; in many ways the parallel between the operant Skinner, B. F. Verbal Behavior. New York: Appleton-Cen¬

chamber in psychology and the microscope in biology tury-Crofts, 1957.

Williams, D. R., & Williams, H. Auto-maintenance in
is justified. The advantages of operant methods were
the pigeon: Sustained pecking despite contingent non¬
recounted a deeade ago in the introduction that corre¬
reinforcement. Journal of the Experimental Analysis of
sponds to this article, These advantages have not Behavior, 1969,12, 511-520.
diminished. We hope that this handbook reaffirms the
value of operant methods as well as the vitality of
the empirical questions to which they continue to be
applied.
 1

Levels
of Integration
of the Operant*

Philip Teitelbaum

INTRODUCTION material in a variety of fields (Anderson, 1967; Skin¬

ner, 1961). In mental hospitals, therapists apply the
As a practical approach to the control of behavior, principles of reinforcement by using token economies
B. F. Skinner’s operant psychology is clearly a success. to shape up socially acceptable behavior patterns
His approach has been embodied in distinctive atti¬ (Kazdin 8c Bootzin, 1972). In the therapist’s office,
tudes toward the study of learned motivated behavior, impulsive behavior is brought under control (Halmi,
which in turn have generated a specialized termi¬ Powers, 8c Cunningham, 1975; Stunkard, 1974; for
nology and have led to the design of automated and possible perils, however, see Bruch, 1974). Experi¬
computerized equipment for detecting an individual’s mental communities, presaged in Walden Two (Skin¬
behavior, for reinforcing it according to particular ner, 1948a), are being explored as ways of solving the
schedules, and for recording the way the behavior is problems of social living. New journals, both theoret¬
shaped by the process (Ferster 8c Skinner, 1957). In ical and applied, are devoted to the operant approach,
psychopharmacology, these techniques have been used and the number of adherents continues to grow.
to generate stable base lines to assess the behavioral However, some aspects of the concept of the oper¬
effects of drugs. Physiological psychologists use them ant have come under attack. Laboratory learning
to interpret the effects of localized brain damage phenomena such as autoshaping (Brown 8c Jenkins,
(Honig, 1966). In human education, teaching ma¬ 1968) look like operants but seem not to fit the prin¬
chines and programmed texts are being developed to ciples of operant conditioning, and there are other
individualize and enhance the learning of conceptual examples of the “misbehavior of organisms” (Breland
8c Breland, 1961, 1966). Ethologically oriented workers
# This paper was written with the support of funds from
encounter biological constraints on learning, in which
NIH Grant #R01 NS 11671. The author wishes especially to
thank Evelyn Satinoff for incisive editorial revision and con¬ specialized evolutionary adaptations, either in non-
ceptual sharpening. Helpful criticisms were also received from traditional physiological systems (taste-aversion learn¬
Donald Davidson, Robert Hendersen, Werner K. Honig, Howard
ing) or in nonmammalian species, suggest to some
Rachlin, B. F. Skinner, John E. R. Staddon and Herbert S.
Terrace. that the search for general laws of learning may be

7
8 LEVELS OF INTEGRATION OF THE OPERANT

premature or even unwarranted (Garcia, Hankins, & that the “operant” approach is incompatible with a
Rusiniak, 1974; Hinde 8c Stevenson-Hinde, 1973; physiological analysis of behavior.
Rozin 8c Kalat, 1971; Shettleworth, 1972, 1975). As Descartes (1637) pointed out, the experimental
Such scientific paradoxes indicate that our present approach to understanding involves two intellectual
thinking may need re-evaluation (Teitelbaum, 1974). or experimental processes: (1) breaking down a phe¬
We must go back to the history of our ideas to dis¬ nomenon into simpler elements (analysis) and (2) re¬
cover how to revise them. In this chapter, I shall combining those parts to make sure that they are
therefore discuss how the operant came to be, and sufficient to reconstitute the original phenomenon
then some of the phenomena that seem to pose diffi¬ (synthesis). It is in the choice of simpler elements, and
culties for it. These difficulties are related to similar in the methods of using them to account for behavior,
problems in our thinking about all motivated be¬ that the various “schools” differ. For instance, physio¬
havior, Finally, I will summarize new evidence from logical psychologists have long tried to simplify be¬
the physiological study of brain-damaged animals and havior by chopping the nervous system into smaller
people that suggests that it may be fruitful to look at chunks. This is the classic levels-of-function approach
the operant in terms of levels of integration. to the nervous system used by Flourens (1824), Sher¬
The operant philosophy has structured, not only rington (1906), and many others. Such experimental
our thinking about behavior, but also our ideas about analysis in animals yields direct evidence for simple
how to study it. Implicit in it has boon tho rejection subcomponents of behavior, such as spinal reflexes,
of alternative approaches, particularly physiological and for their more complicated integration as postural
analysis. I will try to characterize the various forms of and movement patterns, like those described by Mag¬
analysis and synthesis used m the experimental ap¬ nus (1926) in decerebrate animals. Comparative psy¬
proach to understanding behavior in order to see why chologists, like their European counterparts, the
operant analysis went one way, while physiological ethologists, use the simpler nervous system of insects,
analysis took another. Then I will point out that there birds, and fish to study reflexes and the more complex
are new physiological approaches that can meet the hormonally controlled instinctive patterns such as
objections raised by Skinner and are compatible with feeding, fighting, and mating. In general, they all
his thought and work. The time may be ripe to merge agree with Descartes that to understand, one must
the operant and physiological methodologies in a con¬ simplify.
certed intellectual and experimental attack upon the However, these surgically or phylogenetically sim¬
levels of integration of the operant. Through be¬ plified preparations yield phenomena (reflexes, in¬
havioral analysis of developing infants and of adults stinctive patterns) and theoretical constructs based on
recovering from brain damage, we may extend our them that do not seem to help much in understanding
understanding of the operant. The result can be a the phenomena of language, thought, neurosis, and
basic behavioral approach that preserves the values of psychopathology that fascinate us in human behavior.
the operant while linking it, by a set of physiological We still do not know how to use our knowledge of
principles, to the fields of neurology, physiological these simple phenomena synthetically to predict or
psychology, developmental psychology and ethology. control very much that is significant in everyday hu¬

In short, we must bridge the gap between Sherrington man life (Skinner, 1957).
and Skinner. We all face this dilemma very early in our study of
behavior, and it is at this point that we split up. Some
of us (the physiological types) go toward the molec¬
ular. We say that behavior reflects the action of the
HISTORICAL BACKGROUND nervous system, so we must understand the latter be¬
fore anything else. Although we keep human be¬
All psychologists assert a common goal—the attempt havior in mind to return to eventually, we work on
to understand human behavior. However, apart from animals and concentrate on understanding molecular
differences in topics of interest, they adopt different phenomena such as synaptic transmission (with pos¬

fundamental beliefs about the best way of reaching sible relevance to mechanisms of learning and mem¬
this goal. Such beliefs, which are characteristic of all ory), and sensory physiology (how does the nervous
the academic subdivisions of our field, have created system transform a stimulus into a sensation?). Some
different “schools,” many of which have become so use electrical and chemical stimulation and ablation
insular that they hardly communicate with one to study brain mechanisms of motivation and rein¬
another any more. For example, it is commonly felt forcement. Others try to identify areas of the brain
Philip Teitelbaum 9

concerned with learning and memory. Because func¬ on spinal reflexes. He realized that when such a sim¬
tion must depend on structure, many spend a great plification was achieved, the main scientific value
deal of time mapping functional systems neuroan- for behavior was that for each reflex the adequate
atomically. stimulus could be identified, and because of its close,
All physiological experimenters share the belief virtually invariable association with the motor act, the
that the most fruitful experimental analysis is real, laws governing the S-R correlation could be worked
not hypothetical. By a real analysis, I mean using out. The variables governing any reflex can be classi¬
experimental techniques to isolate physically a frac¬ fied into two types: (1) environmental (the strength,
tion of a more complex system, yielding a system for number, and duration of stimuli and their spatial
study that has fewer variables acting on it, yet which and temporal interaction) and (2) organismic (the
still preserves the essential phenomena that are of in¬ central states—such as hunger, fatigue, and hormonal
terest; hence the emphasis on developing finer elec¬ conditions). In spinal reflexes, the organismic states
trodes to measure the activity of fewer cells, or even do not seem to affect the S-R correlation very much
one cell at a time, and on finding ever simpler nervous and for most purposes are largely ignored. Because
systems (e.g., the horseshoe crab Limulus or the sea such reflexes seem relatively uninfluenced by learning
hare Aplysia). Because human behavior is so compli¬ or the central “motivational” states that affect learn¬
cated, most feel they cannot now make much progress ing, the phenomena most germane to human behavior
with it, and they study molecular phenomena, firm in do not appear to have any obvious similarities to
their belief that such knowledge is fundamental to reflexes.
human behavior and will eventually pay off. As a When Pavlov (1927) discovered conditioned re¬
consequence, they deemphasize the synthetic applica¬ flexes, many psychologists believed that they had a
tion of their understanding. Many do not even try to simple system that could reveal the laws of animal
extrapolate their findings to people, feeling that there and human learning. If an unconditioned reflex such
is too great a gap between their observations on ani¬ as salivation at the sight or taste of food could come
mals and analogous phenomena in humans, in whom to be elicited by any arbitrary stimulus, such as a
cultural and social factors loom so large. flash of light or the sound of a buzzer, and if this
When faced with this dilemma—dealing with sub¬ association could be remembered for long periods of
components of behavior that are real but too simple time, then perhaps the laws of learning could be
to use in controlling or predicting the interesting quickly worked out. Many still have faith in this
aspects of human behavior—many psychologists reject paradigm (e.g., Moore, 1973). To Skinner, however, it
the physiological approach entirely. They pick impor¬ seemed clear that much of the behavior of animals
tant aspects of human behavior—phenomena of lan¬ and people was not based on autonomic responses,
guage, modes of problem solving, associative thought evoked automatically by stimuli. Most of their be¬
processes, social attitudes toward others, etc. They havior seems to be emitted as an act that modifies an
bring them or their analogs into the laboratory and environment in which no eliciting stimulus is readily
try to figure out the environmental and constitutional identifiable, rather than automatically evoked as a
variables that determine them. From each particular respondent like salivation at the sight of a stimulus
approach they deduce hypothetical variables for use paired with food. Furthermore, attempts to synthesize
in explanatory and predictive theories. For example, an understanding of complex behavior from the con¬
people have thought in terms of frequency and cepts of the reflex, conditioned or unconditioned, led
recency of associations, perceptual dispositions or to fruitless “physiologizing” (a tendency to push ex¬
“sets,” stimulus-response (S-R) bonds in habit planation back to the level of neural phenomena,
strength, tendencies to increase or decrease “cognitive without any proof that such hypotheses are valid), or
dissonance,” ego, id, or superego, etc. All are highly to a great deal of speculation about theoretical con¬
abstract, and even though they may be useful in try¬ structs involved in learned behavior. In the extreme,
ing to deal with real human problems, to a physiolog¬ the latter can be compared to the uselessness of medi¬
ical psychologist they do not seem very tangible or eval scholasticism—i.e., how many S-R bonds can
relevant to known phenomena in the nervous system. dance on the head of a pin? To Skinner (1950, 1972a,
So we drift further apart. 1972b), both forms of hypothetical synthesis led away
In his book The Behavior of Organisms B. F. Skin¬ from direct contact with real phenomena and there¬
ner (1938) grappled with the same dilemma. He very fore did more harm than good.
carefully considered the value of simplifying behavior In order to evaluate Skinner’s solution to the prob¬
by neurosurgery, as in the work of Sherrington (1906) lem of the analysis and synthesis of behavior, we
10 LEVELS OF INTEGRATION OF THE OPERANT

should first briefly review the common experimental this process of analysis and synthesis, and eventually,
methods of scientific synthesis. After an analysis, real as is now well known, you will discover that the hor¬
or theoretical, that has broken a phenomenon down mone insulin, manufactured by the islets of Langer-
into simpler parts, how do we put them back together? lians in the pancreas, counteracts the otherwise fatal
As far as I have been able to determine, there are five disease of diabetes mellitus.
methods, four of which I have described earlier
(Teitelbaum, 1967). In what seems an increasing order 3. Synthesis by Model
of abstraction (with, therefore, an increasing possibil¬
We can test our understanding of a phenomenon
ity of error in their application), they are as follows:
by constructing a model. We build into the model the
elements we think are important and also our concep¬
1. Direct Synthesis
tion of the way these elements interact to produce the
This method is often used in chemistry. When a phenomenon. Such models can be purely theoretical,
chemist wishes to determine the nature of an un¬ as in mathematical models of learning behavior, in
known substance, he breaks it down into its com¬ which after a theoretical analysis we postulate the
ponents. If his analysis is correct, he should be able to essential elements and processes involved in such a
synthesize the original substance by taking the indi¬ way that they can be described quantitatively. Then,
vidual components from completely different sources in situations which are simple enough to handle
and putting them together under the appropriate mathematically, we attempt to predict in an equation
environmental conditions. A classic example of this how the behavior will change as the variables are
in physiology wa§ carried out by the Nobel Prize manipulated. We can also construct physical anal¬
winner George Wald in collaboration with Ruth ogies, as an engineer does when he tries to simulate
Hubbard (Hubbard & Wald, 1951). After years of human behavior by building a computerized robot. All
working out the experimental analysis of rhodopstn, such models are attempts to synthesize a behavioral
they took the individual subcomponents from com¬ phenomenon through a model which embodies its
pletely different sources and put them all together. essential elements.
Purified opsin from the retinas of cattle, crystalline
alcohol dehydrogenase derived from horse liver, vita¬ 4. Synthesis by Prediction
min A from fish liver oil, and cozymase (now called
DPN) from yeast when brought together in solution If we have correctly analyzed the elements of a
formed a compound with all the properties of natural given form of behavior, we should be able to predict

rhodops&iii This is a beautiful example of the proof of which variables will control them and the way the
an analysis by direct synthesis. behavior will change as these variables are manip¬
ulated experimentally. This is the most common test
2. COUNTEREXPERIMENT! SYNTHESIS of an analysis by synthesis. It lends itself very easily
AFTER FRACTIONATION to theoretical analysis and in formal versions much
akin to mathematical models has played a prominent
This wa§ the favorite method of Claude Bernard role in learning theory (e.g., Clark Hull’s [1943] liypo-
(i86$), the great French physiologist. In essence, the thctico-deductive approach to laws of learning).
principle is: when a change occurs after you remove
something, put hack a fraction of what you have re¬
5. Svnthesi§ by Parallel
moved. If you restore the original state, the fraction
contains the essential ingredient, (If ft i§ tllQ only So far, none of the above methods has been very
sufficient ingredient, the remainder will not restore successful in enabling us to reconstitute complex ani¬
the original state.) For instance, in a famous example mal and human behavior from the real, simpler sub¬
of Bernard’s application of this method, after removal components of behavior isolated so far (reflex, in¬
of the pancreas, rabbits waste away and die. If a differ¬ stinct). We may not be able to do so until we know a
ent pancreas is transplanted anywhere into the body great deal more about how reflexes and instincts work.
of such a pancreatectomized rabbit, it lives relatively However, there is another method of synthesis, rather
normally. Therefore, the transplanted pancreas, even little exploited, which can supplement the previous
without its normal nervous connections, can maintain ones. It allows immediate useful application to com¬
life. If an extract of pancreas is injected daily into a plex behavior of any knowledge we have obtained
pancreatectomized rabbit, it too will live normally. about simpler, experimentally isolated behavior sys¬
Therefore, something in the extract is vital. Continue tems. It is “synthesis by parallel,” which in essence
Philip Teitelbaum 11

says that something new is like something else that is newly isolated “operant”—the cumulative record. Us¬
already familiar. A parallel is a similarity, and the ing the most advanced telephone relay circuitry then
more detailed it is the more confidence we have that available, he was able to detect the correct response (a
the similarity is not mere coincidence. One uses this bar press or a key peck sufficient to close a micro¬
method from the conviction that nature is parsimo¬ switch) automatically, to deliver the reinforcement
nious: if a given phenomenon works in a particular instantaneously, and to record each response and rein¬
fashion, it is likely that the same method is used to forcement as they occurred. By recording such re¬
produce other phenomena which up to now we have sponses cumulatively, one could almost see the devel¬
not recognized as being the same. Therefore, look for oping shape of an animal’s expectations—for instance,
a parallel. After evaluating Skinner’s use of analysis on a fixed-interval schedule (where a reinforcement
and synthesis, I shall illustrate the use of parallels as only occurs after a fixed time has elapsed since the
a possible way of increasing our understanding of previous one), the animal uniformly pauses in its re¬
operant behavior. sponding immediately after reinforcement, then grad¬
ually accelerates its responding as the probability of a
What was B. F. Skinner’s solution to the problem of reinforcement increases with time.
how to apply analysis and synthesis to the understand¬ Skinner used the cumulative record to insure the
ing of learned motivated behavior? Following firmly purity of his simplified preparation. When he got
in the footsteps of Descartes, Sherrington, and Pavlov, smooth curves he believed he had pure operant be¬
Skinner opted for a real simplification of behavior. havior under the control of the reinforcement contin¬
However, the physiological method of transecting the gencies. In a way, this is like the electrophysiologist
nervous system yielded preparations whose behavior who, when dissecting a nerve bundle to isolate one
was too simple—spinal or decerebrate reflexes and fiber, watches the oscilloscope and dissects until the
postural changes seemed unsuitable to reveal the laws preparation responds with impulses all of the same
of learning because these preparations could no longer amplitude. The all-or-none law (that a single neuron
learn. Moreover, trying to build theories of behavior always fires with impulses of the same amplitude) thus
from these overly simple preparations proved in most assures the purity of the dissected nerve preparation.
instances to be a waste of time. Therefore, instead of In a similar fashion, a smooth cumulative record is
transecting the nervous system to purify the S-R corre¬ taken to mean that we have only one kind of behavior
lation between environment and behavior, Skinner being recorded—each response follows the next so reg¬
chose to simplify the environment. He put the organism ularly (here uniform frequency rather than amplitude
into an isolated environment—an opaque, sound-insu¬ is used) that they add up smoothly rather than dis-
lated chamber where one or more stimuli could be in¬ continuously. (If the curve is irregular, as is often true
troduced whenever the experimenter desired (Skinner, early in training, it frequently indicates that we have
1956). In this, Skinner followed Pavlov, whose work more than one act being used to press the bar. How¬
on conditioned reflexes had demonstrated that in ever, see below for further discussion of the adequacy
order to reveal lawful correlations between stimuli of this method.)
and conditioned reflexes it was absolutely essential to Having experimentally isolated and purified the
eliminate extraneous stimuli. Respondent autonomic operant, Skinner then faced the problem of using it to
responses do not act on the world; therefore, Skinner understand the laws of learned motivated behavior.
chose an arbitrary act (but only one), like pressing a In his early work (1938) he used prediction as a
bar or pecking a key, and rewarded the hungry or method of synthesis. He formulated concepts such as
thirsty animal with a tiny amount of food or water the reflex reserve to embody the idea of a reservoir of
each time it performed the desired act. In a simplified responses that is affected both by central organismic
world of one stimulus and one response, it imme¬ states and past experience with reinforcement. The
diately becomes apparent that the presentation of a level of the reflex reserve determined the probability
reinforcing stimulus to an appropriately motivated of an animal’s behavior in particular instances.
animal powerfully shapes its behavior. As will be dis¬ Similar hydraulic models of instinctive behavior have
cussed more fully below, since the response and the been used in psychoanalysis (Freud, 1912) and in
reinforcement appear to be completely arbitrary, we ethology (Lorenz, 1952).
have thus achieved a simplified prototype of adaptive But Skinner soon came to feel that such attempts at
high-level motivated behavior—a unit of behavior theoretical prediction possessed the same drawbacks of
whose laws we can now study. fruitless speculation and lack of contact with the real
Skinner then devised a “microscope” to study the phenomena of behavior that were met with in at-
 12 LEVELS OF INTEGRATION OF THE OPERANT

Francis Bacon’s (1620) dictum that we should engage

in “experiments of fruit” as well as “experiments of
light”—making useful working application of the
scientific laws we are formulating.
Bacon also suggested that we draw up “tables of
discovery” describing the relationships we have
worked out. By classifying these relationships, similar
within category and different between categories,
fundamental generalities governing each category
should leap to the eye and mind, and valuable data
will be gathered in the process. This nonspeculative
form of data gathering is an empirical approach to
experiment—it is concrete, not abstract; therefore, no
one can dispute the facts generated by it. Such “bota¬
nizing” of behavior has great value when applied simul¬
taneously over many species of animals, as in compar¬
ative physiology or taxonomic ethology. However,
Fig 1. Ti*3£ittgS fit three curves which report in re-
because taxonomy has not been explicit in the operant
sponse to a multiple fixed-interval fixed-ratio schedule. One
ef them was made by a pigeon, one by a rat, and one by a approach (but see Skinner, 1966)? some behaviorally
monkey, (From Skinner, (g) 19i?6 by the American Psychological oriented workers become impatient with it and sus¬
Association. Reprinted by permission.^
pect its practitioners of application of the operant
method in trivial instances. Because it is still in its
tempting to reconstruct complex behavior from phys¬ early stages, the separate categories of operant analysis
iological or hypothetical simplifications. “A purely of behavior are not yet clearly apparent, so many do
descriptive science is never popular* For the man not see its theoretical value. Also, because operant
whose curiosity about nature is not equal to his inter¬ terminology is not widely used, many psychologists do
est in the accuracy of his guesses, the hypothesis is tho
not see how the operant “school” adds more than
very life-blood of science17 (Skinner, 1938, p, 426). He simple technology for providing stable behavioral base
ihsretare broke with traditional forms of theory (Skin¬ lines. But operant methods work, so their application
ner, 1950). The operant was simple—but not too becomes more widespread.
simple. It was of sufficient complexity to embody the
interesting phenomena of learning and motivation.
Environmental analy§i§ assured sufficient simplicity to THE OPERANT AS A CRITERION POR
reveal reliable 5-R. relationships that could form the MOTIVATION
framework of a scientific description of behavior. In¬
stead of using a mathematical model or prediction for
In an earlier discussion of this subject (Teitelbaum,
synthesis, Skinner used the successful control of be¬
1966), I pointed out:
havior ag his validation criterion. The phenemenQn
we are interested in (learned motivated behavior) is
When we speak of purposive acts in humans,
clearly evident in the simplified world of the Skinner
wc mean behavior that is directed toward a goal
box, so all wc have to do is describe the way the re¬
and is accompanied by a corresponding motiva¬
sponse varies as stimuli and internal states are varied.
tion to obtain that goal. The essential quality
The fact that they do control the probability of re¬
is the motivational state—the physiological state
sponse guarantees their validity. I call thi§ ‘Vynthesis of events that corresponds to the urge to per¬
by success.77 A? §h<?wn in Figure 1, a given fixed-inter¬ form a particular act, to obtain a certain object,
val type of schedule (this is actually a multiple sched¬ or to produce a desired outcome. If we could
ule: fixed intervals combined with fixed ratios) pro¬ be sure that such a state exists in animals during
duces virtually identically shaped smooth curves in a given act, we could justifiably call that act
several species; therefore, the laws being formulated motivated behavior.
have great generality. The procedure is a variant of Clearly, if the response is a completely auto¬
matic consequence of the stimulus, we cannot
“synthesis by model”—our simplified laboratory model,
speak of motivation. As long as a fixed built-in
derived from animals, works when applied to humans
relation exists between a stimulus and a re¬
in the real world. Skinner’s method thus applies sponse, we have no justification for inferring the
Philip Teifelbaum 13

additional existence of a motivational state un¬ both in the laboratory, and in applied situations, work
derlying that response to the stimulus. Such a on animals has continued apace, and, with increasing
state may exist, but we can have no positive frequency, seemingly paradoxical phenomena are be¬
proof of it. (p. 566) ing demonstrated. Early reports of the “misbehavior
of organisms” came from Breland and Breland (1961,
By definition, therefore, a reflex excludes motiva¬ 1966), students of Skinner who applied operant
tion. It is unconscious, unlearned, and involuntary methods to the training of a wide variety of animals
(Skinner, 1931). To infer motivation we must break in situations designed to entertain the public.
the fixed reflex connection between stimulus and re¬ Chickens were taught to swing a bat to hit a ball out
sponse. By its very nature, the operant appears to onto a “playing field,” raccoons or pigs to “save”
do so. woodeir coins or dollar bills in piggy banks, whales or
porpoises to play with rubber balls, and so on. How¬
In effect, in any operant situation, the stimulus, ever, particularly as such animals became better and
the response and the reinforcement are com¬ better trained, their performance very often deteri¬
pletely arbitrary and interchangeable. No one orated. Chickens would run out onto the “playing
of them bears any biologically built-in fixed field” to chase the baseball they had just hit with a
connection to the others. We arrange the experi¬
bat; raccoons would “wash” their coins instead of
mental situation so that the response produces
dropping them in the box, and pigs would root and
the reward and the animal learns the connection
toss their dollars rather than depositing them in the
between them. Once having learned this rela¬
tionship, the animal reveals its motivation by piggy bank. In all these instances, the interfering be¬
the fact that it works to obtain the reinforce¬ havior delayed the reinforcement, sometimes to the
ment. This is what all operant conditioning point where the animals underwent serious weight
situations have in common: the animal’s motiva¬ loss, since the conditioned acts were their sole means
tion to obtain the reinforcement ... If an of obtaining food.
operant occurs, motivation exists, (p. 567) Hineline and Rachlin (1969) pointed out that in
many circumstances there is great difficulty involved
Thus by using learning to break the fixed reflex in training a pigeon to peck a key to avoid electric
connection between stimulus and response, Skinner shock, though it could readily learn to do so for food.
created an emergent unit of behavior—the operant— Pigeon key pecking seemed still more perplexing
which could be experimentally isolated, whose laws when Brown and Jenkins (1968) demonstrated that
could be studied in their own right, and which could contingent reinforcement with food was not necessary
serve as the prototype of all learned motivated be¬ to train a pigeon to peck an illuminated key—merely
havior. Because the degree of environmental control using the key to signal the opportunity to eat food
over the operant in the Skinner box is so great, the at brief intervals was sufficient to induce them to peck
close S-R correlation is preserved and, with it, the the key light, independent of the reinforcement, at
scientific power of the laws describing it. Indeed, to very high rates. One might conceive of such “auto¬
avoid the pitfalls of speculation and to eliminate the shaping” as an example of “superstitious” responding
“idols of the marketplace” described by Francis Bacon (Skinner, 1948b), but the work of Williams and
(1620) (the tendency to use words with surplus mean¬ Williams (1969) on “negative autoshaping” (where a
ings to describe simpler phenomena), Skinner pre¬ pigeon will learn and continue such key pecking even
ferred to eliminate entirely such constructs as motiva¬ when each response actually prevents the reinforce¬
tion and awareness. The operant embodies them in ment) makes this less tenable. (However, for evidence
clear-cut S-R relations, and a categorization of those of operant control of autoshaped behavior, see Bar¬
relationships should form an adequate scientific basis rera, 1974.) These behaviors seem to fit in the operant
for the control of operant behavior. category but can be extremely difficult to shape, occur
without reinforcement, despite reinforcement, or dete¬
riorate rather than improve with training.
PUZZLING OPERANTS Puzzling phenomena are being found in other types
of learning situations. When a rat feels sick after
As described above, the laws of the operant are re¬ poisoning or exposure to X-rays, it will develop an
markably well suited for application to humans (Mil- aversion for a novel taste (such as saccharin) but does
lenson, 1967) and have been highly successful when not link the illness to other stimuli such as lights or
applied to human learning and motivation. However, sounds, which were equally available for association
 14 LEVELS OF INTEGRATION OF THE OPERANT

(Garcia 8c Koelling, 1966). However, in the identical By examining motivation, we may gain greater insight
situation, if the negative reinforcement is the pain of into operant behavior.
electric shock, the light or sound becomes the danger Since the fundamental work of W. R. Hess (1932,
signal, whereas the novel taste is ignored. So taste for 1954, it has been known that electrical stimulation of
a rat seems physiologically tied to the nausea and the brain of an unanesthetized animal can elicit in¬
malaise of poisoning and X-ray exposure, but lights, stinctive behavior patterns such as mating, feeding,
sound, and locations seem linked to the peripheral drinking, or fighting. A sated rat stimulated through
pain caused by electric shock. More visual species, implanted electrodes in the lateral hypothalamus will
such as birds, link the sickness of poisoning to the eat large quantities of food (Hoebel, 1971). If this is
color of a nutrient solution rather than to its taste done every day, the rat will overeat and even become
(Wilcoxon, Dragoin, 8c Krai, 1971). obese (Steinbaum k Miller, 1965). Such an animal will
Taste aversion learning is special in other ways as learn a new operant or perform a previously learned
well, It seems far more powerful than the instrumen¬ one (e.g., running a maze or pressing a bar to get
tal learning which has served as the paradigm of con¬ food) during stimulation, thus supporting the idea
ditioning for so many years. In traditional condition¬ that this is truly motivated behavior, not merely some
ing experiments, an association between an arbitrary kind of motor automatism (such as chewing), where
stimulus and the sight 6f food Can be formed only if the mgestion of food is an accidental by-product of
they occur virtually simultaneously* If there is more the behavior, rather than a desired outcome (Coons,
than a few 3cconds> delay, it is very difficult to produce Levak, & Miller, 1965* Mendelson & Chorover, 1965;
a learned linkage between them. Rats, however, can Miller, 1971). The same is true of thirst (Andersson 8c
associate sickness with a novel taste, even if the taste Wyrwicka, 1957) and other species-typical behaviors
occurred as much as 12 hours earlier (Smith & Roll, (Roberts, 1970).
1967). {Histamine secreted by the body in reaction to However, a fundamental problem in our concep¬
X-ray exposure seems to be involved in such learning tion of motivated behavior has been identified in the
and may be related to these powerful effects—Levy, work of John Flynn and his colleagues (e.g., Flynn,
Carroll, Smith, 8c Hofer, 1974.) 1973). They implanted electrodes in the lateral hypo¬
To ethologically oriented workers, such phenomena thalamus of cats. Many normal cats do not ordinarily
indicate that it may be premature to seek general laws kill rats. However, when stimulated in the lateral
of learning, including those of the operant. They sug¬ hypothalamus, such cats chase and strike or bite a rat,
gest that a deceptive generality may result when work usually killing it if the current is left on. Two forms
is limited to too few species (Beach, 1950; Bolles, of such attack were seen. One was accompanied by a
1970; Rozin 8c Kalat, 1971; Seligman & Hager, 1972; display of rage (retraction of the lips, exposure of the
Shettleworth, 1972; Tinbergen, 1951). For some canine teeth, piloerection and arching of the back,
species the innate connections between certain stimuli hissing, growling, pupillodilation-the typical "Hallo¬
and responses may be too reflexive to serve in operant ween cat”). In the other form, which Flynn and co¬
behavior. They involve stimulus-bound, nonarbitrary, workers called the “quiet biting attack,” the cat
and nomnhibitable acts. Yet in an operant paradigm moved swiftly about the cage with its nose low to the
the probability of their occurrence can often be ground, back somewhat arched, and hair slightly on
manipulated by reinforcement contingencies. Are they end, and usually went directly to the rat and bit it
operants? If so, what is wrong with our concept of viciously. In the absence of an attack object, neither
operants? If not, what are they, and why do many of form of directed attack would occur. Therefore, stimuli
them often seem to obey the laws of reinforcement? provided by the rat are necessary before electrical
stimulation can evoke attack behavior. What are
these stimuli?
In their early studies, Flynn and his colleagues
SIMILAR PUZZLES IN MOTIVATED took a Sherringtonian approach to this problem. The
BEHAVIOR
Anal component in the attack sequence is the killing
bite. Which stimuli elicit it? They restrained the in¬
As described above, an operant act is proof of the tact cat so that the animal could lunge, or turn its
existence of motivation—if a completely arbitrary head and bite, but could not otherwise walk around.
operant occurs, motivation exists. If we are running Without hypothalamic stimulation, touch around the
into difficulties with our conception of the operant, mouth and on the lips evoked no response. However,
the same must be true of our concept of motivation. during electrical stimulation, touch around the
Philip Teitelbaum 15

the sight of the rat (its color, small size, and move¬
ment) and its smell activates stalking behavior; (2)
when close to the rat, the cat is stimulated to pounce,
to swipe at the rat with its paws, and to lower its head
for the killing bite; (3) when the cat’s whiskers or
muzzle contact the rat’s fur or skin, its head turns and
brings the lips into contact; (4) touch on the lips
Fig. 2. (Left) The cat’s muzzle. (Center) Maximum extent of elicits mouth opening; and (5) touch and taste stimu¬
the maxillary sensory field for head-orienting responses during
lation of the mouth and tongue evoke biting and
relatively intense hypothalamic stimulation. A similar mandi¬
bular field has not yet been mapped in detail. (Right) Maximum swallowing.
extent of the sensory field for the jaw-opening response during This description dovetails very well with the find¬
relatively intense stimulation. (From MacDonnell & Flynn,
ings of Flynn and co-workers described above and
1966a. (c) 1966 by the American Association for the Advance¬
ment of Science.) their additional work on the releasing effects of visual
and other stimuli on the components of electrically
mouth, on the side contralateral to the stimulation, evoked attack (Bandler & Flynn, 1972; Flynn, Ed¬
evoked head turning toward the stimulus (see Figure wards, & Bandler, 1971). The ethological view is
2). When the lips contacted the stimulus, mouth open¬ strongly supported in studies of birds and fish in
ing and biting occurred. Increasing the strength of which removal of a stimulus in the S-R chain aborts
hypothalamic stimulation increased the extent of the the instinctive behavior pattern (Tinbergen, 1951).
sensory field around the mouth and on the lips from But this view implies that the instinctive act is not
which the response could be evoked (MacDonnell 8c outcome-dependent. The eliciting stimulus, not the
Flynn, 1966a). Conversely, if these sensory fields were reinforcement, determines the response. How can we
denervated by section of the appropriate branches of reconcile this with our view that rat killing by a cat
the trigeminal nerve, touch stimuli were no longer is outcome-dependent?
effective in evoking head turning, mouth opening, and Perhaps electrically evoked attack is not controlled
biting, but electrical lateral hypothalamic stimulation by all the variables controlling rat killing in a normal
could still evoke attack; the cat pounced on the rat cat, Electrically evoked eating in rats seems more
and lowered its head for the killing bite—but then did stimulus-bound and more stereotyped than normal
not open its mouth when it contacted the rat—“kiss¬ hunger (Valenstein, 1973; Valemtein, Cox, & Kako-
ing” rather than biting. The cat could open its mouth lewski, 1968). If this is correct and if normal rat kill¬
(it did so normally when eating food spontaneously), ing is determined by the reinforcement rather than
but not when stimulated to attack and kill (MacDon the sign stimulus, then an unstimulated eat killing
nell 8c Flynn, 1966b). spontaneously, even with denervated mouth and lips,
Implicit in this finding is a paradox with poten¬ should open its mouth, bite, and kill a rat when
tially important implications for our thinking about hungry or when provoked to rage by pain. This
motivated behavior. On the one hand, if the operant experiment is theoretically very important and should
is a learned arbitrary act, its occurrence depends upon be carried out.
(1) the memory of past response-reinforcement con¬ Suppose that, in the trigeminal-sectioned cat kill¬
tingencies* (2) the central motivated state which makes ing a rat spontaneously, the killing kiss rather than a
that outcome reinforcing, and (3) the expectation that bite occurs. Does this mean that killing is not moti¬
the operant will continue to produce the reinforcing vated— i.e., not outcome-dependent? Not necessarily—
stimulus. A cat is motivated to kill a rat if the cat will our view of the reinforcement may have been incor¬
press a lever or run a maze to be presented with a rat rect. Perhaps each sign stimulus in the ethological
which it then kills (Roberts 8c Kiess, 1964). chain may be reinforcing. Part of its releasing action,
On the other hand, we can take an ethological view particularly in the experienced animal, may be due to
of a cat’s rat killing. We can assume that there are the memory of the reinforcement provided by that
specific fixed action patterns built into the cat’s ner¬ stimulus in the past. This means that such an animal
vous system. They are selectively potentiated by hor¬ should press a lever to gain the opportunity to swipe
monal or other internal states and released by partic¬ at a rat, to pounce on it, or merely to see and chase
ular, somewhat complex stimuli, called sign stimuli. it. In fish, for instance, sign stimuli can be reinforcers.
Each S-R fixed action pattern forms a segment in a A Siamese fighting fish will learn to press a lever for
chain of behavior which we call the instinctive act. the mere sight of another fighting fish (Thompson,
To account for rat killing by a cat, we assume that (1) 1963). In their mating dance, a male stickleback will
16 LEVELS OF INTEGRATION OF THE OPERANT

perform an operant for the opportunity to see a recep¬ simple neurological tests to normal rats. A normal rat
tive female whom he then courts (Sevenster, 1973). investigates a stimulus by orienting its head toward it.
There is still another way of reconciling sign stimuli This natural response was used to determine the
with reinforcement, but it must wait till we consider responsivity of rats before and after lateral hypothal¬
the evidence on motivation as revealed during re¬ amic damage. For example, to test vision on each side
covery from brain damage. of the body, a 2-in. square (5x5 cm) piece of white
or yellow cardboard was moved in front of each eye.
Normal rats typically turn their heads toward this
RECOVERY FROM LATERAL visual stimulus. To test olfaction, they looked for
HYPOTHALAMIC LESIONS head orientation to a 1,4-in. cube of chocolate held in
forceps or to a cotton swab soaked in Mennen shaving
As we have seen from the work of MacDonnell and lotion (both of which elicited approach) or to an
Flynn (1966a), electrical stimulation in the lateral ammonia-soaked swab (which elicited approach fol¬
hypothalamus opens peripheral sensory fields around lowed by turning the head away).
the mouth whose stimulation then yields head orien¬ In an exact converse of the results of MacDonnell
tation and biting, reflexive components of the cat’s and Flynn (1966a); damaging the lateral hypothal¬
instinctive attack pattern. Increasing the intensity o£ amus on one side profoundly impaired the rat’s abil¬
the stimulation expands the fields. Does lateral hypo¬ ity to orient to stimuli on the side contralateral to
thalamic damage shrink such fields and prevent nor¬ that Of the lesion (see Figure 3). Rats with unilateral
mally £ff£etivg stimuli from acting on them? The lesions initially showed no orientation to contralateral
lateral hypothalamus is involved not only in attack, visual, olfactory, whisker=touch, or somatosensory
but also in eating. Does the lateral hypothalamic syn¬ stimulation, whereas they responded promptly to the
drome of aphagia and adipsia depend, in part, on 1qs§ same stimuli presented ipsilaterally. Rats with bi¬
of responsiveness to sensory stimuli? lateral lesions showed impaired responsivity to sensory
In order to answer these questions, Marshall, stimuli on either side.
Turner, and Teitelbaum (1971) applied a series of Although the precise nature of such sensory neglect

I PSILATERAL CONTRALATERAL
SIDE SIDE

WHISKER
TOUCH

ODOR
Fig. 3. A rat with unilateral
(right) lateral hypothalamic
damage shows precise head ori¬
entation and biting to various
kinds of stimuli (whisker touch,
odor, body touch) on the ipsi-
lateral side (pictures at left)
while neglecting the same
BODY stimuli presented contralater-
ally (pictures at right). (From
TOUCH Marshall, Turner, 8c Teitelbaum,
1971. © 1971 by the American
Association for the Advancement
of Science.)
Philip Teitelbaum 17

needs further behavioral analysis (see Turner, 1973), stage i Stage H Stage HE Staged

several observations suggest that the orientation im¬ ADIPSIA,

APHAGIA
ADIPSIA.
ANOREXIA
ADIPSIA,
DEHVDRATION)- RECOVERY
APHAGIA
pairment is neither a total motor paralysis nor an in¬
o O O O
ability to sense the stimuli. In normal grooming, a
rat usually starts by grooming its face and head,
turns and grooms one side and flank, and terminates
the sequence by grooming the opposite side and flank.
Thus during normal grooming the rat’s head is turned
and oriented to one or the other side side just as it is
during normal orientation to tactile stimuli from that
side. After unilateral lateral hypothalamic damage,
the rat grooms first the ipsilateral, then the contra¬
lateral side of the body. However, even seconds after
Fig. 4. Stages of recovery seen in the lateral hypothalamic
grooming the side contralateral to the lesion, the rat syndrome. (The critical behavioral events which define the
ignores tactile stimuli to that side and fails to turn stages are listed on the left.) (From Teitelbaum & Epstein,
1962.)
toward them. Such failure to respond to stimuli, even
though the animal can perform the necessary head
movements, suggests that the deficit is more sensory ing, or attack. But where does the operant fit into this
than motor. However, the deficit does not resemble picture?
deafferentation, because autonomic (respiratory
changes) and skeletal reflex (eye closure, tooth chatter¬
ing) behaviors often occurred when a stimulus was PARALLEL BETWEEN RECOVERY AND
presented on the contralateral side. The deficit seems DEVELOPMENT IN THE LATERAL
to be more of an inability on the rat’s part to inte¬ HYPOTHALAMIC SYNDROME
grate the sensory information with the adaptive motor
patterns involved in orienting to a stimulus (Marshall Some insight into the role of the operant comes
8c Teitelbaum, 1974; Turner, 1973). from the analysis of the stages of recovery from the
Such sensory neglect can drastically affect the in¬ aphagia and adipsia that result from bilateral lateral
stinctive behavior patterns involved in eating, drink¬ hypothalamic damage. The pattern of behavioral re¬
ing, and attack. Bilateral lateral hypothalamic lesions covery is summarized diagrammatically in Figure 4.
produced total aphagia and adipsia lasting as long as (For a detailed analysis of the homeostatic mecha¬
9 days, followed by the usual stages of recovery. nisms in the lateral hypothalamic syndrome, see
Analysis of the recovery of orientation to sensory Epstein, 1971.) The striking fact about this syndrome
stimuli showed that the transition from Stage I (com¬ is that every lateral hypothalamic animal shows the
plete aphagia) to Stage II (accepting only highly same sequence of recovery. Depending on the lesion
palatable foods) occurred on the same day or shortly size and accuracy of placement, animals recover more
after direct head orientation to olfactory stimuli and or less rapidly. Also, although they may start their
whisker touch first appeared. Rats with unilateral recovery at any point in the sequence, the progression
lesions that were tested for side preference in feeding from that point follows an invariable pattern. There¬
generally took more food from the container located fore, the pattern of recovery could indicate a basic
in the ipsilateral field, though preoperatively no such process of neural reorganization.
preference had existed. Similarly, after unilateral A newborn suckling rat ingests milk but refuses
lateral hypothalamic lesions, rats that normally killed water. At weaning, although they eat dry food and
mice ignored the mouse when it was in the contra¬ drink water, infants still do not respond fully to de¬
lateral field. However, as soon as the mouse moved hydration (Adolph, 1957; Heller, 1947, 1951; KrScek
across the midline into the ipsilateral field, the rats 8c Kr£cekova, 1957). This resembles some of the symp¬
showed oriented biting attack. toms seen during the various stages of recovery of food
In summary, the evidence from electrical stimula¬ and water regulation in the adult rat after lateral
tion or ablation strongly suggests that the role of the hypothalamic damage (Teitelbaum 8c Epstein, 1962).
lateral hypothalamus in the control of motivated be¬ In a sense the lateral hypothalamic rat during part of
havior is at least in part due to its ability to potentiate its recovery is like an infant rat.
the action of peripheral stimuli in eliciting the fixed If a parallel exists between adult recovery and in¬
instinctive action patterns involved in eating, drink- fant development, each stage in recovery from the
 18 LEVELS OF INTEGRATION OF THE OPERANT

Development o£ Eating £ Dnnkunq They were finicky, and although they ate more in the
Parallels Its Recovery
cold they did not eat more in response to glucopriva-
tion. As shown in Figure 5, depending on the degree
decrease in lesion size i Stages
INCREASE IN WGT AT WEANING I II HI IV of retardation, every stage of the adult lateral hypo¬
thalamic syndrome was seen in thyroidectomized rats
^ recovery
IX DEVELOPMENT at weaning.
EATS WET
PALATABLE FOODS
In a separate experiment (Cheng, Rozin, & Teitel-
CVOLUNTARY EATING)
baum, 1971), we severely starved neonatal rats through¬
out the suckling period by limiting their access to the
mother. Control litters with unlimited access to their
mother developed normally. The starvation-retarded
runts, when tested at weaning (21 days of age), showed
every stage of the lateral hypothalamic syndrome.
Like the thyroidectomized infants,, if they survived
they progressed through the various stages as they
PRIMMING EATING developed.
ft&CVNJDI AL OSMOTIC (JiyCOSTATlC TliERMP&TATIQ
One may immediately object to drawing a parallel
PRIM MINS REGULATION
nuieuiwcss
RCOULMTIOW REGULATION between recovery from lateral hypothalamic damage
and stages of development of the regulation of food
and water intake in infancy. After all, lateral hypo-
Tig, 5, Comparison of the development of eating arid drinking thalamic animals die of starvation and thirst, whereas
in rats’ infancy and their recovery after hypothalamic lesion infants do not, But we do not ask newborn infants to
ii\ adult. (The upper right half of each blogk represents the
eat like adults—they nurse at the breast until they can
recovering lateral hypothalamic vat and the lower left half
the growing thyroidectomized or starvation-stunted rats. Uni- be weaned. It follows that adult lateral hypothalamic
forrn ccloring in each full block indicates similar responses in animals should nurse like infants, and, without any
recovery and development.) (From TCit<?lbaum, Cheng, & Rozill, other maintenance, should be able to keep them¬
1969a.)
selves alive until they recover additional regulatory
capacities. Figure 6 shows such an experiment. Merely
lateral hypothalamic syndrome should be reflected in by offering repeated access to a substitute mother (a
proper sequence during development in infancy. To milk bottle with a modified drinking spout, provided
study this in detail, we slowed down the course of at frequent intervals day and night). Dr. Cheng (un¬
normal development by thyroidectomizing infant rats
at birth or shortly thereafter and studied their regu¬
latory capacity at the normal weaning age of 21 days
(Teitelbaumi Cheng, 8c Rozin, 1969a, 196%).
At that age, thyroidectomized weanling rats dis¬
played every stage of the lateral hypothalamic syn¬
drome. If greatly retarded in development (a§ reflected
in thfir body weights at weaning), some were com¬
pletely aphagic and adipsic when offered wet palatable
foods or ordinary food and water. Others, more fully
developed at weaning, accepted wet palatably foods
but did not eat enough to maintain their weight, If
this stage lasted too long, they died. Other weanlings,
even less retarded, gained weight and regulated their
caloric intake of a liquid diet (they at least doubled
their volume intake when the caloric density was on^-
third as great), but were still adipsic and would have
died (some did die) when offered only dry food and
water. Finally, the least retarded weanlings accepted
rig. 6. A lateral hypothalamic adult rat, otherwise totally
dry food and water, but drank only when they ate; aphagic for nine days, nurses reflexively at a milk-containing
like recovered adult rats, they were prandial drinkers baby bottle (with modified nipple). Frequent feedings allowed
it to ingest sufficient quantities to stay alive and recover to
and did not drink in response to body dehydration. the anorexic stage.
 Philip Teifelbaum 19

published results) showed that an otherwise com¬ ment following cerebral hemiplegia in human pa¬
pletely aphagic and adipsic adult lateral hypothalamic tients. His work stemmed from the earlier observation
rat in Stage I was able to ingest sufficient quantities of Seyffarth and Denny-Brown (1948), who pointed
to keep itself alive for 9 days. Then it progressed to out that several types of reflexive movements could
Stage II; i.e., it ate wet palatable baby foods from a be elicited from a paralyzed limb although the limb
dish. Thus otherwise aphagic adult animals, like in¬ could not be used for voluntary movement. In the
fants, can ingest sufficient food to stay alive by nursing hand, for instance, they identified three types of re¬
reflexively. flexive grasping—the traction response, the true grasp,
and the instinctive grasp reaction. In studying the re¬
covery of stroke patients, Twitchell (1951) discovered
STAGES OF RECOVERY AND that these grasping automatisms could always be
DEVELOPMENT OF THE HUMAN GRASP identified and that they represented distinct sequen¬
tial stages in the recovery of voluntary control of
If the parallel between adult recovery and infant movement.
development emerged from the study of another brain In subsequent studies, Twitchell (1965, 1969, 1970)
system, independent of feeding and drinking, one found that very similar stages to those seen in recov¬
could be more sure of its validity. From work on a ery from hemiplegia in adults can be demonstrated in
different species and on a different brain system, a the normal development of voluntary control of grasp¬
very similar parallel has independently been demon¬ ing in newborn infants (Figure 7). In a human sys¬
strated. T. E. Twitchell studied the recovery of move¬ tem as well as in the rat, adult recovery recapitulates
infantile ontogeny.

TRANSFORMATION OF SENSORY CONTROL

OVER AN APPROACH RESPONSE

The concept of stages of sensory control over a re¬

sponse is implicit in the parallel between recovery
and development. During recovery of the grasp, spinal
AT BIRTH proprioceptive mechanisms in the form of tendon
TRACTION RESPONSE
Stimulus: Stretch shoulder adductors and flexors jerks and increased stretch reflexes (spasticity) occur
Response: All joints flex
first. These are modified by tonic neck and vestibular
body-righting reflexes into the traction response. The
recovery process then proceeds to the next stage, in
which flexion of the fingers can be obtained reflex¬
ively by a distally moving tactile stimulus to the
medial palm. This is the true grasp reflex. In a sense,
as Twitchell (1951) points out, the tactile grasp reflex
facilitates the proprioceptive grasp (the traction re¬
sponse). Eventually, the sight of a stimulus is enough
GRASP REFLEX
(INITIAL COMPONENT) (FULLY FORMED) to cause the hand to reach out and grasp it (Twitchell,
S. Contact between thumb and index S. Distally moving contact medial palm
R. Thumb and index adduct alone R. All fingers flex 1970). Presumably, with sufficient recovery, true “op¬
erant” use of the hand returns.
The sequence of development and recovery of
sensory controls over an approach reaction may be
more general than is now suspected. For instance, a
similar sequence seems to govern the gaping response
of the newborn thrush. Immediately after hatching,
the stimulus modalities that appear to elicit the gape
INSTINCTIVE GRASP REACTION
S. Contact radial or ulnar side S. Contact hand (any part) S. Contact hand (any part) are proprioceptive and vestibular; the chicks gape
R. Hand orients R. Hand gropes R. Hand grasps
when the parent bird alights on the nest, which
Fig. 7. Evolution of the automatic grasping responses of in¬ shakes the head and body of each chick. The gape is
fants. (Twitchell, 1965.) directed vertically upward (vestibular control), inde-
20 LEVELS OF INTEGRATION OF THE OPERANT

dogs. As in other animals, such lesions produce

aphagia and adipsia with eventual recovery. Operant
responding for food after such lesions also disappears
and eventually recovers (Rodgers, Epstein, 8c Teitel-
baum, 1965). When a normal dog swallows food, gas¬
tric contractions are inhibited and the stomach re¬
laxes. This is its action as an unconditional stimulus.
After training sessions presenting the sound of a
metronome for 50-60 sec preceding the presentation
of food, the sound of the metronome alone can pro¬
duce stomach relaxation and inhibition of motility.
After bilateral lateral hypothalamic damage, Glav¬
cheva et al., showed that such conditioning is com¬
pletely abolished. Stages of recovery of conditioning
can then be demonstrated. At first, there is stomach
atoma—a complete loss of tonus and spontaneous
hunger contractions. This stage generally corresponded
fig, §. Nestling tLirusKes gaping upward, in response to visual to complete aphagia or anorexia with adipsia. Then
stimulus blit Rot directed by it. TiRbergfift, 1951. By per¬
a stage of rhythmic automatic gastric contractions ap¬
mission of Oxford University Press.)
pears, but the stomach still seemed completely cut off
from influence by the rest of the nervous system—the
pendent of the position o£ the parent bird, Touching unconditioned relaxation effect of food in the stom¬
the side of the face near the mouth also elicits the ach and the conditioned effect of the sound were both
gape, which is still directed upward. Later, the sight absent. In the next stage of recovery of control of
of th£ parent bird (sr th$ approaching hand of the gastric motility, the unconditioned relaxation effect
experimenter—see Figure 8} elicits gaping, but it i§ of food reappeared, but the conditioned stimulus
still directed upward. Finally, the gape is directed (sound) was still ineffective. Finally, in the last stage,
toward the visual stimulus (visual control now not the conditioned stimulus regained its effectiveness.
only triggers the response hut also guides its orienta¬ It w<?uld b^ interesting to investigate, both in re¬
tion—Tinbergen & Ku£n£nr 1930). Although Tin¬ covering lateral hypothalamic dogs and in newborn
bergen and Kuenen ebd not investigate it, it 1$ possi¬ puppies, whether there may be a similar sequence of
ble that* still later* operant gaping as an emitted stages of conditionability of sensory control over gas¬
response would develop in such birds. If so, then, as tric motility (kinesthetic, vestibular, touch, and vision).
in recovering hemiplegic humans and developing nor¬ Since salivary conditioning is also impaired after
mal infant rats, the end result of such a sequential lateral hypothalamic damage (Rozkowska 8c Fonberg,
transformation of control of a reaction pattern would 1972), the sequence of recovery and development of
be voluntary or self-initiated action—an operant re¬ its conditioned sensory control should be investigated
sponse. If there is generality in the sequence of sensory as well.
transformation of a reaction pattern, we may have a It is suggestive (see Table 1), as support for such
beginning insight into a mechanism of transformation a sequence, that Russian investigators find a rather
of reflexes into operants during infant development or similar, immutable sequence of sense modalities in
adult recovery of function. the development of the capacity for Pavlovian condi¬
tioning in human infants (Kasatkin, 1960).

STAGES OF RECOVERY AND STAGES OF ENCEPHALIZATION

DEVELOPMENT OF LEARNED BEHAVIOR OF THE OPERANT

If these concepts apply to the study of operant be¬ To understand any behavioral phenomenon, we
havior, we should be able to demonstrate stages of must view it as we would a stage of development or
integration of learned responses during recovery from recovery. These only make sense as a transformation
brain damage and in early infantile development. from the stage of integration that preceded it (a lower
For instance, Glavcheva, Rozkowska, and Fonberg level of encephalization) toward a higher level of inte¬
(1970) studied the lateral hypothalamic syndrome in gration. Like the nervous system whose action it re-
Philip Teitelbaum 21

Table 1 Developmental Sequence of Conditionability as a Function of Type of Conditional Stimulus

SENSORY ANALYZERS

Vestibular Auditory Tactile Olfactory Taste Visual

SIMPLE
CONDITIONED
RESPONSE
(e.g., change (e.g., complex (e.g., tickle, (e.g., oil of (e.g., 5% sugar (e.g., colored
of body 65-db tone) sole of foot) roses or solution) light)
position) lavender)
First
appearance 8d 15d to 24d 28d 28d 35d 40d
Semistable
response 15d 40d 45d 45d 45d 2m
100% stable
response 20-24d to lm 35d to 2 m 2m 2m 2.5m 3m

SIMPLE DIS¬
CRIMINATION
(e.g., up- (e.g., 1 octave (e.g., right (e.g., roses (e.g., 1% from (e.g., red from
down from higher or from left foot) from lavender) 5% solutions) green or blue)
sideways) lower from
CS+)
First
appearance lm 2m 2m 2m 2.5m 3m
Semistable
response 1.5m 2.5m 2.5m 2.5m 3m 3.5m
100% stable
response 2m 3m 3m 3m 3m 3.5m

Note: Ages appear as days (d) or months (m) (Kasatkin, 1960).

Reprinted with permission of Macmillan Publishing Co., Inc. from Infancy and Early Childhood, Y. Brackbill (ed.). @ 1967 by the
Free Press, a Division of the Macmillan Company.

fleets, behavior is a hierarchically organized structure. tion of the urinary sphincter. With recovery from
This point of view may help resolve some o£ the spinal shock in the adult, such control by distension
apparent paradoxes in behavior which now exist. A returns. Apparently, in the newborn infant the tactile
striking example of such a resolution comes from lordosis reflex promotes urination. Later in develop¬
the study of sexual behavior (Beach, 1966). Adult ment, this reflex disappears and bladder distension
female rodents in estrus show lordosis when stroked becomes sufficient to produce urination. In the adult
around the flank or genital region. This has long female, estrus hormones reinstate the lordosis reflex.
been used as a criterion of adult sexual receptivity. It Just as hypothalamic stimulation involved in killing
was therefore extremely surprising to discover that opens and expands tactile sensory fields around the
newborn guinea pigs (male or female) displayed such mouth (MacDonnell 8c Flynn, 1966a), estrual hor¬
lordosis in the first few hours, or even for some days, mones elicit lordosis through tactile sensory fields
after birth. Was this due to the surge of female estrual around the genitals (Komisaruk, Adler, 8c Hutchison,
hormones in the mother that occurs at parturition? 1972). Therefore, the reappearance of infantile re¬
No—pups born of mothers ovariectomized midway flexes can occur not only in brain damage, but also in
through their pregnancy showed the same lordosis the normal action of hormones which produce in¬
when stroked (Boling, Blandau, Wilson, 8c Young, stinctive behavior.
1939). Careful behavioral analysis revealed that in in¬ Other instinctive behavior patterns go through suc¬
fant guinea pigs, lordosis is the basic posture assumed cessive levels of ontogenetic transformation of nervous
reflexly during urination evoked by the mother’s lick¬ control. Each stage of transformation in the hierarchy
ing of the infant’s genital region as it stands and allows new controls to modify the action of the lower
nurses underneath her. Indeed, such tactile stimula¬ stage of integration. For instance, as McGinty (1971)
tion is necessary for survival in infancy (Beach, 1966), has pointed out (see Figure 9), the various stages of
because, as in spinal adult animals shortly after sleep may be conceived of as separate stages of en-
surgery, bladder distention does not produce relaxa¬ cephalization of the sleep system. It is well known that
22 LEVELS OF INTEGRATION OF THE OPERANT

place, circling and curling up, lowering the head, clos¬

ing the eyes, etc., and progresses through successive
stages to the consummatory act—the achievement of
REM sleep. Like food and water intake, the amount
of REM appears to be regulated homeostatically; e.g.,
after deprivation of REM, more of it occurs (is con¬
sumed?) when free access is once again possible
(Dement, Henry, Cohen, & Ferguson, 1967).
Perhaps every adult motivated behavior pattern
proceeds in a similar sequence from operant and Pav-
lovian approach behavior to the reflex consummatory
act. In the study of instinctive behavior, ethologists
have roughly distinguished such levels of nervous
control by referring to appetitive versus consumma¬
tory behavior (Tinbergen, 1951). Similarly, in man,
Appetitive Lekavlor ckalti In tke aJuli. |From McCinty, the appetitive act of reaching for food is more operant
1071) in its character (less stereotyped, more subject to
willed inhibition) than is a later act in the chain-
newborn infants are capable mainly of rapid eye reflexive chewing—and much more so than swallow¬
movement (REM) sleep. This seems a relatively reflex¬ ing, which is completely reflexive once the food has
ive, obligatory form of sloop; the mfant has little con¬ reached a point far enough back in the mouth. Each
trol over it (it cannot inhibit sleep) and lapses readily behavioral event is followed by the next more auto¬
into it when its stomach is full and it is not cold or matic one until the final molecule-to-molecule match
wet, When its nervous system matures, slow-wave is achieved (Breland 2c Breland, 1966).
sleep emerges as a separate stage with consequent in¬ Operant behavior may have a similar hierarchical
hibition at th@ amount of REM sleep. (In the adult structure, built up by successive levels of transforma¬
decorticate animal, REM represents about 40 percent tion in development. The adult operant may be
of total sleep, versus about 20 percent for a normal viewed as the initial, most arbitrary (learned) part of
cat—Jouvet, 1962: and personal communication.) It an approach sequence toward a fixed consummatory
i§ a§ though each higher stage of encephalization limits aet. During that sequence, the behavior becomes pro¬
access to the less encephalized stage preceding it in gressively deencephalized (more stimulus-bound, less

ontogeny. This may be adaptive; the more reflexive subject to willed inhibition), culminating in reflexive
form of sleep (REM) is suppressed and partially re¬ consummatory behavior.
placed by §low-wav@ sleep which in turn can he more Of what use is it to think this way if the sequence
readily inhibited, thus allowing more opportunity for reek itself off so quickly in an operant act that we
adaptive waking behavior. cannot isolate its parts? In a way, this iS like the prob¬
When the normal adult animal goes to sleep, it lem faced by George Wald in isolating the transition
goes through a fixed sequence of stages=it must enter states in the transformation of rhodopsin to vitamin
slow-wave sleep before it can proceed to the stage of A (Wald, 1968). Lumi-rhodopsin and meta-rhodopsm
REM sleep. Perhaps, therefore, not only can slow- were isolated by literally freezing the reaction—at a
wave s leep inhibit REM, hut also during each nOrm&l temperature below — 409 C, the reaction only pro¬
bout of adult §kep, dow-wavc sleep may serve as a ceeded to hi ml-rhodopsin. Gradual warming allowed

selective facilitating mechanism in the transition from it to proceed to meta-rhodopsm and eventually to
waking to REM sleep. This resembles the way the reach its final transformation state—vitamin A plus
true grasp allows touch to act as a facilitator in the opsin. In like manner, stages of encephalization of
release of the more infantile proprioceptive grasp— behavior can be "frozen” in recovery after brain dam¬
the traction response (Twitched, 1951, 1970). age or in normal development in infancy. Starvation
The above example of sleep behavior is important or thyroidectomy in infancy can slow the development
because it suggests that in adult behavior, the normal process still further.
act of going to sleep involves a rapid, reversible de- Is it possible to reveal such stages during normal

encephalization of function: sleep, as an instinctive motivated acts, where, if they exist, they are so fleeting
behavior pattern proceeds from the appetitive behav¬ and evanescent that we cannot now isolate them? Per¬
ior patterns involved in searching for an appropriate haps the apparent errors in operant behavior can
 23
Philip Teitelbaum

help to reveal the stages of transformation. In the reward may be very closely tied physiologically to the
ethological study of instinct, “mistakes” in apparently act of eating and may not be very arbitrary at all
purposive acts are useful diagnostic indicators. For (Moore, 1973). Pressing a bar for a rat may be an act
instance, when rolling its egg back into the nest, the borrowed from some other instinctive pattern, but by
greylag goose must complete the head-withdrawal pat¬ the power of reinforcement or through Pavlovian con¬
tern even if the egg slips out from under its beak ditioning may be shaped up to act as the initial ap¬
during the act (Tinbergen, 1951). To the ethologist proach segment of the chain leading to the consump¬
this proves that egg rolling is a fixed, built-in, instinc¬ tion of food or water. A lighted key that signals food
tive action pattern rather than an outcome-determined or water may become a sign stimulus releasing the
operant. Likewise, the fixed response of a fish or a consummatory act of pecking in a pigeon, thus yield¬
bird to a dummy model defines the existence of a sign ing some of the phenomena of autoshaping (Tenkins
stimulus and proves that the act it releases is an in¬ & Moore, 1973).
stinctive, rather than learned, pattern. As we have Even if a given act satisfies the criteria of inter¬
seen, such “errors” in adult mammalian behavior changeability that assure its arbitrariness at the be¬
(called abnormalities) are often diagnostic of damage ginning of training (in other words, if the act is so
to the central nervous system. In a similar way, what highly encephalized that it bears little resemblance to
seem today like paradoxes in learned behavior may a reflex), it may become rapidly deencephalized under
turn out to be diagnostic indicators of the level of many circumstances: as the consummatory act is ap¬
encephalization of that behavior. proached; with routinization in overtraining (the
“misbehavior” of organisms); or with the fatigue,
frustration, conflict, and thwarting (lack of reinforce¬
SUMMARY AND CONCLUSIONS; ment) that are the inevitable accompaniments of any
LEVELS OP OPERANT BEHAVIOR reinforcement schedule. If we assume that fatigue or
conflict can inactivate the higher-level neural controls
As discussed earlier, several phenomena in operant characterizing the appetitive components of an in¬
behavior (the “misbehavior” of organisms, negative stinctive behavior chain, then the very same act may
automaintenance, etc.) may seem puzzling because we become less encephalized (more stimulus-bound, stereo¬
assume the operant to be an indivisible, emergent typed, synergistic, and exaggerated in its intensity) as
unit, with laws different from those governing the the behavior is constantly repeated. These are all
simpler subcomponents of behavior (reflex, instinctive attributes of release phenomena usually seen most
fixed action pattern). In its idealized form, perhaps clearly after brain damage. Displacement phenomena
best expressed in human behavior, it is the very anti¬ in instinctive behavior might be viewed as manifesta¬
thesis of a reflex. A reflex is an unconscious, unlearned, tions of deencephalization. Indeed, components of in¬
involuntary, built-in fixed response to a particular stinctive patterns that are normally considered inap¬
kind of stimulus. As idealized, the operant is a con¬ propriate (i.e., that are normally suppressed during
scious arbitrary act which, through learning, has be¬ the behavior in question) may be released as displace¬
come associated with an arbitrary stimulus and whose ment activities. As Falk (1971) has pointed out, many
frequency is maintained by an arbitrary reinforce¬ of the adjunctive behaviors seen in operant situations
ment. In principle, all stimuli, all operants, and all (psychogenic polydipsia, pica, etc.) resemble displace¬
reinforcements are interchangeable because no one of ment activities. Similarly, during the frustration in¬
them bears any biologically built-in connection to the duced by thwarting on a food-reinforcement schedule,
others. a pigeon or squirrel monkey may display release of
In the laboratory, or in real life, when reinforce¬ attack normally elicited by painful external stimuli
ment is used to shape behavior, paradoxes may appear (Azrin, Hutchinson, & Hake, 1966; Hutchinson, Azrin,
when one or more of these idealized assumptions is & Hunt, 1968).
violated. As Skinner (1938) pointed out, one source of Many of these paradoxes that seem so perplexing
error may arise in the act chosen as the operant. Per¬ arise from our view of the operant as an indivisible
haps, especially in lower animals like pigeons and emergent unit whose laws differ from the simpler sub¬
rats, the operants usually used to study the laws of components of behavior revealed by physiological and
reinforcement are not as arbitrary as we have thought. ethological analysis. It is possible that the operant
In autoshaping, orienting toward the key is more undergoes many transformations, and we must devise
readily influenced by food reinforcement than is the methods to isolate its transition forms. We must in¬
act of pecking it (Wessells, 1974). Pecking for food crease the magnifying power of our behavioral micro-
24 LEVELS OF INTEGRATION OF THE OPERANT

scopes. The smooth curves in our cumulative records Azrin, N. H., Hutchinson, R. R., 8c Hake, D. F. Extinction-
induced aggression. Journal of the Experimental Analysis
may no longer appear smooth if we speed up the
of Behavior, 1966, 9, 191-204.
motor of the recorder. As Schwartz and Williams Bacon, F. The new organon (ed. and trans. J. Spedding, R.
(1972) have demonstrated, an autoshaped peck has a L. Ellis, 8c D. D. Heath) in Works, Vol. 8. Boston:
different duration from a peck that obeys the rein¬ Taggard and Thompson, 1963. (Reprinted, F. H. Ander¬
forcement contingencies. Slow-motion photography in son, Ed., Indianapolis: Bobbs-Merrill, 1960.) (Originally
the operant setting may reveal further changes in the published, 1620.)
Bandler, R. J., 8c Flynn, J. P. Control of somatosensory
topography of the key peck during thwarting. We
fields for striking during hypothalamically elicited attack.
should use evolutionary taxonomy as a microscope to Brain Research, 1972, 38, 197—201.
reveal differences in the character of operants that will Barrera, F. J. Centrifugal selection of signal-directed
eventually fit into Baconian tables of similarities and pecking. Journal of the Experimental Analysis of Be¬
havior, 1974, 22, 341-355.
differences. In my opinion, these will correspond to
Beach, F. A. The snark was a boojum. American Psy¬
developmental and phylogenetic levels of encephaliza-
chologist, 1950, 3, 115-124.
tion. Beach, F. A. Ontogeny of “coitus-related” reflexes in the
Our search for general laws of learning is indeed female guinea pig. Proceedings of the National Academy
but not unwarranted. Tafite aveyfiion learn¬ of Sciences, 1966, 36, 526-533.
Bernard, C. Introduction to the study of experimental
ing now seems bizarre and difficult to encompass in
medicine (trails. H. C. Greene, 1926). Republished, New
our previous conceptions of learning. However, it may
York: Dover Press, 1957. (Originally published, 1865.)
illustrate that learned associations between stimuli Boling, J. L., Blandau, R. J., Wilson, J. G., 8c Young, W.
and physiological states undergo transformations and C. Post-parturitional heat responses of newborn and
become less arbitrary as they are tied to sensory and adult guinea pigs: Data on parturition. Proceedings of
the Society for Experimental Biology and Medicine,
motor systems which develop earlier in ontogeny or
1939, 42, 128-132.
recover differentially after brain damage. Perhaps, as Bolles, R. C. Species-specific defense reactions and avoid¬
in the grasp reflex, such sensory controls develop in a ance learning. Psychological Review, 1970, 77, 32—48.
characteristic sequence. Breland, K.,, 8c Breland, M. I lie misbehavior of organisms.
We have a long way to go in our study of motiva American Psychologist, 1961, 16, 681-684.
Breland, K., 8c Breland, M. Animal behavior. New York:
tion, learning, and operant behavior. We must use all
Macmillian, 1966.
the techniques of real simplification! physiological Brown, P. L., 8c Jenkins, H. M. Auto-shaping of the
(the study of brain damage), developmental (the study pigeon's key-peek. Journal of the Experimental Analysis
of maturing infants), and ethological (the taxonomic of Behavior, 1968, 11, 1-8.
study of the similarities and differences seen in op¬ Bruch, H Perils of behavior modification in treatment of
anorexia nervosa. Journal of the American Medical
erants). It is not yet possible to use direct synthesis to
Association, 1974,130, 1419-1422.
assemble the real, simpler subcomponents of behavior GHENG, M. F., Roein, P., & TeiTelbaum, P. Semi-starvation
to prove that our analysis is valid. However, the study retards the development of food and water regulations in
of the parallel transformations in behavior during re¬ Infant rats. Journal of Comparative and Physiological

covery and development can provide a real synthesis. Psychology, 1971, 76, 206—218.
COONS, E. E., LEVAK, M., & MILLER, N. E. Lateral hypo¬
The behavioral description of 8-R correlations at
thalamus; Learning of food-seeking response motivated
each level of encephalization are as much in accord by electrical stimulation. Science, 1963, ISO, 1320-1321.
with the approach of Sherrington as they are with Dement, W, C., Henrv? P.? Cohen, H., 8c Ferguson, J.
Skinner. They fit Skinner’s criteria for a scientific ap¬ Studies on the effect of REM deprivation in humans and
proach to behaytovj yet enable us to perfect our un¬ animals. In S. 5. Kety, E. V. Evarts, 8c H. L. Williams
(Eds.), Sleep and altered states of consciousness. Balti¬
derstanding of the levels of integration of the operant.
more; Williams and Wilkin?, 1967-
Descartes, R. Discourse on method (2nd ed.) (trans. L. J.
Lafleur). New York: Liberal Arts Press, 1956. (Originally
published, 1637.)
REFERENCES Epstein, A. N. The lateral hypothalamic syndrome; Its
implications for the physiological psychology of hunger
Adolph, E. F. Ontogeny of physiological regulations in the and thirst. In E. Stellar 8c J. M. Sprague (Eds.), Progress
in physiological psychology (Vol. 4). New York: Academic
rat. Quarterly Review of Biology, 1957, 32, 89-137.
Anderson, R. G. Educational psychology. Annual Review of Press, 1971.
Psychology, 1967, 18, 129-164.
Falk, J. L. The nature and determinants of adjunctive be¬
Andersson, B., & Wyrwicka, W. The elicitation of a drink¬
havior. Physiology and Behavior, 1971, 6, 577-588.
ing motor conditioned reaction by electrical stimulation Ferster, C. B., 8c Skinner, B. F. Schedules of reinforcement.
of the hypothalamic “drinking area” in the goat. Acta Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1957.
Physiologica Scandinavica, 1957, 41, 194-198. Flourens, M. J. P. Recherches experimentales sur les
 Philip Teitelbaum 25

proprietes et les fonctions du systeme nerveux, dans les (Ed.), Infancy and early childhood. New York: Free
animaux vertebres. Paris: Crevot, 1824. Press, 1967.
Flynn, J. P. Patterning mechanisms, patterned reflexes, and Kazdin, A. E., 8c Bootzin, R. R. The token economy: An
attack behavior in cats. In J. K. Cole 8c D. D. Jensen evaluative review. Journal of Applied Behavior Analysis,
(Eds.), Nebraska Symposium on Motivation (Vol. 20). 1972, 5, 343-372.
Lincoln: University of Nebraska Press, 1973. Komisaruk, B. R., Adler, N. T., 8c Hutchison, J. Genital
Flynn, J. P., Edwards, S. B., & Bandler, R. J. Changes in sensory field: Enlargement by estrogen treatment in fe¬
sensory and motor systems during centrally elicited male rats. Science, 1972, 178, 1295-1298.
attack. Behavioral Science, 1971, 16, 1-19. Krecek, J.,8c Krecekova, J. The development of the regula¬
Freud, S. Types of neurotic nosogenesis. In J. Rickman
tion of water metabolism, III: The relation between
(Ed.), A general selection from the works of Sigmund water and milk intake in infant rats. Physiologia
Freud. Garden City, N.Y.: Doubleday, 1957. (Originally Bohemoslovenica, 1957, 6, 26-33.
published, 1912.) Levy, C., Carroll, M. E., Smith, J. C., 8c Hofer, K. G.
Garcia, J., Hankins, W. G., 8c Rusiniak, K. W. Behavioral
Antihistamines block radiation-induced taste aversions.
regulation of the milieu interne in man and rat. Science, Science, 1974, 186, 1044-1046.
1974,185, 824-831.
Lorenz, K. The past twelve years in the comparative study
Garcia, J., 8c Koelling, R. A. Relation of cue to con¬
of behavior. In C. H. Schiller (Ed. and Trans.), Instinc¬
sequence in avoidance learning. Psychonomic Science, tive behavior. New York: International Universities
1966, 4, 123-124. Press, 1957. (Reprinted from the Report of the Zoo¬
Glavcheva, L., Rozkowska, E., 8c Fonberg, E. The effect of
logical Convention, Freiburg, 1952.)
lateral hypothalamic lesions on gastric motility in dogs. MacDonnell, M. F., 8c Flynn, J. P. Control of sensory
Acta Neurobiologiae Experimentalis, 1970, 30, 279-293. fields by stimulation of hypothalamus. Science, 1966, 152,
Halmi, K. A., Powers, P., 8c Cunningham, S. Treatment of 1406-1408. (a)
anorexia nervosa with behavior modification. Archives of MacDonnell, M. F., 8c Flynn, J. P. Sensory control of
General Psychiatry, 1975, 32, 93-96,
hypothalamic attack. Animal Behaviour, 1966, 14, 399-
Heller, H. The response of newborn rats to administration 406. (b)
of water by stomach. Journal of Physiology, 1947, 106,
McGinty, D. J. Encephalization and the neural control of
245-255.
sleep. In M. B. Sterman, D. J. McGinty, 8c A. M. Adinolfi
Heller, H. The water metabolism of newborn infants and (Eds.), Brain development and behavior. New York:
animals. Archives of Disease in Childhood, 1951, 26, 195— Academic Press, 1971.
204.
Magnus, R. Some results of studies in the physiology of
Hess, W. R. Die Methodik der lokalisierten Reizung und
posture. The Lancet, 1926, 211, 531-536; 585-588.
Ausschaltung subkortikaler Hirnabschnitte. Leipzig: G. Marshall, J. F., 8c Teitelbaum, P. Further analysis of
Thieme, 1932.
sensory inattention following lateral hypothalamic dam¬
Hess, W. R. Diencephalon: Autonomic and extrapyramidal
age in rats. Journal of Comparative and Physiological
functions. New York: Grune and Stratton, 1954. Psychology, 1974, 86, 375-395.
Hinde, R. A., 8c Stevenson-Hinde, J. (Eds.). Constraints on
Marshall, J. F., Turner, B. H., 8c Teitelbaum, P. Sensory
learning. New York: Academic Press, 1973.
neglect produced by lateral hypothalamic damage.
Hineline, P. H., 8c Rachlin, H. Escape and avoidance of
Science, 1971, 174, 523-525.
shock by pigeons pecking a key. Journal of the Experi¬
Mendelson, J., 8c Chorover, S. L. Lateral hypothalamic
mental Analysis of Behavior, 1969, 12, 533-538.
stimulation in satiated rats: T-maze learning for food.
Hoebel, B. G. Feeding: Neural control of intake. Annual
Science, 1965, 149, 559-561.
Review of Physiology, 1971, 33, 533-538.
Millenson, J. Principles of behavioral analysis. New York:
Honig, W. K. (Ed.). Operant behavior: Areas of research
Macmillian, 1967.
and application. Englewood Cliffs, N.J.: Prentice-Hall,
Miller, N. E. Neal E. Miller: Selected papers. Chicago:
Inc., 1966.
Aldine-Atherton, 1971.
Hubbard, R., 8c Wald, G. The mechanism of rhodopsin
Moore, B. R. The role of directed Pavlovian reactions in
synthesis. Proceedings of the National Academy of
Sciences, 1951, 37, 69-79. simple instrumental learning in the pigeon. In R. A.
Hull, C. L. Principles of behavior. New York: Appleton- Hinde 8c J. Stevenson-Hinde (Eds.), Constraints on learn¬
Century-Crofts, 1943. ing. New York: Academic Press, 1973.
Hutchinson, R. R„ Azrin, N. H., 8c Hunt, G. M. Attack Pavlov, I. P. Conditioned reflexes: An investigation of the
produced by intermittent reinforcement of a concurrent physiological activity of the cerebral cortex (ed. and
operant response. Journal of the Experimental Analysis trans. G. V. Anrep). Oxford: Oxford University Press,
of Behavior, 1968, 11, 489-495. 1927. (Republished, New York: Dover Press, 1960.)
Jenkins, H. M., 8c Moore, B. R. The form of the autoshaped Roberts, W. W. Hypothalamic mechanisms for motivational
response with food or water reinforcers. Journal of the and species-typical behavior. In R. E. Whalen (Ed.), The
Experimental Analysis of Behavior, 1973, 20, 163-181. neural control of behavior. New York: Academic Press,
Jouvet, M. Recherches sur les structures nerveuses et les 1970.
mecanismes responsables des differentes phases du som- Roberts, W. W., 8c Kiess, H. O. Motivational properties of

meil physiologique. Archives of Italian Biology, 1962, hypothalamic aggression in cats. Journal of Comparative
,
100 125-206. and Physiological Psychology, 1964, 58, 187-193.
Kasatkin, N. I. 1960, as cited in Y. Brackbill 8c M. M. Rodgers, W. L., Epstein, A. N., 8c Teitelbaum, P. Lateral
Koltsova, Conditioning and learning. In Y. Brackbill hypothalamic aphagia: Motor failure or motivational
26 LEVELS OF INTEGRATION OF THE OPERANT

deficit? American Journal of Physiology, 1965, 208, 334- modification. In G. A. Bray 8c J. E. Bethune (Eds.),
342. Treatment and management of obesity. New York:
Rozin, P., 8c Kalat, J. Adaptive specializations in learning Harper 8c Row, 1974.
and memory. Psychological Review, 1971, 18, 459-486. Teitelbaum, P. The use of operant methods in the assess¬
Rozkowska, E., 8c Fonberg, E. Impairment of salivary re¬ ment and control of motivational states. In W. K. Honig
flexes after lateral hypothalamic lesions in dogs. Acta (Ed.), Operant behavior: Areas of research and applica¬
Neurobiologia Experimentalis, 1972, 32, 711-720. tion. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966.
Schwartz, B., 8c Williams, D. R. Two different kinds of key Teitelbaum, P. Physiological psychology: Fundamental
peck in the pigeon: Some properties of responses main¬ principles. Englewood Cliffs, N.J.: Prentice-Hall, 1967.
tained by negative and positive response-reinforcer con¬ Teitelbaum, P. The encephalization of hunger. In E.
tingencies. Journal of the Experimental Analysis of Stellar & J. M. Sprague (Eds.), Progress in physiological
Behavior, 1972, 18, 201-216. psychology (Vol. 4). New York: Academic Press, 1971.
Seligman, M. E. P., 8c Hager, J. L. Biological boundaries of Teitelbaum, P, Discussion: On the use of electrical stimula¬

learning. New York: Meredith, 1972. tion to study hypothalamic structure and function. In
Sevenster, P. Incompatibility of response and reward. In A. N. Epstein, H. R, Kissileff, 8c E. Stellar (Eds.), The
R. A. Hinde & J. Stevenson-Hinde (Eds.), Constraints on neuropsychology of thirst, New York: V. H. Winston,
learning. New York: Academic Press, 1973. 1974.
Seyffarth, H., & Denny-Brown, D. The grasp reflex and Teitelbaum, Cheng, M. F., 8c Rozin P, Stages of re¬
the instinctive grasp reaction. Brain, 1948, 71, 109-183. covery and development of lateral hypothalamic control
Sherrington, C. The integrative action of the nervous of food and water intake. Annals of the New York
system. Forge Village, Mass.: Murray Printing Go., 1906. Academy of Science, 1969, 137, 849-860. (a)
(Republished, New Haven: Yale University Press paper¬ Teitelbaum, P,, Cheng, M. F., 8c Rozin, P, Development of
back, 1961). feeding parallels its recovery after hypothalamic damage.
Shettleworth, 3. J. Constraints on learning. In D. S. Journal of Comparative and Physiological Psychology,
Lehrman, R. A. Hinde, & E. Shaw (Eds.), Advances in 1969, 67, 430-441. (b)
the study of behavior (Vol. 4). New York: Academic Teitelbaum, P., 8c Epstein, A. N. The lateral hypothalamic
Press, 1972. syndrome: Recovery of feeding and drinking after lateral
Shettleworth, S. J. Reinforcement and the organization of hypothalamic lesions. Psychological Review, 1962, 69,
behavior in golden hamsters: Hunger, environment, and 74-90.
food reinforcement. Journal of Experimental Psychology : Thompson, T. I. Visual reinforcement in Siamese fighting
Animal Behavior Processes, 1975,104, 56-87. fish. Science, 1963, HI, 55-57.
Skinner, B. F. The concept of the reflex in the description Tinbergen, N. The study of instinct. London: Oxford Uni¬
of behavior. Journal of General Psychology, 1931, 3, versity Press, 1951. [Reissued in 1969.]
427-458. Tinbergen, N., 8c Kuenen, D. J. fiber die Auslosendcn und
Skinner, B. F. The behavior of organisms; An experimental die richtunggebenden Reizsituationen der Sperrbewe-
analysis. New York; Applsten-Century, 1088. gung yon jungen Drosseln (Turdus m. merula T. und
^KINNERj B. P. Walden Ttoo. New York: MacMillafi, 1948. T. e. ericetorum Turton). Zeitschrift filY TiCYpSycholOglC,

W 1939, 3, 37-60. (Trans, and reprinted in C. H. Schiller

[Ed.], Instinctive behavior. New York: International
Skinner, R. F. Superstition in the pigeon. Journal of Experi¬
mental Psychology, 1948, 38, 168-17§» (b) Universities Press, 1957.)
SKINNER, B, F, Are disarms necessary) Psyeho- Turner, B- N. A sensorimotor syndrome produced by
logical J1050, VJS 108-216. lesions o£ the amygdala and lateral hypothalamus.
Skinnerj B. E. A case history in sd^ntiflc method. American Journal of Comparative and Physiological Psychology,
Psychologist 1956, IE 221-233. 1973, 82, 37-47.
Skinner, B. F. The experimental analysis of behavior. Twitchell, T. Ei. The restoration of motor function fol

American Scientist, 1957, i3, 343-371. lowing hemiplegia in man. Brain, 1051, 94, 443—480.
Skinner, B* Y. Why we need leashing machines. Harvard Twitchell; T. E. The automatic grasping responses Of In¬
Mdueatlonal PcyletVy 1061, 31, 377-308. fants. Ncuropsychologia, 1965, 3, 247—259.
SkinneRj B. P. The phylogeny and ohtOgCny Of behaVlOT. Twitchell, T. E. Early development of avoiding and
Kctmoc, 1966, 153, 1203-1213. grasping reactions. In 5. Locke (Ed,), Modern neurology.
Skinner, B. F. Cumulative record (3rd ed.). Englewood Boston: Little, Brown, 1969.
Cliffs, N.J.: Prentice-Hall, Inc., 1972. (a) Twitghell, T. E. Reflex mechanisms and the development
Skinner, B. F. The flight from the laboratory. In B. F. of prehension. In EL J. Connolly (Ed.), Mechanisms of
Skinner, Cumulative record. Englewood Cliffs, N.J.: motor skill development. New York: Academic Press,

Preruiee-Hall, Inc., 1972- (b) 1970.

Smith, J. C.? 2c Roll, D. L. Trace conditioning with X-rays Valenstein, E. S. Invited comment: Electrical stimulation
as an aversive stimulus. Psychonomic Science, 1967, 3, and hypothalamic function: Historical perspective. In
11-12. A. N. Epstein, H. R. Kissileff, 8c E. Stellar (Eds.), The
Steinbaum, E. A., & Miller, N. E. Obesity from eating neuropsychology of thirst. Washington, D.C.: V. H.
elicited by daily stimulation of hypothalamus. American Winston, 1973.
Journal of Physiology, 1965, 208, 1—5. Valenstein, E. S., Cox, V. C., 8c Kakolewski, J. W. The
Stunkard, A. J. New treatments for obesity: Behavior motivation underlying eating elicited by lateral hypo-
Philip Teitelbaum 27

thalamic stimulation. Physiology and Behavior, 1968, Wilcoxon, H. G., Dragoin, W. B., & Kral, P. A. Illness-
159, 1119-1121. induced aversions in rat and quail: Relative salience of
Wald, G. Molecular basis of visual excitation. Science, 1968, visual and gustatory cues. Science, 1971, 171, 826-828.
162, 230-239. Williams, D. R., 8c Williams, H. Auto-maintenance in the
Wessells, M. G. The effects of reinforcement upon the pre¬ pigeon: Sustained pecking despite contingent non-rein¬
pecking behaviors of pigeons in the autoshaping experi¬ forcement. Journal of the Experimental Analysis of Be¬
ment. Journal of the Experimental Analysis of Behavior, havior, 1969, 12, 511-520.
,
1974, 21 125-144.
 2

The Operant Revisited*

George Collier,
Edward Hirsch, and Robin Kanarek

INTRODUCTION in a freely feeding animal. In consequence, therefore,

Skinner devised a measure of the strength of feeding
Operant psychology started with a meal. Using eat¬ behavior based upon the rate of eating within a meal
ing as a means of studying reflexes and reflex chain¬ (Skinner, 1932a). The assumption underlying this strat¬
ing, Skinner developed the apparatus, the conceptual egy was that knowledge of the strength of the chain of
framework, and the methodology of operant analysis reflexes within a meal would make it possible to pre¬
(Skinner, 1930, 1931, 1932a, 1932b, 1935). dict both the onset and termination of a meal and
The focus on eating stemmed from the search for a thus the pattern of meals. Skinner’s interest in the
recurrent, lawful behavior for analysis. The “orderly strength of the feeding reflexes within a meal led to a
periodicity in * . . eating activity” reported in Rich¬ shift from this continuous sampling of a nondeprived
ter’s 1927 study of meal taking by the rat provided animal’s behavior to a sample gained in a short, con¬
Skinner with such a phenomenon. Skinner sought to strained experimental session in a food-deprived an¬
account “for the appearance or nonappearance of a imal. With his parsimonious, Baconian devotion to
given set of behavior at a given time.” Richter’s “sim¬ the observable, Skinner sought the laws of eating
ple observation of whether a rat eats” was, “after all, solely m the relations between reflex probabilities and
only an all-or-none measure” (Skinner, 1930, 1932a). such operations as fasting and feeding which changed
His analysis focused on meal frequency and duration these probabilities (Skinner, 1931, 1932a). He eschewed
* This research was supported by Research Grants HD-00941
any dependence upon hypothetical, neurological, or
and HD-03279 from the National Institutes of Health, Bethesda, physiological structures or states. The behavior laws
Maryland, and the Rutgers University Research Council. The were considered to be self-sufficient. There was no
authors are indebted to Nicholas Mrosovsky and Sarah Shettle-
worth at the University of Toronto and Robert Bolles at the
need to reduce them to or explain them by phenom¬
University of Washington (Seattle) for their critical reading of ena from some other domain. Today, this tradition,
the early manuscript and to Carolyn K. Rovee of Douglass further refined, is continued by such investigators as
College, Rutgers University, for both her intensive editing and
her critical commentary. Herrnstein (1970), Morse and Kelleher (1970), Pre-

28
George Collier, Edward Hirsch, and Robin Kanarek 29

mack (1959), and Timberlake and Allison (1974). Eating as a Reflex

To study feeding behavior an eaiometer was de¬
The basic assumption of the operant analysis of
vised! This apparatus consisted of a door which the
behavior is that current consequences control future
rat had to push open in order to seize a single pellet
performance. An important, interesting, and very diffi¬
from a food magazine. Following a period of time
cult question concerns the specification of the units of
without food, the rats were placed within the ap¬
behavior on which these consequences act. The char¬
paratus and allowed to consume food until eating
acterization of these units can vary from simple
ceased. Each door opening was recorded on a cumula¬
muscular movements devoid of meaning (Guthrie,
tive recorder which produced a record of the rate of
1935) to complex patterns of responses whose dynamic
eating. Rate, then, was considered to be the measure
interaction is intrinsic and is shaped by their conse¬
of reflex strength (Skinner, 1932a).
quences (Kohler, 1929). As the first step in his analysis
The behavior was so orderly that Skinner suc¬
of this problem, Skinner chose the reflex as the func¬
cumbed to temptation and fitted a curve to the data
tional unit.
[AT = Ktn; where N is the amount (number of pellets)
The reflex as a unit of analysis has a long history
eaten, t the time in session, and n and K curve-fitting
constants]. (Fearing, 1930; Skinner, 1931). In 1662, an era when
physical science was in the first flush of its initial suc¬
At this juncture Skinner, still under the influence
cesses in devising mechanical models of the physical
of the Sherringtonian definition of the reflex, was in¬
universe, Descartes introduced the concept of the re¬
terested in analyzing the rate of occurrence of the
flex as an attempt to explain animate motion with a
various members of the reflex chain. In particular, he
mechanical model (Jaynes, 1970). He derived his in¬
was concerned with the influence of eating time (chew¬
spiration from the hydraulically actuated dolls in the
ing and swallowing) on the refractory phase of the
gardens of St. Germain which executed intricate pat¬
initial reflex in the chain (seizing) and sought to dis¬
terns of movement when “stimulated” by someone
cover if “the law expressed in the equation N — Ktn
treading upon a concealed pedal (Fearing, 1930;
is independent of the particular reflex that initiates
Jaynes, 1970; Skinner, 1931). In Descartes’s view, an¬
the eating behavior” (Skinner, 1932b). To deal with
imals were automatons. Only the voluntary activity of
this question, Skinner introduced an “arbitrary initial
humans was excepted from this categorization. Be¬
member” to the reflex chain, the lever press.
havior could be exhaustively duplicated by sufficiently
complex machines obeying only physical principles.
The food tray is accordingly replaced by a re¬
The reflex has been accepted by physiologists and
peating “problem box” which delivers a pellet
psychologists of all persuasions as an accurate charac¬
of food into an open trough each time a hori¬
zontal lever is pressed downward. terization of at least some aspects of behavior. There
are at least three major reasons for this attraction;
Thus the Skinner box was born. The results obtained
from the Skinner box showed that “the rate of change 1. Once Pavlov (1927) had demonstrated the condi-
tionability of reflexes, it was relatively easy to con¬
of the rate of eating is independent of the nature of
ceive of reflexes as the building blocks for complex
the particular reflex with which eating behavior be¬
sequences of behavior (Guthrie, 1935; Hull, 1937;
gins.” The delight in this conclusion, however, was
Skinner, 1938). These sequences could be assembled
soon surpassed by the excitement generated by new by an organism’s phylogenetic or ontogenetic inter¬
discoveries concerning the arbitrary initial member action with its environment (Skinner, 1966). In this
and the subsequent interest in elaborating the con¬ view, order in behavior only reflects order in the
cept of the operant. This diversion left the original environment, not structure in the organism.
problems unexplored. 2. Mentalistic or nonphysical language can be avoided
In this chapter we wish to focus on three points in in the analysis of behavior by defining the stimulus
Skinner’s original analysis of feeding. First, we shall component of the reflex as any arbitrary change in
consider whether Skinner’s use of the reflex as the the environment and the response component as
unit of analysis was a felicitous choice. Second, we any arbitrary movement of the organism without
shall examine the “orderly periodicity” of eating re¬ reference to intention, purpose, or function. This
stratagem precludes any biological or psychological
ported by Richter which stimulated Skinner’s analy¬
terms at the level of the data language. The psy¬
sis. And third, we shall explore whether operations
chological meaning of a behavior sequence derives
other than fasting and feeding control the pattern of from such criteria as establishing “smooth curves
meal taking. for dynamic laws” (Skinner, 1938). Skinner cau-
30 THE OPERANT REVISITED

tioned against the use of vernacular, biological, episode of eating by introducing the operation of fast¬
neurological, or mentalistic concepts for other than ing, which insured initiation of eating. That is, it
heuristic purposes in the search for useful classes of placed initiation under the experimenter’s control and
variables (Skinner, 1938). made the description of within-meal behavior the
3. Finally, acceptance of the reflex requires little or no locus of analysis. The basic data of the within-meal
theoretical commitment to mental or physiological analysis was the initial rate of ingestion and the rate
processes as behavioral substrates. In this search for of change of rate of ingestion within a session as
functional laws devoid of causal explanations, Skin¬
functions of such operations as fasting and feeding.
ner followed Mach and Bridgman (Skinner, 1931).
Since only a single episode of eating, defined by the
experimental session, was observed, no data on the
Historically, the major disadvantage of a reflex frequency, duration, and distribution of meals in time
analysis is that description of behavior has, in fact, was obtained.
become circumstantial and complex. It has become Even after interest in eating per se had shifted to
necessary to elaborate complex concepts such as the the effects of schedules on behavior, the same experi¬
“observing response” (Wyckoff, 1952) or “pure stimu¬ mental paradigm was maintained; Animals were de¬
lus acts” (Hull, 1930) to deal with phenomena such as prived of food, and their behavior was measured dur¬
attention or intention. ing an experimental session. The analysis still focused
Reflex-based descriptions of behavior are also on “reflex strength” as Skinner originally defined it
plagued by the ultimately difficult problem of the in¬ (Skinner, 1935). Complex behavior was constructed
finite number of movements and stimuli which might from the simplest reflex components. The laws of
occur at any given cross section in time and place. combination are derived from the laws of reflex
Skmner!s concept of the generic nature of the stimu¬ strength. Thus since in this experimental paradigm
lus and response allowed him to circumvent the tedi¬ only a single meal, initiated and terminated by the
ous problem of botanizing stimuli and responses experimenter, was observed, the analysis of the results
(Skiiinerj 1935). The stimulus class was defined by the obtained was of within-meal behavior. The question
experimental context* and the response class by its raised by employing this tactic is whether the same
effect on the environment rather than by its detailed variables which predict within-meal behavior also
topography. In fact, Skinner advocated watching the predict between-meal behavior, and if so, are the func¬
recorder rather than th§ animal. Conceptual utility tions the same? We shall attempt to demonstrate that
was determined by the extent to which a given set of this analysis does not predict the pattern of feeding in
environmental variables resulted in “simple” laws the freely feeding, nondeprived animal and has led to
when a given category of behavior was used (Skinner, the neglect of several important variables m the study
1938). of animal learning and motivation.
Skinner’s generic characterisation of a response was Skinner followed in the footsteps of the early
a historically important step away from his original physicists in developing his research strategy. Having
analysis of the “strengths” of the specific reflexes borrowed a mechanical model (i.e., the reflex) from
(seizing, chewing, swallowing) involved with the in¬ physics for the study of behavior, it was also natural
gestion of food (Skinner, 1932^). His finding that the to borrow a methodology in the form of the refine¬
“rate of change of the rate of eating i§ independent of ment experiment. Early physics was hampered by
tbo nature of the particular reflex with which eating poor tools and materials and a lack of coherent theory
begins” (Skinner, 1982b) made possible his use as an for identification of variables. Relations established
arbitrary initial member, a specific environmental between variables were subject to large amounts of
event—that is, a bar press—to represent the whole error, and the program was one of successive refine¬
chain of behavior. ment of the experiment to reduce input from “ex¬
Richter (1927) had shown that a freely feeding rat traneous” sources in order to eliminate error and dis¬
exhibited periodic episodes of eating, defined as cover the “true” law. The best experiment was one in
meals, which excluded other activities. His rats were which the effects produced by the variable(s) being
not deprived in the usual sense, and the initiation or studied were large relative to all other effects. Operant
termination of a particular bout of eating was not analysis has not strayed from this path (Sidman, 1960).
predictable from the variables he analyzed. Rather, he To study an automaton, one needed to reduce ex¬
was only able to characterize the number, duration, traneous stimuli and restrict the response possibilities
and distribution of these episodes of behavior. Skin¬ so that one could find the law hidden in the variety
ner approached the problem of determining such an of activities observed. Hence a highly inbred, docile
 , ,
George Collier Edward Hirsch and Robin Kanarek 31

animal, limited in historical inputs (naive), was chosen which some parameter is regulated around a priv¬
as the object of experimentation. He was placed in a ileged value. Regulation of energy balance is inferred
box isolated from the sight, sounds, and smells of his when average energy expenditure and intake are
neighbors, where the only question was “to press or matched. This condition must be met by any or¬
not to press.” It was a study of performance in soli¬ ganism maintaining a constant size or fixed pattern
tary confinement. Different animals and different situ¬ of growth or senescence. In addition, regulation can
ations were not required to discover basic laws (Skin¬ be the result of many different mechanisms, both be¬
ner, 1938). The conviction was that laws derived using havioral and physiological (Yamamoto & Brobeck,
the reflex as the unit of analysis are universal. That 1965). In the strict sense, homeostasis implies nega¬
is, such laws are invariant across species, response tive feedback, specialized receptors, and moment-to-
classes, and reinforcers (see Skinner, 1966, for a quali¬ moment monitoring of energy balance. In the case of
fication of this thesis). It should be emphasized that feeding, the process is assumed to consist of successive
the experimenter exerts close control over the behav¬ depletion and repletion phases (DeRuiter, 1967). De¬
ior that is exhibited within this paradigm. The an¬ pletion occurs as a result of metabolism. When energy
imal is (purposely) restrained from exhibiting its full stores are depleted below a threshold or critical value,
repertoire of behavior. Although these procedures re¬ feeding behavior (search, seizure, ingestion) preempts
duce “extraneous” sources of variation and limit the other ongoing activities. Ingestion leads to repletion,
behavior displayed, they offer little opportunity for and, when an upper threshold of the energy stores or
observing the kinds of “solutions” the animal might some surrogate of these stores is exceeded, ingestive
make to a similar problem occurring in his “natural,” behavior ceases. This depletion-repletion cycle differs
noisy, and (to the experimenter) confusing environ¬ from the more usual homeostatic systems since the
ment. Rather, it is assumed that knowledge of an item being controlled (e.g., food or water) is discon-
animal's evolutionary history, his classification, his tinuously present in most environments in contrast to
current situation, his ecological niche, and his present an item such as oxygen which is continuously present
habitat does not contribute in any fundamental way (cf. Cannon, 1932). Thus feeding can only occur in
to the understanding of the principles of behavior episodes rather than continuously. It is important to
acquisition and maintenance. note that in this model, feeding is initiated in re¬
It will be the argument of the present chapter that sponse to a substantive deficit. The character of this
the meal, as originally defined by Richter (1927), is a deficit is still unspecified and is the locus of most
better unit of analysis than the reflex for the study speculation and current research activity in feeding.
of hunger, because, in the original sense of Skinner The depletion-repletion model has been invoked to
(1938), it “gives smooth curves for dynamic laws.” explain a variety of motivated behaviors, including
This unit will best reveal its utility in an environ¬ those for which there is no obvious biological sub¬
ment in which the animal himself schedules the ini¬ strate (e.g., curiosity). In fact, the view of necessity as
tiation and termination of behavior. It seems possible the driving force of behavior is an important part of
that the discovery of the important variables in such the conventional wisdom of Western civilization.
an environment is most likely to solve Skinner’s Western laws and customs are based on the notion
original problem of accounting “for the appearance of that a person or animal will only perform some act if
a given act of behavior at a given time” (Skinner, some essential requirement is taken from him (depriva¬
1930, 1932a—see above). tion) and given back (reinforcement) in small units
contingent upon the individual performing the re¬
quired act. This historical fact may account for past
Feeding and Fasting
failure to consider alternative models in analyzing
One of the great conceptual difficulties with a motivation.
mechanical model of animate behavior is the provi¬ The depletion-repletion model of motivated be¬
sion of a motive force and direction for the organism. havior has generated two main problems yet to be re¬
The solution to this problem for the early mechanists solved: (1) What is the nature of the signals which are
was homeostasis (cf. Pavlov, 1927). The theory of correlated with depletion and repletion? and (2)
homeostasis derived from the concept of equibrium What mechanisms detect and interpret these signals?
systems developed in thermodynamics. It was orig¬ A variety of physiological processes have been pro¬
inally used to describe the dynamic constancy of the posed to serve as the signals, from stomach contrac¬
fluid matrix of the cells (Cannon, 1932). Its meaning tions or dry mouths (Cannon, 1932) to circulating
has since been extended to include any steady state in metabolites and/or electrolytes (cf. Fitzsimons, 1971;
32 THE OPERANT REVISITED

Hoebel, 1971; LeMagnen, 1971; Mayer, 1955). The the animal determines the initiation and termination
most popular “interpreter” of these hypothetical sig¬ of a meal. This becomes clear when the six parameters
nals has been the hypothalamus, with the two proc¬ which exhaustively described meals are considered:
esses, depletion and repletion, being represented in frequency, duration, amount, rate, intermeal interval,
the lateral and ventromedial nuclei, respectively (cf. and choice of items. The meal as a unit of analysis
Hoebel, 1971). This simple, elegant, two-stage model satisfies the criteria of reliability. When a minimum
has generated most of the research on feeding and amount of ingestive activity (e.g., 10 sec in the feeder
most theories of food-based motivation, with the result or 3 pellets consumed) is used to define meal initia¬
that there has been little systematic research on feed¬ tion^ and a period of time without ingestive activity
ing outside this framework or on animals other than (e.g., 10 min of no eating) defines meal termination,
rats. meals prove to be discrete events which are relatively
The pattern of this research has been to test the insensitive to changes in the criteria (e.g.. Baker, 1953;
implications of various versions of the homeostatic- Hirsch, 1973; Kissileff, 1970; Levitsky, 1970; Pank-
hypothalamic model. This pattern of research ex¬ sepp, 1973; Richter, 1927; Thomas & Mayer, 1968;
emplifies explanation in terms of a conceptual nervous Wiepkema, 1968). It is clear that meals can be con¬
system decried by Skinner (1931). For example, home¬ sistently measured. The question of interest is whether
ostatic models dominated research in the area of feed¬ systematic laws can be found using meals as the units
ing long before any of the requisite physiological and of analysis.
neurological measurements were possible (e.g.7 Hull, The current revival of interest in meals stems from
1943). Only recently have strong reservations about the hypothesis that meals reflect the momentary physi¬
the underlying assumptions of this model been ex¬ ological state of the organism (cf. LeMagnen, 1971;
pressed (e.g., Collier, Hirsch, 8c Hamlin, 1972; Falk, Teitelbaum 8c Campbell, 1958; Thomas 8c Mayer,
1971; Fitzsimons, 1971; Kissileff, 1973; Oatley, 1970). 1968). An animal is presumed to initiate a meal fol¬
The Skinnerian analysis of hunger avoided these lowing a period of time without eating when the level
problems of homeostatic theorizing and can best be of circulating metabolites, hormones, or reserves re¬
summarized in a quotation from The Behavior of flects a critical level of depletion, Similarly, the meal
Organisms (1938, ppf 342 f; sec also Skinner, 1932a, is terminated when ingestion effects some critical
1932b); change in physiological condition. A logical implica¬
tion of the depletion-repletion model is the existence
In dealing with the kind of behavior that gives of significant correlations between size of meals and
rise to the concept of hunger we are concerned intermeal intervals. There are two possible correla¬
with the strength of a certain class of reflexes tions. The first is between the intermeal interval pre¬
and the two principal operations that affect it— ceding a meal and the size of the meal. The second is
feeding and fasting. between the size of the meal and the following inter¬
meal interval. In the first case, if the amount eaten in
This proposition asserts the primacy of behavioral
a meal is a function of the degree of depletion, it
analy§i§. It i§ p?§§ibK that history may show that more should reflect time since the last meal. That is, the
progress in the study of hunger would have been meal following a long period of no eating should be
made had the laws of feeding behavior been investi¬ larger than one following a short intermeal interval.
gated within the positivistic framework advocated by If, however, the time lapse between meals is too short
Skinner rather than within those of the ever-changing
for the depletion threshold to be exceeded, some
models of the central nervous system (see also Adolph,
other mechanism must instigate meals. Similarly in
1947; Brody, 1945; Kleiber, 1961; Richter, 1927; the second case, following a large meal, the intermeal
Young, 1936), interval should be longer than that following a small
meal, reflecting the influence of a “satiety” mech¬
Meals
anism.
A significant correlation between meal size and the
Richter’s early observations showed that rats dis¬ premeal interval has not been demonstrated under
tributed their feeding in episodes which can be re¬ free-feeding conditions (Baker, 1953; Balagura &
garded as discrete meals. Over a 24-hr period of free Coscina, 1968; Booth, 1972; Hirsch, 1973; LeMagnen
feeding, he observed 8-10 meals (Richter, 1927). The & Devos, 1970; LeMagnen 8c Tallon, 1966; Levitsky,
definition of the term free in these studies does not 1970; Levitsky 8c Collier, 1968; Snowden, 1969; Thomas
imply cost, quantity, or availability, but rather that 8c Mayer, 1968; Wiepkema, 1968; Zeigler, Green, 8c
George Collier, Edward Hirsch, and Robin Kanarek 33

Lehrer, 1971). It is important to note that this finding correlation simply as a result of intersubject differ¬
is in marked contrast to the results obtained when an ences in meal frequency and duration. Panksepp
animal undergoes substantial deprivation preceding a (1973), in an elaborate analysis, has shown other sta¬
meal (Adolph, 1947; Bolles, 1967; Stellar 8c Hill, tistical artifacts in computation of the correlation.
1952). Whenever weight loss exceeds 7-10% of ad lib There is, further, the suggestion that both diet com¬
weight, there is a linear relation between body weight position (Levitsky, 1974) and texture (Thomas 8c
loss and many different measures of performance (cf. Mayer, 1968) may affect the size of the correlation.
Collier, 1969). The fact that meal size is a function of The failure to find either correlation consistently, the
the deprivation interval when depletion exceeds a continuity of intestinal load, and the small weight loss
certain critical size reflected in body weight but not in the intermeal interval all suggest that the opera¬
for the minor weight loss occurring between meals in tions of fasting and feeding (Skinner, 1932a, 1938) are
freely feeding animals suggests that different proc¬ not the sole determinants of the appearance or non-
esses may be involved in initiating eating in these two appearance of eating in freely feeding animals. A new
cases. Studies of stomach and intestinal contents in class of variables must be sought in order to discover
freely feeding rats and guinea pigs which have average lawful relations.
intermeal intervals of 2-4 hrs indicate that there is a Richter (1927) found that the daily pattern of
continuous and relatively constant intestinal load, meals was sensitive to a variety of environmental
even though the stomach load fluctuates with meals. variables. The availability of alternative activities
Thus any fluctuations in the input across the in¬ such as climbing on towers, running in wheels, and
testinal lumen would be endogenous in origin (Collier, nesting in boxes substantially affected meal frequency
Hirsch, 8c Hamlin, 1972). It would seem that in environ¬ and duration. Another such effect is the duirnal
ments in which the commodity whose intake (e.g., rhythm in eating. That is, a rat is most likely to be
food or water) is being controlled is discontinuously found eating in the dark phase of the light-dark cycle,
present, animals have met the problem of maintain¬ irrespective of the time between meals or the size of
ing a constant milieu interne by establishing a con¬ the previous meal (Baker, 1953). These observations,
stant milieu externe in the gut. The gut acts as a buttressed by recent laboratory results (Collier, Hirsch,
reservoir which buffers the episodic pattern of intake. 8c Hamlin, 1972; Hirsch, 1973; Hirsch 8c Collier, 1974a,
This is most obvious in the large ruminants. An adult 1974b; Kanarek, in press; Levin 8c Levine, 1974; Mar-
dairy cow, for example, usually has approximately 60 wine, 1974) and field studies (Bell, 1971; Estes, 1967a,
gallons of fluid in the rumen. The existence of these 1967b; Kruuk, 1972; Schaller, 1967, 1972), suggest that
continuous intestinal loads suggests that meals in an analysis of the relations between the parameters of
freely feeding animals might not be initiated by de¬ meals and environmental variables may be fruitful.
pletion, but rather by some endogenous process. The One can speculate that species have evolved feed¬
null hypothesis in this case would be that there is a ing patterns through the course of evolution that
base rate of meal initiation in the freely feeding an¬ reflect their niche (Schoener, 1971). Animals have
imal generating a random sequence of meals which, specialized in the exploitation of specific food sources,
on the average, result in an adequate intake. In¬ which vary in (1) availability, (2) nutritional quality,
dividual meals occur independently of the state of and (3) caloric density. If the energy budgeted for the
the organism (Premack 8c Kintsch, 1970). Only the procurement and ingestion of food represents an im¬
parameters of this distribution, not the individual portant portion of the total energy budget, it seems
events, would reflect regulatory control. improbable that animals could afford the risk of sub¬
On the other hand, a significant correlation be¬ stantial depletion before instigating feeding behavior.
tween meal size and the interval following a meal has This might create a condition of insufficient energy
been reported by LeMagnen (LeMagnen, 1971; Le- for successful feeding activity. Further, it is a striking
Magnen 8c Devos, 1970; LeMagnen 8c Tallon, 1966) observation that animals living in undisturbed eco¬
and a number of other investigators (e.g., Balagura 8c logical systems appear to match their numbers to re¬
Coscina, 1968; Booth, 1972; Levitsky, 1974; Snowden, sources. Since resource matching implies that the be¬
1969; Thomas 8c Mayer, 1968). Recent papers have havior of animals must “anticipate” their needs rather
raised statistical, methodological, and theoretical ob¬ than respond to them, variables other than immediate
jections to the validity of this correlation (Collier, physiological state must mediate the initiation and
Hirsch, 8c Hamlin, 1972; Hirsch 8c Collier, 1974a, termination of feeding. Hungry or at least starving
1974b; Panksepp, 1973). For example, the pooling of animals are very seldom observed in the wild except
subjects (LeMagnen, 1971) can produce a significant for reasons of illness, age, or social status (Wynne-
34 THE OPERANT REVISITED

LIGHT DARK LIGHT CAT 1

_n_n_n_u_n nn n 0
j—ljl ... n in n in ni_n
-in_n_ _n_n n n
i—i _n_n_n_ _n_
j—|__tl
_n_ r“i
i_
_1_1_1_1_1_■■»■!« ii.ii.it
1_J _n_i_i_i fi . ciT2
n_j_i _n_(i_ii n fi
n , n n non
H_L
i_ J]_P_n n n
li Of! n n n
n n n
Fig. 1. Temporal sequence of the
meals of two cats at several ratio
requirements. The width of pip
i——« ..*—.-«—«—«—«—*—*—«—*— .j
ff i$ 4 5 § TAM § shows the duration of the meal.
TIME |From Kanarelc, 1975.}

Edwards, 1962). This fact suggests that animals in un¬ analysis. Such an analysis may well reveal new classes
disturbed habitats have developed behavior, both so¬ of variables and relationships controlling feeding.
cial and unsocial, which insures an adequate intake

of food. For example, when the available food is in¬

sufficient, the dominance hierarchy in a flock of chick¬ FREE FEEDING
ens insures that the dominant birds consume their

usual rationgj while the subdominants receive the The environments in which free feeding has been
remainder. The subdominants will starve to death in studied have varied widely. One which has been ex¬
the presence of food rather than challenge the domi¬ tensively used provides a caged laboratory animal
nant birds. There is no direct competition for food, with nutritionally complete food (as the experimenter
only for status (WynneEdwards* 1962)* Similarly* perceives it) and water. Both commodities are con¬
when food is seasonally abundant, both birth rate and tinuously available. The ad lib lines in Figure 1 show
consumption are constrained in such a fashion that sample 24-hr records of such feeding behavior in two
the population density matches the period of least house cats (Kanarek, 1975). The food was Purina
availability of food (TA/yiiiie-Edwards? 1962). A final Cat Chow available in a hopper attached to a large
example can he drawn from ruminants, Tli§se animalg cage. The feeding episodes were distributed as discrete
have a large storage capacity in the rumen and a long meals of varying size. The majority of the meals were
transit time from ingestion to absorption. A consistent consumed in the dark. Approximately 9-10 meals per
intake must be maintained to provide the raw ma= 24 hr were taken. There were no consistently signifi¬
terials for fermentation. Tbe end product only gradu¬ cant meal-intermeal correlations. The largest meals
ally becomes available for regulatory information. It typically preceded and followed the intervention of
seems unlikely tfiat intake can directly track the the experimenter for purposes of weighing and daily
momentary metabolic state under these circumstances. maintenance. This can be clearly seen in Figure 1,
The considerations discussed above suggest that it where management occurred at 5 r.M. Similar effects
may be useful to return to Richter’s original meth¬ of experimenter intervention on meal size have also
odology and study animals in environments in which been seen in rats and guinea pigs (Hirsch, 1973; Levit¬
they can exhibit the behavior which they have evolved sky, 1970; Marwine, 1974).
to solve both the economics as well as the physiology Although the characteristic number of meals taken
of feeding. This would require a return to the situa¬ within a 24-hr period varies widely among species and
tion in which the animal initiates and terminates may be species-typical, the same general pattern of
feeding and in which the meal is the useful unit of meal-taking behavior has been described in animals
George Collier, Edward Hirsch, and Robin Kanarek 35

also remained constant over this time period, with as

ADULTS many meals taken in the light as in the dark.
The relative insensitivity of meal frequency to ex¬
PURINA CHOW CELLUFLOUR PURINA CHOW
30 DILUTION perimental modification is also illustrated by the fact
that water restriction did not influence the daily num¬
ber of meals. Adolph (1947) first pointed out the
20
strong interrelation between food intake and water in¬
take. There is a voluntary reduction in food intake
if) when water is given in a limited ration, resulting in a
_J
< linear relation between the size of the water ration
LU and food intake (Collier 8c Knarr, 1966; Collier 8c
2
Levitsky, 1967). For our present purposes, the surpris¬
Ll
O ing observation is that both rats (Levitsky, 1970; Mar-
CL wine, 1974) and guinea pigs (Hirsch, 1973) initiate as
LlJ
DD many meals during periods of water restriction as
Z)
they do when water is freely available. The reduction
Z in food intake occurs because smaller meals are taken.
These results are obtained when the water is given in
a limited ration (Levitsky, 1970), for a limited amount
of time (Hirsch, 1973), or if a reduction in intake
occurs when animals are required to lever-press for
their daily water allotment over the 24-hr period
(Hirsch & Collier, 1974b; Marwine, 1974). These find¬
ings are surprising in light of the close temporal rela¬
tion between feeding and drinking in animals fed
and watered ad lib,
DAYS
Another example of the insensitivity of meal fre¬
Fig. %. number of meals for young and adult guinea
Mean daily quency to manipulation is seen following lesions that
pigs. Thedarkened portion of the histogram shows the number
destroy the ventromedial region of the hypothalamus.
of meals taken at night, and the unfilled portion shows the
number of meals taken during the day. (From Hirsch, 1973.) The overeating that results from these lesions is due
entirely to changes in meal size (Teitelbaum 8c Camp¬
bell, 1958; Thomas 8c Mayer, 1968). These observa¬
as diverse as rats (e.g., Richter, 1927), mice (Wiep- tions indicate that, under a variety of conditions, an¬
kema, 1968), guinea pigs (Hirsch, 1973), gerbils imals with free access to food modulate intake by
(Kanarek, unpublished observations), pigeons (Ziegler, adjustments in meal size rather than frequency.
Green, 8c Lehrer, 1971), chickens (Duncan, Duncan, The preceding description is not meant to suggest
Hughes, 8c Wood-Gush, 1970), cats (Kanarek, 1975), that regulatory changes in feeding behavior cannot
dogs (Robinson 8c Adolph, 1943), and cockroaches be mediated by adjustments in feeding frequency.
(Faber, 1975). Changes in meal frequency occur following olfactory
When food and water are readily available, meal bulbectomy (LaRue 8c LeMagnen, 1971) and lateral
frequency is the parameter of free feeding that re¬ hypothalamic lesions (Kissileff, 1970), during chronic
mains invariant under a variety of experimental con¬ diabetes (Booth, 1972; Panksepp, 1973), and during
ditions. This constancy is seen clearly in a develop¬ intragastric nutrient infusion (Thomas 8c Mayer,
mental study of free-feeding behavior in the guinea 1968). Further, those species which make compensa¬
pig (Hirsch, 1973). Figure 2 shows meal frequency for tory changes in food intake in response to manipula¬
two groups of five guinea pigs each. The young an¬ tions of caloric density of diet appear to accomplish
imals were 10 days of age at the start of the experi¬ these primarily by changes in meal frequency (Kan¬
ment, and the adults were 100 days old. Meal fre¬ arek, 1974; LeMagnen, 1971; Thomas 8c Mayer, 1968).
quency did not change from 10 days to almost 6 For our present purposes, we simply wish to empha¬
months of age. Growth-related changes in food intake size that (1) a wide range of intact animals with food
were accomplished entirely by increases in meal size. and water readily available distribute their feeding
The day-night (shown respectively by the unfilled and behavior into distinct episodes which can be classified
filled portions of the histogram) distribution of meals as meals; and (2) the daily frequency of the meals is
36 THE OPERANT REVISITED

relatively insensitive to a number of experimental ulated in the laboratory by the use of operant tech¬
manipulations which produce large changes in daily niques, several important theoretical questions arise:
food intake, the latter changes being accomplished
solely by changes in rate and duration of feeding. 1. Will nondeprived animals tolerate demanding in¬
Drinking can be analyzed in the same fashion as strumental response requirements to obtain their
eating, but there is much less information available daily allotment of food or water in the free-feeding
regarding the pattern of drinking that occurs under paradigm? That is, when food is delivered on
schedules requiring large numbers of responses and
conditions of unlimited access to water. Siegel and
long periods of time in which to execute them,
Stuckey (1947) were the first to quantify Richter’s
must animals undergo “substantial” deprivation in
(1927) observation of a diurnal pattern in rats. They
order to initiate a meal?
found that rats consume approximately 75% of their
2. Is it necessary for the initial training of the re¬
water at night. This diurnal drinking pattern has
sponse to take place under deprivation? This ques¬
been found to persist during food deprivation (Fitz- tion speaks to the old controversy concerning the
simons & LeMagnen, 1969; Oatley, 1971), following relation between habit and drive (Hull, 1943) and/
bilateral nephrectomy when water intake is sharply or the conditions for reinforcement (Skinner, 1938).
reduced (Fitzsimons, 1969), and even when the an- 3. Do animals undergo substantial weight loss be¬
IffiaF§ fluid requirements have been met by continu¬ tween meals when large response requirements are
ous intragastric water infusion (Fitzsimons, 1957). In imposed?

the rat (Fitzsimons & LeMagnen, 1969; Kissileff, 1969; 4. Will schedules and parameters of reinforcement
Mafwinej 1974), dog (Robinson & Adolph, 1943), exert typical effects given the modifications of the
guinea pig (Hirsch, 1978; Hirsch & Collier* 1974b), experimental environment conditions?
and eat (Kanarek, 1975), drinking occurs in dis¬ 5. What seem to be the most appropriate units of
crete bouts with a strong temporal association be¬ analysis for assessing these conditions?
tween feeding and drinking. In a carefully detailed
analysis, Kissileff (1969) has shown that in the rat, A number of experiments were carried out in an
bouts of drinking are small, with 78% of them being attempt to answer these questions. The resulting data
between .5 and 2*5 mL Approximately 75% of these will be considered below.
bouts occur either 10 min prior to a meal, during the
meal itself, or 10 min after the meal (see also Mar-
Meal Frequency a Function
wme^ 1974).
of Ratio Size

In the first experiment that explored an environ¬

AVAILABILITY ment resembling natural feeding conditions, rats were
continuously boused in the experimental chamber
Another possible environment iS Oiie in Which (Collier, Hirsch, & Hamlin, 1972). Both food and

food and water are not readily available and con¬ water were always available during the first stage of
siderable time or effort is associated with obtaining the experiment, and the pattern of feeding and drink¬
food and water. One obvious example of this situation ing was monitored. After a stable base line of feeding
in the natural environment is the feeding pattern ex¬ and drinking was established, a lever was introduced
hibited by carnivores which must pursue and capture and access to food was made contingent on complet¬
reluctant prey varying m distribution and numbers ing a fixed-ratio (FR) schedule o£ reinforcement. Food
(Schaller, 1972), A second example is provided by remained available after a reinforced lever press for
those herbivores which exploit vegetation dispersed as long as the animal was eating and until 10 addi¬
widely over their pasturage (Bell, 1971; Westoby, tional consecutive minutes after feeding had termi¬
1974). A final example is visits to the water hole. nated. This criterion for a meal is similar to that
Animals which must make periodic pilgrimages of which is commonly used in other research of feeding
varying distances to a water hole are, as a result, sub¬ (e.g., Kissileff, 1970; LeMagnen, 1971; Thomas 8c
ject to varying degrees of predation and competition. Mayer, 1968). This reinforcement paradigm differs
The timing of the trip may be determined more by from the classical one in which a fixed amount of
the factors of predation, competition, and effort than food or a fixed time of access is made contingent upon
by water and electrolyte needs (MacFarlane 8c How¬ the required behavior. In the present situation, the
ard, 1972). animal both initiated and terminated the meal. The
When environments of varying availability are sim¬ following ascending series of FR requirements were
George Collier, Edward Hirsch, and Robin Kanarek 37

Fig. 3. Frequency of meals for three rats as a function of ratio

size. (From Collier, Hirsch, & Hamlin, 1972.)

used: 1, 5, 10, 20, 40, 80, 160, 320, 640, 1,280, 2,560,
and 5,120. Each schedule remained in effect for 10
days. No special shaping or training procedures were
necessary. The animals were not deprived by any
experimenter-controlled procedure, and at no time
did their body weights deviate by more than 5-6%
from their initial body weight. The data presented
here were taken from the last five days under each
condition.
This set of requirements produced orderly changes
in both instrumental and consummatory behavior.
Figure 3 shows that these requirements led to a reduc¬
tion in meal frequency that was linearly related to
log FR size. Under free-feeding conditions these
animals had been eating 9-14 meals per day. This
value decreased to 1 meal per day at FR 5,120. The
performance of subject 2 began to deteriorate at FR
320, and his ratio was not increased over 640. The
reductions in meal frequency were associated with
compensatory increases in meal size. The latter re¬
sulted from an increase in the amount eaten during a
meal rather than from changes in rate of eating (see Fig. 4. Meal size and rate of eating for three rats as a function
of ratio size. (From Collier, Hirsch, & Hamlin, 1972 )
Figure 4). Figure 5 shows that insignificant amounts
of weight were lost under these conditions, even
though there were small declines in daily food intake. bar-pressed in excess of the schedule requirement; by
The animals were run subsequently on a random FR 5, however, the number of reinforcements was an
sequence of these ratios, and the function was exactly accurate reflection of the number of meals consumed.
recovered. There were no discernible practice or The rate of responding was not systematically related
order effects. to FR size. One animal showed ratio strain at FR 320
Several noteworthy features of instrumental per¬ with long pauses between bursts of responding. At FR
formance were apparent when free feeding was con¬ 5,120 a second animal began to show signs of ratio
strained in this manner. First, all animals acquired strain. The third animal never showed evidence of
the lever-pressing response without shaping within ratio strain. At the higher ratios the animals tended
hours after the contingency was imposed. At FR 1 all to respond at steady rates for periods of up to 2-3 hr.
38 THE OPERANT REVISITED

under strong schedule control. This conclusion is

strengthened by the fact that a significant correlation
between meal size and the intermeal interval under
x these conditions has not been found (for contrary
e> .
□ *0(* data, see Levitsky, 1974).
£
There are large individual differences in the size
8 35C* of the ratio tolerated. It is possible that much higher
m
300*- ratios could be sustained if the meal definition were
FF CRF 5 10 20 40 80 160 320 640 1280 2560 5120
changed to allow pauses greater than 10 min. Such a
E procedure would allow the animal to extend a meal
Ot

and slow ingestion rate without facing reinitiation.

However, this variable remains to be explored.
1 .0| The effect of food availability (FR size) on fre¬
quency and duration of meals is not unique to rats.
fV^CRF D io ZQ #0 90 160 3ZQ 6^0 IZOO 2500 3K0
Adolescent cats (Kanarek, 1975) and young, grow¬
RATIO
ing guinea pigs (Hirsch & Collier, 1974a), who not
Fig, Ik !Bo4y weight and food Intake for three rats as a function only must maintain intake at the base line level but
of ratio size. jFrom Collier, Hirsch, 2j Haittlift, 1972.)
also must increase intake in order for growth to
progress m a normal manner, respond to the regula¬

The most striking feature of these data is the size of tory problems posed by constraints on food availabil¬
the ratios tolerated by these nondeprived animals. ity in remarkably similar ways. Figure 6 shows the
Ratios of this size have seldom been reported to sus¬ growth curves of six male guinea pigs tested under
tain stable performance (Ferster & Skinner, 1957), and these conditions. Each successive FR represents a
when they have, the data were obtained from deprived block of four days. It is apparent that normal rates of
animals with conditioned reinforcers built into the growth, established by a control group of six animals
schedule (Findley, 1962: Findley & Brady, 1965). It maintained concurrently on noncontingent feeding,
should also be noted that m the Findley situation the were maintained in all animals until at least FR 1,280.

reinforcement was of a fixed size determined by the The guinea pig adopts a somewhat different strategy
experimenter rather than by the animal. It is con¬ than the rat for conserving normal levels of food in¬
ceivable that the limits on ratio size in the present take. His increase in meal size results from increases
situation ars n<?t determined by the demands of the m both meal duration and eating rate (see Figure 7)-
schedule of reinforcement, but by the inability of rats
to process large volumes of food in short time periods.
When food availability is restricted to an hour or less
a dayr rats do net maintain ad lib levels of intake and
los£ weight (Ehrenfreundj 19505 Lawrence & Masoii,

1955),.
As previously stated, there was little tendency for a
rat to interrupt a run of responding. This may be ex=
plained by the fact that each schedule requirement
has associated with it a certain number of initiations
per day. For example, at FR 80, the rats typically ate

six meals per day, Thus it would appear that the

effect of a long pause in a run of responses would be
similar to requiring another initiation of respond¬
ing and at any given ratio reduce the actual number
of meals the animal would obtain. This consequence of
pausing would tend to enforce uninterrupted runs of

responses to the completion of the requirement.

The reduced frequency of meals resulting from in¬
creased ratio requirements is associated with a com¬
pensatory increase in the size of the meals. Thus it
Fig. 6. Body weight of six guinea pigs at successive ratios. Each
appears that both meal initiation and termination are ratio was in effect for four days. (From Hirsch 8c Collier, 1974a.)
George Collier, Edward Hirsch, and Robin Kanarek 39

o
FOOD CONSUMED / MEAL

FIXED RATIO SIZE

Fig. 8. Frequency of meals for two cats as a function of ratio

size. (From Kanarek, 1975.)

ADLIB 1 I 5 10 20 40 80 160 320 640 1280 2660 SI20

The strategy of the cat is more similar to that of the
FIXED RATIO rat: increases in meal size are accomplished solely by
changes in meal duration rather than in rate of inges¬
tion. Particularly notable was the fact that one cat
MEAL DURATION (minutes)

was able to sustain stable performance at FR 10,240.

Both the reduction in meal frequency (Figure 8) and
the increase in meal size (Figure 9) as a function of the
ratio requirement were completely recoverable during
a random sequence of FR values for one cat and dur¬
ing a descending sequence for the other.
In these three species, the relationships between
food availability (FR size), the parameters of feeding
behavior, and instrumental performance are also ob¬
served when the nature of the operant is varied. In an

AD-LIB I 5 10 20 40 80 160 320 640 1280 2560 5120

FIXED RATIO
RATE OF EATING (gms./minutes)

FIXED RATIO
Fig. 7. Food consumed per meal, meal duration, and rate of
eating for six guinea pigs as a function of ratio size. (From Fig. 9. Food intake per meal for two cats as a function of ratio
Hirsch 8c Collier, 1974a.) size. (From Kanarek, 1975.)
40 THE OPERANT REVISITED

attempt to manipulate the energy cost of a meal, descending sequence of FR values. These shifts in
Kanarek (1973, unpublished research) made access to patterns of drinking are not completely successful in
the feeder contingent on wheel turns rather than maintaining total water intake at control levels. At
lever presses. The same functional relations were ob¬ the higher ratios (FR 80 and above) there is a small
tained. Similarly, Levitsky (1974) found a decrease in decline in total intake, but this does not reduce intake
meal frequency and an increase in meal size as a func¬ below the obligatory requirement. That is, food in¬
tion of the amount of time that rats were required to take and weight gain do not differ from controls.
hold down a lever to gain access to a feeder when the Two other observations illustrate the generality of
rat controlled the initiation and the size of the meal. the behavioral effects that are observed when the
It should be noted that his data are reported and in¬ availability of a reinforcer is constrained. Rats readily
terpreted solely in terms of meal size and intermeal learn to leyer press on FR schedules (Collier 8c Hirsch,
interval. 1971) or to lick (Premack, Schaffer, & Hundt, 1964) for
Almost identical relations are observed in rats access to a running wheel. They will also lever-press
when water availability is restricted by increasing FR to start a voluntary treadmill (Collier & Hirsh, 1971).
requirements. There *§ a menetomc decreasing rela¬ When the duration of the running episodes is under
tionship between FR size and the number of bouts of the animal's control, increases in the FR size lead to a
drinking that occur over a 24-hr period (Marwine, decrease in the number of bouts of running (Figure
1974)* The paradigm used to study drinking is equiv¬ 10), but a compensatory increase in the amount of
alent in all major respects to that previously described running per bout. Although these changes conserve
for the folding environment. Following the comple¬ total amount of running, they are insufficient at the
tion of the FR requirement, a drinking tube is in¬ highest ratios to maintain the level of running of the
serted into the cage. The tube remains available for controls. Also, Adair and Wright (1973) have shown
as long as the animal is licking plus & mm. There is a that behavioral thermoregulation is sensitive to the
regular decrease in bout frequency and an increase in effort involved in controlling environmental tempera¬
bout size as a function of the ratio requirement. Both ture. When the force required to pull a chain that
functions are completely recoverable during a changes environmental temperature was increased,

FREE WHEEL WHEEL TURNS

tlWiW wheel turns

T ' —rr-i r ..-r*—i-•.. BAR PRESSES

FR 10 WHEEL TURNS
1 I 1—I---T f-f I BAR PRESSES

FR 20 “mm-—mm—wheel turns
T*---T---r-1-1- BAR PRESSES

FR 40 wit—w.H_EE_Lp
---^ . -- - ■ BAK rKtjjLj

FR 80 WHEEL TURNS
BAR PRESSES

Fig. 10. Wheel turns for one rat as a function of ratio size. (From Collier 8c Hirsch, 1971. © 1971 by the American Psychological Associa¬
tion. Reprinted by permission.)
 George Collier, Edward Hirsch, and Robin Kanarek 41

squirrel monkeys tolerated greater extremes in the

amplitude of the air temperature. Not only did they
allow a cold environment to become much colder at
high force requirements, but also they tolerated a
much warmer temperature before chain pulling was
initiated.

Rate of Ingestion

Another version of a free-feeding environment is

one in which the meal consists of discrete units—for
example, nuts, berries, or pellets. The laboratory ver¬
sion is accomplished via the pellet dispenser in which
the rate of delivery of pellets is usually under the
animal's control (Balagura 8c Coscino, 1968; Kissileff,
1970; Teitelbaum 8c Campbell, 1958). Under these
circumstances, meals consist of a number of pellets.
This is similar in some respects to the classic rein¬
forcement paradigm in which a deprived subject is
provided small units of the appropriate commodity
contingent upon a specified response sequence. The
size of the unit (weight, volume, concentration, access
time) is fixed. The rate at which the item is delivered
to the subject is dependent upon rate or pattern of
responding on some schedule (e.g., ratio) and is inde¬
pendent of response rate on others (e.g., interval). In
our major departure from this traditional procedure,
the animals live in the experimental space and obtain
all their food or water over the 24-hr period by satis¬
fying the schedule requirements. On this 24-hr
regimen the animal can initiate the required response
sequence at any time and by successive repetitions of
this sequence can control the amount consumed. As
stated before, a meal consists of a series of discrete
pellets, each obtained on an FR schedule. The end of
a meal was defined by the passage of 10 min without the
initiation of a ratio run. The rate of ingestion is con¬
strained by both the size of the reinforcer and the
schedule on which it is delivered.
Adult rats and growing guinea pigs were tested on
an ascending series of FR schedules with reinforce¬
ment consisting of single 45-mg pellets (Collier,
Hirsch, 8c Hamlin, 1972; Hirsch 8c Collier, 1974b). The
sequence of FR sizes used was 1, 5, 10, 20 followed by
increments of 20 up to FR 240. Each schedule re¬
mained in effect for 10 days. Figure 11 shows the
changes in instrumental performance for two rats
tested under these conditions. Rate of responding in¬
creased sharply as a function of FR size. The changes
in momentary rate were actually larger than the figure
indicates because the rate measure included the post¬
reinforcement pause, which also increased substan¬ Fig.n. Number of responses, rate of responding, and number
of reinforcements for two rats as a function of ratio size. (From
tially with FR size. Although not measured directly, Collier, Hirsch, & Hamlin, 1972.)
42 THE OPERANT REVISITED

the increase in the postreinforcement pause was vis¬

ually evident in the cumulative records (see Collier,
NUMBER OF MEALS

Hirsch, Sc Hamlin, 1972). Response output, which

stabilized between 60,000 and 70,000 responses per day
at FR 160, was a monotonic increasing function of FR
size. At asymptote, animals lever-pressed for almost
14 hr per day. Despite the large increases in response
output and rate of responding, the daily number of
reinforcements was a decreasing function of FR size.
Figure 12 shows concomitant changes in the param¬
eters of feeding behavior under the same conditions.
At low ratios, meal frequency was somewhat higher
than that normally observed when powdered chow is
fed. Both animals showed a gradual reduction in
MEAL SIZE (g)

feeding frequency from FR 1 to FR 160. That is, the

increasing number of responses required to obtain a
pellet extended the time required to obtain a fixed
number of pellets but did not affect the number of
pellets consumed per meal. The increase in meal size
may be an artifact of meal duration, in that successive
meals necessarily began to overlap since so much time
was spent obtaining meals as the length of the meals
increased. Meal duration was an increasing function
UEAL DUIRATION (rnirtuUa}

of the ratio size, which reached asymptote at 60 min at

FR 160. Meal size showed a small increase to FR 160
and then decreased. The function relating the length
of the intermeal interval to FR size was less regular
but tended to increase up to FR 160 and then to de
crease. The changes in the parameters of free feeding
were small and in many ways were secondary relative
to changes in the level and rate of responding in con¬
serving the animal's rate of ingestion, pattern of feed¬
ing, and level of intake, Figure IS shows that food in¬
take and body weight were maintained at a constant
INTERMEAL l»TTERMU_Onimu«-**>

level until FR 80 under these conditions. The control

animals were maintained in the same housing as the
experimental group, but two had noncontingent ac¬
cess to a dish of pellets and four were fed powdered
Purina Chow.
Growing guinea pigs tested under very similar con¬
ditions showed the same general pattern of feeding
and changes in instrumental performance (Hirsch &
Collier, 1974). However, performance tended to
deteriorate at somewhat lower ratio requirements.
When small amounts of water rather than food
were used as the reinforcer in the paradigm requiring
animals to complete an FR requirement for a fixed-
size reinforcer, rats (Marwine, 1974) and guinea pigs
(Hirsch 8c Collier, 1974b) showed a similar profile of
Tig. 12. Number of meals, meal size, meal duration and inter¬ adjustment and maintained intake at the control
meal intervals for two rats as a function of ratio size. (From
Collier, Hirsch, & Hamlin, 1972.)
levels until approximately FR 50. Under the latter
condition, the daily number of drinking bouts stayed
relatively constant, and there was a small reduction in
George Collier, Edward Hirsch, and Robin Kanarek 43

Regulation

Regulation of energy balance implies a matching

of caloric intake to caloric expenditure. One common
test of regulation is to present animals with diets of
varying caloric density. Regulation is inferred when
the animal adjusts volume intake in such a fashion
that total caloric intake remains constant across diets.
When the availability of the source of calories is con¬
currently varied such that meal frequency and size
both vary, the test becomes more powerful. Using
both availability and caloric concentrations as vari¬
ables, Kanarek (1974) tested the ability of rats and
cats to maintain body weight by adjusting volume
food intake. Availability was manipulated by the use
of ratio schedule, and the reinforcer consisted of un¬
limited access to a tunnel feeder. In the rat, low- and
high-caloric-density food increased and decreased, re¬
spectively, the frequency of feeding in such a fashion
that caloric intake remained constant. These relations
were obtained at each level of availability and were
additive rather than interactive.
On the other hand, cats faced with the same prob¬
lem did not control total intake appropriately at any
level of availability; rather, they appeared to eat solely
for bulk. Guinea pigs similarly tested alse> tended to
hold bulk intake constant, However, they surpassed
FIXED RATIO
the cats in their ability to vary the efficiency of food
Fig. 13. Body weight and food intake for two rats as a func¬ utilization, and their weight did not fluctuate as
tion of ratio size, (From Collier, Hirsch, & Hamlin, 1972.) widely when their calorie intake varied to the same
degree (Hirsch, unpublished observation! Kanarek,
1974). The above results show that the nutritive prop¬
the size of these bouts at the higher ratios when total erties of the reinforcer are important in determining
intake declined. The temporal association between both the pattern of feeding and the ancillary instru¬
feeding and drinking (Kissileff, 1969) was largely un¬ mental behavior. These species differences have im¬
affected by this constraint on drinking behavior. In portant implications for an analysis of reinforcement
the free-feeding paradigm, when the rate of ingestion which will be considered subsequently.
was constrained there were several noteworthy
features of instrumental performance. Again, it was
observed that nondeprived animals required no shap¬ Self-Selection
ing. Also, nondeprived animals defended their total
intake by meals of large response outputs at very high Faced with a variety of dietary items, some animals
rates of responding, over very long (14-hr) periods of are able to select a nutritionally adequate diet. Rats
time. are well known for this ability (Lat, 1967). An experi¬
mental paradigm that permits study of the strength of
the tendency of an animal to balance its diet has been
developed by Hirsch (unpublished data). In this pro¬
CALORIC REGULATION AND cedure, one component of the diet (e.g., a carbohy¬
CHOICE OF DIETARY ITEMS drate source such as sucrose) is presented freely while
a second component (e.g., a protein source such as
Species differ widely in their ability to control Purina Chow) is presented contingent upon an oper¬
caloric intake and to select a nutritionally adequate ant. The effort requirement can be varied and its
diet. These differences reflect both their ecological effect on the intake of the two items explored. In a
niche and position in the food chain. first experiment, rats pressing for Noyes pellets (ap-
44 THE OPERANT REVISITED

proximately 23% protein) and offered “free” sucrose 1973; Panksepp and Krost, 1975). Thus the under¬
made as many as 50,000 responses per day maintaining lying physiological substrate such as that demon¬
at least a minimal (6%) protein intake at the highest strated by LeMagnen et al. (1973) can be easily
ratio. Similarly, when the protein-containing com¬ disassociated from the pattern of meal taking by
ponent of the diet was free and the rat worked for environmental constraints.
carbohydrate (sucrose), a large number of responses
were expended to maintain the protein/carbohydrate
ratio (Collier, unpublished data). Thus under free- RESPONSE STRENGTH
feeding conditions, some species will expend a large
number of responses to maintain their dietary bal¬ One of the assumptions of Skinner’s original anal¬
ance. ysis was that rate of responding is the measure of the
strength of the response components in the consum-
matory chain,
OTHER ENVIRONMENTAL CONSTRAINTS

Response Rote
In fclis two prsesdmg sections* we have shown that
constraints on two parameters of meals, frequency and One interesting finding of the present studies is the
mt& 6f mgsstioft, l£&d to compensatory changes in of both average and local rates
differential sensitivity

other parameters such that total intake is conserved, of responding to the two experimental paradigms.
Constraints on the distribution of meals have similar With unrestricted access to the food magazine, no in¬
effects. Far £5£ampl£f many animals show diurnal crease in the average rate of responding during a ratio
cycles of feeding, These cycles have been the inspira¬ run was observed in cats or rats, and only a small in¬
tion for the study of a wide variety of biological crease was seen in the guinea pig. This finding con¬
rhythms (Richter? 1027). Most r@cently? fading cycles trasts with the dramatic increase in response in the

haye been hypothesized to reflect changes in the basic more typical "within-meal’5 (i.e., between-pellet) pro¬
metabolic processes involved in the control of food cedure in which each ratio run is followed by a single
intake. For example, TeMagnen and co-workers (Le- pellet or a small drop of water. The differential effects
Magnetij D^vogj (iaiichlli£r£j Loiiis-Sylvesfcrej Tal¬ of FR size on rate of responding can be understood if
lin, 1978) have argued that rats eat in excess of their one considers the utility of rate increases in the two
requirements in the dark and store energy (i.e., arc cases. An increase in the rate of responding when the
lipogenic), whereas in the light they eat less than their duration of access to food is not constrained has no
requirements and consume energy are lipolytic). effect oft food availability or rate of ingestion. The
Levin and Levine (1974) placed rats on a Irgc-lgcding only contingency between rate and the parameters of
regime pressing a leyer for pellets and found that con meal taking is that increases in rate will shorten the
straining the rate of ingestion by introducing a ratio time from the initiation of a ratio run to the delivery
requirement profoundly affected the temporal distri¬ of food. This contingency appears to exert some con¬
bution of free-feeding behavior. Under conditions in trol on the guinea pig which eatg faster when access is
which the ratio requirement was the same in both constrained (c£. Hirsdh 2c Collier, 1974a, 1974b). On
parts of the 1§-hr light-dark cycle^ the rat§ mg^st^d the other hand, in the pellet or drop paradigm when
approximately 70% of their food in the dark phase, only single pellets of food or drops of water are avail¬
the commonly reported value. However, this pattern able, a long FR chain between pellets or drops de¬
of eating could be completely reversed by program^ creases the rate of ingestion. In this instance, increases
ming food to be available on a multiple schedule with in the rate of responding serve to counteract the con¬
continuous reinforcement during the 12-hr light pe¬ straint on rate of ingestion. This contingency exerts
riod alld various FR sizes during the dark. The strong control in the two species tested. Thus we have
larger the FR, the greater the shift of eating to the two different situations. In one, changes in ratio size
light component. It appears that, given a choice, rats do not affect rate of instrumental responding; in the
prefer to obtain food in the most “efficient” manner other, they do. These appear to result from the effect
available, even if this requires altering their customary of the schedule on the rate of ingestion. This differ¬
diurnal patterns. Boice (1972) had previously reported ence might be dramatized if the two paradigms were
similar data for water intake. Finally, increasing the combined in such a fashion that the subject would be
palatability of the diet in the light phase also effected required to complete a ratio to gain access to a meal.
a shift in the diurnal distribution of intake (Panksepp, The meal would consist of pellets delivered on a
George Collier, Edward Hirsch, and Robin Kanarek 45

second ratio requirement which would remain in reinforcement (e.g., volume, concentration, duration)
effect until 10 min of no responding had passed. In and rate of responding (Collier 8c Meyers, 1961; Col¬
this situation, each ratio would be expected to have lier 8c Willis, 1961). Here again we see an important
different effects on the meal parameters. The size of difference between the freely feeding animal and the
the first ratio would influence the frequency of initiat¬ depleted one.
ing meals and the size of the meals and would not
affect the rate of responding between pellets.
Ratio Tolerance
Preliminary results confirmed these expectations.
Thus there appears to be a direct relationship be¬ In the situation in which access to the feeder is un¬
tween the rate of responding and the utility of a restricted until the meal is completed, it can be asked
given rate in the food economy. This suggests that the whether the tolerance of very large ratios is simply a
classical notion of rate as a measure of response matter of the size of the reinforcement taken follow¬
strength only obtains under carefully circumscribed ing the response chain, or whether this tolerance em¬
conditions. bodies other principles. The most attractive conjecture
is that the subject’s termination of the meal (run¬

Magnitude of Reinforcement
ning burst, etc.) is the critical event. That is, consider
a meal to be not simply a collection of reflexes waxing
The rate analysis can be explored further by con¬ and waning in strength, but rather a dynamic unit,
sidering “magnitude of reinforcement.” In the con¬ the initiation and termination of which reflect cen¬
ventional account, increasing the concentration, qual¬ tral processes under the control of the animal’s total
ity, amount, or time of access to reinforcement economy. From this perspective, then, the completion
increases the rate (e.g., Collier, 1962; Collier 8c Siskel, of the meal may control response strength, rather
1959; Collier 8c Willis, 1961; Guttman, 1953) or than the summation of strengths of the individual re¬
reciprocal latency (e.g., Bolles, 1967) of responding. sponse units which make up the meal. It is quite clear,
In the 24-hr procedure in which the rat earns his total for example, that those features of the environment
intake by bar-pressing for pellets and in which there which cause an animal to eat fewer meals also cause
are no constraints on the initiation and termination him to eat longer meals. Total intake is conserved.
of meals, the prediction is exactly the opposite. For Thus at the same body weight, eating the same food
example, an adult rat will eat about 500 45-mg stan¬ item, meal length covaries with frequency of initia¬
dard Noyes pellets per day. A ratio requirement of tion. How does the animal measure the amount con¬
200 bar presses per pellet would require an output of sumed? It is unlikely, particularly when frequency
100,000 presses per day to maintain this intake. has only been partially reduced, that the stomach will
Neither the rat nor the guinea pig defends its intake be emptied or the reserves depleted to such a degree
completely, but each comes close: the rat makes on the that they can act as signals for extending a meal. The
order of 70,000 presses per day at this ratio, the guinea likelihood of immediate feedback in the free-freeding
pig somewhat less. If the pellet size is increased to 90 situation seems low enough to encourage speculation
mg, it would require one-half the responses to main¬ about a guidance system which anticipates rather than
tain intake, and if it were reduced to 22 mg, it would responds to needs (cf. Oatley, 1970).
require double the number of responses. It is not sur¬
prising that this actually occurs. It is surprising, how¬
Unit of Analysis
ever, that the average between-pellet rate of respond¬
ing (including the postreinforcement pause) is higher One last argument that can be made with respect to
for small pellets and lower for large pellets. This is response strength derives from the question of the
contrary to usual magnitude-of-reinforcement predic¬ appropriate unit of analysis in feeding behavior. From
tions. Although the local rate in this situation has not studies in which the “effort” to obtain a meal is
been analyzed in detail, preliminary evidence suggests varied, it is clear the relations between frequency,
that even local rate is higher for the smaller pellet duration, amount, and rate of eating covary in such a
and lower for the larger. This negative relation be¬ fashion that total intake over a given 24-hr period is
tween pellet size and rate appears to be the case only conserved. The implication of these facts is that Skin¬
for nondeprived animals whose total intake is gained ner’s focus on the dynamics of the within-meal reflex
in the experimental situation. When the animal is chain as the way in which to study total intake may
depleted (i.e., greater than 7-10% body weight loss), have been misdirected. He argued that all-or-none
there is a positive relation between the parameters of events such as meals were most appropriately analyzed
46 THE OPERANT REVISITED

in terms of the within-meal strength of the reflex ates and of how they are reflected in general problems
chain involved in ingestion. The frequency of occur¬ of behavior. One approach to these problems is taken
rence and size of meals should be predictable from a by the ecologists. For example, quoting from Schoener
knowledge of the “strength” of the components. How¬ (1971, p. 369):
ever, it would appear from the present data that total
intake over the feeding cycle is the important bound¬
Natural history is replete with observations on
ary condition. Feeding behavior is path-independent
feeding, yet only recently have investigators be¬
in the sense that the final state, total intake, does not
gun to treat feeding as a device whose perfor¬
depend on the particular responses and their rate of mance—as measured by net energy yield /feeding
occurrence within a meal. Thus one would not expect time or some other units assumed commensurate
correlations among the more typical measures of re¬ with fitness—may be maximized by natural selec¬
sponse strength such as rate, extinction, or ratio toler¬ tion. . . . The primary task of a theory of feed¬
ance except under particular circumstances. The ing strategies is to specify for a given animal that
path-independence argument is similar to Skinner’s complex of behavior and morphology best suited
(1935) argument for the generic nature of the reflex in to gather food energy in a given environment.

which the class is defined by its effect rather than by

the order and topography of its members. If we ac¬ Schoener views the problem as one of optimization
cept the path-independence argument, we must con¬ which can be further “trisected” into tasks of: (1)
cur with Richter’s (1927) original conjecture that the choosing a currency, (2) choosing the appropriate cost-
meal is the appropriate unit of analysis of feeding benefit function, and (3) solving for the optimum. For
behavior. our present discussion, the most important idea in this
view is that an animal does not approach the problem
of feeding naively; rather, he brings a strategy to the
FOOD ECONOMY problem—a strategy which psychologists typically do
not see because of the restrictions of their experi¬
Psychologists have long paid lip service to the mental situations. The usual laboratory setting is
theory of evolution, but they have seldom paid atten¬ specifically arranged to minimize or prevent the dis¬
tion to its consequences for psychological theory ex¬ play of any pattern of behavior other than that which
cept in the most global terms. Most have accepted the the experimenter is measuring.
notion that the morphology of a species reflects its In an animal’s natural environment, both the
adaptation to a particular ecological niche, but until initiation and termination of feeding are under the
recently few have accepted the notion that particular control of the animal. “When/7 “how often/7 and “the
patterns of behavior similarly represent §uch adapta¬ size and type of item” are the basic parameters of any
tions (Barash, 1974). feeding strategy which have to be adjusted to the
Historically, the most important variables in the density (i.e., probability of encounter) of the item, its
study of feeding related to depletion and repletion of caloric and nutritional content, and energy or time
some nutritional item. One had only to find the sub¬ cost of obtaining the item. The basic problem for the
strate which the hypothalamus was monitoring and animal is to partition his time and energy between
one could predict the onset and offset of feeding be¬ the many different activities which insure his reproduc¬
havior from knowing its current state. Aside from tive success. An animal which did not consider the
slight differences In equipment, all species would be¬ density, cost, and character of the item to be procured
have similarly—that is, obey the same laws. Behavior but waited to initiate feeding when it was “hungry”
successful in meeting nutritional needs Was thereby m the physiological sense would simply be unlikely to
strengthened; unsuccessful behavior was thereby weak¬ survive in any but the most permissive environment.
ened. It seems more likely that animals must, in some sense,
Our suspicion that animals were economists as well feed in anticipation of, rather than in response to,
as physiologists led to the research discussed in this their needs. Obvious examples o£ such anticipating
chapter. It is obvious that feeding patterns (i.e., item behavior are premigration hyperphagia (Odum, 1960)
choice and the frequency, duration, rate, and distribu¬ and hibernation (Mrosovsky, 1971). To investigate
tion of meals) differ widely from species to species. Of these speculations, it is necessary to introduce ecolog¬
greater interest and import, however, are the ques¬ ical variables into the laboratory—that is, to construct
tions of how these different patterns relate to the analogues of niches in laboratory situations. We view
ecological niche in which the animal naturally oper¬ the present research as such an attempt.
George Collier, Edward Hirsch, and Robin Kanarek 47

Some insight into the role of these biological vari¬ straint on meal frequency can be viewed as a labora¬
ables can be gained by examining a class of phenom¬ tory analogue of a niche in which food varies in
ena not typically considered by psychologists. From availability. The usual situation of the carnivore
the work of ecologists, it is clear that there are many exemplifies this niche (Estes, 1967a, 1967b; Kruuk,
environmentally defined events which control feeding. 1972; Schaller, 1967, 1972). The most typical circum¬
If one examines the feeding pyramid, there is a chain stance is one of low availability of prey. Carnivores
of exploitation of resources. Animals can be grouped typically, but not always, have a pattern of infrequent
into three major categories: herbivores, carnivores, large meals. Since it appears, however, that the carniv¬
and omnivores. Herbivores harvest the readily avail¬ orous mode of feeding is in the repertoire of many
able plant energy, but they pay for this in terms of species (Collier, Hirsch, 8c Hamlin, 1972; Hirsch 8c
low nutritional quality and low caloric density of Collier, 1974a, 1974b; Kanarek, in press; Westoby,
their food. Their foods require intensive mechanical 1974), an important generalization might be made re¬
degradation (e.g., rumination) and chemical conver¬ garding the effect of food availability on ingestive
sion (fermentation) before digestion to produce a behavior.
nutritionally adequate diet. Because the varied in-
gestants of the generalized herbivore (Westoby, 1974)
Law of Availability
are modified by fermenting into a nutritionally ade¬
quate diet, it is unnecessary for him, except in the case As commodities (e.g., food, exercise, heat) become
of certain minerals, to eat a highly selected diet. At less readily obtainable, the frequency of initiating be¬
the other extreme, the carnivores exploit a niche in havior which procures them decreases, but tbe amount
which the food is of the highest quality and caloric taken per occasion increases in such a fashion that the
density but is not readily available. The carnivore’s total amount consumed is conserved. Thus the law of
problem is one of procurement. Bulk intake alone availability appears to relate to the animal’s efficiency
can accurately regulate both calories and nutritional of allocation of resources. For example, as food be¬
quality. Finally, omnivores, by being opportunists, can comes less readily availably the amount of time,
occupy a wide variety of habitats. This poses quite effort, and/or energy expended to obtain it increases.
different problems for these animals. The variability As a result, it becomes more efficient to expend this
in both caloric density and nutritional quality of the amount of time, effort, or energy less often and to take
omnivore diet requires the omnivore to adjust his in¬ l&i'gOr amount's on any given occasion. On the other

take to his needs by choice of item and amount con¬ band, when the commodity is readily available* fre¬
sumed on a day-to-day basis (Schoener, 1971; Westoby, quency of initiation increases; This suggests that there
1974). is a “tradeoff’7 between the cost of procurement and
Ilie extensive use of the rat in the study of feed¬ the cost of use, the cost of use being related, for
ing and nutrition has bhndod many researchers to this example, to such variables as thg cogt of ingcgtion or

variety of feeding patterns and strategies. As a result, absorption and thdr effect on the efficiency of utiliza¬
the physiological models on feeding based on the be¬ tion. The feeding behavior of large carnivores, such
havior of the rat are limited to the omnivore model as the lion, provides a elassie example. These animals
and a very restricted experimental procedure* must expend considerable effort and undergo high
In addition to considerations of the environmental risks of Injury in procuring their usual game. As a re¬
niche, it is clear that social, seasonal, and climatic sult they live on a “fe&gt-or-f&mms” regime (Schalle^
factors have important impacts on feeding schedules 1972) which fluctuates with the density and size of prey.
(Westoby, 1974). Feeding strategies are intensively Large and/or scarce prey lead to infrequent large
conditioned by environmental considerations (cf. meals, whereas small and/or numerous prey lead to
Schoener, 1971). Operant psychologists, by their im¬ frequent small meals (Schaller, 1972). It would seem
plicit acceptance of the rat feeding model, have failed that the processing cost of large infrequent meals is
to consider such ecological variables in their analyses higher than the cost of small frequent meals (cf.
of behavior. Attention must be paid to the structure Morrison, 1968), such that animals revert to small fre¬
of the environment and its interaction with response quent meals when possible.
patterns. These factors may specify the units of be¬ The law of availability is parallel to Schoener’s
havior upon which reinforcement operates. (1971) principle that the ratio of energy yield/time
Considering only the variables arising from an expended is maximized. Both principles raise, again,
animal’s niche, we can make a very preliminary anal¬ the difficult question of the appropriate dimensions
ysis of their role in feeding patterns. The FR con¬ for their terms. For example, what is the measure of
48 THE OPERANT REVISITED

the cost of “search time”? Is it the time, or the energy a member of the file came slightly closer than
expended per unit of time, or the amount of other the rest. But still they waited, until a zebra
behavior excluded during the search, or all of these? spotted them and snorted an alarm.
Similarly, what are the dimensions of availability (cf.
Westoby, 1974)? Are they dispersion, density, diffi¬ Schaller did not label this as a contingency differential
culty, effort required, or danger? reinforcement, at low rates of responding (DRL), but
In any case, it is clear that this law cannot be ob¬ it is difficult not to. If the lioness were to move too
served in the usual experimental situation, since both soon or be seen, the entire behavior sequence would
initiation and termination of meals are constrained. have to be reinitiated. This aspect of carnivore be¬
In fact, the usual situation in which a deprived animal havior could be simulated in the laboratory by im¬
works for reinforcers of fixed size on a schedule for a posing a DRL contingency and allowing the animal
fixed session length amounts to a single meal. What is to control the size of the meal that became available
being studied is the effect of various experimental when the DRL contingency was satisfied. The DRL
variables on the course of a single meal. This is Skin¬ contingency would functionally reduce itself to con¬
ner's original paradigm. The failure of this paradigm tinuous reinforcement (CRF) if the interval require¬
to generate the laws which govern an animal’s usual ment did not exceed the animals typical intermeal
pattern of feeding has led to a questioning of the interval length. For the rat, this time requirement
&§§umptid>iis oil which it is based. In the more 'Nat¬ would be approximately 90-120 min. Based on DRL
ural” situation* irrespective of the scarcity or abun¬ performance in short test sessions, one would be
dance of resources and the circumstances of their forced to predict that this contingency exceeds the
availabili£y; the animal initiates and terminates meals rat’s capacity for temporal discrimination. However,
and controls the amount of the commodity consumed. behavior of this type is observed routinely on the
It is the circumstances of availability which appear to part of carnivores that rely on stealth and short
ll£ criicial. bursts of speed to capture prey. Such an experiment
(DRL 2 hr or more) has not yet been conducted, but

we would predict this type of performance to be in

Schedules the repertoire of some animals. If so, increases in DRL
length should lead to compensatory increases in meal
At the present time a set of unrelated principles
size. Thus an analysis of the function of schedules in an
and laws specifies what is known about schedules of
animal’s economics may provide some order to sched¬
reinforcement, Attempts to organize these principles
ule effects. These accounts may not be unique, as the
into a coherent whole and reduce them to the deduc¬
two different effects of FR in the meal and pellet
tive consequences of a few axioms have mainly been
paradigms reported above show, and may be best
directed toward probabilistic relations defining rein¬
understood from the perspective of the niche in which
forcement density (Schoenfeld, 1970). However, sched¬
the schedules are employed. Thus two schedules iden¬
ules of reinforcement may also have biological sig¬
tical in terms of temporal and numerical structure
nificance, and examination of schedules in light of may be functionally different depending upon the
biological variables may yield the organization investi¬
environment in which they occur.
gators have been seeking. For example, one aspect of
the carnivore's feeding which a schedule in the labora¬
tory may possibly simulate is Characteristic Dietary Selection and Regulation
of some hunting patterns are long periods of waiting,
Niche variables can also be used to predict “nutri¬
Schaller (1972) documented this point in a description
tive” behavior. As a result of their particular diet and
of hunting lionesses:
their mechanics and chemistry of digestion, herbivores
must consume large quantities of food at relatively
What impressed me most was the patience and low rates. Thus one would predict not only a high
incredible fussiness they displayed. On a num¬
meal frequency, as previously pointed out, but also a
ber of occasions I have sat for more than an hour
relative insensitivity to the nutritional quality or
watching a lioness or a young lion waiting for
caloric density of their food. Studies using guinea
a herd of gnus or zebras, already close, to come
pigs, a monogastric herbivore, have supported these
closer. Once, two lionesses let a file of gnus and
zebras trek within fifty yards of where they conjectures (cf. Hirsch, 1973). Studies on the ability

crouched, in plain sight but unnoticed. Several of herbivores to self-select a balanced diet are not
times one or the other tensed for a spring when known to the authors.
George Collier, Edward Hirsch, and Robin Kanarek 49

Cats likewise appear to be bulk eaters and rela¬ within the niche. It is obvious that for many species
tively insensitive to the caloric content of the diet these strategies are in whole or in part constructed
(Hirsch, unpublished data; Kanarek, 1974). Their and/or modified by interaction with the environ¬
ability to select a nutritionally adequate diet also re¬ ment.
mains unexplored. Like their wild counterparts, Returning to the problem of the unit of analysis
domestic cats can go for long periods without food and the question of the units upon which the conse¬
and then consume an amount sufficient to maintain quences of behavior act, let us examine the question
growth or adult body weight. Rats, on the other hand, of reinforcement. The classic view is that complex be¬
are well known for their ability to regulate caloric havior is assembled from reflexes and that conse¬
intake and select an adequate diet. quences operate directly on the individual reflex
Thus it seems that exogenous variables play a units. Another possibility, however, is that behavior is
dominant role both in feeding and in determining the preassembled into dynamic units and that the changes
effect of schedules on response patterns. Further, it induced by the outcome of a behavior sequence affect
appears possible to translate the parameters of feeding not only the occurrence of the larger unit, but also the
in the wild into operant laboratory procedures. interrelations of members within the unit. For ex¬
ample, hyenas hunt in packs. When the available
game hunted is small and the hunt results in a few
CONCLUSIONS well-fed and several frustrated and quarreling mem¬
bers of the pack, the pack will break into smaller
At the outset of this chapter, we considered two units suitable to the game size. If the available game
questions raised by Skinner’s original formulation of again becomes numerous, the pack will reassemble
an analysis of feeding behavior. The first concerned (Kruuk, 1972). This example illustrates how the con¬
the definition of a unit of analysis: Is the reflex, an sequences, insufficient food in a single meal, can
arbitrary unit which acquires its behavioral meaning affect a complex social structure. We would conjecture
by its appearance in “simple” laws, the most appro¬ that a common principle underlies feeding patterns of
priate unit with which to study feeding behavior, or individual animals as well. That is, the feeding strat¬
would larger units such as the meal or the feeding egies of all animals are matched to the niches in
cycle lead to a more general principle which might be which they operate. Further, these feeding strategies
translatable within a natural context? Data presented are adjusted as a unit when the parameters of the
in this chapter showed that, using the meal as the niche change in such a fashion that the ratio of cal¬
basic unit of analysis, such laws can be stated, and sug¬ ories gained per meal to the time and/or effort pro¬
gested exploration of a different class of variables curing the meal is maximized. Where do learning and
which reflect the structure of the animal’s environ¬ motivation enter into this process? We think that
ment. ‘learning’’ in this situation is the process by which the
The second question concerned the model of moti¬ animal modifies the parameters of his habitat, and
vation (reinforcement) which would be most useful in motivation is basically the maximization function. We
studying feeding behavior. The classic model of feed¬ have shown that animals behave in such a fashion as
ing is one in which behavior is generated, shaped, and to maximize their ration and to reduce the effects of
maintained in response to physiological needs (defi¬ constraints placed on their normal feeding patterns.
cits). This model has led to the search for underlying We assume that modifications of feeding patterns
physiological perturbations as the occasion for all be¬ which increase the probability of finding suitable food
havior, whether it be to meet other environmental de¬ in a given habitat will also occur, thus improving the
mands (e.g., temperature regulation) or social behavior niche. Consider, for example, the problem of a sit-and-
(e.g., courtship or maternal care). Recent descriptions wait predator whose food consists of two sizes of prey.
by ethologists have emphasized the elicitation of such When large prey predominates, the optimal strategy
behavior by particular concatenations of events in the is to concentrate on the large prey; but as the ratio
environments rather than by the push of physiological of large to small prey decreases, the frequency of at¬
needs. In the present analysis, we have suggested that, tack on small prey should increase (Schoener, 1971).
except in cases of emergency, the ethological model A second strategy, different from optimization, also is
which describes social behavior is also appropriate for possible. The predator can move to a part of his en¬
feeding behavior. That is, animals possess a repertoire vironment in which the ratio of large to small prey
of feeding strategies which are appropriate to the is more favorable. Thus the animal can either maxi¬
niche they occupy and which vary with varying habitats mize within a given habitat or he can improve the
50 THE OPERANT REVISITED

habitat by modifying its parameters. It is the latter Collier, G., & Levitsky, D. Defense of water balance in

skill which is so characteristic of the human animal. rats: Behavioral and physiological responses to depletion.
Journal of Comparative and Physiological Psychology,
It is our conjecture that the essence of reinforcement
1967,57, 59-67.
lies in “controlling” the environment, where control is Collier, G., 8c Myers, L. The loci of reinforcement. Journal
evaluated in terms of its effect on the total economy of Experimental Psychology, 1961, 61, 57-66.
of the animal. Collier, G., 8c Siskeu, M., Jr. Performance as a joint func¬
Although the present theoretical speculations are tion of amount of reinforcement and inter-reinforcement
interval. Journal of Experimental Psychology, 1959 57,
crude; we think that a fruitful research strategy is one
115-120.
which attempts to study variables derived from eco¬ Collier, G., 8c Willis, F. N. Deprivation and reinforcement.
logical descriptions in a controlled laboratory situa¬ Journal of Experimental Psychology, 1961, 62, 377-384.
tion using operant analogues. This approach may DeRuiter, L. Feeding behavior of vertebrates in the natural
environment. In Code & Heidel (Eds.), Handbook of
yield a more generalized understanding of the mecha¬
physiology (Vol. 1). Washington, D.C.! American Physio¬
nisms underlying learning and motivation than other¬
logical Society, 1967.
wise has previously been possible. Duncan, I. H. H., Duncan, A. R., Hughes, B. O., & Wood-
Gush, D. G. M. The patterns of food intake in female
brown leghorn fowls as recorded in a Skinner box.
RiFiRih^Si Animal Behavior, 1970, IS, 245-255.
Ehrenfreund, D. The relationship between weight loss
during deprivation and food consumption. Journal of
ARAIft, & WflifiHT, B. B£h§vIoral thermoregulation: Task
Comparative and Physiological Psychology, 1959, 52,
difficulty as a measure of motivational strength. Paper 123 124*
presented at the 14th Annual Meeting of the Fsy- £§te§. R, B« Predators and scavengers, I, Natural History,
chonomic Society, 3t. Louis, 1973. 1967, 76, 20 29 (a)
Adolfii, E. F. Urges to cat and drink in rats. American Estes, R. D. Predators and scavengers, II. Natural History,
Jmmml of Physiology, 1017, 151) 110— 1967. 76, 38-47. (b)
?AK™j A: A periodic feeding behavior in th§ albino Faber, B. The effects of several environmental factors on
rat. Journal of Comparative and Physiological Psy-
feeding behavior in the American cockroach, Periplaneta
1953. 422-426. americanci (L). Unpublished dissertation, Rutgers Uni¬
balacura, 3.. & Coscina, D. v. Periodicity of food intake in versity* 1975,
the rat as measured by an operant response. Physiology Balk, J. L. The nature and determinants of adjunctive be¬
and Behavior) 1968* h 611-613. havior Physiology and Behavior, 1971, 6, 577-588.
Barashj B. P, The evolvdon tff marmot societies; A general Fearing, F. Keflex action: A study in the history of physio¬
theory. Science f 1074, 18hf 4IS—421.
logical psychology. Baltimore; Williams & Wilkins, 1930.
Bell R H. V. A gracing ecosystem in the Serengeti. Ferster* G. B.* 8c Skinner* B. F. Schedules of reinforcement.
Scientific American, 1971, 224, 80-93. Englewoo d Cliff?, Nj,: Prentice-Hall, Inc,, 1957*
Boice, B. Some behavioral tests of domestication in the IiNDLev, J. An experimental outline for building and
Norway rat, Behaviour, 1072, 12, 108-231. exploring multioperant behavior repertoires. Journal of
Bolles, R. C. Theory of motivation. New York: Harper 2c
the Experimental Analysis of Behavior, 1902, JJ; 113-100.
ROW, 1967.
Findley, J. D., & Brady, J. Y. Facilitation of large ratio
Booth, D. A. Caloric compensation in rats with continuous performances by use of conditioned reinforcement.
or intermittent access to food. Physiology and Behavior, Journal of the Experimental Analysis of Behavior, 1963,
1078* §91-899: St 125-129.
Brody, S. Bioenergehcs and growth. New York: Reinholdf
Fitzsimons, J. T. Normal drinking in rats. Journal of
194B.
Physiology (London), 1957. 158, 39P.
Oaimnon, W. B. The wisdom of the body. New York:
Fitzsimons, J. T. Effect of nephrectomy on the additivity of
Norton, 1932.
Collier, G. Consummatory and instrumental responding certain stimuli of drinking in the rat, Journal of Com¬
as functions of deprivation. Journal of Experimental parative and Physiologica l Psychology j 1969, 68, 308-314.
Fitzsimons, J. T. The physiology of thirst: Extraneural
Psychology, 1902, 04, 410-414.
aspects of the mechanisms of drinking. In E. Stellar &
Gollier, G. Body weight loss as a measure of motivation in
hunger and thirst, Annuls of the New York Academy of J. M. Sprague (Eds.), Progress in physiological psychology
Science, 1969, 257, 694-609. (Yol. 4). New York: Academic Press, 1971.
Collier, G., & Hirsch, E. Reinforcing properties of spon¬ Fitzsimons, J. T., 8c LeMagnen, J. Eating as a regulatory
taneous activity in the rat. Journal of Comparative and control of drinking in the rat. Journal of Comparative
Physiological Psychology, 1971, 7, 155-160. and Physiological Psychology, 1969, 67, 273—283.
Collier, G., Hirsch, E., 8c Hamlin, P. The ecological de¬ Guthrie, E. R. The psychology of learning. New York:
terminants of reinforcement in the rat. Physiology and Harper, 1935.
Behavior, 1972, 9, 705-716. Guttman, N. Operant conditioning extinction, and periodic
Collier, G., 8c Knarr, F. Defense of water balance in the reinforcement in relation to concentration of sucrose
rat. Journal of Comparative and Physiological Psy¬ used as a reinforcing agent. Journal of Experimental
chology, 1966, 61, 5-10. Psychology, 1953, 46, 213-224.
George Collier, Edward Hirsch, and Robin Kanarek 51

Herrnstein, R. J. On the law of effect. Journal of the Ex¬ LeMagnen, J., 8c Tallon, S. La Periodicite spontanee de la
,
perimental Analysis of Behavior 1970, 13, 243-266. prise d’aliments ad libitum du rat blanc. Journal of
Hirsch, E. Some determinants of intake and patterns of Physiology (Paris), 1966, 58, 323-349.
,
feeding in the guinea pig. Physiology and Behavior 1973, Levin, R., 8c Levine, S. Ecological determinants of circadian
,
11 687-704. rhythms in rats. Paper presented at the meeting of the
Hirsch, E., 8c Collier, G. The ecological determinants of Eastern Psychological Association, Philadelphia, 1974.
reinforcement in the guinea pig. Physiology and Be¬ Levitsky, D. Feeding patterns of rats in response to fasts
havior, 1974, 12, 239-249. (a) and changes in environmental conditions. Physiology
Hirsch, E., 8c Collier, G. Effort as determinant of intake and Behavior, 1970, 5, 291-300.
and patterns of drinking in the guinea pig. Physiology Levitsky, D. Feeding conditions and intermeal relation¬
and Behavior, 1974, 12, 647-655. (b) ships. Physiology and Behavior, 1974, 12, 779-787.
Hoebel, B. Feeding: Neural control of intake. Annual Re¬ Levitsky, D., 8c Collier, G. Effects of diet and deprivation
view of Psychology, 1971, 33, 533-568. on meal eating behavior in rats. Physiology and Be¬
Hull, C. L. Knowledge and purpose as habit mechanisms. havior, 1968, 5, 291-300.
Psychological Review, 1930, 37, 511-525. MacFarlane, W. V., 8c Howard, B. Comparative water and
Hull, C. L. Mind, mechanism, and adaptive behavior. energy economy of wild and domestic mammals. Sym-
,
Psychological Review 1937, 44, 1-32. posium of the Zoological Society of London, 1972, 31,
Hull, C. L. Principles of behavior. New York: Appleton- 261-296.
Century-Crofts, 1943. MARWINE, A. G. Patterns of intake m the rat as a function
Jaynes, J. The problem of animate motion in the seven¬ of the effort required to obtain water. Unpublished
teenth century. History of Ideas, 1970, 31, 219-234. doctoral dissertation, Rutgers University, 1974.
Kanarek, R. B. The energetics of meal patterns. Unpub¬ Mayer, J. Regulation of energy intake and the body
lished dissertation, Rutgers University, 1974. weight; The glucostatic theory and the lipostatic hypoth¬
Kanarek, R, B, Availability and caloric density o£ the diet esis. Annals of the New York Academy of Science ,
as determinants of meal patterns in cats. Physiology and 1955, 63 ? 15-42.
Behavior, 1975, 15, 611-618. Morrison, S. D. Regulation of wator intake by rats de¬
Kissileff, H. R. Food-associated drinking in the rat. Jour¬ prived of food. Physiology and Behavior, 1968, 5, 75-81.
,
nal of Comparative and Physiological Psychology 1969, Morse, W. H., & Kelleher, R. T. Schedules as funda=
67, 284-300. mental determinants of behavior* In W. Schocnfcld
Kissileff, H. R, Free feeding in normal and “recovered (Ed.}? The theory of reinforcement tichcdukdt EnglcWOOd
lateral” rats monitored by a pellet detecting eatoraeter. Cliffs, N.J.i PrOntice-Hall, Irxo? 1970.
Physiology and Behavior, 1970, 5, 162-173. Mrosovsky, N. Hibernation and the hypothalamus. Englo-
Kissileff, H. R. Nonhomeostatic controls of drinking. In wood Gliffs, N.J.: Prentice-Hall, Inc., 1971.
Epstein et al. (Eds.), Neuropsychology of thirst. Wash¬ Oatley, K. Brain mechanisms and motivation. Nature,
ington, D.G.: Y. H. Winston, 1973. ),
1979, 22 ftft-801.
Kleiber, M. The fire of life. New York: Wiley* 196L Oaxleyj K. Bissgdarion of the circadian drinking pattern
Kohler, W. Gestalt psychology. New York; Horace Live- from eating. Nature} I97L --))
right, 1929. Odum, E. P. Premigratory kyperphagia |n birds, American
Kruuk, H. The spotted hyena : A study of predation and Journal of Clinical NutritionI960* 8, 621—629.
social behavior. Chicago: University of Chicago Press, PANKSEPP, J. Rganalysis of feeding patterns in the rat.
1972. Journal of Comparative and Physiological Psychology ^
Larue, C. G., & LeMagnen, J. The olfactory control of 1973, 82, 78-94.
meal patterns in rats. Physiology and Behavior, 1971, 9 ? PANKSEPP, J., & KRO£T, K. Modification of diurnal feeding
817-821. patterns by palatability. In Physiology and Behavior,
Lat, J. Self-selection of dietary components. In Code & 1975, 15, 673—677.
Heidel (Eds.), Handbook of physiology (Vol. 1). Wash¬ Pavlov, I. P. Conditioned reflexes. Oxford University Press,
ington, D.G.; American Physiological Society, 1967. 1927.
Lawrence, D. H., 8c Mason, W. A. Food intake in the rat Premagk, D. Toward empirical behavior laws, I: Positive
as a function of deprivation intervals and feeding reinforcement. Psychological Review, 1959, 66, 219-233.
rhythms. Journal of Comparative and Physiological Psy¬ Premack, D., 8c Kintsch, W. A. Description of free respond¬
chology, 1955, 48, 267-271. ing in the rat. Learning and Motivation, 1970, 1, 321—
LeMagnen, J. Advances in studies on the physiological con¬ 336.
trol and regulation of food intake. In E. Stellar & J. M. Premack, D., Schaeffer, R. W., 8c Hundt, A. Reinforce¬
Sprague (Eds.), Progress in physiological psychology (Vol. ment for drinking by running: Effect of fixed ratio and
4). New York: Academic Press, 1971. reinforcement time. Journal of the Experimental Analy¬
LeMagnen, J., 8c Devos, M. Metabolic correlates of meal sis of Behavior, 1964, 7, 91-96.
onset in the free food intake of rats. Physiology and Richter, C. P. Animal behavior and internal drives.
Behavior, 1970, 5, 805-814. Quarterly Review of Biology, 1927, 2, 307-343.
LeMagnen, J., Devos, M., Gaudilliere, J. P., Louis-Syl- Robinson, E. A., 8c Adolph, E. F. Pattern of normal water
vestre, J. 8c Tallon, S. Role of a lipostatic mechanism drinking in dogs. American Journal of Physiology, 1943,
in regulation by feeding of energy balance in rats. 139, 39-44.
Journal of Comparative and Physiological Psychology, Schaller, G. B. The deer and the tiger: A study of wild¬
1973,57,1-23. life in India. Chicago: University of Chicago Press, 1967.
52 THE OPERANT REVISITED

Schaller, G. B. The Serengeti lion: A study of predatory- Journal of Comparative and Physiological Psychology,
prey relations. Chicago: University of Chicago Press, 1969, 69, 91-100.
1972. Stellar, E., 8c Hill, J. H. The rat’s rate of drinking as a

Schoener, T. W. Theory of feeding strategies. Annual Re¬ function of water deprivation. Journal of Comparative
view of Ecology and Systematics, 1971, 2, 369-404. and Physiological Psychology, 1952, 45, 96-102.
Schoenfeld, W. N. Theory of reinforcement schedules. Teitelbaum, P., 8c Campbell, B. A. Ingestion patterns in

Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1970. hyperplragic and normal rats. Journal of Comparative
Sidman, M. Tactics of scientific research. New York: Basic and Physiological Psychology, 1958, 51, 135-141.
Books, 1960. Thomas, D. W., 8c Mayer, J. Meal taking and regulation of
Siegel, P. S., 8c Stuckey, H. L. The diurnal course of water food intake by normal and hyperphagic rats. Journal of
and food intake in the normal mature rat. Journal of Comparative and Physiological Psychology, 1968, 66,
Comparative and Physiological Psychology, 1947, 40, 642-653.
365-370. Timberlake, W., 8c Allison, J. Response deprivation: An
Skinner, B. F. On the conditions of elicitation of certain empirical approach to instrumental performance. Psy¬
eating reflexes. Proceedings of National Academy of chological Review, 1974, 81, 146-164.
Science, 1930, 16, 433-438. Westoby, M. An analysis of diet selection by large general¬
Skinner, B. F. The concept of the reflex in the description ist herbivores. American Naturey 1974, 108, 290—304.
of behavior. Journal of General Psychology, 1931, 5, Wiepkema, P. R. Behavior changes in CBA mice as a

427-458. result of one goldthioglucose injection. Behavior, 1968,

Skinner, B. F. Drive and reflex strength. Journal of General 32, 179-210.
Psychology, 1932, 6, 22-37. (a) Wyckoff, L. B., Jr. The role of observing responses in
Skinner, B. F. Drive and reflex strength, II. Journal of discrimination learning, I. Psychological Review, 1952,
General Psychology, 1932, 6, 38-48. (b) 59, 431-442.
Skinner, B. F. The generic nature of the concepts of stim¬ Wynne-Edwards, V. C. Animal dispersion in relation to
ulus and response. Journal of General Psychology, 1935, social behavior. London: Oliver and Boyd, 1962.
72,40-65. Yamamato, W. S., & Brobeck, J. R. (Eds.). Physiological
Skinner, B. F. The behavior of organisms. New York: Ap- controls and regulations. Philadelphia: W. B. Saunders,
pleton-Century-Crofts, 1938. 1965.
Skinner, B. F, The phylogeny and ontogeny of behavior. Young, P. T. Motivation of behavior. London: Wiley, 1936.
Science, 1966, IS?, 1205-1913. Zeicler, H. P., Green, H. L., 8c Lehrer, R. Patterns of
Snowden, C. T. Motivation, regulation, and the control of feeding behavior in the pigeon. Journal of Comparative
meal parameters with oral and liitragastnc feeding. and Physiological Psychology, 1971, 76, 468-477.
 3

Pavlovian Control
of Operant Behavior
an analysis of autoshaping and its implications
for operant conditioning*

Barry Schwartz
and
Elkan Gamzu

INTRODUCTION gaged, did nc>t appeav t<s be elicited all. XAhile it was

ea§y to specify the stimulus which produced salivation

In the beginning, there was the reflex. Payloy em¬ or flexion of a hind limb, it was difficult indeed to
ployed the reflex arc as a model in establishing the find the stimulus which triggered walking, or writing,
laws of classical conditioning. Under a variety of or playing the piano. These behaviors appeared vol¬
conditions, stimuli which had previously had no rela¬ untary and decidedly unelicited. They could not be
tion to particular reflexes could be made to trigger or captured by the reflex arc concept or by the principles
elicit them. This process of classical (or respondent or of Pavlovian conditioning. A new learning principle
Pavlovian) conditioning was taken by Pavlov as the was required to explain their development and con¬
basic constituent of all learning—of all adaptive modi¬ tinued occurrence. This principle was Thorndike's
fication of behavior. law of effect.
From the outset, however, it was clear that Pavlov’s The rest, of course, is history. It remained for Skin¬
phenomenon could not explain all he hoped. The ner to highlight the distinction between the two kinds
problem lay in the reflex arc model itself. Some, in¬ of learning, to emphasize the particular importance of
deed most, behaviors in which complex organisms en- the law of effect in learning—operant or instrumental
* The authors wish to thank Robert Boakes, Len Cook, conditioning—and to develop a brilliant set of
Francis Irwin, Barry Polsky, Daniel Reisberg, Elias Schwam, methods for the study of operant conditioning. The
Myrna Schwartz, Sara Shettleworth, Alan Silberberg, Herb Ter¬
race, Teresa Vollmecke, and Edward Wasserman for providing research which is the fruit of Skinner's pioneering
critical insight at various points in the preparation of this work is prodigious. The study of animal learning cur¬
chapter. We wish to thank Herb Jenkins, J. A. Nevin, Howard
rently, at least in the United States, is heavily focused
Rachlin, Herb Terrace, and David Williams for providing
prints of figures which appeared in their journal articles. Most on operant conditioning. This volume and its prede¬
especially, though, we wish to thank David Williams, whose cessor (Honig, 1966) are testimony to the rather rapid
contribution to our thinking over the years has been profound
and, we hope, obvious in these pages. Preparation of the chapter
development of sophisticated analyses of the principles
was facilitated by NSF grant BMS 73-01403 to the first author. which govern operant behavior.

53
54 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

Meanwhile, the study of Pavlovian conditioning that Pavlovian and operant principles represent good
has also progressed, although more slowly. Most of analytic tools though they are problematic as classi-
Pavlov’s initial findings are reproducible, and the ficatory categories.
domain over which Pavlovian conditioning extends In the first part of the chapter we shall review the
has been enlarged. In addition, there have been occa¬ evidence that Pavlovian procedures are sufficient to
sional new ideas about Pavlovian conditioning which produce and maintain pecking in the pigeon. Most of
have changed our understanding of its basic nature this evidence centers on the study of autoshaping and
(e.g., Kamin, 1969; Rescorla, 1967). automaintenance. In the second part of the chapter
In the midst of this atmosphere of progress, there is we shall show that the Pavlovian control of key peck¬
one problem which has consistently resisted solution: ing revealed by autoshaping studies enters signif¬
the problem of the relation between Pavlovian and icantly into many standard operant conditioning pro¬
operant conditioning. How are the two types of learn¬ cedures. The bulk of this section will be a discussion
ing to be defined? What are the processes which of multiple schedules of reinforcement. It will be a
underly them? Are they mutually exclusive, or can kind of case study, designed to illustrate the pervasive
they operate simultaneously on the same class of be¬ influence of Pavloyian conditioning on the control of
havior? Are there any empirical findings which un» operant behavior. The chapter will not provide an
cfjmvora I ly allow on? to distinguish between them? exhaustive discussion of either autoshaping or multi¬
These questions do not have secure answers. The ple schedule performance. Very thorough recent re¬
history of the problem has seen attempts to reduce one views have been provided by Hearst and Jenkins
type of learning to the other. The characteristics Of the (1974) of the former and by Mackintosh (1974) or the
two types of learning have been exhaustively specified latter.
in an effort to highlight their differences (Kimble,
1061). What emerged from this analysis, however, was
their remarkable similarity. More recent efforts to dis¬ AUTOSHAPING AND AUTOMAINTENANCE
tinguish between the two types of learning (Rescorla
& Solomon, 1907) have been similary frustrated. De¬
Autoshaping! Necessary and
fining tfie two types of learning iii terittg of the pro¬
Sufficient Conditions

cedures used to produce them is the best our current

understanding allows. In i960 Brown and Jenkins reported the following
Consider an example. The prototypic experimental experiment. Deprived, magazine-trained, but other¬
context for the study of operant conditioning cur¬ wise naive pigeons were placed in a dimly illuminated
rently is the study of the pigeon pecking at a response chamber. Once every 60 sec} on the average, a response
key. The key peck is presumably a voluntary behavior, key was illuminated for 8 see and followed by the
governed hy the law of effect. Yet in 1968 Brown and delivery of grain. The surprising result of this pro¬
Jenkins showed that Pavlovian Conditioning proce¬ cedure was that even though food delivery was inde=
dures could also generate and sustain pecking. pendent of the pigeons’ behavior, all 36 subjects
Does this mean that peeking is both voluntary and began pecking at the illuminated key after between

reflexive? Does it mean that pecking is sometimes 8 and 119 key-food pairings. Once pecking occurred,
voluntary and sometimes reflexive? The best one can each peck at the illuminated key extinguished the key
do at present is suggest that peeking which is pro¬ light and produced immediate food delivery. Brown
duced by a Pavlovian procedure is r£fl£xlV£ While and Jenkins called this procedure autoshaping (be¬
pocking wlucfi produced by an operant procedure cause it was automatic7 the pigeon shaped itself). As
is voluntary. The experimental procedure defines the §uch, it represented an important technical advance
learning process. Unfortunately, this is demonstrably over the previously used “method of successive ap¬
false. Pecking which is produced by a Pavlovian pro¬ proximations” (e.g., Ferster 8c Skinner, 1957), which
cedure is also governed by the law of effect. It is just was as inexact as it was artful.
these interactions between Pavlovian and operant However, autoshaping has represented consider¬
conditioning variables in the control of key pecking ably more than technological improvement. The
in the pigeon which is the concern of this chapter. phenomenon has raised theoretical issues which go to
Rather than endeavor to classify instances of learning the heart of the experimental analysis of behavior.
on the basis of the two procedures, we shall try to The reason is that key pecking in the pigeon has been
assess the joint influence of each on the occurrence of considered a prototypic operant—an arbitrarily de¬
a single class of behavior. We hope to demonstrate fined class of skeletal behavior which is sensitive to
Barry Schwartz and Elkan Gamzu 55

and controlled by its consequences. Indeed, most of SUMMARY OF RESULTS

our present understanding of the control of behavior NO.S»*/»OFSs MEAN
Ai 4 e'A NUMBER
EMITTING A
PECK WITHIN
TRIAL
OF 1®t
by its consequences has come from the study of key PROCEDURE OF Ss 160 TRIALS PECK RANGE
pecking. Yet the autoshaping paradigm is Pavlovian: KEY
LIGHT
An arbitrary conditional stimulus or CS (key light) 36 36-100*/. 45 6-119

precedes an unconditional stimulus or US (grain). As (FORWARD FAIRING)

in other Pavlovian procedures, the sequence of events
is completely unaffected by the organism’s behavior, a 12 2-17*/. 54 50-57

characteristic which does not fit well with our defini¬ (REVERSE FAIRING)
tion of operants. Moreover, Pavlovian procedures are
usually said to influence reflexive, nonskeletal, non¬
operant activities. Thus autoshaping seems to repre¬ (TRIALS ONLY)

sent Pavlovian conditioning of a prototypic operant

response (Gamzu Sc Williams, 1971; Hearst Sc Jenkins,
r 12 4-337, NOT APPLICABLE

1974; Jenkins, 1973; Jenkins Sc Moore, 1973; Moore, (TRAY ONLY-CONSTANT LIGHT)

1973). How can the same response be both operant

and reflexive? How secure is the distinction between
n 22 21 • 95*/. 47 10*112

operant and Pavlovian conditioning? Is susceptibility (FORWARD FAIRING -3 5EC.TRIAL)

to Pavlovian influence unique to key pecking, or do [white

6 2-33*/. 141 140-142
all operants share this property? Are there other
(FORWARD FAIRING -DARK KEY)
phenomena which demonstrate Pavlovian control of
key pecking which have been overlooked or misinter¬
preted in the past? These and other questions will be
addressed in the sections which follow. The first order
_n
(FORWARD FAIRING-RED KEV)
6 6-1007. 33 14-66

of business, however, is an assessment of whether auto¬

11-98 7* 35 86-133
shaping is unequivocally the product of a Pavlovian,
(FORWARD PAIRING - FIXED TRIAL)
stimulus-reinforcer association. In this section we shall
consider the conditions which are necessary and suffi¬ Fig- I- Schematic representation of the procedures used by
Brown and Jenkins and summary acquisition results. The top
cient for the acquisition of key pecking in autoshap¬
procedure is the standard autoshaping paradigm. (From Brown
ing procedures. & Jenkins, 1968. @ 1968 by the Society for the Experimental
Just as in standard studies of Pavlovian condition¬ Analysis of Behavior, Inc.)
ing, a variety of control procedures are required to
ascertain whether an association of CS (key light) and only), 4 of 12 pigeons pecked the key (fourth panel).
US (food) is what produces autoshaping. Many of When the key light was illuminated for 3 rather than
these control procedures were included in Brown and 8 sec, 21 of 22 pigeons pecked the key (fifth panel).
Jenkins’s demonstration of autoshaping. The results When a dark key was paired with food, only 2 of 6
and procedures are shown in Figure 1. Brown and pigeons pecked the key, while when a red key (which
Jenkins reported only the trial on which the first key was otherwise white) was paired with food, all 6
peck occurred. The reason for this is that in all but pigeons pecked the key (sixth and seventh panel).
the last of their procedures, a key peck immediately These control conditions appear to demonstrate that
produced food. Thus once the first peck occurred, the pairing is necessary and sufficient (except when a dark
response-reinforcer relation would exert control over key is the CS) to produce autoshaping.
pecking and increase its likelihood. In most later However, in the province of standard Pavlovian
studies of autoshaping, food delivery has always been conditioning, Rescorla (1967) has persuasively argued
independent of responding. This makes the study of that none of the control procedures employed by
pecks after the first one of interest. The first panel Brown and Jenkins are sufficient to allow an un¬
shows the basic procedure already described. The equivocal conclusion that autoshaping is the result of
second panel shows that when key and food were CS-US relation. What is needed is a procedure in
paired in reverse order (backward conditioning) only which CS and US are neither paired nor unpaired, but
2 of 12 pigeons pecked the key. When the key was independent of one another. This, of course, is the
illuminated without food (CS only—third panel), no truly random control procedure. In such a procedure,
pigeons pecked the key. When the key was continually pairings of CS and US occasionally occur, but the
illuminated and food was presented periodically (US occurrence of the CS provides no information about
56 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

the likelihood of occurrence of the US; i.e., the prob¬ KEY

ability of the US given the CS equals the probability FOOD
A. AUTO-SHAPING
of the US given no CS [P(US/CS) = P(US/CS)]. Peter¬
son (1972), Bilbrey and Winokur (1973), and Wasser- KEY
man, Franklin, and Hearst (1974) have employed this FOOD
+ +
procedure and have shown that it does not result in I /sec. PG
SAMPLE B DIFFERENTIAL
autoshaping. The procedure has been studied most
extensively, however, by Gamzu and Williams (1971,
KEY
1973), and it is to their work we now turn in discuss¬
FOOD
ing the importance of the informativeness of the CS I /sec PG + + + + + + +
SAMPLE
for autoshaping. C. NON-DIFFERENTIAL

KEY
f_i ,_,
Informativeness of the CS FOOD
+ + + + +
I /sec. PG
SAMPLE
D. DIFFERENTIAL- ABSENCE
Contingency vs. Pairing
KEY
Gamzu and Williams (1971, 1973) studied a variant FOOD
of the autoshaping procedure similar to that used in E. NO-REINFORCEMENT

recent studies of Payloyian ayersiye conditioning

Fig. 2. Schematic representation of (a) the basic autoshaping
(Rescorla, 1968). A response key was periodically
paradigm and (b-e) procedures in which food presentation was
illuminated for 8.6-sec trials, with a variable mtertnal determined by sampling a probability generator (PG); (b) The
interval (ITI) with a mean of 30 sec. Once every differential procedure, in which food was presented randomly
in time, but only during illuminated key trials, (c) The non¬
second during the trial a random probability generator
differential procedure, in which food was presented randomly
was sampled, and an output occurred with a probabil in time during trial and intertrial intervals, (d) The differential-
ity of .03 (p = .03). Each output operated the feeder for absence procedure, in which food was presented only in the
intertrial interval, (e) The nonreinforcement procedure, in
4 see. Thus food wa§ delivered onee every ZZ §ee or once
which food was never presented. (From Gamzu & Williams,
every fourth trial, on the average. This differential 1973. © 1973 by the Society for the Experimental Analysis of
procedure is depicted in the second panel of Figure 2. Behavior, Inc.)
It differs from standard autoshaping in two respects:
autoshaped key pecking depended on the information
first, food is not delivered in every trial; second? food
the CS conveyed about the US, rather than a mere
can be delivered at any moment during the trial, not
pairing of the two. These conclusions parallel those
just atthe end. Key pecking was acquired and maim
drawn by Rescorla (1972) about the crucial relation¬
tamed at high rates in all pigeons tested. The pro¬
ships in Pavlovian conditioning.
cedure was then modified §o that the probability gen¬
erator wa§ sampled (and food delivered) at the same
Redundancy
rate during intertrial intervals as during trials; the key
was nondiffcrentially related to food, i.e., food was as Egger and Miller (1962, 1963) suggested that the in¬
likely in its absence as in its presence. This procedure formativeness of a stimulus with respect to reinforce¬
is equivalent to the truly random control procedure ment determines its power as a secondary reinforcer.
(Rescorla, 1967) and is depicted m the third panel of They pointed out that just because a stimulus always
Figure 2. Pigeons exposed to this procedure did not precedes reinforcement does not guarantee that it is
peck the key. The two other control procedures de¬ informative. For example, if a series of pairings al¬
picted in Figure 2 were also run. In one (differential- ways consists only of tone followed by food, then the
absence) food was presented only in the ITI, and in tone is informative. If, however, the tone is always
the other (no-vein forcement) food was never pre¬ preceded by a light, then the tone is now redundant
sented. Neither procedure generated consistent key because the light reliably predicts reinforcement, and
pecking. The birds on these three procedures were that “information” is available before the tone is
then exposed to the differential procedure, and peck¬ sounded. However, if the light is sometimes followed
ing was obtained in all of them. Thus key pecking was by tone and food and sometimes not, then the mere
reliably obtained only when the illuminated key was presentation of the light does not “guarantee” rein¬
differentially associated with food. These data led forcement, while the tone does. Thus, the tone is once
Gamzu and Williams to conclude that the acquisition again informative. Egger and Miller substantiated this
(and indeed, as we shall see later, the maintenance) of argument by showing that the relative efficacy of two
Barry Schwartz and Elkan Gamzu 57

stimuli as secondary reinforcers depended on whether train is the one you want to board. As the waiting
they were informative or redundant in the sense out¬ time (trial duration) gets shorter until the train stops
lined. only on demand, the vigilance required increases. The
Allaway (1971) tested the Egger and Miller concep¬ relevance of this analogy to autoshaping is as follows:
tualization of informativeness in autoshaping. Three we would expect that with ITI held constant, shorter
groups of pigeons were exposed to a basic fixed-trial trials will convey greater information, and will thus
autoshaping procedure in which 6-sec key illumina¬ engender more pecking. Similarly, with trial duration
tions were always followed by access to food. For one constant, longer ITIs will generate more pecking dur¬
group (key-only) no additional manipulations were ing a trial.
involved. For the second group, 2 sec before each In their very first experiments. Brown and Jenkins
illuminated key trial a tone was sounded for 8 sec (1968) compared an 8-sec trial with a 3-sec trial but
after which food was delivered. For this group the did not find much difference in rate of acquisition.
illuminated key was redundant (key-redundant). The Ricci (1973), however, reported that with a constant
third group had the same conditions as the key-redun¬ mean ITI of 4 min, autoshaping was much more
dant group, but in addition an' equal number of 8-sec rapid when the trial duration was 30 sec than when it
tone trials that were associated with neither key nor was 120 sec.
food (tone-irrelevant). Throughout the experiment, An extensive study of trial duration has been made
reinforcement was delivered response-independently. by Baldock (1974). When ITI duration was held con¬
In general, key pecking was far less frequent when the stant, the rate of acquisition of autoshaping was in¬
key was redundant than in either of the other two versely related to trial duration; that is, autoshaping
procedures. Indeed, some naive birds failed to auto¬ was most rapid when the CS was brief (4 sec) and was
shape when the lit key was always preceded by a tone. gradually less rapid as the trial duration increased
Allaway’s data confirm the fact that the important (up to 32 sec).
feature of the CS-US relationship in autoshaping is Terrace, Gibbon, Farrell, and Baldock (1975) in-
the informativeness of CS with respect to the US and
not merely the pairing of the two. Wasserman and his
co-workers have obtained data consistent with Alla¬
way’s with a variety of similar procedures (Wasser-
man, 1972, 1973b, 1974; Wasserman 8c Anderson,
1974; Wasserman & McCracken, 1974).

Trial and ITI Duration Effects

Another aspect of informativeness which is not cap¬

tured by measures of conditional probability is the
importance of relative trial and intertrial interval
(ITI) durations. To understand the informativeness
of these events, consider the following analogy. Imag¬
ine waiting at a train station, through which five
differently numbered trains continually pass. Only one
number will take you to your destination. If the trains
arrive once every minute, you can afford to read your
copy of this volume and only look up occasionally.
However, if the trains come in once every 10 minutes,
you will certainly pay more attention to the train
arrivals (trials). Consider now the difference between
waiting at the terminal and waiting at a through sta¬
tion. At the terminal the trains roll in and wait for,
say, a minute, so that once you notice that a train is
in, you can go over and inspect the number. How¬
ever, at the through station the train only stops for Fig. 3. Median number of reinforcements prior to first trial
20 seconds, so that it is necessary to be constantly on which a peck occurred as a function of the mean intertrial
interval. Trial durations were always 10 sec. (From Terrace et al,
alert in order to determine whether the incoming 1975.)
58 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

vestigated the other side of this coin. They used a signals food (Gardner, 1969). Squier (1969) in a discus¬
constant 10-sec trial and varied the mean ITI from sion of autoshaping in different species of fish, stated
5 to 400 sec. The rate of acquisition was a direct that “the topography of key responses varied, each
monotonic function of the ITI. Some of the data are type closely resembling the consummatory response”
presented in Figure 3. Thus both intuitions about the (p. 178). Smith and Smith (1971) demonstrated
role of trial and ITI durations are confirmed. The autoshaping in the dog, and S. G. Smith (personal
next logical step was to combine them. Baldock communication) remarked that licking the response
(1974) showed that the important dimension in deter¬ key was observed. Rats lick and gnaw at laboratory
mining the rate of acquisition of pecking is the ratio food, and indeed Peterson, Ackil, Frommer, and
of trial to ITI time. Over a wide range of trials (4 to Hearst (1972) reported that rats react in the same way
32 sec) and mean ITIs (8 to 78 sec), acquisition was to a response lever when it is the signal for food in an
always more rapid when the ratio of trial to ITI was autoshaping paradigm. Stiers and Silberberg (1974)
smaller. have made similar observations. The two exceptions
We can summarize briefly what is presently known to this general trend were reported by Sidman and
about the necessary CS-US relationships for auto¬ Fletcher (1968), who autoshaped rhesus monkeys for
shaping to occurj specific pairings are neither neces¬ food7 and Gamzu and Sehwam (1974), who studied
sary nor sufficient. Rather, the CS must provide in¬ squirrel monkeys. Sidman and Fletcher noted that “al¬
formation (broadly construed) about the occurrence though the monkey used its fingers both to press the
of th§ US* Th§ more informative the CS is, the more key and to pick up the pellet, the topography of these
rapid acquisition is.1 two behaviors is quite different” (p. 308). Gamzu and
Sehwam found similar results and reported that some
monkeys eventually made nose-pressing responses.
Types of US and the Relation
They suggested that these response-topography differ¬
S@fwthe US and Response
ences resulted from the fact that the consummatory
The apparent similarity between the conditions behavior of the monkey is quite variable in compar¬
necessary to produce autoshaping and the necessary ison to that of the pigeon. It is possible that in species
conditions for more standard demonstrations of Pav- with varied feeding behaviors, the particular behavior
lovian SOIidlUonmg l§ad§ to look for further one observes will be governed primarily by the manip-
evidence that autoshaping is an instance of Pavloyian ulandum, For example, Moore (1973) reported that
conditioning. One obvious thing to investigate is the monkeys grasped and bit a protruding key “as if it
relation between the US or reinforcer and the con¬ were food” (p. 187).
ditioned response. Pavlovian conditioned responses
typically are some component of the unconditioned
Food Versus Water
response to the US. In this section we review evidence
that the same is true of autoshaping. Jenkins and Moore (1973), Moore (1973), Morri¬
Iii the most common ailtosliaping Situation^ With son (1974), and Woodruff (1974) have all autoshaped
subjects and food as the US, the condi¬ pigeons using water rather than food as the reinforcer.
tioned response is key pecking, Pecking is o£ course All found that the autoshaped response resembled

the major component of the pigeon’s feeding reper¬ drinking movements, Perhaps the most elegant demon¬
toire (e.g., Craig, 1912). Other species also show this stration of the relationship between US and autoshaped
similarity between the response elicited by the rein- response can be found in Jenkins and Moore’s (1973)
forcer and th£ autoshap^d r£§pon§£. For cvample, study. First they made high-speed films and videotapes
b<?bwhxte quail peck to feed and peck a light that of the unconditional behaviors to food and water. Us¬
ing these as prototypes^ judges were then asked to eval¬
1 Throughout this discussion we hare finessed the problem uate the response form of birds autoshaped for food or
that the concept “informativeness" cannot at present be defined
water. The judges who were presented with videotapes
to include all the senses In which we have used it here. There
have been a few attempts to provide such a definition (Bloom¬ of the learned response only (but not with the whole
field, 1972: Gibbon, Berryman, & Thompson, 1974; Rescorla, sequence, which would have allowed them to see the
1972), but none have been complete, Most recently, Gibbon et al.
reinforcer itself) correctly identified the approach and
(1974) have proposed a metric for evaluating contingencies in
classical and instrumental conditioning. Their model can in¬ contact movements as grain-related or water-related
corporate the conditional probability studies and trial and ITI on 87% of the trials. The response was either like
duration studies described above. It cannot account for the
grain pecking or like drinking. Sample photographs of
redundancy effects. It is, however, the most thorough and
complete account to date. the two types of key pecking are presented in Figure 4.
Barry Schwartz and Elkan Gamzu 59

for food was appropriately like a grain peck. Because

the stimuli were presented randomly and equally
often and because both deprivation states were in¬
duced, these data represent the clearest evidence for
the dependence of the autoshaped response on the
actual consummatory response.

Other USs

Farris (1967) conditioned the courting behavior of

three male Japanese quail. On four separate occasions
each day a buzzer was sounded for 10 sec at the end of
which a female quail was introduced to the cage and
was left there until copulation occurred or 1 min had
passed. The CS overlapped the presence of the female
by 5 sec. Within as few as 5 pairings, part of the male
display began to occur in the presence of the buzzer.
After 32 pairings all components of the characteristic
male display were reliably elicited by the CS in all
three birds. In a similar experiment, Rackham (1971,
cited in Moore, 1973) exposed pigeons to repeated
pairings of a stimulus light and a sexual reinforcer.
He, too, found that the behavior to the signal (and in
this case it was directed to the visual signal) strongly
resembled the unconditioned response that would be
elicited by the forthcoming reinforcement.
Similar findings have been reported for Pavlovian
conditioning of aggressive display in Siamese fighting
fish (Adler & Hogan, 1963; Murray, 1974; Thompson
& Sturm, 1965). Thompson and Sturm, for example,
paired a red stimulus light with a mirror presentation
Fig. 4. Photographs of pigeons pecking at response keys. Pictures and found gradual acquisition of conditioned be¬
(a) and (b) show the consummatory topographies obtained when
key responses are autoshaped with water (a) and food (b) rein¬ havior that was identical to the unconditioned aggres¬
forcement; (c) and (d) show a food-reinforced instrumental sive display. A study by Rachlin (1969) could also be
response. The spot of light is a discriminative stimulus. Part (e)
interpreted as an instance of autoshaped aggressive
shows the consummatory reaction which often arises even when
lever pressing rather than key pecking is reinforced. When behavior. Rachlin exposed pigeons to an autoshaping¬
presses are reinforced only in the presence of some positive cue, like procedure with shock as the US. A response on
the cue itself may elicit consummatory reactions, as shown in (f).
the key, which was fitted with a transparent hemi¬
(From Moore, 1973.) (photo by Roy DeCarava)
spheric extension, turned off the shock. Rachlin found
that key responding could be conditioned, but that
In their second and third experiments, Jenkins and some birds pecked the key while others struck it with
Moore showed that the determinant of the response their wings. Moore (1973) has suggested that since
form was the reinforcer itself and not the depriva- wing flapping and pecking are both parts of the
tional state. As long as the key predicted food and not pigeon’s normal aggressive behavior pattern, and since
water the response form was foodlike, even if the shock elicits aggression, Rachlin’s data are another in¬
dominant deprivational state was changed to thirst. stance of reinforcer-appropriate conditioned behavior.
Finally, in a most convincing experiment, Jenkins and Wasserman (1973a) recently reported that 3-day-old
Moore exposed food- and water-deprived pigeons to a chicks would peck at a key, the illumination of which
procedure in which two different colored key illu¬ always preceded the illumination of an overhead heat
minations were used, one to signal food and the other lamp. In order to insure the reinforcing quality of
to signal water. In most pigeons the response to the heat, the experiments were conducted in a cold cham¬
stimulus predicting water reinforcement was a drink¬ ber (5-15°C). Appropriate control groups were run,
inglike movement, while the response to the signals and the subjects in those groups seldom pecked the
60 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

key. In the group exposed to the pairings of key and altogether indistinguishable from key pecks auto¬
illumination and heat, seven of eight chicks pecked shaped with water presented in the standard fashion.
the CS within the first 20 trials, although on some Thus it appears that neither the sight of the US nor
trials they contacted the key with a “snuggling” re¬ the occurrence of US-appropriate consummatory be¬
sponse. The unconditioned response to the heat lamp havior is a necessary feature of autoshaping with
was described as a reduction of activity accompanied water.
by an extension of the wings and the emission of
twittering sounds. Other aspects of the response were Summary

less uniform. Both Hearst and Jenkins (1974) and

A wide variety of reinforcers in a number of species
Wasserman (1973a) regard this experiment as showing
have been studied in autoshaping experiments. The
autoshaping of a response that is different in topog¬
overwhelming impression derived from these data is
raphy from the response to the US. Hogan (1974) has
that the autoshaped response usually bears a remark¬
pointed out that peeking and snuggling are part of
able resemblance to the response elicited by the rein¬
the normal heat-seeking repertoire of chicks, however.
forcer. There are two types of exceptions to this state¬
Peterson, Ackil, Frommer, and Hearst (1972) im¬
ment, The first are the counterexamples, particularly
planted electrodes in the lateral hypothalamus of rats,
in the primates (but also Wasserman^ experiment
After it was determined that the sites were positively
with heat as the US in chicks). Moore (1973) feels that
rsm forcing, the rats were exposed to autoshaping.
the u§e of an appropriate stimulus^manipulandum
One illuminated retractable lever (CS+) was inserted
would convert the primate work into positive examples
for 15-gee periods after which a train of rewarding
of response similarity. Gamzu and Schwam (1974)
stimulation was presented, A second lever (CS-) was
and Schwam and Gamzu (1075), on the other hand,
presented equally often but was never paired with
have argued that in primates one ought to expect dis¬
reinforcement. Each rat soon began to approach the
similarities between conditioned and unconditioned
CS4- and “sniff” it, making contact with the lev£r
responses, because the latter are so variable. Indeed,
with its whiskers. The C8~ was generally ignored*
it is difficult in squirrel monkeys to specify what
Peterson et al. reported that the exploratory behavior
skeletal behavior will be elicited by the presentation
in the vicinity of the signal far brain stimulation was
o£ food. The second type of exception is less severe.
quite constant for a given rat, but varied among rats.
Certain reinforcers (e.g., brain stimulation and water
However, “there seemed to be a definite relation be¬
directly in the beak) are not localizable in the cham¬
tween the behaviors directed at the C3+ and those
ber. Yet in both cases the autoshaped behavior is
elicited by the brain stimulation? if an animal sniffed
clearly directed toward the signal. It should be
or displayed certain postural adjustments during US
pointed out that it is only the directed aspect of
presentation, we often noticed fragments of the same
these behaviors that is problematic. In both cases the
general pattern during presentation of the OS+” (p.
autoshaped response is clearly similar to the uncon¬
1011). In experiments with a design similar to the
ditioned behavior.
Jenkins and Moore (1973) experiment Peterson
(1972) conftrmed and extended these findings. Rats
Pissvssisn Qnd Gsnslystem
that were both hungry and had electrode implants
Autoshaping as Pavlovian Conditioning
showed directed responses to a GS+ lever that were
classified either as licking-gnawing or sniffing=explor= Brown and Jenkins (1908) in their discussion of
mg. These were perfectly correlated with the type of the autoshaping phenomenon posed the possibility
r^inforcer arid corresponded to the topography Of the that key pecking might have emerged as a result of
response to the US. Pavlovian conditioning, although it seemed unlikely
Finally, Woodruff (1974) employed an ingenious to them at that time. Since then, autoshaping has
technique to study the importance of localizability of been interpreted as an instance of Pavlovian condi¬
the US. A small hole was made in the upper mandible tioning with varying degrees of reservation (cf.
of pigeons and a chronic cannula implanted. Through Gamzu, 1971; Peterson et al., 1972). Perhaps the most
this cannula small amounts of water were delivered as wholehearted adoption of the Pavlovian conditioning
the USs m an autoshaping procedure. When water account of autoshaping can be found in Moore
was placed directly in the beak, the pigeons drank it, (1973), who brings together a great deal of evidence in
usually without any peckinglike behavior. Nonethe¬ support of this approach. The data that have been
less, key pecking was autoshaped, and, not surpris¬ presented here overwhelmingly suggest that autoshap¬
ingly, the topography of the peck was “drinking”-like ing is Pavlovian. First of all, in all cases of auto-
Barry Schwartz and Elkan Gamzu 61

shaping that have been extensively studied the crucial effect on autoshaping (e.g., Farrell, 1974; Gamzu 8c
variable has always been the signaling relationship Williams, 1971, 1973; Gonzales, 1973, 1974; Schwartz
between the CS and US. Autoshaping occurs if and 8c Williams, 1972a), although it is claimed severely to
only if the CS reliably predicts a period of relatively retard acquisition of classically conditioned responses.
higher density of reinforcement than otherwise ob¬ Indeed, Spence (1966) suggested that this effect of
tains. Control procedures such as the truly random partial reinforcement be used to distinguish between
control, CS-alone, US-alone, and backward-pairing all Pavlovian and operant conditioning. However, Gor-
fail to generate the autoshaped behavior. Indeed, mezano and Moore (1969) summarized the literature
when a CS- is paired with the absence of food, as being equivocal. Grant and Schipper (1952) found
pigeons tend to move away from it (Jenkins 8c Boakes, conditioning to be unimpaired by partial reinforce¬
1973; Wasserman et al., 1974) and one can indepen¬ ment. More representative are findings that Pavlovian
dently demonstrate that such CS-s have the inhib¬ conditioning (of eye blink reflex on the whole) will
itory properties (Wessells, 1973) that are predicted occur at partial reinforcement (even as low as 25%),
from Pavlovian theory. Secondly, the responses that but that the magnitude of the effect is smaller than in
are autoshaped are far from being arbitrary. On the 100% control groups (e.g., Ross, 1959). Another effect
contrary, they tend to be very constrained within a of partial reinforcement is to increase resistance to
given species and are demonstrably similar to the re¬ extinction. This phenomenon is well documented in
sponses that are unconditionally elicited by the par¬ operant learning (cf. Lewis, 1960) but it is not always
ticular reinforcer being used. seen in Pavlovian conditioning in infrahuman sub¬
Given the quality and quantity of the data, it is jects (e.g., Thomas and Wagner, 1964). However, the
surprising that there is still considerable reluctance to original report of this partial reinforcement effect
accept the notion that autoshaping is Pavlovian (cf. (PRE) by Humphreys (1939) was based on classical
Herrnstein 8c Loveland, 1972; Hursh, Navarick 8c Fan- conditioning. Recently Hilton (1969) has shown a
tino, 1974). Other than dogma, there appear to be clear-cut PRE using a conditioned emotional response
three reasons for this reluctance: the directedness of (CER) paradigm which is most commonly interpreted
the response, the absence of a deleterious effect of as resulting from Pavlovian conditioning (cf. Rescorla
partial reinforcement, and a continuing dispute about 8c Solomon, 1967). Indeed, although Hilton provides
the process presumed to underlie Pavlovian condi¬ no details of acquisition, he does indicate that all the
tioning. groups (consistent and partial reinforcement) showed
Among the many aspects of autoshaping that make equally effective complete suppression to the CS. This
it an interesting phenomenon is the directedness of very cursory review should be sufficient to indicate
the response. Indeed, this may be the only way of that the effects of partial reinforcement in Pavlovian
distinguishing autoshaping phenomena from more conditioning are sufficiently equivocal that they cannot
familiar instances of Pavlovian conditioning. Tradi¬ possibly be used to distinguish it from operant con¬
tionally, salivation, galvanic skin response (GSR), ditioning or to define a phenomenon as an instance of
heart rate, and eye blinking have been the behaviors Pavlovian conditioning. Thus it seems illogical to
studied in Pavlovian conditioning experiments. None refute a Pavlovian conditioning approach to auto¬
of these behaviors could be called directed. However, shaping on these grounds.
when other components of behavior are noted, Finally we come to the mechanism presumed to
directed behavior is often observed. For example, underlie Pavlovian conditioning. If there were one or
Pavlov noted that dogs licked an electric bulb that more clear-cut mechanisms that were unequivocally
was a CS for food (Pavlov, 1955); indeed, if the stim¬ acceptable as an explanation of Pavlovian condition¬
ulus was within reach, the dog usually tried to touch ing, then it would indeed be fair to ask that all the
it with its mouth (Pavlov, 1941). Similar findings autoshaping data be encompassed by one or more of
were reported by Zener (1937). Thus directedness of these mechanisms. Unfortunately, this is not the case,
behavior has been observed in Pavlovian conditioning and thus we are left with an important theoretical
experiments but simply was not the focus of the re¬ problem for behavioral psychology.
search. As a result, it has been more or less ignored. Stimulus substitution is the most commonly cited
Autoshaping redresses that wrong and provides a mechanism and is often the only mechanism con¬
vehicle for the study of Pavlovian control of directed sidered (cf. Terrace, 1973). It refers to the view that
skeletal behavior. the CS in a Pavlovian conditioning experiment comes
The reader will have already noted that partial to substitute for the US and generate responses iden¬
reinforcement does not seem to have a deleterious tical to those produced by the US. The lack of well-
62 PAVLOViAN CONTROL OF OPERANT BEHAVIOR

established alternatives is surprising, since it is quite able approach or whether they clearly differ from one
clear that a literal interpretation of stimulus substi¬ another are matters for further exploration. Nonethe¬
tution is inadequate to explain even the most un¬ less, this debate clearly revolves around facts that indi¬
equivocal examples of Pavlovian conditioning. Typ¬ cate that Pavlovian conditioning—whatever its under¬
ically one observes CRs which (a) omit portions of the lying mechanisms may turn out to be—is the major
unconditioned response (UR) and (b) include com¬ learning process in the acquisition of autoshaping.
ponents which are absent from the UR. Because of The next section will indicate that this is also true
the documented similarity of autoshaped and con- in the maintenance of autoshaped behavior, where
summatory responses, stimulus substitution has been operant relations also exert a powerful role—some¬
the suggested mechanism for autoshaping. The short¬ times in opposition to Pavlovian relations.
comings of explanations of autoshaping that a re based
on Pavlovian conditioning are often related to the
Automaintenance: Interaction of
weakness of the stimulus-substitution concept, It
Pavlovian and Operant Contingencies
seems unreasonable to expect conceptualizations of
autoshaping to be more preeise than the theory on The phenomenon of autoshaping seeing like such a
which they are based. straightforward example of Pavlovian conditioning
A more reasonable approach is to use autoshaping that one wonders why it has generated so much inter¬
phenomena ag a tool for better understanding the est and research activity. While it is true that key
mechanisms of Payloyian conditioning. To some ex¬ pecking has not traditionally been viewed as a mem¬
tent this has already happened. For one thing, alterna¬ ber of the class of behaviors which is susceptible to
tives to stimulus substitution have begun to be artic¬ Pavlovian procedures, there has been virtually no
ulated. Hearst and Jenkins (1974) have pointed out systematic investigation of key peck acquisition in the
that Favlov himself probably thought of the CS as a past, and the discovery of a method other than shap¬
surrogate rather than as a substitute for the US. In¬ ing by successive approximation for instituting key
deed, Camzu (1071) preferred the term stimulus .mr- pecking might well have been viewed as just another
rogation as capturing the essence of the idea without fact and a methodological convenience. In relating
the limitations of stimulus substitution. Bindra (1972) autoshaping to the operant conditioning literature,
considered the US an unconditional incentive stim¬ one might reasonably adopt this model: Pavlovian
ulus and the CS a conditional incentive stimulus. conditioning procedures may be used to produce the
This apparently eliminates some of the problems first key peck. This peck is followed by food, and the
posed by the stimulus-substitution concept (cf. Hearst law of effect then takes over. Thus autoshaping and
Jenkins, 1974? Moore, 1973), but still leaves the CS the control of behavior by its consequences reflect
as surrogate for the US. Hearst and Jenking (1974) independent processes. This, indeed, is not unlike the
have coined the term object substitution as capturing initial tentative explanation of autoshaping put forth
Bindra’s approach. The most recent attempt to specify by Brown and Jenkins (1968). However, such an ac¬
the mechanism arises from the work of Woodruff and count is dramatically inadequate.
Williams on autoshaping with water injected directly The experiment which demonstrated that the pro¬
into the beak. Williams (1974) and Woodruff (1974) cesses which underlie autoshaping extend beyond
have referred to this experiment as demonstrating that the acquisition of key pecking, and which is probably
the key in autoshaping is a “learned releaser,” Wil¬ responsible for the enormous research interest in the
liams points out that food on the tongue and not the phenomenon, was conducted by Williams and Wil¬
sight of food is the important feature of Pavlovian liams (1969). Naive pigeons were exposed to trials in
salivary conditioning. Likewise, grain in the mandi¬ which the brief illumination of a key light was fol¬
bles and not the sight of grain is the US in autoshap¬ lowed by food. Until the first peck occurred, the pro¬
ing. Via Pavlovian conditioning the sight of grain cedure was almost identical to the Brown and Jenkins
(paired with gram m the mandibles as the US) re¬ procedure. The crucial difference was this: if the
leases pecks at grain. Similarly, by Paylovian condi¬ pigeon pecked the key, the trial was terminated and
tioning the response key (paired with the sight of food was omitted. What might one expect the results
grain) releases pecking. of such an experiment to be? The Pavlovian contin¬
Most of these approaches to the mechanism of auto¬ gency would generate the initial key peck (auto¬
shaping can be transposed to other approaches with shaping). However, key pecks would not be followed
appropriate assumptions. Which is the most accept¬ by food. Indeed, they would prevent food delivery,
Barry Schwartz and Elkan Gamzu 63

while any other behavior that the pigeon engaged in Stimulus-Reinforcer Relations in

would be followed by food. The expectation is clear: Automaintenance and Omission

the negative contingency between pecking and food

Many of the studies already discussed in connec¬
would quickly eliminate pecking. The result, how¬
tion with the Pavlovian control of acquisition of
ever, was that key pecking was maintained at substan¬
responding are also relevant to the question of
tial frequency over many hundreds of trials in vir¬
maintenance. The simplest demonstration of auto¬
tually all pigeons. As Williams and Williams (1969)
maintenance is a study by Schwartz and Williams
noted, this phenomenon is virtually identical to what
(1972b). Pigeons were exposed to 6-sec trials which
Sheffield (1965) observed in studying the effects of a
terminated with food. Responses had no programmed
similar procedure on salivation in dogs. Sheffield
consequence. Key pecking was nevertheless main¬
labeled the phenomenon omission training. Williams
tained at rates between 8 and 15 responses per trial
and Williams labeled their finding automaintenance.
over many sessions in all pigeons. While this simple
It has since been referred to in the literature as nega¬
demonstration of automaintenance seems to suggest
tive automaintenance to highlight the negative re¬
Pavlovian control of pecking, there is, of course, an
sponse-reinforcer contingency. This, however, is a
alternative explanation. We can presume that the
rather cumbersome term, and so we shall hereafter
control over initial responding is Pavlovian. How¬
refer to it as omission training or the omission effect
ever, once these responses occur, they are followed in
(cf. Hearst 8c Jenkins, 1974).
time by food. There is thus an adventitious response-
The Williams and Williams study made it clear
reinforcer relation which may contribute substantially
that the Pavlovian pairing of key and food con¬
to the maintenance of responding once initiated
tributed to the maintenance of responding, and not
(Herrnstein, 1966; Skinner, 1948). A series of studies
just to its acquisition. It demonstrated a clear viola¬
by Gamzu and Schwartz (1973), Gamzu and Williams
tion of the law of effect. The phenomenon has since
(1971, 1973), and Schwartz (1973a) strongly suggest
been demonstrated in numerous experiments (e.g.,
that such an account is inadequate, Let us suppose
Herrnstein 8c Loveland, 1972; Schwartz, 1972, 1973b;
that responding during trials was being maintained
Schwartz & Williams, 1972a, 1972b), though occasional
by an adventitious response-reinforcer relation. What
investigations have failed to obtain it reliably (e.g.,
influence would the delivery of food during the inter¬
Hursh et al., 1974). Jenkins (see Hearst & Jenkins,
1974) has also reported a variant of the effect. In a
trial interval have on this putative relation? It seems
clear that these extra reinforcements might be ex¬
study described as the 'long box” experiment, pigeons
pected to increase pecking, or perhaps not influence
were exposed to autoshaping trials in an unusually
it. Certainly, they would not be expected to d6cr£&§6
long chamber. Response keys, which signaled food,
pecking. On the other hand, from the Pavlovian point
were located at the ends of the chamber, and the
of view, food deliveries during the ITI would decrease
feeder was located in the center of the chamber. Key
the differential predictiveness of the trial stimulus
pecking was acquired and maintained under these
and, as a result, decrease Pavlovian control over peck¬
conditions despite the fact that when the pigeons
ing. The Pavlovian view makes the paradoxical pre¬
pecked the key, it took them so much time to move
diction that increasing the rate of food delivery will
from the key to the feeder that food presentation usu¬
decrease responding. This indeed is what occurs. In
ally terminated before they arrived. Thus, in this pro¬
the Gamzu and Williams studies, in which trials were
cedure, key pecking did not prevent food presenta¬
tion, but it effectively prevented food consumption. 8.6 sec long and ITIs averaged 40 sec, food delivery
What is one to make of the omission effect? The during the ITI at the same rate as during the trial
maintenance of key pecking in the face of a negative virtually eliminated responding. In the Gamzu and
response-reinforcer contingency strongly supports the Schwartz (1973) study, which involved the regular al¬
view that stimulus-reinforcer relations dominate re¬ ternation of two key colors for 27-sec periods, when
sponse-reinforcer relations in controlling key pecking food was presented in only one key color substantial
in autoshaping-type procedures, and even suggests responding was maintained to that color. When food
that response-reinforcer relations might exert no con¬ was then presented in both colors with equal fre¬
trol at all. We shall evaluate these possibilities in the quency, responding was substantially decreased,
following sections, focusing first on the role of stimulus- though not eliminated. The Schwartz (1973a) study
reinforcer relations in automaintenance, and second was similar to that of Gamzu and Schwartz except that
on the role of response-reinforcer relations. decreases in responding were even more dramatic. It
64 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

should be noted that these studies cannot logically scheduled reinforcement. From this result, Williams
rule out the possibility that adventitious response- and Williams argued that the key pecking which oc¬
reinforcer relations contribute to the maintenance of curred on the omission procedure was sensitive to its
responding on autoshaping procedures. Only pro¬ consequences. Unfortunately, the design of the Wil¬
cedures employing a negative response-reinforcer de¬ liams and Williams experiment permits an alternative
pendency can do that. However, they do make it clear interpretation. Let us assume that on each trial, the
that such relations are not sufficient. pigeon looks at only one key, and that the pigeon is
likely to peck at the key it looks at. With this assump¬
tion, one can explain the Williams and Williams re¬
The Response-Reinforcer Relation
sults in purely Pavlovian terms. Despite the fact that
The mere demonstration of the omission effect sug¬ both keys are simultaneously illuminated, if the
gests that autoshaped pecking is insensitive to its con¬ pigeon only looks at one key per trial, it is experiencing
sequences. However, while responding on automain¬ automaintenance trials and omission trials separately.
tenance procedures is maintained at levels of 80-120 Automaintenance trials always terminate with food.
pecks per mm, responding on the omission procedure Omission trials terminate with food only if the pigeon
is maintained at substantially lower levels, often only does not peck the key. Since the pigeon does peck the
lh—30 ir£ep6ils6s per mm (Schwartz 8c William^, 1972a, key, there are fewer omission key-food pairings than
1972b), This discrepancy suggests that the response- automaintenance key-food pairings. Moreover, the
reinforcer dependency does exert control over re** omission key-food pairings are effectively on a partial
spondmg, it i§ secondary to the control exerted reinforcement schedule, which often weakens Pav-
by the stimulm-rcinforccr dependency. Williams and lovian conditioning. There might thus be stimulus-
Williams {1969} examined this possibility. A response reinforcer relations of different strengths between each
key wa§ illuminated for 6-see trials and followed by of the keys and food, and this might account for the
food unless a key peck occurred (omission). A second observed difference in levels of responding.
response key was Illuminated whenever the first one To test this account of the Williams and Williams
Pe^ks oii this key had no programmed conse- study, Schwartz and William$ (1972a) did an CXperi
cpaences. Pigeons qwivkly R&med to peek exclusively at ment in which the frequency of automaintenance key-
this second key, and thus obtain nearly all of the food pairings and omission key-food pairings was kept

Tig, 3, Percentage of trials with

at least one response and re¬
sponses per minute throughout
the experiment, averaged across
all eight subjects in four-session
blocks and separated according
to key color. To the left of the
dotted vertical lines, the red key
was associated with the omission
condition, the white key with
the yoked control. To the right
of the dotted vertical lines, the
significance of the key colors
was reversed. (From Schwartz
& Williams, 1972a. © 1972 by
the Society for the Experimental
BLOCKS OF 4 SESSIONS Analysis of Behavior.)
Barry Schwartz and Elkan Gamzu 65

equal. The two types of pairings occurred on separate the response key, pigeons also “bowed” and “rooted”
trials indicated by different key colors. If an omission when the signal was present. Morrison then syste¬
trial terminated with food, a subsequent, yoked, auto¬ matically examined the effects of a negative response-
maintenance trial also terminated with food. If the reinforcer contingency on each of these behaviors. The
pigeon pecked the key during the omission trial and general result was that a negative contingency sup¬
prevented food, food was not delivered at the end of pressed, but did not eliminate, the target behavior,
the yoked automaintenance trial. In this way, the two with the result that each of the other behaviors in¬
types of trials differed only in the relation between creased in frequency. When a negative contingency
key pecks and food. Under these conditions pigeons was established simultaneously between each of the
pecked the automaintenance key on many more trials behaviors and food, the pigeons emitted all of the be¬
and at twice as high a rate as they pecked the omission haviors rather than suppressing them all. Murray
key. The data are presented in Figure 5. From this, (1974) recently observed the same sort of effects in a
Schwartz and Williams concluded that autoshaped study of Siamese fighting fish (Betta splendem). The
responding was to some degree sensitive to its conse¬ reinforcer was presentation of a mirror, which reliably
quences. produces display in the fish. Murray observed four dis¬
Some more recent research on the omission effect tinct behaviors conditioned to the signal. When a
has examined in more detail the behaviors which are negative contingency was imposed between any of the
actually conditioned by the contingency relation be¬ behaviors and the mirror, that behavior was sup¬
tween the key light and food. Wessells (1974) observed pressed, while the others continued to occur. Like

that what was conditioned in addition to pecking was Morrison’s observations, Murray observed that while
orientation toward and approach to the response key. the negative contingency reduced the behavior, in all
When a negative contingency was established between cases it failed to eliminate the behavior.
any approach to the key and food, Wessells observed The study by Murray is the first one we have dis¬
the gradual elimination of approach behavior. The cussed which examined the omission procedure iil a
response elimination required a good deal of experi¬ species other than the pigeon. Actually, the omission
ence. After about 10 sessions, the subjects were still procedure has also been studied with rats, chicks, and
approaching the key on 30-40% of the trials. Never¬ squirrel monkeys, and not always yielded the same
theless, what is of prime interest in this context is that results. Wasserman (1973a) studied chicks with heat
approach toas eliminated by the negative response- as the reinforcer. Pecking and nuzzling the response

reinforcer contingency. It should be noted, in addi¬ key were maintained despite the negative contingency.
tion, that orientation toward the key continued at Stiers and Silberberg (1974) observed maintenance of
high frequency even after approach had been elimi¬ responding in rats in the face of a negative contin¬
nated. A similar study by Browne and Peden (see gency. The signal for food was the insertion in the
Hearst & Jenkins, 1974) produced somewhat different chamber of a retractable lever. The response was not
results. Imposition of a negative contingency between lever pressing, but lever contact, typically with the
approach and food never completely eliminated ap¬ mouth or vibrissae. Interestingly, Stiers and Silber¬
proach behavior. After 35 sessions, subjects were still berg observed licking, pawing, and biting in an auto¬
responding on 12-60% of the trials. In another study, maintenance procedure, but nose contact of the lever
Barrera (1974) observed that key pecking occurred at on the omission procedure. Finally, Schwam and
a lower rate under a negative contingency than under Gamzu (1975) studied the omission procedure with
standard automaintenance conditions. However, he squirrel monkeys as subjects. Across different index
observed that pecking per se was occurring at as high responses, the uniform result was that responding was
a rate as ever. The effect of the negative contingency not maintained in the face of the negative contin¬
was to displace pecking off the key. Interestingly, gency. Schwam and Gamzu explained the discrepancy
Dunham, Mariner, and Adams (1969) observed the between the squirrel monkey and the pigeon on omis¬
same phenomenon when key pecks were punished sion procedures by arguing that the omission effect
with electric shock. occurs only in species for which there is a relatively
There is more evidence that responses which occur rigid pattern of reinforcer-appropriate consummatory
in the face of a negative response-reinforcer contin¬ activity. In these cases the stimulus-reinforcer associa¬
gency are nevertheless sensitive to their consequences. tion determines the response which will occur. On the
Morrison (1974) studied automaintenance in the other hand, in species which have a large repertoire
pigeon with water as a reinforcer. In conducting the of consummatory activities (e.g., in the squirrel mon¬
research, he noticed that in addition to pecking key, biting, mouthing, pawing, licking, etc.) there is
66 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

no inflexible link between any one of these activities trial. Each trial terminated with food. Finally, the
and food, so that any of them can be eliminated by fourth group was yoked to this DRL group. Re¬
the negative response-reinforcer relation. sponses had no programmed consequences, and trial
duration was determined by obtained trial duration
in the DRL group. Each of the pigeons in the non-
Stimulus-Reinforcer and
yoked groups was exposed to both DRL and omission
Response-Reinforcer Interaction
procedures, in counterbalanced order. Each of the
In the preceding discussion, we have shown that pigeons in the yoked groups was exposed to both
both stimulus-reinforcer and response-reinforcer rela¬ yoked procedures, in counterbalanced order. Each
tions exert control over behavior in automaintenance group contained eight pigeons.
and omission studies. In automaintenance the two What prediction might one make about the levels
types of relation may be mutually facilitative. The of responding maintained in the four groups? Let us
stimulus-reinforcer relation generates pecking, which consider first the response-reinforcer contingency. The
may then be adventitiously reinforced with food. DRL group is exposed to a positive contingency, the
While adventitious reinforcement is not itself suffi¬ omission group to a negative contingency, and the two
cient to maintain responding, it may make a substan¬ yoked control groups to no contingency (except per¬
tial contribution to the high rates at which pecking haps an adventitious one). On this basis alone, we
occurs. In omission training, the stimulus-reinforcer would expect responding to be strongest in the DRL
and response-reinforcer relations are antagonistic. group and weakest in the omission group, with the
While the stimulus-reinforcer relation generates peck¬ other two groups somewhere in between.
ing, the response-reinforcer relation works, with only Now let us consider the stimulus-reinforcer rela¬
partial success, to eliminate peeking. Indeed, it is the tion. As we have already mentioned above, increases
lack of success of the negative response-rcinforccr in the trial/ITI ratio (i.e., increases in trial length
contingency which has generated such interest in the with ITI constant) reduce the level of responding
phenomenon. It is a tribute to the remarkable pQW€F maintained on autoshaping procedures (Raldock,
of the stimulus-rcinforcer relation that it can compete 1974). We would therefore expect that responding
effectively with a response-reinforcer contingency for controlled by the stimulus-reinforcer relation in these
control of a behavior which i§ ordinarily quite §ensi groups will be inversely related to the trial duration
tire to its consequences. We would like to be able to obtained. Since the DRL should maintain more re¬
specify the variables which determine the relative sponding than the omission procedure, DRL trials
contributions of th£sO two typos of contingency to the should be longer than omission trials. Hence stimulus-
outcome of omission training and automaintenance reinforcer control should be weaker in the DRL and
studies. Unfortunately, there is little evidence on this DRL-yoked groups than in the other two groups.
point. One systematic investigation of the problem is Since trial durations are equal in experimental groups
a §tudy by Williams (1974). The study involved four and their yoked partners, we would expect no differ¬
groups of pigeons. One group was exposed to a vari¬ ence in stimulus-reinforcer control between each ex¬
ant of the omission procedure. The response key was perimental group and its yoked partner.
illuminated periodically, and if the pigeon did not The results of the experiment, in asymptotic re¬
peck the key for £ sec, the key light was extinguished sponses per minute, were as follows; DRL—14.0; omis¬
and food was presented; Each time the pigeon pecked sion—4.0; DRL-yoked—6.2; omission-yoked—25.0. Thus
the key, the trial was restarted. Thus a trial did not all predictions were confirmed. Omission-yoked pi¬
terminate until 6 sec without a peck had elapsed, and geons responded more than DRL-yoked pigeons (equal
every trial terminated with food. Conditions for a response reinforcer contingencies but longer trials for
second group were determined by conditions obtained the DRL-yoked group); DRL pigeons responded more
by the first group. Trials were identical in length. In than omission pigeons (difference in the response¬
this group, however, there was no programmed rela¬ reinforcer dependency); DRL pigeons responded more
tion between responding and trial duration or food. than their yoked partners, and omission pigeons re¬
We shall call this group the “yoked omission” group. sponded less than their yoked partners (in both cases
A third group was exposed to a discrete trials DRL as a result of equal trial durations but different
(differential reinforcement of low rate) procedure. response-reinforcer dependencies). The data provide
When the key was illuminated, the pigeons had to clear evidence that stimulus-reinforcer and response¬
peck the key after 6 sec elapsed to obtain food. Pre¬ reinforcer relations are both importantly involved in
mature pecks reset the DRL timer and prolonged the the control of responding by these procedures.
Barry Schwartz and Elkan Gamzu 67

Are the Behaviors Influenced by reviewed above suggests? There are two possibilities.
Stimulus-Reinforcer/Response-Reinforcer One is that the apparent interaction is not an inter¬
Relations Different? action at all. Instead, stimulus-reinforcer relations
control short-duration pecks and response-reinforcer
Throughout the discussion thus far we have been relations control long-duration pecks, and the “inter¬
assuming that the pecking which is influenced by action" simply results from the fact that in most stud¬
stimulus-reinforcer relations and occurs on automain¬ ies both kinds of key peck are lumped together as
tenance and omission procedures belongs to the same instances of switch closure. A second and more in¬
class as the pecking which is maintained by response- triguing possibility is that the two kinds of peck do
reinforcer relations and occurs in standard free op¬ interact, but only indirectly. This possibility has been
erant procedures. It is possible, however, that the two discussed by Gamzu (1971) and by Schwartz and Wil¬
types of contingency control two different types of liams (1972b). We might call it the “minimal-unit”
peck which are as independent of one another as hypothesis. Briefly, the argument is this: short-duration
salivation and panel pushing in the dog. Schwartz and pecks are generated and controlled by stimulus-
Williams (1972b) conducted a series of experiments reinforcer relations. Moreover, they comprise the basic
designed to explore this possibility. The property of biological units out of which long-duration, operant
key pecks they measured was their duration, since pecks develop. Furthermore, the long-duration pecks
Wolin (1968) had previously observed that key peck continue to depend for their occurrence upon the
duration was influenced by the nature of the rein¬ simultaneous occurrence of short-duration pecks. Thus
forcer, either food or water. Schwartz and Williams the operant key peck may be viewed as an “anaclitic
found that key pecks maintained on the omission pro¬ operant" (Kimble and Perlmutter, 1970), since it is
cedure were of almost uniformly short duration—less built from, and depends upon, the members of a
than 20 msec. Distributions of response duration on different response class. From this account, the inter*
standard fixed-interval and fixed-ratio schedules also action between stimulus-reinforcer and response-
included short duration pecks, but the majority of reinforcer relations reflects the dependence of reflexive
pecks were long in duration—greater than 40 msec.
pecks on the former, the dependence of operant pecks
This suggested that there might indeed be two differ¬ on reflexive pecks, and, finally, the sensitivity of oper¬
ent classes of key peck, one of which (short-duration) ant pecks to their consequences. It should be noted that
was reflexive, controlled by stimulus-reinforcer con¬ this account almost entirely unsupported, and at
tingencies and representing the dominant response on
present it raises more questions than it answers. How,
omission procedures, while the other (long duration) for example, do operant pecks develop out of re¬
was nonreflexive, sensitive to response-reinforcer con¬ flexive pecks? This question is merely a specific re¬
tingencies and representing the dominant response on statement of a question which has haunted experimen¬
free operant reinforcement schedules. To test this pos¬ tal psychology since its inception: how does voluntary
sibility, Schwartz and Williams tried to differentially behavior emerge out of the collection of infantile re¬
reinforce both short- and long-duration responses. The flexes? Also, one might wonder whether enough ex¬
rationale behind the study was this: short-duration perience with operant contingencies eventually frees
pecks, if insensitive to their consequences, would not the operant peck from its reflexive origins. These and
increase in frequency when differentially reinforced, other questions require empirical investigation. For
while long-duration pecks, if sensitive to their conse¬
the present, let us discuss one finding which offers
quences, would increase in frequency if differentially some support for the minimal-unit hypothesis.
reinforced. The results obtained by Schwartz and We discussed above, in the section which addressed
Williams supported this hypothesis. However, other the conditions necessary for the acquisition of peck¬
investigations (cf. Moore, 1973) have failed to find ing in autoshaping procedures, a series of studies by
duration differences across different procedures and Gamzu and Williams (1971, 1973). The reader will
have offered alternative interpretations for the data recall that when food presentation was as likely dur¬
observed by Schwartz and Williams. Thus the argu¬ ing the intertrial interval as during the trial, key
ment for two kinds of key peck must be taken as pecking either did not develop or, if already devel¬
tentative. oped, was eliminated. Gamzu and Williams observed
If there are two different kinds of key peck, each that pigeons initially exposed to a differential pro¬
sensitive to different variables, how is one to explain cedure (food presented only during the trial) would
the apparent interaction of response-reinforcer and peck the key at least 60 times per min. However, they
stimulus-reinforcer contingencies which the research also observed the curious phenomenon that if the
68 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

pigeons were initially exposed to a nondifferential exposed to the differential procedure from the outset.
procedure (food equally likely during trial and ITI) Here response durations are short early in training,
during which they did not peck the key and were sub¬ but by later sessions there are substantial numbers of
sequently exposed to a differential procedure, key long-duration responses. These data support the views
pecking developed at a normal rate but reached a that (a) there are two different kinds of key peck,
much lower asymptotic level. This effect persisted identifiable on the basis of duration; (b) the two types
seemingly indefinitely. The explanation offered by of peck are controlled by different variables; and (c)
Gamzu (1971) was that during the nondifferential short-duration pecks seem to occur initially while
procedure a feeding-related behavior other than key long-duration pecks only develop with experience.
pecking occurs and is maintained (Staddon and Sim- Moreover, the fact that a procedural shift from differ¬
melhag, 1971). When the differential procedure is ential to nondifferential conditions eliminates pecking
introduced and key pecking develops, the other be¬ entirely, despite the fact that only the stimulus-rein¬
havior continues to occur and to be reinforced. Thus forcer relation is changed, suggests that instrumental
the development of operant pecks is essentially blocked responses are indirectly controlled by stimulus-rein¬
by the occurrence of these adventitiously reinforced forcer contingencies.
other behaviors, and the pecking one does observe is
strictly under the control of the stimulus-reinforcer
Autoshaping and Automaintenance:
relation. The implication of this account is that the
Theoretical Analysis
duration of key pecks which occur under these con¬
dition 8 should be almost exclusively short. In an un¬ In the sections above we have suggested that auto-
published portion of his doctoral dissertation Gamzu shaping is best described as Pavlovian conditioning
(1971) measured response durations. In Figure 6 and that automaintenance entails the joint action of
durations are presented for a pigeon exposed to the Pavlovian and operant contingencies. There have been
differential procedure after the nondifferential pro¬ two attempts to capture the autoshaping literature
cedure. It can be seen that both early and late in theoretically, one by Hearst and Jenkins (1974) and
training, response durations are exclusively short. one by Williams (1974).
Contrast this with the data in Figure 7 for a pigeon Hearst and Jenkins treat autoshaping and auto-

Fig. 6. Relative-frequency histo¬

gram of response durations of
a single pigeon during two ses¬
sions of the differential proce¬
dure. Prior to the introduction
of this procedure the pigeon
was exposed to the nondiffer¬
ential procedure for 14 days.
DURATION (MSECS) (From Gamzu, 1971.)
 69
Barry Schwartz and Elkan Gamzu

ing. Wasserman’s explanation is that in a dark

chamber the key light provides general illumination
which can be seen anywhere, i.e., it is a nonlocalized
CS. On the other hand, when the chamber is lit, the
pigeon must look at the key to see the stimulus
change, i.e., the CS is localized.
Thus the sign-tracking view places most of its
emphasis on the stimulus side of the phenomenon. It
does not explain why pigeons peck the key rather than
engage in some other directed behavior. Hearst and
Jenkins (1974) acknowledge the fact that in almost all
autoshaping studies the conditioned response is a
component of the unconditioned response to the
reinforcer. However, the implication of their account
is that if the signaling stimulus were somehow inap¬
propriate for directed consummatory activity, some
UJ other activity (e.g., approach) would nevertheless be
directed at it. In the standard pigeon autoshaping
experiment, there is no way to test this view. The CS
is typically response key illumination, and thus is in
many ways an ideal stimulus for pecking (Cruze, 1935;
Fantz, 1957; Hunt 8c Smith, 1967; Padilla, 1935).
There are some autoshaping studies, however, in
which tones rather than lights were used as signals.
Gamzu (1968) and Schwartz (1973a) both failed to ob¬
serve pecking at the tone source in such experiments.
More significantly, the pigeons in these studies also
DURATION (MSEGS)
failed to reliably approach the tone. After a brief
Fig. 7. Relative-frequency histogram of response durations of a period of orientation to the tone early in the experi¬
single pigeon during two sessions of the differential procedure, ments (owing, presumably, to its novelty), orientation
which was the first procedure to which the bird was exposed.
ceased, and in later tone presentations pigeons simply
(From Gamzu, 1971.)
moved to the feeder. Jenkins (in Hearst 8c Jenkins,
1974) did manage to condition approach to a tone
maintenance as an instance of sign tracking, which is source and even conditioned pecking, when the tone
defined as “behavior that is directed toward or away was localized behind a continuously illuminated
from a stimulus as a result of the relation between perforated hemisphere. Thus the efficacy of a tone as
that stimulus and the reinforcer or between that stim¬ a signal is still debatable. What is clear is that a tone
ulus and the absence of the reinforcer” (p. 4). They is far less effective than a key light.
suggest that sign tracking is a general phenomenon In our view, the sign-tracking account of auto¬
and may contribute substantially to many observa¬ shaped key pecking places too much emphasis on the
tions in the study of discrimination learning. Our key and not enough emphasis on pecking. The auto¬
primary concern here, however, is with sign tracking shaping phenomenon raises two questions: Why does
as an account of autoshaping. the pigeon peck, and why does the pigeon peck the
In Hearst and Jenkins’s view, as long as a stimulus key? Hearst and Jenkins may have provided a satisfac¬
can be localized, so that behavior can be directed tory answer to the latter question. However, the an¬
toward or away from it, such behavior will develop as swer to the former question is different. Pecking is
a function of the relation between that stimulus and conditioned because it is the central component of
a reinforcer (US). The importance of localizability of the feeding pattern of the pigeon. As Staddon and
the CS has been indicated by Wasserman (1973b). One Simmelhag (1971) have shown, pecking is observed in
group of pigeons was exposed to an autoshaping pro¬ pigeons when food is presented at regular intervals
cedure with the houselight illuminated. A second with no signal. Thus it is the mere presentation of
group was exposed to the same procedure but with the food which engenders pecking. The key light in the
houselight off. Only the first group acquired key peck¬ autoshaping procedure directs pecks but does not gen-
70 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

It is just the S-S* relation which generates the be¬

havior R, and this process is labeled f(As) in Figure
8. Williams argues that the form which a behavior
takes as a function of the S-S# relation is a problem
which is quite distinct from characterizing the as¬
sociation itself. He refers to this problem, which in¬
volves the relation between R and R*, as the “biology
of association/' In general, the answer to this question
will be specific to the species, reward, and situation
under investigation. Williams argues, in the case of
autoshaping, that the R set is a collection of un¬
learned consummatory behaviors. The learning which
occurs in autoshaping is the development of control
over response R by a new stimulus, S, such that S
becomes capable of releasing the behavior R. Thus
Williams identifies his account as a “learned release”
hypothesis. Whether all Pavlovian conditioning ex¬
periments should be viewed as learned release experi¬
ments or whether autoshaping is unique in this regard
Fig. 8. Schematic outline of the CfitiC&l events underlying bicon¬
is discussed in detail by Williams. As we suggested
ditional behavior. See text for details. (From Williams. 1974.)
above, the question of what is actually conditioned in
Pavlovian conditioning experiments has been more
crate them. Moreover, the nature of the reinforcer evaded than analyzed in the past. Whether stimulus
(US) in an experiment determines net only what be¬ substitution, stimulus surrogation, learned release, or
havior will occur, but also which conditional stimuli some other label aptly characterizes most or all condi¬
will be maximally effective. Both Shettleworth (1972b) tioning results cannot at present be addressed with
with chicks and Foree and LoLordo (1973) with any confidence.
pigeons have shown that visual CSs dominate auditory The feature of Williams’s account which is unique
ones only when food rather than electric shock is the to the autoshaping literature is the relation between
US, the R set and the S4 event, labeled Ar and g(Ar) in
second and somewhat more comprehensive ac¬
A Figure 8. This of course reflects an operant contin¬
count of autoshaping and automaintenance has been gency between response and reinforcer. Williams's
provided by Williams (1974). Autoshaped pecking, account incorporates the fact that once behavior is
to Williams, is an instance of “biconditional behav¬ generated by an S-S4 association, it is subsequently
ior.” This term refers to the fact that both stimulus- further enhanced by an R-S* relation. Both f(As)
reinforcer and response-reinforcer contingencies play and g(Ar) feed into the R set and increase the prob¬
significant roles in controlling key pecking. Williams* ability of R. Williams's model is thus an explicit

§shems presented in Figure where S effort to account for the joint action of the two types
designates a stimulus, 3 4 the reinforcing event, R4 the of contingency in the determination of a particular
consummatory response, and R the conditioned be¬ class of behavior. The function g(AP) is a positive
havior; As refers to the associative link between 5 and feedback loop which, once the behavior has occurred,
S*. As we discussed above, the formation of an associ¬ will insure its continued occurrence. This feature of
ation seems to require a differential predictive relation Williams's model is specific to autoshaping, as W6

between S and S*, and in thi§ respect autoshaping have said, This, however, is more historical accident

does not differ from standard Pavlovian conditioning, than logical necessity. The fact that key pecking has
Precise specification of the conditions necessary for for so long been studied in operant situations and has
the formation of an association is a problem Williams so clearly been shown to be sensitive to its conse¬
refers to as the “psychophysics of association.” It is a quences demands that an account of Pavlovian con¬
general problem of definition and specification of trol of pecking also include a vehicle for control by
contingency spaces and has already been discussed response-contingent reinforcement. It is entirely pos¬

above. The main point, again, is that autoshaping is sible that other behaviors which have been tradition¬
in no sense unique in terms of the nature of the S-S* ally studied in Pavlovian contexts are also sensitive to
association. R-S* links which are built into experimental pro-
Barry Schwartz and Elkan Gamzu 71

cedures. This possibility has simply not been sys¬ ules operate in alternation, each in the presence of a
tematically explored. different stimulus” (Catania, 1968, p. 339).
Multiple-schedule procedures are instances of what
has traditionally been called successive discrimination.
THE ROLE OF STIMULUS-REINFORCER Conditions of reinforcement in the presence of one
RELATIONS IN THE CONTROL OF stimulus typically differ from conditions of reinforce¬
BEHAVIOR MAINTAINED BY ment in the presence of a second stimulus. Of major
RESPONSE-REINFORCER RELATIONS interest is the extent to which the two stimuli control
behavior appropriate to their correlated reinforce¬
Having reviewed above the phenomena of auto¬ ment conditions and the extent to which the com¬
shaping, automaintenance, and omission, and having ponent schedules interact. The emphasis in the study
suggested that key pecking in these Pavlovianlike of multiple schedules is on maintenance of inter¬
procedures is influenced by both stimulus-reinforcer mittently reinforced behavior rather than its acquisi¬
and response-reinforcer relations, it seems appropriate tion. These characteristics set the study of multiple
to ask now whether the same kind of interaction can schedules apart from most other successive discrimina¬
be demonstrated in operant procedures which bear tion studies.
no obvious formal relationship to Pavlovian ones. An
understanding of the influence of autoshapinglike
Interactions in Multiple Schedules
phenomena in standard operant situations is essential
before the full significance of autoshaping for the The feature of the control of behavior by multiple
experimental analysis of behavior can be evaluated. schedules which has attracted the greatest research
In the remainder of the chapter we shall explore the interest is the interaction of the component sched¬
possibility that in situations in which control of ules. Suppose, for example, a pigeon is pecking a
behavior by response-reinforcer contingencies is dra¬ response key illuminated by a red light for reinforce¬
matic and unequivocal, stimulus-reinforcer contingen¬ ments programmed on a variable-interval 2-min (VI 2-
cies nevertheless play a vital role. Most of the phe¬ min) schedule. When the pigeon’s behavior is stable,
nomena we shall discuss come from the literature on the procedure is changed so that 3-min periods of red
multiple schedules of reinforcement. We shall first key illumination alternate with 3-min periods of
review the phenomena observed on multiple schedules green key illumination. The same VI 2-min schedule
and the standard accounts of these phenomena. We is in effect in the presence of both key colors. This
shall then apply some of the principles which have procedure is defined as a multiple VI 2-min VI 2-min
developed out of the study of autoshaping and auto¬ schedule (mult VI 2-min VI 2-min). It differs from the
maintenance to these phenomena. We shall argue that preceding only in that there are two alternating stim¬
no account of multiple-schedule phenomena will be uli instead of one. Suppose responding to the red key
accurate unless it includes an analysis of autoshaping¬ in the mult differs in some way from responding in
like stimulus-reinforcer relations and that, indeed, the the previous procedure when the key was always red.
most dramatic findings in studies of multiple sched¬ This difference would be presumed to result from an
ules result from the influence of these relations. interaction between the two component schedules.
That a prototypic operant, key pecking, can be Indeed, there is evidence that responding is main¬
influenced by Pavlovian operations does not mean tained at a higher rate by a simple VI schedule than
that it must be so influenced. It is possible that, in by a mult VI VI, in which the value of the VI sched¬
standard operant situations, the control of behavior ules is the same as in the simple VI (Bloomfield,
by response-reinforcer relations simply dwarfs the in¬ 1967).
fluence of stimulus-reinforcer relations, even though Now suppose the behavior of our pigeon in the
certain operant conditioning procedures have Pav¬ mult VI 2-min VI 2-min schedule has stabilized. The
lovian stimulus-reinforcer contingencies built into procedure is then changed to mult VI 2-min extinc¬
them. The remainder of this chapter will be con¬ tion (EXT). When the key is red, the same VI 2-min
cerned with assessing the influence of these Pavlovian schedule is in effect as before, but when the key is
contingencies on behavior which is already controlled green, no reinforcement is scheduled. The effects of
by operant contingencies. This issue will be discussed this procedural change are twofold. First, responding
mainly in the context of the control of behavior by on the green key decreases. This effect is not consid¬
multiple schedules of reinforcement. A multiple sched¬ ered the product of an interaction, since the schedule
ule is one “in which two or more component sched¬ on the green key has changed from VI to EXT. Sec-
 72 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

ond, responding on the red key increases. This in¬ VI 1-min EXT schedule. Responding in the un¬
crease must be attributed to an interaction between changed VI component increases (positive contrast).
the two components of the multiple schedule, since Is this a clear indication of schedule interaction? It
the reinforcement schedule in the presence of the red certainly seems to be, since the VI component has not
stimulus has not been altered. This particular type of changed while the behavior in that component has
interaction is called positive behavioral contrast changed. However, what has also changed is the
(Reynolds, 1961a), and it will command most of our amount of exposure to the procedure. It is possible
attention in the remainder of this chapter. Before that responding changes merely as a function of ses¬
actually reviewing research on interactions in multi¬ sions of exposure to a procedure. Thus in order
ple schedules, however, we shall discuss some general clearly to identify the effect as an interaction, it must
issues regarding the terminology and measurement of be shown that if the pigeons are returned to the mult
interactions. VI 1-min VI 1-min procedure, responding in the
Interactions between components of a multiple unchanged component will decrease and return to its
schedule can never be directly assessed, If one is inter¬ initial levels. In short, in order to demonstrate com¬
ested in the effects of component B of a multiple ponent interaction in a multiple schedule, the inter¬
schedule on responding m component A} then one action effect must be reversible, i.e., the base line
alters conditions in component jB, keeps conditions in against which interactions are assessed must be re¬
component A con§tant; and looks for changes in re¬ coverable. Many studies which have purported to
sponding in component A, demonstrate schedule interaction have failed to satisfy
If responding in component A changes when com¬ this baseline recovery criterion (cf. Gonzalez & Champ-
ponent B conditions are changed, then one might lm, 1974),
infer that i3 has been influencing A all along. Thus To summarize, the ideal procedure for demon¬
assessments of interaction in multiple schedules typi¬ strating multiple-schedule interactions contains three
cally require within-subject, across procedure compari stages; first, exposure to a mult with equal compo¬
son. Usually one component schedule remains constant nents until behavior is stabilized; \ second, alteration
from one procedure to the next while the other of one component schedule; and third, return to the

component changes* While interactions probably oc¬ first procedure to recover the baseline.
cur on all multiple-schedule procedures, an inter¬
action can only b$ unequivocally demonstrated and
Types of Interaction
categorized in a multiple-schedule component which
is unchanged. While the strategy for demonstrating There are four possible types of schedule inter¬
interactions seems straightforward, there are some action; positive and negative contrast and positive and
complexities. Suppose pigeons are exposed first to a negative induction. Part A of Figure 9 presents sche¬
mult VI 1-min VI 1-min schedule, and then to a mult matic diagrams of positive contrast and negative in-

Fig. 9. Schematic diagrams of the four types

of behavioral interaction. In A, after be¬
PROCEDURE PROCEDURE PROCEDURE PROCEDURE havior has stabilized on a mult VI 3-mm VI
mult vis1 yi^1 mult YI31 EXT mult via' YI3' mult YI3' VII* 3-min, the schedule associated with the sec¬
ond component is changed to EXT1 and
response rate decreases (dashed line). The
upper portion of the figure demonstrates
RESPONSES PER MINUTE

^iViViViViViV»S‘i;ri"iViViVi*iViYi*OVr 3 / _ positive contrast (shaded area). The lower

^^ViViYriYiYiYiViViYiViVi'iViVi'iY portion of A demonstrates negative induction
\ i
s- (shaded area). In B, after stabilization on
UJ
POSITIVE CONTRAST £ mult VI 3-min VI 3-min, the schedule in the
NEGATIVE CONTRAST
LJ second component is changed to VI 1-min,
to
Z i.e., more reinforcements are available per

U ...I...•.•.•.KW.V.W.W.W.W
toQ_
III
y^YiY.YiYiYiYiY.'iYl'iY.Y.Yi'iYiYiYi
jrnmrnmrnmxmm
unit time (solid line). The shaded
the upper portion of B is an example of
area in

Vi’iViViV.V'ViViViV.V.ViViViViViViV Ou
1 ,jY.V.YiV.V>ViViV»Vr.V*V»'iV.Vl'l"fS' negative contrast, while the shaded area in
the lower portion of B is an instance of
positive induction. Note that in all four
NEGATIVE INDUCTION POSITIVE INDUCTION cases the schedule in the first component
does not change, nor does reinforcement
SESSIONS SESSIONS
density. Thus these changes are the result of
A B interactions with the adjacent schedule.
Barry Schwartz and Elkan Gamzu 73

duction. Positive contrast is defined as an increase in min the color of the key changed and the second
responding in an unchanged component of a multiple component was in effect. This cycle was repeated 30
schedule with decreases in responding in the other times each session. During the first component the
component. Negative induction is defined as a de¬ schedule of reinforcement for pecking was VI 3-min.
crease in responding in an unchanged component of Several different schedules were used in the second
a multiple schedule with decreases in responding in component, but we shall focus on the transition from
the other component. Positive induction and negative VI 3-min to EXT and back to VI 3-min.
contrast are diagramed in part B of Figure 9. Positive Figure 10 shows what happens to rate of respond¬
induction is an increase in responding in an un¬ ing in the unchanged component of these different
changed component of a multiple schedule with in¬ schedules. (The data are approximations from Rey¬
creases in the other component, while negative nolds’s figures and are plotted for only three of the
contrast is a decrease in responding in an unchanged pigeons, since the rate of responding for the fourth
component of a multiple schedule with increases in pigeon did not appear to be stable.) The introduction
the other component. These four types of interaction of the EXT component resulted in a large increase in
are not independent. In any particular experimental responding (positive contrast) in the unchanged VI 3-
manipulation only two types of interaction are pos¬ min component. Reintroduction of the VI 3-min
sible. Decreases in responding in the changed compo¬ schedule in the second component reversed this effect
nent of a multiple schedule can produce either and restored the original response rate.
increases (positive contrast) or decreases (negative in¬ Many of the features of Reynolds’ experiment
duction) in the other component. Similarly, increases have become standard parts of investigations of posi¬
in responding in the changed component can produce tive contrast. First, the reinforcement schedule in the
either decreases (negative contrast) or increases (posi¬ unchanged component is typically a VI, though occa¬
tive induction) in the unchanged component. It sionally fixed schedules (Reynolds 1961b, 1961c), fixed-
should be noted that all four types of interaction are interval schedules (Reynolds & Catania, 1961; Stad-
defined in terms of response rate rather than some don, 1969) and DRL schedules (Reynolds, 1961b) have
other feature of the experiment. This is meant only to been used. Second, the reinforcement schedule in the
be descriptive. It does not imply that response rate changed component is typically also VI, and the
changes in one component cause rate changes in the change is to EXT. However, many other procedures
other component. We shall see below that while some have been used (Brethower & Reynolds, 1962; Terrace,
investigators have argued this position, the matter is 1968), and one of the uncertainties in the contrast
still quite controversial. The definitions given here literature centers on what procedure changes are nec¬
are not meant to imply support for any particular essary to produce contrast. Third, usually the same
causal account of schedule interaction. response is required and the same remforcer delivered
In the sections to follow we first describe reported in both components of the multiple schedule. An
instances of the different types of schedule interaction. exception is a study by Scull and Westbrook (1973)
The remainder of the chapter will focus almost ex¬ in which key pecking was required in one component
clusively on behavioral contrast. We discuss the differ¬ and bar pressing in the other. This experiment failed
ent accounts in the literature offered to explain to result in positive contrast.
behavioral contrast. We next review some of the Finally, the use of pigeons as subjects is a crucial
temporal properties of contrast. Next the relation feature of the basic demonstration of positive con¬
between schedule interactions in multiple and con¬ trast. Species differences are evident in contrast ex¬
current schedules is discussed. Finally, a new account periments, and indeed the theory of contrast that we
of contrast, based upon the phenomena of autoshap¬ shall propose predicts that this should be so. When
ing and automaintenance, is presented and evaluated. rats are subjects, the results are equivocal. These
studies of contrast with rats fall into three categories
according to the experimental results, which occasion¬
Positive Behavioral Contrast
ally give evidence of contrast in rats, more often are
The classic demonstration of positive behavioral equivocal, and sometimes clearly fail to find contrast
contrast was Reynolds’s (1961a) experiment in which in this species.
pigeons were exposed to a series of multiple schedules The first, small, category includes studies that pro¬
with two alternating components. Each component vide positive evidence for contrast. Coates (1972) ex¬
was in effect for 3 min during which the key was posed rats to a mult VI 30-sec VI 30-sec until response
illuminated by a specific color. At the end of the 3 rates had stabilized. Then the procedure was changed
74 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

Fig. 10. Individual response rates

in the unchanged VI 3 min com
ponent of a series of multiple
0 1 2 3 4 5 6 7 8 9 10 II 12 15 14 15 16 17 18 19
schedules for three pigeons. (Es¬
SI2§|0N§ timated from Reynolds, 1961a.)

ta a mult PUN T EXT, VI 50-acc, EXT, VI 30-act, in schedules; VI; mult VI EXT; mult VI VI. The VI
the first component of which not only was bar press- schedule was always VI 30-see. In the transition from
mg not reinforced but eaeh response resulted in shock. simple VI to mull VI EXT, hve of eight rats showed
Response rat<*§ m the VI components increased, with positive behavioral contrast, However, two of the
rats in the VI following PUN+EXT slightly higher other three rats showed negative induction. A final
than in the VI following regular EXT. Wilkie (1972) complicating factor is that the elevated VI rate in
exposed four rats to a mult VI 30-see VI 30-see and mult VI EXT shown by two rats did not return to
then ehanged the second component to EXT. In all base line in stage III, when both components were
four subjects response iut<,'v m the VI component in¬ equal Vis.
creased. Henke, Allen, and Davison (1972) studied Other mixed results come from a set of experiments
four rats on a mult VI 1-min EXT after a mult VI employing shock. In these studies rats are first exposed
Emin VI l-min, and again all four subjects showed to a mult VI 30-sec fixed ratio (FR) 10. Subsequently
positive eontrast in the unchanged component. Thus shock is introduced to the second component, so that
positive contrast can he observed in rats. each tenth response results in both a food pellet and a
Vor§ chara<£t£ri§tis of the outcome of contrast shock (Cook 8c Davidson, 1973; Cook 8c Sepinwall,
studies using rats is the second category, in which 1974; Davidson k Gook, 1969; Sepinwall, 1973). The
cquiyocal results are obtained. Pear and Wilkie (1970) initial effect of introducing shock is negative induc¬
exposed two rats to six sessions of VI 30-sec, followed tion (Cook & Davidson, 1973; Davidson & Cook, 1969),
by a number of sessions on MIX VI 30-sec EXT. (A but after a few sessions there is often evidence of posi¬
mixed schedule is just like a multiple schedule, except tive contrast (Sepinwall, personal communication).
that components are not signaled.) After 10 sessions The third class of rat studies either failed to find
one rat showed positive contrast, but the other showed positive contrast or found negative induction. The
negative induction to the extent that the VI 30-sec procedures used have varied from mult VI EXT after
had to be changed to a VI 20-sec to maintain respond¬ simple VI (Freeman, 1971a; Jaffe, 1973; Weiss, 1971)
ing. In a second experiment (Pear & Wilkie, 1971) or after mult VI VI (Dickinson, 1973) to mult VI VT
eight rats were exposed to the following sequence of (a VT or variable-time schedule is one in which rein-
Barry Schwartz and Elkan Gamzu 75

forcements are delivered at irregular intervals inde¬ same phenomenon. Investigators in the past have
pendent of responding) after either simple VI or mult failed to notice this interdependence between the two
VI VI (Freeman, 1971a; Lattal & Maxey, 1971). types of contrast because they have explicitly designed
experiments to look for one or the other type.
Demonstrations of Terrace (1968) exposed three pigeons to a mult
Negative Contrast VI 5-min VI 5-min and then to a mult VI 1-min VI 5-
min. Only one pigeon showed negative contrast (de¬
There are certain logical problems in evaluating
creases in responding in the unchanged VI 5-min com¬
negative contrast. Rachlin (1973) has suggested that
ponent). Nevin (1968) observed negative contrast in a
positive and negative contrast are the same phenom¬
mult with a VI 3-min schedule in the constant com¬
enon, but that they occur at different points in an
ponent and different DRO (differential reinforcement
experimental sequence. Consider the top of part A in
of other behavior) schedules in the changing com¬
Figure 9, which depicts positive contrast schematically.
ponent. There are a number of demonstrations of
Suppose a third procedure, the return to VI VI after
local negative contrast (contrast with temporal charac¬
VI EXT, to recover base line, were added there and
teristics) which will be taken up in detail in a later
the base line recovered. Base line recovery means that
section (e.g., Bernheim & Williams, 1967; Nevin 8c
responding in the unchanged component decreases as
Shettleworth, 1966; Williams, 1965). The only clear
responding in the changed component increases. But
demonstration of large negative contrast effects in
this defines negative contrast. Thus an unambiguous
multiple schedules was obtained by Schwartz (1974a,
demonstration of positive contrast entails a later
1975). Pigeons were shifted from mult 3-min VI 3-min
demonstration of negative contrast, and there is a log¬
to mult VI 3-min VI 72-sec. Responding in the un¬
ical argument for treating them as instances of the
changed component is shown in Figure 11. Data
points below the dashed horizontal line indicate nega¬
tive contrast.

Explanations of Multiple-Schedule
Interaetidns

Contrast has received far more theoretical attention

than induction in the literature. The reason for this
asymmetry is probably historical. Induction effects
follow from classical Hull-Spence discrimination
theory (Spence, 1936), while contrast explicitly contra¬
dicts it (Allen, Capehart, 8c Hebert, 1969). In that
context induction is the normal, expected effect and
contrast is the surprise (cf. Bloomfield, 1969).
From the outset, explanations of contrast have in¬
cluded some notion of inhibition (Pavlov, 1927—
though he labeled what is now called contrast “induc¬
tion”). Almost all of the major current alternative
explanations of contrast on multiple schedules include
an inhibition component (Bloomfield, 1969; Catania,
1969; Malone 8c Staddon, 1973; Staddon, 1969; Ter¬
Fig. 11. Response rate to the operant key in the unchanged
race, 1966a, 1966b, 1968, 1972). The point of conten¬
(VI 3-min) component of a mult VI 3-min VI 72-sec as a pro¬
portion of response rate in the same component in a prior mult tion among the different accounts is the source of
VI 3-min VI 3-min. Points below the dashed line are indicative inhibition, not its existence. In this section we shall
of negative contrast; points above the dashed line are indicative
outline the different accounts of contrast and review
of positive induction. The data in the left-hand panel are from
a series of conventional multiple schedules and show clear nega¬ the evidence which makes a distinction among them
tive contrast. In the right-hand panel both components were possible. Our treatment will roughly parallel recent
signaled on a separate key. Responses to the second key were
recorded but had no experimental consequences. (From Schwartz,
reviews of the problem (Freeman, 1971b; Terrace,
1974a.) 1972).
76 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

The Reinforcement turn energizes responding in neighboring situations.

Frequency Account Thus in Terrace’s view any manipulation which sup¬
presses responding and is demonstrably aversive will
Reynolds’s pioneering research on contrast (1961a,
result in contrast in the adjacent component of a
1961b, 1961c, 196Id) suggested that the variable re¬
multiple schedule. The response suppression pro¬
sponsible for contrast in the unaltered component of
cedure may, but need not, involve reinforcement re¬
a multiple schedule was a change in reinforcement
duction. Similarly, reinforcement reduction may not
frequency in the other component. Contrast was a
involve response suppression. Thus reinforcement re¬
change in response rate in one component in the
duction is neither necessary nor sufficient to produce
opposite direction from a change in reinforcement
contrast.
frequency in the other component. Thus increases in
A related account has been proposed by Bloomfield
reinforcement in component B of a multiple schedule
(1969). He argued that any procedure change which
result in negative contrast in component A, decreases
"worsens” conditions will result in contrast, and that
in reinforcement in component B result in positive
any procedure which does not worsen conditions will
contrast m J, and changes in B which do not in*
not result in contrast. The major difference between
fluence reinforcement frequency will not result in
Bloomfield’s view and Terrace’s is that Bloomfield
change in responding in A. The role of inhibition
looks to the antecedent conditions which produce re¬
in this account of contrast has been most explicitly
sponse suppression rather than to the suppression it¬
and generally stated by Catania (1969), Reinforce¬
self to explain contrast. This has the logical advantage
ment of one class of behavior is argued to have an
of substituting a causal relation for Terrace’s correla¬
inhibitory effect on all other classes of behavior. Thus
tional one. However, in terms of specific application,
reinforcement in one component of a multiple sched¬
the two accounts are not very different.
ule will inhibit responding in the other component,
and the procedural change from mult VI VI to mult
VI EXT results in an incroaso in VI responding be¬ Experimental Separation OF
cause it is released from previous inhibition by rein¬ Reinforgement Reduction and
forcement in the other component. Response Reduction

Experimental attempts to separate the two theories

The Response Suppression amount
<?i contrast fall into two main classes. One class in¬
Terrace (196$a? 1963b, 1966, 1072) observed that volves the reduction of reinforcement frequency with¬
when a discrimination is learned without errors—i.e., out concomitant reduction in responding. It includes
when a simple VI schedule is gradually transformed Terrace’s research on errorless discrimination learning
into a mult VI EXT in such a way that virtually na (1963a, 1963b, 1966a, 1966b, 1973). Terrace demon¬
responses are made to the EXT stimulus—positive strated that contrast does not occur after errorless dis¬
contrast is not observed in the VI component. This crimination training. Since errorless discrimination
observation led Terrace to propose that response rate involves a reduction in reinforcement without a con¬
reduction, not reinforcement frequency reduction, is comitant reduction in responding. Terrace takes this
responsible for contrast. Note that in the errorless pro¬ as Strong support for his account of contrast. On the
cedure, response rate reduction does not occur7 since other hand, Halliday and Boakes (1974) have observed
errors are never made. Note also that m the most com¬ contrast in procedures in which response rate was not
mon contrast-inducing procedure, shift from mult VI reduced (see Rilling chapter 15 in this volume for
VI to mult VI EXT, response rate reduction and rein¬ additional contradictory evidence). The second and
forcement rate reduction are perfectly confounded, and more diverse class of studies attempts to reduce rate of
the two explanations are not separable. In more recent responding without reducing rate of reinforcement. In
accounts Terrace has refined his view of contrast. Re¬ these procedures Terrace would predict contrast while
sponse reduction is necessary but not sufficient to Reynolds would not. The results of these studies are
produce contrast. Responding must be actively sup¬ mixed and open to methodological criticism (Free¬
pressed, i.e., it must be inhibited. It is the inhibition man, 1971b).
of responding in one component which produces con¬ One class of studies which manipulates response
trast in the other component. This view is much like rate without changing reinforcement rate involves
Amsel’s account of contrastlike effects in very different switching from mult VI VI to mult VI VT (e.g., Halli¬
experimental situations (1962). The inhibition of re¬ day & Boakes, 1972). Such procedures fail to produce
sponding produces an emotional by-product, which in contrast, which seems to support the reinforcement
Barry Schwartz and Elkan Gamzu 77

reduction account. However, Terrace’s recent (1972) the pattern of responding within a multiple-schedule
view is that response suppression, produced by nonre¬ component is not constant. Response rate changes are
inforcement or by some aversive stimulation contin¬ often most dramatic at the beginning of a component.
gent on responding, and not merely response reduction These changes, which are restricted to only a portion
is the necessary condition for contrast. Response-inde¬ of a component, will be referred to as local contrast
pendent reinforcement, while it reduces responding, (Malone 8c Staddon, 1973). They have also been re¬
presumably does not suppress it. Hence this class of ferred to in the literature as transient contrast (Nevin
studies is not decisive. 8c Shettleworth, 1966). However, we wish to reserve the
Another class of studies reduces responding with latter term to describe a different phenomenon, i.e.,
punishment by electric shock. The punishment param¬ that sometimes contrast dissipates with extended ex¬
eters are chosen so as to substantially reduce re¬ posure to a procedure.
sponding, but nevertheless to maintain it at a high To be consistent with the definition of contrast
enough rate so that reinforcement frequency is un¬ that we have already given, local contrast must be de¬
affected. Change from mult VI VI to mult VI VIT fined relative to the adjacent schedule. Indeed, with¬
punishment results in positive behavioral contrast out reference to the prior component, any time an FI
(Brethower & Reynolds, 1962; Terrace, 1968), offering (fixed-interval) or FR schedule is used one would have
clear support for Terrace's view. to call the resulting behavior an example of local
There is finally a class of studies which reduces re¬ contrast, since the pattern of responding maintained
sponse rate by manipulating the schedule in effect on those schedules 15 not constant. Consequently, an

during one of the multiple-schedule components. initial elevation followed by a lower constant response
Mult VI VI is switched to mult VI DRL (differential rate in a given component (A) will be defined as local
reinforcement of low rates) or mult VI DRO (differen¬ positive contrast if the overall response rate in an
tial reinforcement of other behavior). Terrace (1968) immediately prior component (jB) is lower than the
and Weisman (1969) have shown that as response rate overall response rate in component A. If the overall

is reduced in DRL, contrast occurs in the other com¬ response rate in component B is greater than the over¬
ponent. More recently, however, Boakes, Halliday, all response rate in component A, then the local effect
and Mole (1974) failed to observe contrast in a similar Is defined as local positive induction. Conversely, an
experiment. The Boakes et al. study controlled very initial depression in response rate at the beginning of
carefully for differences m local patterns <?£ reinforce- a component i§ d^fin^d &§ ea tjl£r !£>£td neg^tiye con¬

ment between the VI and DRL components, which trast or local negative induction, depending on the
Terrace's study did not. In the mult VI DRO pro¬ response rate in the prior component.
cedures, as response rate is reduced in DRO, contrast Boneau and Axelrod (1962) studied pigeons on a VI
is usually not observed (Boakes, Halliday & Mole, 1-min schedule for six days and then changed the
1974; Nevin, 1968; Nevin 2c Shettleworth, 1966; procedure to a mult VI 1-min EXT. Each component
Reynolds, 1961a; but see Weisman, 1970, for a demon¬ lasted 60 sec. As might be expected, overall contrast
stration of contrast under these conditions). Thus the was found. Responding in the presence of the VI
data from these studies are not decisively in support stimulus more than doubled after the introduction of
of one or the other account of contrast. an extinction component. Local positive contrast was
In summary, we have discussed the major views of found in the next stage, in which instead of alternat¬
behavioral contrast in this section and have seen that ing between VI and EXT components, Boneau and
experimental tests yield contradictory results. We Axelrod introduced either the EXT stimulus or a
shall see below that none of these accounts is suffi¬ time-out (TO) only every ninth component. Thus the
cient. VI component lasted 8 min. Response rate in the first
minute was substantially higher than in subsequent
1-min blocks and showed a gradual decrease until the
Temporal Properties of Contrast
block immediately preceding the EXT or TO. How¬
Our definitions and examples of contrast so far ever, after four sessions of this procedure, these local
have dealt with session-to-session changes in rate of contrast effects were no longer evident. Thus Boneau
responding in the constant component. These com¬ and Axelrod had reported the first instance of tran¬
parisons are typically made in terms of overall rate sient local contrast. It should also be noted that while
responding (total number of responses in a component the overall rates of responding in the VI component
divided by the total session time in which they may be showed a slight decrease during the four days of test¬
emitted). There is, however, substantial evidence that ing, the basic overall contrast effect remained even
78 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

trast. In pigeons both positive and negative local

contrast are reliably observed. In rats positive local
contrast is typically not observed, though negative
local contrast is (Bernheim 8c Williams, 1967; Wil¬
liams, 1965). Third, the necessary prerequisite for
local contrast appears to be a difference in reinforce¬
ment frequency, not response frequency, in adjacent
components (Freeman, 1971a; Nevin 8c Shettleworth,
1966; Williams, 1965). Finally, although the condi¬
tions from which local contrast arises are often iden¬
tical to the conditions which produce overall contrast,
it is clear that the latter is not made up merely of the
SCHEDULE COMPONENT former. Overall positive contrast has been observed
Fig, Performance of one bird oil a 3-play multiple schedule after local contrast has ceased (Boneau 8c Axelrod,
in which a VI 8-mm component alt^n^d With Cither a YI 1965), and local contrast has been observed in a situa¬
2-fniil fit* 3.ii EXT schedule. Response rate in the VI 8-irnn,
tion which produced overall induction (Freeman,
is broken down into six successive 20-sec periods and is averaged
1971a).
over three sessions (10-12). The depression and elevation in the
first 20 sec of YI 8-min are examples of negative and positive
local cQiurswt effects respectively. (Adapted from Nevin &
Shettleworth; 1966. ^ 1966 by th« Society for the Experimental Interaction© In Concurrent Schedules;
Analysis Of Behavior, Inc.)
Relations Between Multiple
and Concurrent Schedules
though tho local effect had disappeared. Catania and
Gill (1964) observed local positive contrast on a mult Concurrent schedules are two or more schedules
FI EXT schedule. However, unlike the Boneau and which are simultaneously in effect, each associated
Axelrod data, these effects persisted for 52 successive with a different response. The prototypic concurrent
daily sessions. Similarly, Arnett (1973) reported local procedure has involved the study of pigeons pecking
contrast effects in a mult VI 3-min EXT schedule for one of two simultaneously illuminated response keys,
up to 65 sessions. with pecks on each key associated with a different
Nevin and Shettleworth (1966) studied a procedure schedule (e.g., Catania, 1966). An alternative pro¬
in which 2*min components of VI 8-min reinforcement cedure involves two keys. Pecks on the operant key
in red were preceded by either a VI 5-mm component produce reinforcement, while pecks on the changeover
(green) or an EXT component (white). Figure 12 pre¬ key change the schedule (and the correlated stimulus)
sents response rates in successive 20=sec segments of the on the operant key (Findley, 1958). A great deal of
VI 8 min component. Local negative contrast was ob¬ research, especially on the quantitative aspects of
served when this component followed VI 2-mm, and schedule control, hag been done with concurrent
local positive contrast was observed when this com¬ schedules, and is discussed in detail by de Villiers
ponent followed EXT. (Chapter 9 of this volume), Catania (1966), and
There has not been a great deal of research on local Herrnstein (1970). Moreover, Herrnstein (1970) has
contrast. It is, therefore, difficult to enumerate with drawn attention to the parallels between phenomena
confidence the conditions necessary to produce it. Observed on multiple and concurrent schedules and
Some generalizations can be stated briefly, however. Suggested that performance on both types of schedules
First, there is good evidence that local contrast effect? reduces to a common explanatory principle. It seems
increase with increase? in the duration of the changed appropriate, therefore, to discuss the relations be¬
component (Staddon, 1969; Wilton 8c Clements, tween these schedules briefly.
1971). These effects parallel the results of studies of The findings obtained in studies of concurrent
trial and ITI duration in autoshaping procedures schedules can be summarized by the word matching.
(e.g., Baldock, 1974; see p. 77, above), if the un¬ Relative rate of responding in component A (rate of
changed component is viewed as a “trial” against the responding in component A /rate in component A
background of the changed component. These studies plus rate in component B) matches relative rate of re¬
found that the rate of responding on autoshaping inforcement in that component (Catania, 1966; Herrn¬
procedures increased with increases m the ITI. stein, 1970). This is the epitome of schedule interac¬
Second, there is evidence that pigeons and rats differ tion. Responding in one component varies inversely
in showing local contrast—at least local positive con¬ with reinforcement frequency in the other component.
Barry Schwartz and Elkan Gamzu 79

If a concurrent VI VI (cone VI VI) procedure is durations, matching is obtained on multiple schedules.

shifted to a cone VI EXT, an increase in responding Indeed, one could view a concurrent procedure as a
in the unchanged VI component occurs (Catania, special case of a multiple procedure, where the sub¬
1969). Similarly, if a cone VI EXT procedure is ject rather than the experimenter controls component
switched to cone VI VI, a decrease in responding in duration. On concurrent procedures, subjects tend to
the unchanged VI component occurs (Catania, 1969). produce short component durations (i.e., change keys
These outcomes are analogous to demonstrations of at a high rate). Hence one observes matching with
positive and negative behavioral contrast in multiple concurrent procedures. Killeen (1972) examined the
schedules. Indeed, the matching law is a general state¬ possibility that multiple and concurrent procedures
ment about schedule interactions which subsumes are intimately related by exposing pigeons to a con¬
positive and negative contrast. current procedure and arranging for switches from
The foregoing is not meant to suggest, however, one component to the other to produce component
that multiple and concurrent schedules always yield changes for other pigeons that were responding on
identical results. First, matching does not usually multiple schedules identical in reinforcement fre¬
occur on multiple schedules. Rather than allocating quency to the concurrent schedules. This enabled
responses in the two components in direct proportion Killeen to compare the control over responding
to reinforcements in the two components, organisms exerted by concurrent schedules and multiple sched¬
tend to undermatch (Reynolds, 1961b). The equation ules which were identical in both reinforcement fre¬
relating relative response rate to relative reinforce¬ quency and component duration. Killeen found that
ment rate has a slope less than 1.0 and a positive inter¬ mult and cone pigeons allocated their responses to the
cept. different components identically, thus strengthening
In a recent quantitative analysis, Herrnstein (1970) the view that multiple and concurrent schedules are
attempted to capture both the similarities and differ¬ not fundamentally different. However, the generality
ences between multiple and concurrent schedules, The of Killeen’s findings has recently been questioned in
quantitative relation he proposed was an experiment by Silberberg and Schrot (1971). They
pointed out that in Killeen’s study, since the concur¬
rent schedule pigeons switched frequently between
rA q_ mr# -j- vQ components, the resulting multiple schedule had short
component durations. Both Herrnstein’s theoretical
account and the data obtained by Shimp and Wheat-
where PA — rate of responding in component A; rA —
ley (1971) and Todorov (1972) suggest that multiple
rate of reinforcement in component A; rB = rate of
schedules and concurrent schedules have similar
reinforcement in component B; To — rate of reinforce¬
effects only when the multiple schedules are short.
ment for responses other than A and B (e.g.; groom¬
Thus Silberberg and Schrot asked whether the simi¬
ing); and m = a constant representing the degree of
larity between mult and cone performances on a Kil¬
interaction between components. In concurrent pro¬
leen-type procedure would persist even when the cone
cedures, m — 1; i.e., interaction is maximal. Matching
pigeons created long mult components. They accom¬
is reflected by the following equation, with m — 1:
plished this by introducing long changeover delays
(CODs). A COD is a contingency which prevents rein¬
Pa _ rA
forcement of a response on one key until some amount
Pa + Pb ~ rA + rB of time has elapsed since the last response on the other
key. Long CODs have been shown to decrease the like¬
In multiple schedules, m approaches 1.0 as component lihood of switching between components in concur¬
duration decreases. rent procedures (Shull 8c Pliskoff, 1971). Thus they
As is apparent from Equation 1, any alternative rein¬ should result in lengthened components for the
forcement for responses other than PA will decrease PA pigeons on mult procedures. Silberberg and Schrot
(contrast). Also apparent is the prediction that as com¬ found that as COD increased, thus increasing com¬
ponent duration in multiple schedules is shortened, ap¬ ponent duration in both cone and mult procedures,
proximations to matching should get closer and closer, differences in response allocation between cone and
since m grows larger and larger. There is substantial mult subjects increased. Thus they argued that since
empirical support for this prediction. Shimp and matching on concurrent procedures is independent of
Wheatley (1971) and Todorov (1972) have shown that component duration, while matching on multiple pro¬
with extremely short (i.e., about 10-sec) component cedures depends upon component duration, there are
80 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

fundamental differences in the control over behavior phenomenon of contrast raised by the Gamzu and
exerted by the two types of procedure. Williams experiments and other studies of autoshap¬
There are other reasons for believing this to be the ing and automaintenance is this: given that stimulus-
case. Despite the gross similarity between responding reinforcer relations control key pecking in the absence
maintained by multiple and concurrent schedules, of response-reinforcer relations, what is their effect
molecular analysis of how matching occurs on concur¬ when response-reinforcer dependencies are also pres¬
rent schedules makes it clear that matching on con¬ ent? To examine this question, let us return to the
current and multiple schedules is different. Rachlin standard sequence of multiple schedules employed to
(1973) has discussed this in detail. On concurrent demonstrate behavioral contrast. One begins typically
schedules, animals distribute the time spent respond¬ with mult VI-VI. In this procedure response-rein forcer
ing in the two components in proportion to the rela¬ dependencies exist in both components. However,
tive reinforcement rate in those components (Baum & there is no differential stimulus-reinforcer relation.
Rachlin, 1969). For example, if reinforcement^ in Food is equally likely in both components of the
component A arc twice as frequent as reinforcements multiple schedule. When the procedure is changed to
in component B, animals will spend twice as much mult VI EXT, the response-r§mforcer dependency
hme responding in component A and make twice as continues in the VI component. Now, however, a
many responses. What this means, however, is that differential stimulus-reinforcer relation is also intro¬
local response rate (responses divided by th§ flHlV duced. The Vi-correlated stimulus predicts food while
available in which to make them) will be equal in the the EXT-correlated stimulus does not. This state of
two components. This cannot be true of multiple affairs is what generated and maintained pecking in
schedules. Since component duration in multiple the Gamzu and Williams experiments. Thus we might
schedules is fixed, increases in responding in com¬ expect two sources of control of pecking to be oper¬
ponent A as a function of decreases in reinforcement ative in the VI component of a mult VI EXT schedule
in component B must result from increases in local (both stimulus-reinforcer and response-reinforcer rela¬
response rate and not from increases in time alloca¬ tions), while only one was operative in the preceding
tion. mult VI VI schedule. By assuming, for simplicity, that
What might be the source of this increase in local the two sources of control interact additively, one
response rate which occurs in multiple schedules? We would expect pecking to increase in the VI component
shall now describe a new theory of behavioral contrast of a mult VI EXT schedule relative to its rate of
which attributes these extra responses to autoshaping- occurrence on a mult VI VI. This, of course, defines
like stimulus-reinforcer relations which are sometimes positive contrast. Thus the additivity theory of con¬
present in multiple schedules, trast is simple: contrast occurs because a differential
stimulus-reinforcer dependency is imposed upon an
An Additivity Ths«ry «f Contrast already existing response-reinforeer dependency, and
the two sources of control combine to increase the
The phenomena of autoshaping and automain¬ rate of key pecking. A number of investigators arrived
tenance have been thoroughly discussed above. The
at roughly this conclusion simultaneously (Boakcs,
available evidence clearly indicates that autoshaping 1973; Gamzu & Schwartz, 1973; Hemmes, 1973; Rach¬
reflects the control of key pecking by Pavlovian, lin, 1973; Staddon, 1972). The theory was first artic¬
stimulusdeinforcer contingencies. One series of experh ulated in its present form by Gamzu and Schwartz
ments, by Gamzu and Williams (1971, 1973), makes (1973) on the basis of an experiment which extended
the parallel between autoshaping and Pavlovian con¬ the findings of Gamzu and Williams (1971, 1973) to
ditioning particularly clear and is especially relevant
procedures employing parameters akin to those em¬
to the theory of contrast we shall propose, Pavlovian ployed on standard multiple schedules. Pigeons were
conditioning depends upon the existence of an in¬ exposed to a multiple schedule with regularly alter¬
formative or differential relation between the CS and nating components signaled by key color. The com¬
the US. Pairing is neither necessary nor sufficient ponents were 27 sec long, and reinforcements were
(Rescorla, 1967). Gamzu and Williams applied this always delivered independently of responses. The pro¬
analysis to autoshaping by showing that unless the re¬
cedures were either differential (mult VT 33-sec EXT)
sponse key was a differential predictor of food or nondifferential (mult VT 33-sec VT 33-sec). As
[P(food/key) > P(food/key)], autoshaping would not Gamzu and Williams found with a discrete-trials
occur and already established key pecking would procedure, pecking was generated and maintained on
cease. The question of primary relevance to the the differential procedure and essentially eliminated
Barry Schwartz and Elkan Gamzu 81

Fig. 13. Responses per minute

in each component of a series of
multiple schedules for Pigeon
85, averaged across blocks of
three sessions. Component sched¬
ules are identified on the
abscissa. (From Gamzu &
Schwartz, 1973. © 1 m by the
Society for the Experimental
BLOCKS or 3 SESSIONS Analysis of Behavior, Inc.)

on the nondifferential procedure. These data are sum¬ sponding is appropriate to the reinforcer—food or
marized in Figure 13. Gamzu and Schwartz argued water pecks with food or water reinforcers (Jenkins &
that it was these responses which summed with those Moore, 1973). When rats are autoshaped to contact
maintained by response-reinforcer dependencies in rather than to press a lever, they contact the lever by
standard multiple schedules to produce contrast. chewing it (Peterson, Acki'l, Frommer, 8c Hearst, 1972;
Stiers & Silberberg, 1974). A second view, which has
been labeled “sign tracking” (Hearst and Jenkins,
Predictions Based Upon the
1974) includes the directedness of the previous view,
Additivity Theory of Contrast
but suggests that within limits, organisms will direct
and Their Confirmation
whatever skeletal activity is possible toward a stimulus
The central component of the additivity theory, which signals food. The form of the response need
stated generally, is that whenever a differential stim¬ not, though it might, bear any clear resemblance to
ulus-reinforcer relation exists, that stimulus will exert the unconditional response to food. It is not clear
control over some class of behavior. The obvious prob¬ which of these views is correct, but they make the
lem which arises is the specification of the class of be¬ same predictions about contrast in standard pigeon
havior which will be controlled. There are at least and rat experiments. In the standard multiple-sched¬
two views on this matter. One view suggests that the ule procedure employed with pigeons, pecking is the
class of behavior which will be controlled by stimulus- measured response (consummatory), and the discrim¬
reinforcer relations is just that class which is appro¬ inative stimuli are located on the response key (sign
priate to the reinforcer—i.e., a class of consummatory tracking). Thus either view of autoshaping would
responses—and that these responses will be directed at predict that stimulus-reinforcer and response-rein-
the signaling stimulus. This view is in keeping with forcer relations will both influence the same behavior
our traditional understanding of Pavlovian condition¬ —key pecking. (The difference between the two views
ing, which is that a conditional stimulus comes to is in whether one underlines “key” or “pecking” in
elicit some component(s) of the unconditional re¬ “key pecking.”)
sponse to the US. The evidence in support of this Now consider the standard multiple-schedule pro¬
view of autoshaping is good, as discussed earlier (cf. cedure employed with rats. Rats press a lever (non-
Moore, 1973). In the pigeon, pecking is a consum¬ consummatory) for food. The discriminative stimuli
matory response (Jenkins 8c Moore, 1973; Staddon 8c are located away from the lever (no sign tracking).
Simmelhag, 1971), and the form of autoshaped re¬ Thus either view would predict that when rats are ex-
82 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

posed to mult VI EXT after mult VI VI (i.e., when a

differential stimulus-reinforcer relation is introduced),
bar pressing will not be enhanced. Indeed, to the ex¬
tent that the stimulus-reinforcer relation is effective,
it will generate some other consummatory behavior,
directed at the signal, which may compete with bar
pressing. This competition would result in negative
induction of bar pressing (reduction in VI rate when
the second component is changed from VI to EXT)
rather than contrast. Thus the additivity theory of
contrast would predict that contrast will not be ob¬ SESSIONS Hz VALUES
tained with bar pressing r&t§ a$ subjects. There are no
features of other theories of contrast already discussed
whi^h would lead one to this prediction. The litera¬
ture suggests that, indeed, with bar-pressing rats, in=
duction rather than contrast is the rule (e.g.f Freeman,
1971b), To summaries! contrast is not expected simply
because rcsponsenreinforcer and stimulus-reinforcer
relations do riot influence the same class of behavior.

If the discriminative stimuli were located on the lever,

then contrast might occur. The stimulus-reinforeer
contingency would generate some form of leyer con¬ SESSIONS Hz VALUES
tact, perhaps biting, which would sum with the lever Fig. 14. Bar presses per minute for two pigeons over the last
presses already being maintained by the response-rem- 7 sessions of mult VI 1-min VI 1-min and 14 sessions of mult
VI l-min EXT (lefthand panels) with results of a generaliza¬
forcer dependency.
tion test around the stimulus associated with EXT (S2) in the
The implication of this theory of contrast is that right-hand panels. Both negative induction and inhibitory
the bulk of demonstrations of positive contrast are the control are demonstrated (From Westbrook, 1973. {^5) 1973 by the
Society for the Experimental Analysis of Behavior. Ine.)
result of a fortuitous procedural convention which (a)
measures as operants consummatory responses and
(b) localizes discriminative stimuli on the response shows that inhibitory control can be established in
key. The theory predicts that if either of these pro¬ such an experimental situation, but that it is not
cedural features is altered, contrast will not appear. sufficient to produce behavioral contrast.
Again, other theories make no such prediction. A A second class of relevant studies employs key peck¬
number of experiments have recently been conducted ing as the operant but removes the discriminative
which put this aspect of the theory to empirical test. stimuli from the response key. Again, the additivity
Westbrook (1973) and Hemmes (1973) exposed theory would predict that the differential stimulus-
pigeons to standard multiple schedules except that reinforcer contingency would control behavior, but
the required operant was treadle hopping in that since the stimulus was off the key the behavior
Hemmes’ experiment and bar pressing in Westbrook’s. would not sum with operant peeks, and hence no con¬
Under these conditions, we would expect the pigeons trast would result. Experiments by Redford and

to behave like rats, The differential stimulus-rein¬ Perkins (1974) and Schwartz (1974a, 1974c, 1975)
forcer relation established during mult VI EXT would confirm this expectation. In Schwartz’s experiments,
not enhance treadle hopping or bar pressing, but pigeons were exposed to a series of multiple schedules,
some other behavior directed at the signaling stim¬ both mult VI VI and mult VI EXT. What varied
ulus. Thus probably negative induction, but certainly from one set of schedules to the next was the location
not positive contrast, should occur. Both studies failed and/or modality of the discriminative stimuli. Figure
to find contrast, and Westbrook’s study demonstrated 15 presents, for a representative pigeon, rates of re¬
large negative induction effects. Some data from West¬ sponding in both components of the series of multiple
brook’s study are presented in Figure 14. An especially schedules. Each exposure to the mult VI EXT sched¬
interesting feature of the Westbrook study is that ule was characterized by a different set of discrimina¬
generalization tests around the stimulus associated tive stimuli, identified in the figure. Contrast was only
with EXT were performed, and inhibitory generaliza¬ observed when the signals for the multiple-schedule
tion gradients were obtained. Thus Westbrook’s study components were on the response key. The data pre-
Barry Schwartz and Elkan Gamzu 83

Fig. 15. Responses per minute

for one pigeon exposed to a
repeated cycle of mult YI VI
(panels 1, %, and 5) followed by
mult VI EXT (panels t, 4, and
6). The location and modality of
the discriminative stimuli were
varied as indicated in each panel,
Contrast was only observed
when the discriminative stimuli
were on the key. (From Schwartz,
1974a.)

sented in the figure were characteristic of all subjects, Initially, the component schedules were VI, VI, and
except that some pigeons evidenced a small contrast EXT. Under these conditions, pigeons pecked at both
effect with tone off as S + . This latter effect has also of the VI stimuli on the signal key. When the pro¬
been reported by Hemmes (1973) and Westbrook cedure was changed to mult VI EXT EXT, Keller
(1973) and may require additional explanatory con¬ observed the following: (1) peeks on the signal key at
cepts. the EXT stimulus which was previously correlated
We shall conclude this section by describing a with VI were substantially reduced; (2) pecks on the
study by Keller (1974) which is a most elegant support signal key at the VI stimulus were substantially in¬
for the additivity theory of contrast. Keller exposed creased; (3) pecks at the operant key during VI were
pigeons to a standard sequence of multiple schedules substantially reduced, i.e., negative induction was
and obtained positive contrast. He then spatially sep¬ observed; (4) despite induction on the operant key, if
arated response-reinforcer and stimulus-reinforcer re¬ operant key pecks and VI signal pecks were summed,
lations. The old response key was always illuminated the uniform result was behavioral contrast. Some of
by the same stimulus, and reinforcement depended these data are presented in Figure 16.
upon pecks on this key (operant key). The stimuli
which signaled components of the multiple schedule
Relations Between an Additivity
were now alternated on a second key (signal key).
Theory of Contrast and
During mult VI VI, responding was maintained as
Existing Theories
normal on the operant key, and there was no respond¬
ing on the signal key. During mult VI EXT, response In the preceding section we outlined a new account
rate on the operant key did not change substantially of behavioral contrast. We shall now consider the
during VI. However, most of the pigeons now started possible relation between this theory and other
pecking at the VI stimulus on the signal key. Re¬ theories of contrast, which focus on the concept of
sponding on the two keys together, if summed, showed inhibition. Logically, the two accounts are not in¬
behavioral contrast. compatible. It is entirely possible that inhibition pro¬
A second study produced more impressive results. duced in some way is a necessary condition for con¬
Pigeons were exposed to three-component multiple trast. The autoshaping theory simply asserts that
schedules of reinforcement. Pecks on the operant key inhibition is not sufficient. As the experiment by West¬
were required for reinforcement, while the multiple- brook (1973), discussed in the last section, clearly
schedule components were signaled on a second key. demonstrates, one can obtain inhibitory stimulus con-
84 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

However, on a mult VI EXT procedure, the EXT

stimulus resembles procedurally a Pavlovian con¬
ditioned inhibitor (Rescorla, 1969b). Thus the stan¬
dard procedural progression from mult VI VI to mult
VI EXT inevitably introduces both excitatory and
inhibitory Pavlovian contingencies.2 The obtained
contrast effect could ostensibly result from either or
both of these contingencies. The mult VI VI pro¬
cedure is the operant analog to the Pavlovian truly
random control (Rescorla, 1967). Neither stimulus is
a differential predictor of food, and there should be
no Pavlovian conditioning.
Support for the possibility that the extinction
stimulus may be a Pavlovian conditioned inhibitor
comes from a study by Jenkins and Boakes (1973).
Pigeons were exposed to three different stimuli on
response keys. One stimulus reliably signaled food.
One stimulus reliably signaled no food. Food delivery
was random with respect to the third stimulus. Auto¬
shaped pecking occurred only to the stimulus which
signaled food. Of interest to the present topic, how¬
ever, is that measured by their position in the chamber
relative to the stimuli, pigeons, preferred the random
stimulus to the stimulus which signaled no food. If
rig. id. Responses per minute during die unchanged component
of a series of three-component multiple schedules. Response the relative aversion to the no-food stimulus in the
rates are segmented Into responses to the Qperant key and Jenkins and Boakes study is taken as an indication of
r£§pOttS6s to the signal key (shaded). The multiple schedules
Pavlovian conditioned inhibition, and if the no-lood
are divided into those with two VI components (regardless of
order of presentation) and those with two EXT components. stimulus is considered the analog of an extinction
The data are averages of data presented ill Eellcr (l97l, Table
stimulus in a multiple schedule, then the Jenkins and
2). In all eassfi th<* overall contrast effect can be completely
Boakes study supports the view that the EXT signal
attributed to increased responding to the signal key.
in a multiple schedule functions as a Pavlovian con
ditioned inhibitor.
trol without obtaining contrast. Contrast requires Having suggested that contrast may depend upon
more than inhibition, i.e., it requires the appropriate the joint action of Pavlovian excitation and inhibition
choice of response and discriminative stimulus by th£ of the same response, let us consider the following
experimenter. Thus the question of whether itthi- experiment. Pigeons are exposed to a mult VI l-min
kitivn is sufficient for contrast has already been an¬ VI l-min schedule, then shifted to a mult VI l-min
swered negatively. The unresolved question is whether VI 5=min (Guttman, 1959; Terrace, 1968). Such a
inhibition is necessary. Current controversies regard¬ procedure results in positive behavioral contrast.
ing inhibitory theories of contrast focus upon the re¬ While it is not unreasonable to suggest that such a
sponse-reinforcer relation. Inhibition produced by procedure shift introduces an excitatory stimulus-rem-
Uoiireiii forced responding results m a rebound in¬
crease in the rate of reinforced responses on one yiew 2 it should be noted that this feature of multiple schedules is
an inherent feature of all Pavlovian conditioning experiments.
(Terrace, 1972). Inhibition of responding in one com¬ If a dilferential positive contingency exists between a CS and a
ponent of a multitude schedule by reinforcement of US, then a differential negative contingency must exist between
the absence of the CS and the US, That this relation may some¬
responding in the other component is eliminated in a
times be crucial is discussed in Seligman's (1969) criticism of
shift from mult VI VI to mult VI EXT, thus increas¬ Rescorla’s truly random control procedure. This fact is a con¬
ing unchanged VI response rate on the other view sequence of the modern view of Pavlovian conditioning as being

(Catania; 1969; Reynolds, 1961a). The additivity dependent on more than the mere pairing of stimuli (Rescorla,
1967). Indeed, Pavlov (1927) expected that in standard experi¬
theory of contrast suggests a different approach to mental procedures background stimuli would become weak
inhibition—one which focuses on stimulus-reinforcer conditioned excitatory stimuli by virtue of being sometimes
relations. It is an excitatory stimulus-reinforcer rela¬ paired with a US. On the modern view, background stimuli
would not be expected to be excitatory, and might even be
tion which produces contrast according to the theory. inhibitory.
Barry Schwartz and Elkan Gamzu 85

forcer relation in the unchanged VI 1-min component, There is one final issue to be discussed with regard
can one argue that the change from VI 1-min to VI to the additivity theory of contrast. How does such a
5-min in the other component results in an inhibitory theory account for negative contrast? As mentioned
stimulus-reinforcer relation? Neither of the two recent above, negative contrast refers to a decrease in re¬
reviews of research on conditioned inhibition (Hearst, sponse rate as response rate in the other component
Besley, Sc Farthing, 1970; Rescorla, 1969b) present any increases. In studies of multiple schedules, negative
relevant data from the Pavlovian conditioning litera¬ contrast is not obtained as reliably, or in as great a
ture. One piece of experimental evidence which is magnitude, as positive contrast. For example, Terrace
available suggests that inhibition is not produced by (1968) shifted pigeons from mult VI 5-min VI 5-min
such a procedure. In the Gamzu and Schwartz (1973) to mult VI 1-min VI 5-min and obtained little nega¬
experiment discussed above, one sequence of pro¬ tive contrast, a finding which he interpreted as sup¬
cedures involved a shift from mult VT 33-sec VT port for his view that contrast results from the frustra¬
33-sec to mult VT 33-sec VT 100-sec. Response rate tion of nonrewarded responses, or, as Bloomfield has
increased dramatically in the unchanged VT 33-sec described it, from a worsening of conditions. There
component—presumably due to excitatory Pavlovian are, however, demonstrations of local negative contrast
conditioning. However, response rate also increased in both rats (Bernheim & Williams, 1967) and pigeons
dramatically in the VT 100-sec component relative to (Nevin 8c Shettleworth, 1966). An additivity theory of
prior rate when the schedule in that component was negative contrast encounters the same problem dis¬
VT 33-sec. This increase in rate is incompatible with cussed above with regard to conditioned inhibition.
the idea that such changes in procedure might pro¬ Multiple VI 5-min VI 5-min schedules presumably re¬
duce inhibition. On the other hand, a study by Weis- sult in no Pavlovian conditioning. The shift to mult
man (1969) suggests that the stimulus signaling re¬ VI 5-min VI 1-min must make the VI 5-min stimulus a
duced reinforcement may be inhibitory. Weisman conditioned inhibitor in order to account for negative
shifted pigeons from mult VI 1-min VI 1-min to mult contrast. As we have seen, such an argument is prob¬
VI 1-min VI 5-min and observed positive behavioral lematic. An alternative possibility is that negative and
contrast in the unchanged VI component. Concur¬ positive contrast are causally unrelated—that despite
rently, he observed gradients of inhibition about the their symmetrical appearance, they are produced by
stimulus correlated with the VI 5-min schedule. Also, different variables. Bernheim and Williams (1967)
a study by Rilling, Askew, Ahlskog, and Kramer demonstrated that positive and negative contrast are
(1969) has shown that pigeons will peck a key to dissociable: they appear to occur independently of
escape from the stimulus correlated with VI 5-min on one another. Some rats in their study showed one type
a mult VI 30-sec VI 5-min procedure. Thus at present and some the other, and the same rats showed each
the question of whether Pavlovian inhibition is neces¬ effect at different points in the experiment. One way
sary for contrast to occur must be left unresolved.3 to investigate this possibility more systematically is
to assess whether manipulation of the variables which
3 We have been using the term inhibition somewhat loosely influence positive contrast (e.g., location of discrimina¬
in this section, and we shall not attempt a rigorous definition of
the term. Reviews by Rescorla (1969) and Hearst, Besley, and tive stimuli or required response) have similar effects
Farthing (1970) have dealt extensively with the problem of on negative contrast. Schwartz (1975) has recently
establishing defining criteria for the presence of inhibition. We
done such an experiment. Pigeons were exposed to a
have cited evidence from Jenkins and Boakes (1973) that pigeons
withdraw from an S-, and from Rilling et al. (1969) that mult VI 3-min VI 3-min schedule with the com¬
pigeons peck a key to escape from a stimulus correlated with a ponents signaled by green and white key color and
relatively low density of reinforcement as instances of inhibitory
control. Technically, they are not. Rather, they are demonstra¬ then shifted to a mult VI 3-min VI 72-sec schedule.
tions of the relative aversiveness of these stimuli. The relation Absolute rates of responding in both components are
between these indications of aversiveness and inhibition is an
presented for one pigeon in Figure 17. It can be seen
open question. For the present purposes, however, the concepts
of inhibition and aversion are functionally equivalent: they both from the second panel that negative contrast occurred.
imply a reduction in responding. There is still a problem, how¬ The pigeons were then returned to the mult VI 3-min
ever. How can an S— reduce responding which is already at
zero, which is the level of Pavlovian responses we would expect
VI 3-min schedule. Now, however, the operant key
to be present in a mult VI VI with equal-density components? A was always blue. A second key, the signal key, alter¬
possible solution to this problem may arise from an evaluation nated between green and white. Recall that with pro-
of the stimulus-reinforcer relations which exist in the chamber
from a broader perspective. While it is true that neither mult
stimulus is a better predictor of food than the other, it is also logical truth is also a psychological one, we would expect both
true that both stimuli are differential predictors of food in mult stimuli to be slightly excitatory, in which case an inhibitory
contrast with the stimuli present outside the chamber. If this operation could be expected to reduce responding.
86 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

Fig. 17. Responses per minute

for one pigeon exposed to a
series of mult VI 3-min VI 3-
min schedules alternating with
mult VI 3-min VI 12-sec sched¬
ules. In panels 1 and 2 the
discriminative stimuli were dis¬
played successively on the same
key. In panels 3 and 4 two keys
were used. One key was con¬
stantly illuminated blue and
pecks at it could be reinforced
by the prevailing VI schedule,
the value of which was signaled
by either green or white illumi¬
nation of a second key. Pecking
at the second key was recorded
but had no experimental effect.
Responses per minute to the
white signal key are shown at
the bottom of the last panel.
(From Schwartz, 1975. © 1975
by the Society for the Experi¬
mental Analysis of Behavior,
Inc.)

cedures designed for assessing positive contrast, no con¬ Additivity Theory and Local Contrast
trast effect appears on the operant key (Keller, 1974).
Rather^ peclcg are directed at fch§ signal k^y. The A prominent feature of demonstrations of contrast

fourth panel of Figure I? indicates that negative con¬ is its time dependence or local character. Much of this
trast does in fact appear on the operant key with this literature has already been reviewed above. Let us
procedure. Incidentally, substantial signal-key peek¬ simply summarize by asserting that the magnitude of
ing occurred during the changed component (VI 7^- contrast decreases with tiriie after th£ preceding com¬

gec)i This is represented at the bottom of the fourth ponent ends and increases as a function of its dura¬

panel of Figure 17. Data from all pigeons in this ex¬ tion.

periment have already been presented earlier (see It is possible that the distinction between local and
Figure 11). overall contrast effects may help reconcile those con-
Thug both logic arid preliminary evidence suggest trast phenomena which are consistent with the ad¬
that the additiYity theory of contrast may not be ap¬ ditivity theory with those which are not (see de
plicable to negative contrast. This conclusion is simi¬ Villiers, Chapter 9 of this volume). The consistent
lar to the one suggested by studies of contrast in very phenomena have already been discussed in detail.
different experimental contexts. Studies of rats in run¬ Some of the inconsistencies are (1) occasional demon¬
ways and mazes, typically involving variations in re¬ strations of contrast with discriminative stimuli lo¬
ward magnitude rather than frequency, have occasion¬ cated off the key (Hemmes, 1973; Schwartz, 1974a,

ally provided evidence for both positive and negative 1974c; Westbrook, 1973); (2) occasional demonstra¬
contrast (Crespi, 1944). However, the most common tions of contrast in bar-pressing rats (Gutman & Sut-

finding in such studies is clear negative contrast but terer, 1974; Pear 8c Wilkie, 1971); (3) demonstration
no positive contrast (e.g., Bower, 1961; Glass & Ison, of contrast in rats with procedures employing only
1966; Spear It Hill, 1965; see Dunham, 1968, for a re¬ aversive stimuli (de Villiers, 1972, 1974); and (4) lack
view of this literature). Both the lack of positive con¬ of contrast in some errorless discrimination procedures
trast and the presence of negative contrast in these (Terrace, 1966). Rachlm has provided evidence which

studies is consistent with the additivity theory, since suggests that the contribution of a stimulus-reinforcer
it suggests that (a) positive contrast is unlikely in contingency to contrast may be largely restricted to
most of the traditional situations to which rats are periods just after a change in multiple-schedule com¬
exposed and (b) positive and negative contrast may be ponents—i.e., that the stimulus-reinforcer effect may
fundamentally different phenomena. be local. Pigeons were exposed to mult VI 2-min VI
Barry Schwartz and Elkan Gamzu 87

Fig. 18. Rates of responding of

two pigeons in successive inter¬
vals of 8-min components of a
mult VI 2-min VI 2-min plus VT
15-sec. The duration of each in¬
terval increases logarithmically
along the abscissa. For instance,
the first points on the abscissa
show rates of responding during
the first 8 seconds; the last 8
points show rates during the
—| 248th to 480th seconds of the
480 component. (From Rachlin,
SUCCESSIVE INTERVALS IN § MIN COMPONENT (SECONDS LOG Scale) 1978.)

2-min schedules. In one o£ the components additional crease in overall response rate (no contrast). The
reinforcements were delivered independent of re¬ shorter the component, the larger the contribution
sponding on a variable time (VT) 15-sec schedule. Let made by the local contrast effect to overall rate-
us consider the expected effect of these free rewards Boakesj Halliday, and Poli (1975) have supported
from the additivity theory of contrast. The free rein¬ Rachlin’s findings. They studied a procedure very
forcements would create a differential stimulus-rein¬ similar to Rachlin s, but with 2-min component dura¬
forcer contingency, which would generate additional tions. They found increases in responding when free
pecks in that component. Thus response rate should reinforcements were added to a component of a mul¬
be higher when free rewards are presented than when tiple schedule under these conditions.
they are not presented. Rachlin observed that when Another study which suggests that the stimulus-
the components were 8 sec long, response rate was reinforcer relation contributes mainly to local con¬
higher in the component which included free rein¬ trast was done by Schwartz, Hamilton, and Silberberg
forcements. However, when the components were 8 (1975). Schwartz et al. exposed pigeons to a sequence
min long, this was no longer true. of multiple schedules in which the signal for the com¬
Figure 18 presents response rates in successive sub¬ ponents was located on a second key (see Keller, 1974).
components of the 8-min multiple procedure. Early in In addition, they recorded the duration of key pecks
each component containing free reinforcement re¬ on both the operant key and the signal key. The
sponding was higher than in the other component. reader will recall that Schwartz and Williams (1972b)
Later in each component containing free reinforce¬ and Gamzu (1971) have obtained evidence which sug¬
ment responding was lower. Rachlin’s study suggests gests that pecks controlled by the response-reinforcer
that the main effect of the stimulus-reinforcer con¬ relation are long-duration pecks, while pecks con¬
tingency may be confined to the part of a multiple- trolled by the stimulus-reinforcer relation are short-
schedule component which borders the. other compo¬ duration pecks. If the Keller procedure spatially
nent, i.e., the stimulus-reinforcer relation may be separates behavior controlled by the two different con¬
responsible primarily for local contrast. If components tingencies, one would expect response durations on
are very long, the local effect will be averaged across the signal key to be substantially shorter than response
the entire component and represent only a small in- durations on the operant key. On the mult VI EXT
88 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

procedure, Schwartz et al. obtained evidence that this conditioning processes. Thus it is appropriate to
was the case. Median response durations on the op¬ examine some of the research explicitly designed to
erant key were about 50 msec, while median dura¬ assess such process interactions in order to shed more
tions on the signal key were about 20 msec. Of greater light on what have been termed schedule interactions.
relevance to the present discussion, however, was the Some of these studies are reviewed in the next section.
finding that most signal-key responses occurred dur¬ In concluding our discussion of multiple-schedule in¬
ing the first 10 sec of each 120-sec VI component. The teractions, it is important to emphasize that we view
four pigeons emitted 14, 32, 59, and 77% of their behavioral contrast merely as an example of how
signal-key responses during the first 10 sec of the VI Pavlovian relations may control operant behavior.
component. If there were no local effect, 8.5% of We have devoted a great deal of space to this example,
responses would have occurred in the first 10 sec. but we expect that once investigators start looking for
These data support both the view that the stimulus- other examples the interactions discussed here will
reinforcer contingency exerts its main effect on local turn out to be pervasive.
contrast and the view that there are two distinct
classes of pecks which may be separated on the basis
of duration (Schwartz 8c Williams, 1972b), If the ad¬ Interactions Between Pavlovian and
Operant Conditioning and the
ditivity theory of contrast offered here accounts
mainly for local contrast, then 6115 might 65£p5Ct 16 Additivity Theory of Contrast

see contrast (though of lesser magnitude) in some The interaction between Pavlovian and operant
situations to which the additivity theory does not conditioning processes has been a traditional concern
apply, since not all contract i§ local. Unfortunately,
in the study of animal learning, especially with regard
since only a small proportion of the studies which to avoidance learning (Herrnstem, 1969; Rescorla &
demonstrate behavioral contrast provide the evi¬ Solomon, 1967). Most of the research which has been
dence necessary for assessing local contrast, it is not
done on this problem has been done in avoidance
possible to say anything yery definitive on this mat¬ paradigms and is thus not the proper concern of this
ter. For example, in studies which offer clear support discussion. The research of interest here involves the
far fch£ additivity theory (e g-, Keller, 1974,' Schwart2,
superimposition of free food reinforcement (either
1975) there are no data availably 9n y/hether the signaled or unsignaled) on a base line of operant re¬
observed contrast effects were local. As we mentioned sponding which is itself maintained by food reinforce¬
above, local positive contrast is almost never reported ment.
with rats a§ subjects, although overall contrast occa¬
sionally is, This one bit of confirming evidence is a
Positive Conditioned Suppression
slender reed on which to support a theory, however,
and until more experimental analysis has been car¬ What might one expect to be the effects of a signal
ried out the resolution of this matter will have to wait. for food delivery on operant responding? The litera¬
ture on Pavlovian-Operant interactions suggests that

Pavlovian excitation will result in an increase in re¬

Additivity Theory ot
sponse rate (Rescorla & Solomon, 1967), a phenome¬
Contrast. Conclusions
non we shall call conditioned enhancement. It is the
We have presented a theory of contrast which re¬ Pavlovian alimentary analog of conditioned suppres¬
lates it to autoshaping and automaintenance and sion (Estes 2c Skinner, 1941). What might one expect
have discussed the findings which leftd support to the on the basis of the additivity theory of contrast out¬

theory. Much of the evidence one would like m or¬ lined in this chapter? Its predictions are more equiv¬

der to evaluate the theory is not presently available. ocal. A stimulus-reinforcer contingency will result in
However, there is enough confirming evidence to in¬ excitation, but whether this excitation is reflected in
dicate that stimulus-reinforcer relations play a sig¬ an increase in operant responding will depend upon
nificant role in the phenomenon of behavioral con¬ (a) the spatial relation between the stimulus and the
trast. This being the case, it becomes clear that the target for operant responses; (b) the nature of the
phenomenon of behavioral contrast has been im¬ operant; and (c) the relation between the Pavlovian

properly labeled a “schedule interaction” (Reynolds, US and the operant reinforcement. If, for example,
1961b). Behavioral contrast reflects not so much sched¬ the operant is key pecking and the CS is located on
ule interaction as it does process interaction. The two the key, we would expect to observe conditioned
processes which interact are Pavlovian and operant enhancement. If, on the other hand, the operant is
Barry Schwartz and Elkan Gamzu 89

treadle hopping and the signal is an illuminated re¬ which is incompatible with the operant. Neither
sponse key, we would expect to observe decreases in study, however, attempted to measure the putative
treadle hopping (called hereafter positive conditioned respondent. A more recent study by Kelly (1973) did
suppression to distinguish it from the more familiar attempt to measure respondent behavior. The lever
demonstration of suppression by a CS which signals pressing of monkeys was maintained on a VI schedule,
electric shock). It is interesting that the existing litera¬ and occasional pairings of tone and food were super¬
ture has precisely the equivocal character which the imposed on the operant procedure. Conditioned
present account would predict. For example, one suppression of bar pressing by the tone rapidly devel¬
recent experiment examined the two hypothetical oped and was sustained over 275 sessions. Concurrent
situations just described, and indeed found condi¬ measurement of heart rate and blood pressure gave no
tioned enhancement of key pecking and conditioned indication that either of these autonomic systems was
suppression of treadle hopping (LoLordo, McMillan, influenced by the Pavlovian contingency. Thus just
& Riley, 1974). This study is described in detail be¬ as Azrin and Hake failed to observe skeletal behaviors
low. In brief, the present theory of contrast would which might compete with lever pressing, Kelley
predict conditioned enhancement of responding by a failed to observe potentially competing autonomic
Pavlovian contingency under just those conditions behaviors.
which would also yield positive contrast with the ap¬ There are a number of demonstrations that the
propriate experimental manipulations. Those situa¬ duration of the CS influences its effects. Henton and
tions which yield negative induction would be ex¬ Brady (1970), Meltzer and Brahlek (1970), and Miczek
pected to result in positive conditioned suppression and Grossman (1971) studied the effects of CS dura¬
when Pavlovian contingencies are applied to operant tion on the positive conditioned suppression effect.
base lines. This formulation is much like one ad¬ Henton and Brady superimposed Pavlovian condi¬
vanced by Staddon (1972) in somewhat less detail. tioning trials on responding maintained on a DRL
The available evidence tends to support these pre¬ 30-sec schedule in squirrel monkeys. At CS durations
dictions. Azrin and Hake (1969) studied the effects of of 20 and 40 sec they observed no effect. At CS dura¬
Pavlovian contingencies on bar pressing in rats. Press¬ tions of 80 sec they observed conditioned enhance¬
ing was maintained by either food or water, and the ment, with most of die responding occurring early in
Pavlovian CS (either a 6-per-sec relay click or a 10-per- the CS. Meltzer and Brahlek exposed rats whose bar
sec blinking light) signaled either food, water, or pressing was maintained on a VI 2-min schedule of
intercranial stimulation (ICS). In cases where food reinforcement (Noyes pellets) to pairings of a CS fol¬
and water were USs? they were five times as large as lowed by access to a sucrose solutiom When the CS
the food and water operant rewards. In all there were was either 12 or 40 sec long they observed conditioned
five groups of rats: three groups bar-pressed for food suppression. However, when the CS was 120 sec long
and received either food, water, or ICS as a Pav¬ they observed about a 5% enhancement of bar press¬
lovian US; two groups bar-pressed for water and re¬ ing. Finally, Miczek and Grossman exposed squirrel
ceived either food or water as a Pavlovian US. All monkeys to a similar procedure, in which the Pav¬
groups but one showed substantial suppression of bar lovian and operant rewards differed. With a 30-sec CS
pressing in the presence of the CS. The water-water they observed large suppression effects. With 1-, 2-, or
group displayed conditioned enhancement. With the 3-min CSs, suppression still occurred, but substantially
exception of this group, Azrin and Hake’s results con¬ less. How do these effects of CS duration—i.e., the
firm expectations from the additivity theory of con¬ longer the CS, the less the suppression—relate to the
trast. However, the authors made a point of noting additivity theory of contrast? According to the addi¬
that no behaviors were observed during the CS which tivity theory, positive conditioned suppression results
might have competed with bar pressing. This observa¬ from Pavlovian conditioning of a response which com¬
tion presents something of a problem, since we might petes with the operant. Thus the less the suppression,
expect orientation toward and perhaps contact of the the weaker the competing response. Seen in this light,
CS to occur and to mediate the suppression effect. Van the CS duration effects merely reflect weaker Pav¬
Dyne (1971) observed similar suppression effects in a lovian conditioning with longer CSs. If the Pavlovian
study of bar pressing in rats much like the Azrin and conditioned response were compatible with the op¬
Hake study. Both Azrin and Hake and Van Dyne erant (e.g., key pecking in pigeons), one would expect
offered interpretations of their results which were increases in CS duration to produce exactly the oppo¬
similar to the present account. The Pavlovian con¬ site effects. At long CS durations Pavlovian condition¬
tingency results in the conditioning of a respondent ing would be weakened and little or no increase in
90 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

pecking would occur. A recent study by Smith (1974) studies by Redford and Perkins (1974) and Schwartz
provides direct support for this argument. A 5-sec CS (1974a, 1974c, 1975), which showed that behavioral
on the key enhanced responding, while 30- and 60- contrast in multiple schedules only occurred when the
second CSs either had no effect or reduced responding. discriminative stimuli were on the response key.
There is also indirect evidence which supports this LoLordo et al. also exposed two other groups of
view. First, it has been demonstrated that in auto¬ pigeons to the same procedures, except that these
shaping situations, increases in trial duration result in pigeons were treadle-hopping for food rather than
decreases in responding (Ricci, 1973). Second, Rach- key-pecking. In this case, the response-key CS sup¬
lin’s (1973) study of the effects of component duration pressed the treadle hopping and autoshaped key
on multiple-schedule performance makes a similar pecking! This study is of course analogous to the
point. Recall that in Rachlin’s study free reinforce¬ demonstrations by Hemmes (1973) and Westbrook
ments were delivered m one component of a multiple (1973) that induction rather than contrast occurs with
schedule, both components of which were correlated pigeons on multiple schedules if treadle hopping is
with identical VI schedules of response-dependent the required operant. Further supportive evidence
reinforcement. Rachlin found that when the compo¬ comes from the study by Boakes, Halliday, and Poli
nent duration was short (8 sec) free reinforcement in¬ (1975) described earlier. Pigeons were exposed to a
creased overall response rate, while when component mult VI 2-min VI 2-min schedule. When free rein¬
duration was long (8 min) freg reinforcement de¬ forcement was delivered on a VT 30-sec schedule in
creased overall response rate. If one views the stimuli one component only, responding increased in that
correlated with the components of the multiple sched¬ component. When it was delivered in both compo¬
ule as being Pavlovian CSs as well (as the additivity nents, there was no consistent effect on responding.
theory of contrast suggests), then Rachlin’s study can When bar-pressing rats were exposed to virtually
be interpreted as showing facilitation of pecking With identical procedures, suppression resulted from the
a short G5, but not with a long CS. differential presentation of free food. One other study
consistent with those just described was conducted by
Schwartz (1976). Pigeons were exposed to a VI 2-min
Conditioned Enhancement schedule of reinforcement for key pecking. Period¬
ically, the response key changed color for 12 sec after
LoLordo (1971) exposed pigeons to a VI 2-min
which response-independent food was delivered. Rate
§drsd\ds with 4-§§c access to grain as the mnk>rc§r,
o£ packing in the presence of this stimulus was almost
Superimposed on this were presentations of 20*sec twice that in its absence. In another procedure the
CSs. The CS+ was terminated in food delivery, with
response-independent food was signaled on a second
ths duration u£ reinforcement varied from 2 to 8 sec. key while the VI key remained illuminated. Under
The second GS, the GS“j was correlated with no food. these conditions, VI responding m the presence of the
Clear enhancement of pecking developed to the CSi- food signal wa§ almost completely suppressed. Mean-
as a direct function of reinforcement duration. No whiki substantial responding was maintained on the
dear evident of suppression of peckmg developed to food signal key. The results from these two procedures
the GS . What makes this study different from the illustrate the difference between conditioned enhance¬
ones described in the preceding section is that the ment studies with pigeons and rats. In rat studies,
Pavlovian contingency and the Operaftt contingency where the signal is located away from the bar, suppres¬
both influenced the same class of behavior—key peck¬ sion is observed. Schwartz similarly observed suppres¬
ing. These are just the conditions under which con¬ sion in the pigeon when the signal was off the key. On
trast is expected by the additivity theory, and Lo* the other hand, when the signal is on the key, en¬
Lordo's data offer strong support for that theory. hancement is observed. No exactly comparable pro¬
In a second, larger study, LoLordo, McMillan, and cedures have been studied with the rat. An approx¬
Riley (1974) provided more complete support for the imation is provided by one final recent study which
additivity theory. Pigeons whose key pecking was offers support for the additivity theory of contrast.
maintained on a DRL schedule were exposed to pair¬ Christoph, Peterson, Karpicke, and Hearst (1973)
ings of CSs and food. For some pigeons the CSs were studied rats as subjects and imposed Pavlovian condi¬
located on the response key; for other pigeons the CSs tioning trials on an operant base line and varied the
were tones. Only the first group showed conditioned location of the CS relative to the bar. When the CS
enhancement of pecking. The second group showed was near the bar, they observed almost no suppression.
no clear effects. This study is exactly analogous to the When the CS was far from the bar, substantial sup-
 Barry Schwartz and Elkan Gamzu 91

pression of bar pressing developed, accompanied by man 8c Hager, 1972; Shettleworth, 1972a; Staddon 8c
approach and contact of the CS. Simmelhag, 1971) have challenged the sometimes tacit
and sometimes explicit assumption that general laws
of learning exist and that they can be discovered by
CONCLUSION
exploring the control of behavior in arbitrary experi¬
mental situations. Seligman (1970) has referred to this
as the “equipotentiality premise.” A better label for it
The research discussed in this chapter suggests the
might be the “assumption of interchangeability.”
following conclusions:
Whatever its label, what it conveys is the view that if
the world of a particular species is partitioned into
1. Autoshaping of the pigeon’s key peck is the result three sets—stimuli, behaviors, and rewards—then any
of Pavlovian stimulus-reinforcer contingencies. member of a set can be substituted for any other
2. The maintenance of responding in autoshaping member without materially altering the relations one
procedures is partly determined by operant re¬ observes among sets. This view is now demonstrably
sponse-reinforcer contingencies. false. There is substantial evidence that some associ¬
3. In standard operant procedures, stimulus-reinforcer ations are more rapidly acquired than others—that
contingencies, when present, will exert an influence some members of the stimulus set, the behavior set,
on behavior which is consistent with Pavlovian and the reward set “belong” together (Bolles, 1970;
theory. Garcia 8c Koelling, 1966; cf. Seligman 8c Hager, 1972;
4. In multiple-schedule procedures, analysis of the Schwartz, 1974b). Indeed, a good deal of the research
interaction between Pavlovian and operant con¬ discussed in this chapter violates the assumption of
tingencies allows one to predict which species in interchangeability. It seems undeniable that there are
which situation will show behavioral contrast. The significant biological constraints on what organisms
substantial differences in the literature between rats can learn, as the ethologists have always argued
and pigeons are consistent with this analysis. (Lorenz, 1965). The problem which now confronts the
5. In procedures in which Pavlovian contingencies study of learning is the determination of what modi¬
are explicitly superimposed on operant contingen¬ fications in both method and theory are demanded by
cies, the present account rationalizes a literature these biological constraints on learning. One possibil¬
which has been marked by inconsistent experi¬ ity is that the search for general laws must be aban¬
mental outcomes as a function of the species and
doned and a “botanizing” strategy adopted. It was
the situation under investigation.
Skinner’s explicit rejection of this approach (1938)
which led to the development of current experimental
In addition, the ideas put forth in this chapter methods. A second possibility is that one should
speak to two broader issues. The first is the distinction search for different types of laws which take account
between Pavlovian and operant conditioning. This of biological constraints on learning (Seligman, 1970).
issue has been discussed in various parts of the chap¬ The research discussed in this chapter suggests that
ter. We have no solution to offer to the problem of at least in some cases neither botanizing nor the de¬
classifying learning phenomena as one or the other velopment of new laws is necessary. The autoshaping
type of conditioning. Rather, the analysis presented literature is unequivocal evidence that the key peck
in this chapter represents a somewhat different ap¬ is a special, biologically relevant behavior. In many
proach. The approach is to choose a particular class areas, the study of key pecking in pigeons, bar press¬
of behavior, and rather than classify it as Pavlovian ing in pigeons, and bar pressing in rats yield different
or operant, analyze the extent to which both types of results. Nevertheless these differences can be inter¬
contingency contribute to its occurrence. With regard preted and understood in terms of learning principles
to the class of behavior under scrutiny here—key peck- which are already well established. They require no
ing-this approach has proven fruitful. Moreover, the new formulation—only new combinations of old
analysis can easily be extended to the behavior of formulations. Whether similar analyses will provide
other species in other situations. sufficient explanations of other biologically con¬
The second major issue addressed by this chapter strained phenomena is an open question. For the
is the biological boundaries or constraints on learn¬ present, it would be ill advised to give up the “gen¬
ing. A number of major recent contributions to the eral principles” which guide our investigations with¬
field (Bolles, 1970; Hinde 8c Hinde, 1973; Rozin 8c out a struggle. They may be far more general than
Kalat, 1971; Schwartz, 1974b; Seligman, 1970; Selig- anyone could reasonably have suspected.
92 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

REFERENCES a variable-interval baseline. Journal of the Experimental

Analysis of Behavior, 1975, 23, 177-191.
Bolles, R. C. Species-specific defense reactions and avoid¬
Adler, N., & Hogan, J. A. Classical conditioning and pun¬
ance learning. Psychological Review, 1970, 77, 32-48.
ishment of an instinctive response in Betta splendens.
Boneau, C. A., 8c Axelrod, S. Work decrement and reminis¬
Animal Behavior, 1963, 11, 351-354.
cence in pigeon operant responding. Journal of Ex¬
Allaway, T. A. Attention, information and auto-shaping.
perimental Psychology, 1962, 64, 352-354.
Unpublished doctoral dissertation, University of Penn¬
Bower, G. H. A contrast effect in differential conditioning.
sylvania, 1971.
Journal of Experimental Psychology, 1961, 62, 196-199.
Allen, D., Capehart, S., & Herbert, S. Spence’s theory of
Breland, K., 8c Breland, M. The misbehavior of organisms.
discrimination: A reevaluation. Psychological Record,
American Psychologist, 1961, 16, 681-684.
1969, 19, 443-456.
Brethower, D. M., 8c Reynolds, G. S. A facilitative effect
Amsel, A. Frustrative nonreward in partial reinforcement
of punishment on unpunished behavior. Journal of the
and discrimination learning. Psychological Review, 1962,
Experimental Analysis of Behavior, 1962, 5, 191-199.
69, 306-328.
Brown, P. L., & Jenkins, H. M. Auto-shaping of the
v\rnett, F. B. A local-rateof-response and interresponse¬
pigeon’s key-peck. Journal of the Experimental Analysis
time analysis of behavioral contrast. Journal of the Ex¬
of Behavior, 1968, 11, 1-8.
perimental Analysis of Behavior, 1973, 20, 489-498.
Catania, A. C. Concurrent operants. In W. K. Honig (Ed.),
Azrin, N. H., & Hare, D. F. Positive conditioned suppres¬
Operant behavior: Areas of research and application.
sion! Conditioned suppression using positive reinforcers
Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966.
as the unconditioned stimuli. Journal of the Experi¬
Catania, A. C. (Ed.). Contemporary research in operant
mental Analysis of Behavior, 1969,12, 167-173.
behavior. Glenview, Ill.! Scott, Foreman, 1968.
Baldock, M. D. Trial and intertrial interval durations in
Catania, A. C. Concurrent performances; Inhibition of one
the acquisition of autoshaped lcey peeking. Paper pre¬
response by reinforcement of another. Journal of the
sented at the meetings of the Eastern Psychological
Experimental Analysis of Behavior, 1969,12, 731-744.
Association, Philadelphia, 1974.
Catania, A. C., & Gill, C. A. Inhibition arid behavioral
Barrera, F. J. Centrifugal selection of signal directed peck*
contrast. Psychonomic Science, 1964, 1, 257=258.
ing. Journal of the Experimental Analysis of Behavior,
GiiRisTorn, G., Peterson, G., Karficke, J., & Hearst, E.
1374, 22, 341-355.
Positive conditioned suppression as a function of GS
Bovm, W, M,j k Rogiilin, H, G« Choice aa time allocation.
location. Paper presented at the meetings of the Pay-
foztrnq] gf Experimental Anelym of Behmwry
vhonomic Society, St. Louis, 1973.
861-894.
Coates, J.The differential effects of punishment and ex¬
BEHMHEIm, J. Wv k WiLLSAMSj, D. 1C. 1 iriiO-dOpOftdOnt COfi-
tinction on behavioral contrast. Psychonomic Science,
frasr effects in a multiple schedule of food reinforce¬
1972, 27, 146-148.
ment. Journal of the Experimental Analysis of Be¬
Cook, L„ 8c Davidson, A. B. Effects of behaviorally active
havior, 1967, 10, 243-249.
drugs in a conflict-punishment procedure in rats. In
Bilbreyj Ji| & WmoiiUR, 8i Controls for the constrainta on
S. Garrattina, E. Mussini, 8c L. O. Randall (Eds.), The
Rttto-ahapingi Journal of the Experimental Analysts of
benzodiazepines, Hew York: Rayen Press, 1973.
Behavior ^ 1073,2o) m-
Cooiq L.? 8c Sepinwall; J. Parameters of Emotion: Psy-
Binbua, D. A urulied account of classical conditioning and
chopharmacological parameters and methods. In L. Levi
operant training. In A. H. Black & W. F. Prokasy (Eds.),
(Ed.), Emotions: The parameters and measurement. New
Classical conditioning II: Current theory and research.
York; Raven Press, 1974.
Englewood Cliffs; NJo Prentice-Hall, Ine«, 1972.
Bloomfieldj T. Af. Behavioral contrast and relative rein¬
Craig, W. Observations on doves learning to drink. Journal
of Animal Behavior, 1912? 2, 273—279.
forcement frequency in two multiple schedules. JOlimdl
Crespi, L. P. Amount of reinforcement and level of per¬
of the Experimental Analysis of Behavior, 1967, 10,
formance. Psychological Review, 1944, 31, 344-358.
151-158.
Gruze, W. W. Maturation and learning in chicks. Journal
Bloomfield, T. M. Behavioral contrast and the peak shift.
of Comparative Psychology, 1935, 19) 371-409.
In R, M. Gilbert & N. S. Sutherland (Eds.), Animal d?V- Davidson, A. B., 8c Cook, L. Effects of combined treatment
crtmination learning, Hew Vork: Academic Press, 1969.
with tri-flubperazine-HCl and amobarbital on punished
BloomfIELd, T. M. Reinforcement schedules: Contingency
behavior in rats. Psychopharmacologia (Berlin), 1969, 13,
or continguity? In R. M. Gilbert & J. R. Millenson 159-168.
(Eds.), Reinforcement: Behavioral analyses. New York: de Villiers, P. A. Reinforcement and response rate inter¬
Academic Press, 1972. action in multiple random-interval avoidance schedules.
Boakes, R. A. Response decrements produced by extinction Journal of the Experimental Analysis of Behavior, 1972,
and by response-independent reinforcement- Journal of 18, 499-507.
the Experimental Analysis of Behavior, 1973, 19? 293— de Villiers, P. A. The law of effect and avoidance: A
302. quantitative relationship between response rate and
Boakes, R. A., Halliday, M. S., & Mole, J. S. Successive shock-frequency reduction. Journal of the Experimental
discrimination training with equated reinforcement fre¬ Analysis of Behavior, 1974, 21, 223-235.
quencies: Failure to obtain behavioral contrast. Sub¬ Dickinson, A. Septal lesions in rats and the acquisition of
mitted for publication, 1974. free-operant successive discriminations. Physiology and
Boakes, R. A., Halliday, M. S., 8c Poli, M. Response ad¬ Behavior, 1973, 10, 305-313.
ditivity: Effects of superimposing free reinforcement on Dunham, P. J. Contrasted conditions of reinforcement: A
Barry Schwartz and Elkan Gamzu 93

selective critique. Psychological Bulletin, 1968, 69, 295- in avoidance learning. Psychonomic Science, 1966, 4,
315. 123-124.
Dunham, P. J., Mariner, A., 8c Adams, H. Enhancement of Gardner, W. M. Autoshaping in bobwhite quail. Journal

off-key pecking by on-key punishment. Journal of the of the Experimental Analysis of Behavior, 1969, 12, 279-
Experimental Analysis of Behavior, 1969, 12, 789-797. 281.
Egger, M. D., 8c Miller, N. E. Secondary reinforcement in Gibbon, J., Berryman, R., 8c Thompson, R. L. Contingency

rats as a function of information value and reliability of spaces and measures in classical and instrumental condi¬
the stimulus. Journal of Experimental Psychology, 1962, tioning. Journal of the Experimental Analysis of Be¬
64, 97-104. havior, 1974, 21, 585-605.
Egger, M. D., 8c Miller, N. E. When is reward reinforcing? Glass, D. H., 8c Ison, J. R. The effects of incentive shifts

An experimental study of the information hypothesis. following differential conditioning. Paper presented at
Journal of Comparative and Physiological Psychology, the meetings of the Psychonomic Society, St. Louis,
1963, 56, 132-137. October 1966.
Estes, W. K., 8c Skinner, B. F. Some quantitative properties Gonzalez, F. A. Effects of partial reinforcement (25%) in
of anxiety. Journal of Experimental Psychology, 1941, an autoshaping procedure. Bulletin of the Psychonomic
29, 390-400. Society, 1973, 2, 299-301.
Fantz, R. C. Form preference in newly hatched chicks. Gonzalez, F. A. Effects of varying the percentage of key

Journal of Comparative and Physiological Psychology, illuminations paired with food in a positive automainte¬
1957, 50, 422-430. nance procedure. Journal of the Experimental Analysis
Farrell, L. The role of partial reinforcement in the acquisi¬ of Behavior, 1974, 22, 483-490.
tion of an autoshaped response. Paper presented at the Gonzalez, R. C., 8c Champlin, G. Positive behavioral con¬

meetings of the Eastern Psychological Association, Phil¬ trast, negative simultaneous contrast, and their relation
adelphia, 1974. to frustration in pigeons. Journal of Comparative and
Farris, H. F. Classical conditioning of courting behavior in Physiological Psychology, 1974, 87, 173-187.
the Japanese quail, Coturnix coturnix japonica. Journal Gormezano, L., 8c Moore, J. W. Classical conditioning. In

of the Experimental Analysis of Behavior, 1967, 10, M. H. Marx (Ed.), Learning: Processes. London: Mac¬
213-217. millan, 1969.
Ferster, C. B., 8c Skinner, B. F. Schedules of reinforcement. Grant, D. A., 8c Schipper, L. M. The acquisition and ex¬

Englewood Cliffs, N.J.: Prentice-Hall, fnc., 1957. tinction of conditioned eyelid responses as a function
Findley, J. D. Preference and switching under concurrent of the percentage of fixed-ratio reinforcement. Journal
scheduling. Journal of the Experimental Analysis of of Experimental Psychology, 1952, 43, 313-320.
Behavior, 1958, 1, 123-144. Gutman, A., 8c Sutterer, J. R. Positive behavioral contrast

Foree, D. D., 8c LoLordo, V. M. Signalled and unsignalled in rats. Paper presented at the meetings of the Eastern
free-operant avoidance in the pigeon. Journal of the Psychological Association, Philadelphia, 1974.
Experimental Analysis of Behavior, 1973, 13, 283-290. Guttman, N. Generalization gradients around stimuli asso¬

Freeman, B. J. The role of response-independent reinforce¬ ciated with different reinforcement schedules. Journal of
ment in producing behavioral contrast effects in the rat. Experimental Psychology, 1959, 58, 335-340.
Learning and Motivation, 1971, 2, 138-147. (a) Halliday, M. S., 8c Boakes, R. A. Discrimination involving

Freeman, B. J. Behavioral contrast: Reinforcement fre¬ response-independent reinforcement: Implications for

quency or response suppression? Psychological Bulletin, behavioral contrast. In R. A. Boakes 8c M. S. Halliday
1971, 75, 347-356. (b) (Eds.), Inhibition and Learning. London: Academic
Gamzu, E. The classical conditioning paradigm in the Press, 1972.
Halliday, M. S., 8c Boakes, R. A. Behavioral contrast
“autoshaping” of the pigeon's key peck. Unpublished
without response rate reduction. Journal of the Experi¬
master’s thesis, University of Pennsylvania, 1968.
mental Analysis of Behavior, 1974, 22, 453-462.
Gamzu, E. Associative and instrumental factors underlying
Hearst, E., Besley, S., 8c Farthing, G. W. Inhibition and
the performance of a complex skeletal response. Un¬
the stimulus control of operant behavior. Journal of the
published doctoral dissertation, University of Pennsyl¬
Experimental Analysis of Behavior. 1970, 14, 373-409.
vania, 1971.
Hearst, E., 8c Jenkins, H. M. Sign tracking: The stimulus-
Gamzu, E., 8c Schwam, E. Autoshaping and automainte¬
reinforcer relation and directed action. Monograph of
nance of a key-press response in squirrel monkeys.
the Psychonomic Society: Austin, Texas, 1974.
Journal of the Experimental Analysis of Behavior, 1974, Hemmes, N. S. Behavioral contrast in pigeons depends
21, 361-371. upon the operant. Journal of Comparative and Physio¬
Gamzu, E., 8c Schwartz, B. The maintenance of key peck¬ logical Psychology, 1973, 85, 171-178.
ing by stimulus-contingent and response-independent Henke, P. G., Allen, J. D., 8c Davidson, C. Effects of lesions
food presentation. Journal of the Experimental Analysis in the amygdala on behavioral contrast. Physiology and
of Behavior, 1973, 19, 65-72. Behavior, 1972, 8, 173-176.
Gamzu, E., 8c Williams, D. R. Classical conditioning of a Henton, W. W., 8c Brady, J. V. Operant acceleration
complex skeletal act. Science, 1971, 171, 923-925. during a pre-reward stimulus. Journal of the Experi¬
Gamzu, E. R., 8c Williams, D. R. Associative factors under¬ mental Analysis of Behavior, 1970, 13, 205-209.
lying the pigeon’s key pecking in autoshaping pro¬ Herrnstein, R. J. Superstition: A corollary of the prin¬
cedures. Journal of the Experimental Analysis of ciples of operant conditioning. In W. K. Honig (Ed.),
Behavior, 1973, 19, 225-232. Operant behavior: Areas of research and application.
Garcia, J., 8c Koelling, R. Relations of cue to consequence Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966.
94 PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

Herrnstein, R. J. Method and theory in the study of literature since 1950. Psychological Bulletin, 1960,57, 2-28.
avoidance. Psychological Review, 1969, 16, 49-70. LoLordo, V. M. Facilitation of food-reinforced responding
Herrnstein, R. J. On the law of effect. Journal of the by a signal for response-independent food. Journal of
Experimental Analysis of Behavior, 1970, 13, 243-266. the Experimental Analysis of Behavior, 1971, 15, 49-55.
Herrnstein, R. J., 8c Loveland, D. H. Food avoidance in LoLordo, V. M., McMillan, J. C., 8c Riley, A. L. The
hungry pigeons and other perplexities. Journal of the effects upon food-reinforced pecking and treadle-pressing
Experimental Analysis of Behavior, 1972, 18, 369-383. of auditory and visual signals for response-independent
Hilton, A. Partial reinforcement of a conditioned emo¬ food. Learning and Motivation, 1974, 5, 24-41.
tional response in rats. Journal of Comparative and Lorenz, K. Evolution and the modification of behavior.
Physiological Psychology, 1969, 69, 253-260. Ghicago: University of Chicago Press, 1965.
Hinde, R. A., 8c Hinde, J. S. [Eds.). Constraints on Learn¬ Mackintosh, N. J. The Psychology of Animal Learning,
ing. London; Academic Press, 1973. London: Academic Press, 1974.
Hogan, J, A. Responses in Pavlovian conditioning studies. Malone, J. C., & Staddon, J. E. R. Contrast effects in
Science, 1974, 186, 156-157. maintained generalization gradients. Journal of the
Honig, W. K. (Ed.). Operant behavior: Areas of research Experimental Analysis of Behavior, 1973, 19, 167-179.
and application. Englewood Cliffs, N.J.: Prentice-Hall, MELTZER, If., &? Brahlek, J. A. Conditioned suppression
Inc., 1906. and conditioned enhancement with the same positive
Humphreys, l. g. The effect of random alternation of UCSl An effect of CS duration. Journal of the Experi¬
reinforcement on the acquisition and extinction of con¬ mental Analysis of Behavior, 1970, 13, 67-73.
ditioned eyelid racoons. Journal of Experimental Psy¬ Miczek, K. A., & Grossman. S. I3. Positive conditioned sup¬
chology, 1939, 15, 111-158. pression: Elfeas of CS duration. Journal of the Experi¬
Hunt, G. L., & Smith, W. S. Peeking and initial drinking mental Analysis of Behavior, 197L 15, 243-247.
responses in young domestic fowl Journal of Gompara Moore, B. R. The role of directed Pavlovian reactions in
dve and Physiological Psychology, 1967, 61, 230-236. simple instrumental learning in the pigeon. In R. A.
Hvf\§u? 3. Rm Nayarios, D, JM k Panting, E, “Automainte Hinde & J. StevensonTIinde (Eds.), Constraints on
nunceT The role o£ reinforcement Journal of the E?c- learning. New York: Academic Press, 1973.
perimental Analysis of Behaznor^ 1Q74j 21 y 112—124? Morrison, Rt R. The effects of a negative reinforcement
J aFTejEj M L. I he effects of lesions in the Y$ntromedial contingency on auto-shaped key pecking and other
nucleus of the hypothalamus on behavioral contrast ill water-associated behaviors. Paper presented at the meet¬
rats. Physiolo gical Psychology, 1973? 1, 191—198. ings q£ the Eastern Psychological Association, Philadel
JENKINS, H. M. Effects of the stimulus-reinforcer relation phia, 1974.
on selected and unselected responses. In R. A. Hinde & Murray, C. S. Conditioning in Bella splendens. Paper
J. Stevenson-Hinde (Eds.), Constraints on learning. New presented at the meetings of the Eastern Psychological
York: Academic Press, 1973. Association, Philadelphia, April 1974.
Jenkins, H. M., & Boakes, R. A. Observing stimulus sources Nevin, J. A. Differential reinforcement and stimulus control
that signal food or no food. Journal of the Experi¬ of not responding. Journal of the Experimental Analysis
mental Analysis of Behavior} 1973* 197-207, of Behavior, 1968, 11, 715-726.
JeNKINSj H, Mi, & Moore, B, 1\? The form of the ante- Nlyin, J. A;, 8c Shettleworth, 8. J. An analysis of contrast
shaped response with food or water reinforcers. Journal effects in multiple schedules, Journal of the Experi¬
of the Experimental Analysis of Behavior, 1973, 20, 163— mental Analysis of Behavior, 1966, 9, 305—315.
191. Padilla, S. G. Further studies of the delayed pecking of
Kamin, L. J. Predictability, surprise, attention, and condi¬ chicks, journal of Comparative Psychology, 1935, 20,
tioning, In B. A. Campbell & 1\« M« Church (Eds*), 413-443. _
Punishment and aversive behavior. Englewood Cliffs, Pavlov, J, p. Conditioned reflexes. London: Oxford Uni¬
N.J.i Prentice-Hall, Inc., 1969. versity Press, 1927-
Keller, k. The role of elicited responding in behavioral Pavlov, I. P. Lectures on conditioned reflexes, Vol. 2:
contrast. Journal of the Experimental Analysis of Be- Conditioned reflexes and psychiatry (trans. W. H.
kavior, 1974, 21, 249-257. Gantt). New York: International, 1941.
Kellv; 13. 13. Long-term prereward suppression m monkeys Pavlov, I, p, Selected Works (trans. S. Belsky). Moscow:
unaccompanied by cardiovascular conditioning. Journal Foreign Languages Printing House, 1955.
of the Experimental Analysis of Behavior, 1973, 20, 93- Pear, J. J., & Wilkie, 13. M. Behavioral contrast in mixed
104, schedules of reinforcement. Psychonomic Science, 1970,
Killeen, P. A yoked-cliamber comparison of concurrent 20, 167-168.
and multiple schedules. Journal of the Experimental Pear, J. J,, & Wilkie, 1), M. Contrast and induction in
Analysis of Behavior, 1972, 18, 13-22. rats on multiple schedules. Journal of the Experimental
Kimble, G. A. Hilgard and Marquis’ conditioning and Analysis of Behavior, 1971, 15, 289-296.
learning, Englewood Cliffs, N.J.; Prentice-Hall, Inc., Peterson, G. B. Auto-shaping in the rat: Conditioned
1961. approach and contact behavior toward signals of food
Kimble, G. A., & Perlmutter, L. C. The problem of voli¬ or brain-stimulation reinforcement. LJnpublished doc¬
tion. Psychological Review, 1970, 11, 361-384. toral dissertation, LJniversity of Indiana, 1972.
Lattal, K. A., 8c Maxey, G. C. Some effects of response- Peterson, G. B., Ackil, J. E., Frommer, G. P., 8c Hearst,
independent reinforcers in multiple schedules. Journal E. S. Conditioned approach and contact behavior
of the Experimental Analysis of Behavior, 1971, 16, 225- toward signals for food or brain-stimulation reinforce¬
23L ment. Science, 1972, 111, 1009-1011.
Lewis, D. J. Partial reinforcement: A selective review of the Rachlin, H. Autoshaping of key pecking in pigeons with
Barry Schwartz and Elkan Gamzu 95

negative reinforcement. Journal of the Experimental ment in the maintenance of key pecking which prevents
Analysis of Behavior, 1969, 12, 521-531. delivery of primary reinforcement. Psychonomic Science,
Rachlin, H. Contrast and matching. Psychological Review, 1972, 28, 277-278.
1973, 80, 217-234. Schwartz, B. Maintenance of key pecking by response-in¬

Rackham, D. Conditioning of the pigeon's courtship and dependent food presentation: The role of the modality
aggressive behavior. Unpublished Master’s thesis, Dal- of the signal for food. Journal of the Experimental
housie University, 1971. Analysis of Behavior, 1973, 20, 17-22. (a)
Redford, M., 8c Perkins, C. C. The role of auto pecking in Schwartz, B. Maintenance of key pecking in pigeons by a

behavioral contrast. Journal of the Experimental Analy¬ food avoidance but not by a shock avoidance contin¬
sis of Behavior, 1974, 21, 145-150. gency. Animal Learning and Behavior, 1973, 1, 164-166.
Rescorla, R. A. Pavlovian conditioning and its proper (b)
control procedures. Psychological Review, 1967, 74, Schwartz, B. Autoshaping and behavioral contrast: The
71-80. key to contrast is on the key. Paper presented at the
Rescorla, R. A. Probability of shock in the presence and meetings of the Eastern Psychological Association, Phila¬
absence of the CS in fear conditioning. Journal of Com¬ delphia, April 1974. (a)
parative and Physiological Psychology, 1968, 66, 1-5. Schwartz, B. On going back to nature: A review of Selig-

Rescorla, R. A. Conditioned inhibition of fear. In W. K. man and Hager’s Biological Boundaries of Learning.
Honig 8c N. J. Mackintosh (Eds.), Fundamental issues in Journal of the Experimental Analysis of Behavior, 1974,
associative learning. Halifax: Dalhousie University Press, 21, 183-198. (b)
1969. (a) Schwartz, B. Behavioral contrast in the pigeon depends

Rescorla, R. A. Pavlovian conditioned inhibition. Psy¬ upon the location of the stimulus. Bulletin of the
chological Bulletin, 1969, 72, 77-94. (b) Psychonomic Society, 1974, 3, 365-368. (c)
Rescorla, R. A. Informational variables in Pavlovian Schwartz, B. Discriminative stimulus location as a deter¬

conditioning. In G. H. Bower (Ed.), The psychology of minant of positive and negative behavioral contrast in
learning and motivation (Vol. 6). New York: Academic the pigeon. Journal of the Experimental Analysis of
Press, 1972. Behavior, 1975, 23, 167-176.
Rescorla, R. A., 8c Solomon, R. S. Two-process learning Schwartz, B. Positive and negative conditioned suppression
theory: Relationships between Pavlovian conditioning in the pigeon: Effects of the locus and modality of the
and instrumental learning. Psychological Review, 1967, CS. Learning and Motivation, 1976, 7, 86-100.
74, 151-182. Seligman,M. E. P. Control group and conditioning: A
Reynolds, G. S. Behavioral contrast. Journal of the Ex¬ comment on operationism. Psychological Review, 1969,
perimental Analysis of Behavior, 1961, 4, 57—71. (a) 76, 474-491.
Reynolds, G. S. An analysis of interactions in a multiple Seligman, M. E. P. On the generality of laws of learning.
schedule. Journal of the Experimental Analysis of Be¬ Psychological Review, 1970, 77, 406-418.
havior, 1961, 4, 107-117. (b) Scull, J., 8c Westbrook, R. F. Interaction in multiple
Reynolds, G. S. Relativity of response rate and reinforce¬
schedules with different responses in each of the compo¬
ment frequency in a multiple schedule. Journal of the
nents. Journal of the Experimental Analysis of Behavior,
Experimental Analysis of Behavior, 1961, 4, 179-184. (c)
1973, 20, 511-519.
Reynolds, G. S. Contrast, generalization, and the process
Schwartz, B., 8c Williams, D. R. Two different kinds of
of discrimination. Journal of the Experimental Analysis
key peck in the pigeon: Some properties of responses
of Behavior, 1961, 4, 289-294. (d)
maintained by negative and positive response-reinforcer
Reynolds, G. S., 8c Catania, A. C. Behavioral contrast with
contingencies. Journal of the Experimental Analysis of
fixed-interval and low-rate reinforcement. Journal of
Behavior, 1972, 18, 201-216. (b).
the Experimental Analysis of Behavior, 1961, 4, 387-391.
Schwartz, B., 8c Williams, D. R. The role of the response-
Ricci, J. A. Keypecking under response-independent food
reinforcer contingency in negative auto-maintenance.
presentation after long simple and compound stimuli.
Journal of the Experimental Analysis of Behavior, 1972,
Journal of the Experimental Analysis of Behavior, 1973,
19, 509-516. 71, 351-357. (a)
Rilling, M., Askew, H. R., Ahlskog, J. E., 8c Kramer, T. J.
Schwartz, B., Hamilton, B., 8c Silberberg, A. Behavioral
Aversive properties of the negative stimulus in a suc¬ contrast in the pigeon: A study of the duration of key
cessive discrimination. Journal of the Experimental pecking maintained on multiple schedules of reinforce¬
Analysis of Behavior, 1969, 12, 917-932. ment. Journal of the Experimental Analysis of Behavior,
Ross, L. E. The decremental effects of partial reinforce¬ 1975, 24, 199-206.
ment during acquisition of the conditioned eyelid re¬ Seligman, M. E. P., 8c Hager, J. L. (Eds.). Biological bound¬
sponse. Journal of Experimental Psychology, 1959, 57, aries of learning. Englewood Cliffs, N.J.: Prentice-Hall,
74-82. Inc., 1972.
Rozin, P., 8c Kalat, J. W. Specific hungers and poison Sepinwall, J. Some pharmacological aspects of benzodiaze¬
avoidance as adaptive specializations of learning. Psy¬ pines. Folha Medica, 1973, 67, 727-735.
chological Review, 1971, 78, 459-487. Sheffield, F. D. Relation between classical condition¬
Schwam, E., 8c Gamzu, E. Constraints on autoshaping in the ing and instrumental learning. In W. F. Prokasy (Ed.),
squirrel monkey: Stimulus dimension and response Classical conditioning. Englewood Cliffs, N.J.: Prentice-
topography. Bulletin of the Psychonomic Society, 1975, Hall, Inc., 1965.
5, 369-372. Shettleworth, S. Constraints on learning. In D. S. Lehr-

Schwartz, B. The role of positive conditioned reinforce¬ man, R. A. Hinde, 8c E. Shaw (Eds.), Advances in the
 96
PAVLOVIAN CONTROL OF OPERANT BEHAVIOR

study of behavior (Vol. 4). New York: Academic Press, Effects of extended discrimination training. Journal of
1972. (a) the Experimental Analysis of Behavior, 1966, 9, 613-617.
Shettleworth, S. Stimulus relevance in the control of
(a)
drinking and conditioned fear responses in domestic Terrace, H. S. Stimulus Control. In W. K. Honig (Ed.),
chicks (Gallus gallus). Journal of Comparative and Phys¬
Operant behavior areas of research and application.
iological Psychology, 1972, 80, 175-198. (b) Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966. (b)
Shimp, C. P. 8c Wheatley, W. L. Matching to relative Terrace, H. S. Discrimination learning, the peak shift, and
reinforcement frequency in multiple schedules with short behavioral contrast. Journal of the Experimental Analy¬
component duration. Journal of the Experimental Analy¬ sis of Behavior, 1968, 11, 724-741.
sis of Behavior, 1971, 15, 205-210. Terrace, H. S. By-products of discrimination learning. In
Shull, R. L., 8c Pliskoff, S. S. Changeover behavior under
G. H. Bower (Ed.), The psychology of learning and
pairs of fixed-ratio and variable-ratio schedules of rein¬
motivation (Vol. 5). New York: Academic Press, 1972.
forcement. Journal of the Experimental Analysis of Terrace, H. S. Classical conditioning. In J. A. Nevin (Ed.),
Behavior, 1971, 16, 75-79.
The study of behavior. New York: Scott, Foresman,
Sibman, M., 8c Fletcher, F. G. A demonstration of auto¬ 1973.
shaping with monkeys. Journal of the Experimental Terrace, H. S., Gibbon, J., Farrell, L., 8c Baldock, M. D.
Analysis of Behavior, 1968, 11, 307-309. Temporal factors influencing the acquisition of an auto¬
Silderderg, A., 8c Schrot, B. A yoked-chamber comparison shaped key peck. Animal Learning and Behavior, 1975,
of concurrent and multiple schedules: The relationship 3, 53-62.
between component duration and responding. Journal Thomas, E., 8; Wagner, A. R. Partial reinforcement of the
of the Experimental Analysis of Behavior, 1974 22 classically conditioned eyelid response in the rabbit.
21-30.
Journal of Comparative and Physiological Psychology,
Skinner, B. f. The behavior of organisms. New York: 1964, 58, 157-158.
Appleton-Gcntury«Crofts, 1938. Thompson, T.; & Sturm, T. Classical conditioning of ag¬
Skinner, B. F. “Superstition” in the pigeon. Journal of gressive display in Siamese fighting fish. Journal of the
Experimental Psychology, 1948, 38, 168-172. Experimental Analysis of Behavior, 1965, 8, 397-403.
Smith, j. B. Effects of response rate, reinforcement fre¬ Iodorov, J. C. Component duration and relative response
quency, and the duration of a stimulus preceding re¬ rates in multiple schedules. Journal of the Experimental
sponse-independent food. Journal of the Experimental Analysis of Behavior, 1972, 17, 45-50.
Analysis of Behavior, 1974, 21, 215-221. Van Dyne, G. G. Conditioned suppression with a positive
Smith, S. G., 8c Smith, W. M., Jr. A demonstration of auto¬ US in the rat, Journal of Comparative and Physiological
shaping with dogs. Psychological Record, 1971, 21, 377- Psychology, 1971, 77, 131-135.
379.
Washerman, E. A. Auto-shaping: The selection and direc¬
Sfear, N. E., & Hill, W. F. Adjustment to new reward:
tion of behavior by predictive stimuli. Unpublished
Simultaneous and successive contrast effects. Journal of
doctoral dissertation, University of Indiana, 1972.
Experimental Psychology, 1965, 70, 510-519.
Washerman, E. A, Pavlovian conditioning with heat rein¬
Spence, K. W. The nature of discrimination learning in
forcement produces stimulus-directed pecking in chicks.
animals. Psychological Review, 1936, 43, 427-449. Science, 1973, 181, 875-877. (a)
Spense, K, W. Cognitive and drive factors in the extinction Wasserman, E. A. The effect of redundant contextual
of the conditioned eye blink in human subjects. Psy¬ stimuli and autoshaping the pigeon’s keypeck. Animal
chological Review, 1966, 445—458.
Learning and Behavior, 1973, 1, 198-206. (b)
Squier, L. H. Auto-shaping key ^ponses in Osh. Psycho-
Wasserman, E. A. Stimulus-reinforcer predictiveness and
nomic Science, 1969,17, 177-178.
selective discrimination learning in pigeons. Journal of
Staddon, J. E. R. Multiple fixed-interval schedules: Tran¬
the Experimental Analysis of Behavior, 1974 103 284-
sient contrast and temporal inhibition. Journal of the 297.
Experimental Analysis of Behavior, 1969, 12, 583-590. Wasserman, E. A., 8; Anuersqn, P. A. Differential autoshap'
Staddo^j J. E. R. Temporal control and the theory of ing to common and distinctive elements of positive and
reinforcement schedules, In R, M. Gilbert 8c J. R.
negative discriminative stimuli. Journal of the Experi¬
Millenson (Eds.), Reinforcement: Behavioral analysis.
mental Analysis of Behavior, 1974, 22, 491-496.
New York: Academic Press, 1972.
Waterman, E., Franklin, S., 8c Hearst. E. Pavlovian ap¬
Staddon, J. E. R., & SlMMELHAC, B. The superstition ex¬
petitive contingencies and approach vs. withdrawal to
periment; A reexamination of its implications for the conditioned stimuli in pigeons. Journal of Comparative
principles of adaptive behavior, Psychological Review, and Physiological Psychology, 1974, 86, 616-627.
1971, 78, 3-43.
Wasserman, E. A., & McCracken, S. B. The disruption of
STIERS, M., & SlLBERBERC, A. Autoshapmg and automainte-
autoshaped key pecking in the pigeon by food-tray
nance of lever-contact responses in rats. Journal of the
illumination, Journal of the Experimental Analysis of
Experimental Analysis of Behavior, 1974, 22, 497-506. Behavior, 1974, 22, 39-45.
Terrace, H. S. Discrimination learning with and without
Weisman, R. G. Some determinants of inhibitory stimulus
errors. Journal of the Experimental Analysis of Be¬
control. Journal of the Experimental Analysis of Be¬
havior, 1963, 6, 1-27. (a)
havior, 1969, 12, 443-450.
Terrace, H. 5. Errorless transfer of discrimination across
Weisman, R. G. Factors influencing inhibitory stimulus
two continua. Journal of the Experimental Analysis of control: Differential reinforcement of other behavior
Behavior, 1963, 6, 223-232. (b)
during discrimination training. Journal of the Experi¬
Terrace, H. S. Behavioral contrast and the peak shift: mental Analysis of Behavior, 1970, 14, 87-91.
Barry Schwartz and Elkan Gamzu 97

Weiss, S. J. Discrimination training and stimulus com¬ without constraint. Unpublished manuscript, University
pounding: Consideration of non-reinforcement and re¬ of Pennsylvania, 1974.
sponse differentiation consequences of SA. Journal of Williams, D. R., 8c Williams, H. Automaintenance in the
the Experimental Analysis of Behavior, 1971, 15, 387- pigeon: Sustained pecking despite contingent non-rein¬
402. forcement. Journal of the Experimental Analysis of
Wessells, M. G. Errorless discrimination, autoshaping, and Behavior, 1969, 12, 511-520.
conditioned inhibition. Science, 1973, 182, 941-943. Wilton, R. N., 8c Clements, R. O. The role of information
Wessells, M. G. The effects of reinforcement upon the in the emission of observing responses: A test of two
pre-pecking behaviors of pigeons in the autoshaping hypotheses. Journal of the Experimental Analysis of
experiment. Journal of the Experimental Analysis of Behavior, 1971, 16, 161-166.
Behavior, 1974, 21, 125-144. Wolin, B. R. Difference in manner of pecking a key be¬

Westbrook, R. G. Failure to obtain positive contrast when tween pigeons reinforced with food and water. In A. C.
pigeons press a bar. Journal of the Experimental Analy¬ Catania (Ed.), Contemporary research in operant be¬
sis of Behavior, 1973, 20, 499—510. havior. Glenview, Ill.: Scott, Foresman, 1968.
Wilkie, D. M. Variable-time reinforcement in multiple and Woodruff, G. Autoshaping: A “learned release’’ hypothesis.
concurrent schedules. Journal of the Experimental Paper presented at the meetings of the Eastern Psycho¬
Analysis of Behavior, 1972, 17, 59-66. logical Association, Philadelphia, April 1974.
Williams, D. R. Negative induction in behavior rein¬ Zener, K. The significance of behavior accompanying con¬
forced by central stimulation. Psychonomic Science, ditioned salivary secretion for theories of the condi¬
1965, 2, 341-342. tioned response. American Journal of Psychology, 1937,
Williams, D. R. Biconditional Behavior: Conditioning 50, 384-403.
 4

The Nature
of Reinforcing Stimuli*

Philip Dunham

Consider for a moment an instrumental environ¬ Shred paper, etc. Imposing a contingency can cause
ment wiiicli ie equipped with several items uf inter¬ an increase, a decrease, or no change in the probabil¬
est fA the small rodent known as the Mongolian, gerbil ity of the instrumental behavior. If a particular con¬
[Merzmqq unqmeukka), The chamber contains a tingency produces a change in the probability of
small box of sand, some sunflower seeds, a drinking instrumental responding, most psychologists would
tube, an activity wheel, and some bristol hoard which agree that some hypothetical process has operated
the animal enjoys shredding. This environment, or {although not all would agree that we should pursue
variations Oil it, will he referred to on §€Y€ial Occa¬ the matter), The term reinforcement is typically used
sions m the discussion which follows, Given uncon¬ to refer to this process with the label reward reserved
strained access to this environment for one hour each for increases in instrumental responding and punish¬
day, the gerbil will distribute much of the available ment for decreases in instrumental responding. The
time among the various items of interest in the cham¬ general purpose of this chapter will he to consider the
ber. Once Stable behavior pattens emerge, a small current status of our attempts to predict the outcome
amount of engineering permits us to arrange any OllC of instrumental contingencies like those which we
of gO possibly instrumental contingencies in the might wish to arrange for the gerbil.
gerbil’s world, We can require the animal to eat in
order to run, run in order to eat, drink in order to

A HISTORICAL PERSPECTIVE
* Research presented in this manuscript was supported by
Grant APA-194 from the National Research Council of Canada.
The manuscript was prepared during the author’s tenure on a If we ignore some of the fine grain of the historical
leave fellowship granted by the Canada Council. I am most record, I would suggest that history has provided the
grateful to Carol Anderson, Sue Cohen, Pat Kelly, Sue
contemporary experimental psychologist with one of
Marmaroff, Robyn Pascoe, and Pat Thomas for assistance in the
various tasks of data collection, analysis, and manuscript prepara¬ three points of departure if one is interested in an
tion. analysis of the reinforcement process.

98
Philip Dunham 99

Traditional Reinforcement Theory rejected the negative side of his classic law of effect.
All subsequent attempts to account for the observa¬
In its initial form, the reinforcement process served
tion that some events will produce a decrease in the
one simple explanatory function. It functioned as the
probability of instrumental responding adopted some
“glue” which cemented together the ubiquitous Ss
variation on what has been called the alternative re¬
and Rs of early associationism—or conversely, as the
sponse assumption (cf. Dunham, 1971). In its weakest
“solvent” which dissolved S-R connections already
form, this assumption suggests that the decrease in
formed. Thorndike's (1914) statement of his sym¬
instrumental responding produced by some contin¬
metrical law of effect is a classic example of this tradi¬
gent events is the indirect result of an increase in some
tion:
alternative behavior. The alternative behavior is as¬
sumed to be developed and maintained by the posi¬
The Law of Effect—to the situation, a modifiable
tive side of the law of effect (i.e., an escape from aver¬
connection being made by him between an S
sive stimulation mediated by either a two-process or
and an R and being accompanied or followed
single-process conditioning model).
by a satisfying state of affairs, man responds,
other things being equal by an increase in the Parallel with the decline in popularity of concepts
strength of that connection. To a connection such as drive reduction, there is also a contemporary
similar, save than an annoying state of affairs disenchantment with the various forms of the alterna¬
goes with or follows it, man responds, other tive response assumption as explanations of the effects
things being equal, by a decrease of the con¬ which aversive events are observed to have upon be¬
nection. (p. 71) havior. Rachlin and Herrnstein (1969) and Dunham
(1971) have discussed both data and conceptual argu¬
Starting from this assumption, the basic task which ments against the alternative response assumption as
confronted the reinforcement theorist was to provide an explanation of the effects which negative contin¬
an indexing system which would tell us the category gent events have upon instrumental behavior.
into which any particular contingent event might fall To summarize, the fundamental task posed by
—“satisfier,” “annoyer,” or “neutral." With an ade¬ traditional reinforcement theorists was to predict the
quate set of rules, we should be able, for example, to effect a particular contingent event might have-
predict the outcome of any one of the 20 contingen¬ positive, negative, or neutral. Although the notions
cies possible in our gerbil environment. generated by this task appear to be in some disfavor
As Premack (1969) has suggested, this traditional at present, it remains as one possible point of de¬
view of reinforcement asserts that there are three parture for the contemporary student of reinforce¬
mutually exclusive categories of contingent event to ment.
be found in nature and assumes, implicitly or ex¬
plicitly, that particular contingent events are incon¬
The "Incentive" Function
trovertible members of a particular category. From
of Reinforcement
this fundamental assumption, the search was initiated
to find the property or properties which would permit As Walker (1969) has suggested, the general trend
us to know the particular category into which a par¬ in the development of theoretical views of reinforce¬
ticular event might fall. It is in this context that such ment has been to expand the explanatory powers of
prominent notions as drive reduction, drive induction, the concept. This expansion provides us with a sec¬
arousal, optimal level, and other major concepts were ond point of departure for studying reinforcement.
developed. The conceptual and empirical deficiencies As stated earlier, the process was initially a “glue”
of the major theoretical schemes in this tradition have in the chemistry of associationism. Primarily under
been extensively reviewed and criticized in a number the direction of Hull, the concept started to mutate
of articles (cf. Miller, 1963; Wilcoxon, 1969) and need into a much more powerful explanatory mechanism.
not be repeated here. Although one might take excep¬ Concerned with such problems as the rate at which
tion to some of the particulars, the general consensus performance changed with changes in magnitude of
of contemporary critics is that concepts such as drive reward, Hull decided that the process not only ce¬
reduction have serious shortcomings as a basis for mented the habit structure together, but pulled the
predicting when a particular contingent event might organism down the runway via K (incentive) and its
function as a reward. machinery, the rg mechanism. Hence by the 1950s the
The search for the nature of negative or aversive reinforcement process was busy with such additional
events took a curious twist when Thorndike (1932) tasks as moving the animal out of the start box and
TOO THE NATURE OF REINFORCING STIMULI

mediating changes in performances which accom¬ Current Status of the Reinforcement Concept
panied changes in the magnitude of reward. The ad¬
In my opinion, contemporary psychologists work¬
dition of incentive motivation to the function served
ing in the context of reinforcement phenomena have
by the reinforcement process added a large and per¬
opted almost exclusively to approach the problem
sisting area of research and theorizing to the rein¬
using the weak law of effect as a justification for an
forcement literature. It permitted us to deal with the
extensive analysis of scheduling effects or to investi¬
tangled problems raised by the incentive concept with¬
gate the incentive function of the contingent event
out being distracted by the questions raised by Thorn¬
(be it fear or the rg mechanism). Attempts to develop
dike and the early associationists. For example, given
a theory of reinforcement which is capable of pre¬
the observation that a hungry rat will run down a
dicting the effects of a particular contingent event
runway for food pellets, it is possible to proceed with
prior to arranging the contingency are sparse. Per¬
an extensive empirical analysis of the effects which a
haps most of us have accepted the statement made by
shift in the number of food pellets might have upon
Meehl (1950) concerning the problems which would
the rat’s performance without ever worrying about
confront such an effort:
the more fundamental question of why the rats ran
for the food in the first place. Any thirst for theoret¬
Finally, it would be very nice if in some magical
ical analysis can be quenched quite readily by con¬
way we could know before studying a given
sidering the rather complicated manner in which the
species exactly what stimulus changes would
incentive mechanism has been suggested to mediate
have the reinforcing property; but I have tried
changes in performance produced by a shift in the to indicate that this is essentially an irrational
number of food pellets (cf. Dunham, 1968). demand, (p. 74)

In support of using the weak law of effect as an

The "Weak Law of Effect"
approach to reinforcement, Meehl promised us some
20 years ago that such an inductive empirical exami¬
Yet a third point of departure in the analysis of nation would lead to some potent predictive rules
reinforcement starts from the assumption which has (even for visiting Martians). I would suggest that the
been called the weak law of effect. Although the weak track record for the past few decades is not as im¬
law of effect has been given a number of different pressive as Meehl hoped, and it is perhaps time for a
labels and definitions, all versions imply that we can more concerted emphasis upon some of the fundamen¬
conduct an empirical analysis of various contingency tal questions posed by traditional reinforcement the¬
operations until, for example, we find that food is an ory. First, the data base generated by the inductive
effective contingent event for training a hungry rat to analysis is dangerously circumscribed. It is difficult to
press a lever. We can then proceed with an analysis find much more than the various combinations of
of variations on this contingency (e.g., schedules of lever press, key peck, food, water, and electric shock
reward), assuming that we have found at least one of upon which to base generalizations. Unlike Meehl’s
several possible cases in which the reinforcement proc¬ Martian, I find little information upon which to base
ess is operating. As discussed by Meehl (1950), one an inductive leap into the multiple-response world of
way in which the weak law of effect can escape circu¬ the gerbil described earlier. Perhaps the problem has
larity is by assuming that food is also an effective con¬ been less with the inductive process than with the
tingent event for a variety of instrumental responses convenience of the manufactured operant chamber.
(i.e., the reward event is transituational). The same Second, and related, is that the critical assumption
arguments can be used, for example, if electric shock that rewards (or punishments) are transituational has
is used as a contingent event which will effectively not been subjected to extensive, rigorous testing,
suppress a number of different behaviors. again perhaps because of the compatibility of key
The contents of this volume provide ample evi¬ pecks and impulse counters.
dence for the popularity of the weak law of effect as a In view of these problems, it would appear that
point of departure in the analysis of reinforcement. there is some justification for returning to the ques¬
The adoption of the empirical approach to rein¬ tion of how we might predict the outcome of an
forcement represented by the weak law of effect and instrumental contingency before it has been arranged.
the transituational assumption also permits one to Hence, the remainder of the chapter will be devoted
ignore the traditional questions about reinforcement to a discussion of two contemporary trends in the
posed by Thorndike and his successors. literature which run counter to the weak law of effect
Philip Dunham 101

as an approach to reinforcement and which have gen¬ forcement theory: How can we predict the outcome
erated data which question the validity of the assump¬ of an instrumental contingency? There is a funda¬
tion that rewards are transituational. The first trend, mental difference in approach, however. Thorndike’s
an attempt to return to a symmetrical law of effect, is analysis implies that there are three different types
represented by the work of Premack (1959. 1965, 1971) of event in nature which have absolute properties
and his students. The second is recent research con¬ termed satisfying, annoying, and neutral. Hence
cerned with the biological constraints which operate Thorndike and those who followed in his tradition
when an organism is exposed to particular instru¬ attempted to find the absolute properties which per¬
mental contingencies. The result of this exercise will mitted certain events to be assigned to fixed cate¬
be to demonstrate that the transituational assumption gories (i.e., satisfiers and annoyers). Premack makes
is not valid and that a symmetrical law of effect based the fundamentally different presupposition that the
upon preference notions represents a more fruitful property of being rewarding or punishing is relative.
approach to reinforcement theory. Food, for example, can be either a reward or a punish¬
ment depending upon the relative value of the in¬
strumental response. If an instrumental running
PREMACK'S REINFORCEMENT THEORY response is less probable than eating, the food will func¬
tion as a rewarding event. If the running response is
The essential features of Premack’s reinforcement more probable than eating, the food will function as
theory are contained in three major papers (Premack, a punishing event. The recognition of the relativistic
1959, 1965, 1971). He suggests that the organism nature of reward and punishment represents a sig¬
places the events in its world on a unitary dimension nificant departure from traditional thought about the
of value or preference. The relative value of a par¬ reinforcement process.
ticular event can be measured in terms of the amount The relativistic position necessarily contradicts the
of time the organism spends engaging in that event, assumption that rewards are transituational. The di¬
If we use time as an indicator of value, it is possible, rect and testable implication of Premack’s rules is that
according to Premack, to predict the outcome of any a given contingent response like eating can function
particular instrumental contingency. If we arrange a as a reward in one context and as a punishment in
contingency in which value of the contingent event is another context. If food is demonstrated to reward
higher than the value of the instrumental event. running in one instrumental contingency and punish
Premack predicts that we will observe an increase in running in another instrumental contingency, food,
the probability of the instrumental response. If we by definition, is not a transituational reward.
arrange a contingency in which the value of the con¬ Finally, Premack?s position can be viewed, in a
tingent event is lower than the value of the instru¬ curious sense, as having continued the trend described
mental event, Premack predicts that we will observe a by Walker (1909) of increasing the explanatory pow¬
decrease in the probability of the instrumental re¬ ers of the reinforcement concept. As a miniature
sponse. It should be noted that the value of a particu¬ modch typical of contemporary psychological theory,
lar event is measured in terms of the amount of time itomits reference to such concepts as habit, incentive,
the organism spends indulging in that event when the fg mechanism, or the “^lue” whiek were so promi¬
permitted unconstrained access to it. These rules are nent m the heyday muHlCOtiCOpt theories 5Uch &S
simple and testable, They suggest that the organism Hull's (1943, 1952). A consequence of these omissions
will increase the probability of a behavior which is that Premack's reinforcement concept is a singte
moves it from a less to a more preferred state, but motivational construct which is assumed to account
that it will not engage in a behavior which moves it for all changes in performance observed when an in¬
from a more to a less preferred state. strumental contingency is instituted, Although I have
yet to learn the Angerthas, I suspect that Walker has

Historical Context of Premack's Position

accused Professor Premack of having found the “one
ring.”
Before looking at the evidence for these assump¬
tions it may be helpful to see where Premack fits into
Evidence for Premack's Assumptions
the three approaches to reinforcement discussed ear¬
lier. Essentially, Premack is concerned with the prob¬ Prior to 1971 most of the evidence in support of
lem that Meehl finds “irrational.” It is Thorndike’s Premack’s position was restricted to the positive side
problem and the question central to traditional rein¬ of the symmetrical rules which he proposes. Most ex-
102 THE NATURE OF REINFORCING STIMULI

periments were designed to test the assumption that relative probability of the two responses was reversed,
the contingent response must be more probable than and running was demonstrated to be an effective
the instrumental response for there to be an increase contingent event for increasing the probability of in¬
in the probability of instrumental behavior. The basic strumental drinking (cf. Premack, 1962; Schaeffer,
design used in all experiments, with minor variations, 1965).
is a three-stage (ABA) procedure. During the first The generality of these findings has been extended
stage the independent probabilities of the responses to include contingencies between intracranial stimula¬
which are to be used in the instrumental contingency tion (ICS) and drinking (Holstein 8c Hundt, 1965)
are measured. The organism is permitted uncon¬ and between running and lever pressing (Hundt 8c
strained access to the appropriate manipulanda (e.g., Premack, 1963). The experiments have also included
a drinking tube and a running wheel), and the both children (Premack, 1971) and college students
amount of time spent engaging in each of these be¬ (Schaeffer, Hanna, 8c Russo, 1966) as subjects.
haviors is measured. During the second stage a con¬ It was not until 1971 that the punishment side of
tingency is arranged in which the organism is re¬ Premack’s formulation was presented in any detail
quired to engage in a specified amount of instrumental (Premack, 1971). Although the punishment rule is the
behavior in order to gain access to a specified amount simple converse of the reward rule, testing it poses
of contingent responding. Any increase (or decrease) some problems. Consider, for example, a thirsty rat
in instrumental responding above (or below) the which is given free access to a running wheel and a
previously measured base line probability indicates drinking tube. With appropriate deprivation param¬
that the reinforcement process has operated. A third eters, drinking can be made more probable than run¬
stage is often included in which the animal is per¬ ning. If a contingency is now arranged in which the
mitted to return to the unconstrained base line con¬ animal is required to drink in order to run (high to
dition. low), the animal will very likely continue to drink
Using this basic procedure, the evidence which until satiated because drinking is the preferred state.
supports the positive side of Premack’s differential To demonstrate that the less probable response is an
probability rules extends over a wide range of species aversive event, the animal must be forced to run; and
and responses. Using Cebus monkeys which were it must be demonstrated that the forcing operation
given unconstrained access to one of four manipu¬ per se is not the effective punishment event.
landa during base line sessions (a door, a plunger, a The initial work on this problem was presented in
vertical lever, and a horizontal lever), Premack (1963) a paper by Weisman and Premack at the meetings of
arranged instrumental contingencies in a manner the Psychonomic Society in 1966 and discussed sub¬
which required the monkeys to respond on a less sequently by Premack (1971). Weisman and Premack
probable manipulandum in order to gain access to a permitted rats free access to a drinking tube and a
more probable manipulandum (low to high), or in motorized running wheel for daily 15-min sessions.
which the animals were required to respond to a more Two rats were maintained on a 23-hr water depriva¬
probable manipulandum in order to gain access to a tion schedule, which made drinking more probable
less probable manipulandum (high to low). In all than running, and two rats were maintained ad lib,
cases, the low-to-high contingency increased the prob¬ which made running more probable than drinking.
ability of the instrumental response and no such After base line probabilities of responding were es¬
increase was observed when the contingent response tablished for all four animals, contingency sessions
was less probable. were initiated in which 15 laps on the drinking tube
In a similar series of experiments in which rats produced 5 sec of motorized running in the wheel. For
were used as subjects and running and drinking were the two deprived animals the lick-to-run contingency
the members of the instrumental contingency, the (high to low) suppressed the amount of drinking, as
reinforcement relation was demonstrated to be re¬ predicted. For the ad lib animals the lick-to-run con¬
versible, as the differential probability rules predict. tingency (low to high) increased the amount of drink¬
Using a low-to-high contingency in which drinking ing, as predicted. The results in the latter condition
was more probable than running, drinking was ob¬ also indicate that the forcing operation per se was not
served to increase the probability of running—a rather aversive. Subsequent recovery of the base line prob¬
typical observation. Less often observed, however, was abilities was followed by a reversal of deprivation
the reverse case in which the running response was conditions for each subject. A crossover design re¬
made contingent upon drinking. With appropriate vealed results identical to those obtained in the first
manipulation of the deprivation parameters, the phase of the study.
Philip Dunham 103

The results of a subsequent, similar study by Ter- to maintain the instrumental response at its free
hune and Premack (1970) also indicate that forced performance level.
running is an aversive event when it is less probable
than instrumental drinking. In addition, Terhune
and Premack reported that the amount of suppression Evidence for a Response Deprivation Hypothesis
produced by forced running was a linear function of
There are three independent lines of experimenta¬
the probability that the animal would not be in the
tion available at the present time which have been
state of running at any fixed time after running had
generated by the preceding rationale. The results of
been initiated.
all three efforts indicate that, contrary to Premack’s
assumptions, under certain conditions a less probable
Problems with Premack's Formulation response can serve as a reward for a more probable
instrumental response.
As is obvious from the preceding discussion, a sub¬ Eisenberger, Karpman, and Trattner (1967) were
stantial amount of data has accumulated to support the first to examine the problem. They reported a
Premack’s differential probability rules (particularly series of experiments in which college students were
the reward side). There are, however, several basic given access to two different manipulanda: a wheel
problems with the supporting data which suggest that which could be hand-cranked and a lever which could
some additional experimentation and a reformulation be pressed. In the critical experiment in the series,
of the position might be in order. each subject was run in a 5-min base line session in
An examination of the data in support of both the which both wheel and lever were freely available.
reward and the punishment assumption reveals two This was followed by a 5-min contingency session in
basic confoundings: which the subject was required to crank the wheel 10
revolutions in order to press the lever once. The re¬
1. With respect to the reward assumption, in every sults of the base line session revealed that most sub¬
procedure in which it has been demonstrated that
jects preferred cranking the wheel to lever pressing.
a more probable response will reinforce a less prob¬
During the subsequent contingency session, if the
able response, the subject was required to increase
leyer-press response was suppressed by the contingency
the probability of the less probable instrumental
requirement, an increase in instrumental wheel crank¬
response in order to maintain the contingent re¬
sponse at the independently measured free perfor¬ ing was observed—even if the contingent lever-press
mance level. response was lass probable than the instrumental
2. With respect to the punishment assumption, every response.
procedure in which it was demonstrated that a less Eisenberger et al. interpreted their results in terms
probable contingent response would punish a more of a response suppression hypothesis which stated that
probable instrumental response, the subject was the necessary condition for an increase in instrumen¬
required to increase the probability of the contin¬ tal responding is the suppression of the contingent
gent response above the independently measured response, independent of the relative probability of
free performance level in order to maintain the instrumental and contingent responses. As they stated:
instrumental response at its free performance level.

The present set of experiments suggests the nec¬

In view of these two basic confoundings, the pos¬
essary and sufficient condition for reinforcement
sibility remains that:
in the contingency situation is the animal’s ne¬
cessity to increase instrumental responding if it
1. A less probable contingent response can be demon¬ is to maintain contingent responding at the free
strated to reinforce a more probable instrumental performance level, (p. 350)
response if the contingency requirements are such
that the subject must increase the probability of
The second line of experimentation designed to
the instrumental response above its free perfor¬
examine this problem consists of two experiments
mance level in order to maintain the free perfor¬
mance level of the contingent response. performed in our laboratory using subjects, procedure,
and apparatus which more closely approximate Pre¬
2. A more probable response can be demonstrated to
punish a less probable instrumental response if the mack’s early experimentation.
contingency arrangements are such that the subject The first experiment was a master’s thesis con¬
must increase the probability of the contingent ducted by Susan Marmaroff (1971) which employed
response above its free performance level in order albino rats as subjects and running and drinking as
104 THE NATURE OF REINFORCING STIMULI

the members of the instrumental contingency. The than once during the six sessions immediately prior
apparatus, identical to that described in a paper by to the contingency training. The running wheel re¬
Dunham (1972), consisted of a single modified activity mained in the free-access state throughout the ses¬
wheel housed in a dark, ventilated, sound-attenuating sion; thus running was possible during both states of
chamber located in a room adjacent to the control the drinking tube, inserted or retracted. Phase 4 lasted
apparatus. A motor-cam mechanism permitted us to for 15 sessions and was followed by a final phase in
insert or retract a drinking tube to which the animal which the contingency was eliminated and free access
had access through a small hole in the stationary wall to the wheel and tube was reinstated.
of the wheel.
The rats were on 23-hr water deprivation, and daily Table 1 Instrumental and contingent response requirements
sessions 60 min in length were conducted 7 days per for each subject during contingency training.
week. The experimental procedure was divided into Phase 3

four phases. During Phase 1 the animals were per¬ SUBJECT INSTRUMENT CONTINGENT CONTINGENCY

mitted free access to the running wheel with the

(No. of 90-deg (No. of licks)
drinking tube retracted for 15 consecutive sessions. revolutions)
Animals were permitted access to water for 1 hr in
the home cage immediately after the session. 1 5 low to high
30
During Phase 2 the running wheel was mechan¬ 2 8 35 no preference
ically locked so that no running could occur. The 3 10 20 high to low
drinking tube was inserted, and a base line level of 4 8 35 no preference
5 10 30 high to low
free operant drinking was observed. Phase 2 continued
6 9 30 high to low
for 15 sessions.
During Phase 3 the subjects were permitted free
access to both the drinking tube and running wheel. The results of the five phases of the experiment for
In all phases of the experiment, drinking was defined each of the subjects are presented in Figure 1. As is
as a single lap on the tube and running as a 90-deg evident from the base line data, the deprivation
revolution of the wheel. Electronic circuitry divided parameters produced low-to-high, high-to-low, and
the entire session into 2-sec intervals, and each inter¬ nondifferential probability cases. The major result
val was scanned for an instance of either drink or run. was that the contingency produced an increase in the
The data were converted to a probability measure by instrumental performance of all subjects. Reinforce¬
dividing the number of intervals in which a response ment was thus observed in low-to-high, high-to-low,
occurred by the total number possible in the session and nondifferential probability cases. A htest for cor¬
(cf. Premack, 1965). related observations indicated that the increase in
During Phase 4, a run-to-drink contingency was
arranged for all animals. To commence each session,
the wheel was available for free-access running (brake ~D
released) and the drinking tube was retracted. A fixed- -R
ratio schedule was selected for each animal in which
a fixed number of 90-deg revolutions in the wheel
(instrumental response) produced the drinking tube
for a fixed number of licks (contingent response),
after which the tube was retracted. The instrumental
and contingent requirements were arranged such that
each animal had to increase the base line amount of
running (as measured in Phase 3) by approximately
50% in order to maintain the contingent response at
its base line level. As indicated in Table 1, the base
line level of responding in Phase 3 shows that run
was consistently more probable than drink for sub¬
jects 3, 5, and 6; that drink was more probable than
run for subject 1; and that no consistent preference
was observed for subjects 2 and 4. A subject was
.
Fig. 1 Probability of running and drinking for each subject
judged inconsistent if it reversed its preference more during each of the five phases of the first experiment.
Philip Dunham 105

the probability of instrumental running reliably ex¬ daily session. This phase continued for 15 days, and
ceeded both the single-response base line (Phase 1) the base line probability of each response was mea¬
and the two-response base line (Phase 3) (single¬ sured with the same method as employed in the first
response base line comparison: t — 6.662, p < .005; experiment.
two-response base line comparison: t = 8.50, p < .005). During Phase 2 a drink-to-run contingency was
The contingency also suppressed the contingent established for all subjects. The contingency required
drinking response. The amount of suppression was the subjects to complete 50 laps on the drinking tube
not great, and drinking tended to recover over the in order to obtain five 90-deg revolutions of the wheel.
course of training. A correlation between the amount This contingency, as in the first experiment, required
of suppression of contingent drinking and the amount each subject to increase the amount of instrumental
of increase in instrumental responding was reasonably responding above its base line level if contingent
high, but not reliable (r = .48; df 5; p > .05). The responding was to be maintained at the base line
presence of some degree of positive correlation sup¬ level. Unlike the first experiment, however, the use of
ports the suggestion of Eisenberger et al. that sup¬ the same contingency requirement for each subject
pression of contingent responding is an important meant that each subject had to increase instrumental
factor in obtaining the reinforcement effect (see also responding by different amounts in order to maintain
Premack, 1965, p. 172), although the sample size is contingent responding at base line level. Phase 2
not large enough to argue persuasively either way. continued for 10 sessions and was followed by 2 ses¬
During the nine sessions of Phase 4, the subjects sions of recovery with free access to both wheel and
returned, in general, to the base line probability of tube.
responding. The results of this experiment are presented in Fig¬
In summary, the data from MarmarofFs thesis also ure 2. In Phase 1 of the experiment subjects 1 and 5
question the validity of Premack’s reward rule. A showed consistent preferences for running over the
knowledge of the independent probability of instru¬ last six sessions of the two-response base line. Subject
mental and contingent responses is apparently not an 3 revealed a consistent preference for drinking, and
adequate basis for predicting reward (and by impli¬ subjects 2, 4, and 6 failed to reveal any consistent
cation punishment) effects in a run-to-drink contin¬ preference.
gency. The introduction of the drink-to-run contingency
A second experiment has since been completed, us¬ in Phase 2 provided three types of probability rela¬
ing a drink-to-run contingency instead of the previ¬ tion, Subjects 1 and 5 participated in a low-to-high
ously employed run-to-drink contingency. In the first contingency; subject 3 participated in a high»to»low
experiment, the run-to-drink contingency reduced the
total amount of water intake during the early sessions
of training. It is possible that such a change in water
intake was responsible for the increase in instrumental
running as a general activity phenomenon (cf. Camp¬
bell 8c Lynch, 1968). By reversing the contingency and
establishing drinking as the instrumental event, the
confounded change in water deprivation is elimi¬

nated. In addition, the generality of the argument

against Premack’s assumptions is increased to include
the drink-to-run situation.
The second experiment followed the same pro¬
cedure used in the first. It was divided into three
phases: a two-response base line phase; a contingency
training phase; and a two-response base line recovery
phase. The single-response base line phases used in
the first experiment were not included as control con¬
ditions, largely because the reinforcement effects ob¬
tained in the first experiment were observed to exceed TWO SESSION BLOCKS
both the single- and the two-response base line.
During Phase 1 the animals were permitted free Fig. 2. Probability of running and drinking during the last
six sessions of base line (B), contingency (C), and recovery
access to the running wheel and drinking tube each (R) phases of the second experiment.
 106 THE NATURE OF REINFORCING STIMULI

contingency; and subjects 2, 4, and 6 represented the The final experiment in the Allison and Timber-
indifferent case. As seen in Figure 2, all subjects in¬ lake series followed the same procedure as the first
creased the probability of drinking to levels which two, except that the contingency arranged did not
exceeded the base line probability. A £-test for cor¬ require an increase in the amount of .4% licking in
related observations revealed this increase to be reli¬ order to receive the base line amount of .1% contin¬
able (t = 6.50, p < .05). The contingency also sup¬ gent licking. This procedure failed to increase the
pressed contingent running, and again the correlation .4% instrumental licking.
between the amount of suppression and the amount The results reported by Allison and Timberlake
of increase in instrumental responding was substan¬ add further support to the data which question Pre¬
tial but not reliable with the same sample (r = .74, df mack’s differential probability rules. To account for
5; p > .05). During the two sessions of base line recov¬ their findings, Allison and Timberlake (1974) and
ery in Phase 3, the responses tended to return to their Timberlake and Allison (1974) have outlined a posi¬
original base line levels. tion called the response deprivation hypothesis. For a
Along with the results reported by Eisenberger et particular contingency arrangement, response depriva¬
al. (1967), the data from these two experiments di¬ tion is identified “if the animal, by performing its
rectly question the validity of Premack’s reward and baseline amount of instrumental response, is unable
punishment assumptions. A contingent response was to obtain access to its baseline amount of the con¬
demonstrated to reinforce an instrumental response tingent response” (Timberlake & Allison, 1974, p.
whether the contingent event was higher or lower in 152). They suggest that the necessary and sufficient
probability than the instrumental response. condition for an increase in instrumental responding
More recently a third series of experiments based is that the instrumental contingency employed pro¬
upon the same rationale has been reported by Allison duce the response deprivation state. The response
and Timberlake (1974). Their first experiment was deprivation hypothesis, as outlined by Timberlake
designed to demonstrate that a less probable response and Allison, does not differ in any essential manner
would reward a more probable instrumental response from the response suppression hypothesis offered
if the contingency was arranged so that the animal earlier by Eisenberger et al. (1967). In future discus¬
had to increase the probability of the instrumental sion these two positions will be treated as identical
response in order to obtain the base line (free-access) and referred to as the response deprivation hypothesis.
amount of contingent responding. In the first phase Taken together, the evidence obtained in the three
of the experiment, albino rats were given daily 10-min independent lines of experimentation which have
base line sessions in which they had simultaneous ac¬ been discussed in this section permit two conclusions:
cess to two drinking tubes. One tube contained a .4% (a) all of the experiments described directly question
saccharine solution, the other a .3% solution. The rats the validity of Premack’s differential probability rules;
consistently preferred the .4% solution during this and (b) the data consistently suggest that the instru¬
base line exposure. Following base line sessions, con¬ mental contingency must force the amount of con¬
tingency sessions were conducted in which the rats tingent responding below its base line operant level
were required to lick the .4% solution for 80 sec in in order to observe an increase in instrumental re¬
order to gain access to the .3% solution for 10 sec. sponding—i.e., the response deprivation hypothesis.
This contingency arrangement required that the rats
increase the amount of .4% instrumental licking if
Some Control Considerations
they were to obtain their usual base line amount of
.3% contingent licking.
The Yoked Control Procedure
Contrary to Premack’s reward rule, this high-to-
low contingency increased the amount of instrumen¬ Before the evidence leading to the two above-
tal .4% licking to levels which exceeded the base line. mentioned conclusions can be completely accepted, a
The amount of contingent licking was suppressed by number of control questions need to be considered.
the contingency, and, curiously, the rats licked the First and perhaps most important, it is obvious that
contingent .3% solution for only a small percentage the introduction of most instrumental contingencies
of the time that it was made available during the will drastically change the manner in which the or¬
contingency sessions. A second experiment using .4% ganism normally distributes the contingent response
and .1% saccharin solutions, as instrumental and in time when compared to the base line performance.
contingent events respectively, corroborated the re¬ It is possible that many of the changes observed with
sults obtained in the first study. the use of Premack’s typical experimental procedure
Philip Dunham 107

can be produced by simply changing the manner in tube and a running wheel during daily half-hour ses¬
which the organism is permitted access to the contin¬ sions (Dunham, 1972). Subsequent removal of access
gent e\ent in time without arranging an instrumental to either one of these responses by braking the wheel
contingency. In order to examine this possibility, the or removing the tube during the session produced an
most appropriate control procedure would be to in¬ increase in the amount of the alternative behavior
clude a group of animals in each experiment which observed during the session. Contrary to the observa¬
were yoked to the instrumental contingency group tions made by Eisenberger et al., it would appear that
and received the reward in the same temporal pattern removal of a response from the animal’s repertoire
without any contingency in effect. With the yoked will, in some cases, produce an increase in an avail¬
control group, one should be able to judge the effects able alternative. In our case, the two responses mea¬
which the change in the temporal distribution, as sured were topographically quite different.
well as the amount of contingent responding per se, Bernstein (1973) has also recently reported data
had upon the designated instrumental behavior. Pre¬ relevant to this control question. He used a fascinat¬
mack (1965, pp. 166-172) recognized this problem ing procedure in which human subjects were placed
and suggested that the reduction in the amount of in a controlled experimental environment contain¬
contingent responding usually produced by an in¬ ing various items of interest (e.g., sewing materials,
strumental contingency may, along with the low-to- reading, art work). The subjects lived continuously in
high probability differential, be necessary for an in¬ this environment for periods as long as 34 consecutive
crease in the probability of instrumental behavior. days. The amount of time which each subject spent
Various approximations to the yoked control pro¬ indulging in a number of selected activities was
cedure described above can be found in some of the measured on a 24-hr basis. Following the measurement
experimentation concerned with Premack’s theoretical of base line durations, Bernstein arranged various con¬
position. Most of them attempt to control for the tingencies in which the subjects were required to
reduced amount of contingent responding and do not engage in a fixed amount of a less probable behavior
consider changes in the temporal distribution of the in order to gain access to a fixed amount of a more
contingent response produced by the contingency. The probable behavior (e.g., read fiction in order to sew).
results of these efforts have been inconsistent. Eisen- All of the contingencies studied, with one exception,
berger et al. (1967) recognized the possibility that sim¬ increased the amount of instrumental behavior ob¬
ple suppression of the contingent response by the served and suppressed the amount of contingent be¬
instrumental response requirement might produce an havior below base line levels. In the case where no
increase in the instrumental behavior. They con¬ increase in instrumental behavior was observed, it is
ducted a control experiment (Experiment III of their interesting to note that the contingent response was
series) in which they removed access to the lever-press not suppressed by the contingency requirement.
response (nominally their contingent response) and More important in the present context is Bern¬
measured the effects of such removal upon the wheel- stein’s use of a matched control procedure in which
turning response (nominally their instrumental re¬ the same subjects were subsequently exposed to ses¬
sponse). Although the complete removal of access to sions in which the previously designated contingent
the contingent response is not as informative as the response was simply restricted in the same manner in
yoked control described earlier, it does control for the which ths previous contingency had restricted the
effects of simply reducing the amount of contingent response, without any instrumental response require*
responding produced by the contingency. The results ment. This procedure come? very close to the yoked
revealed that removal of access to the lever-press re¬ control procedure described earlier. In two of the
sponse reduced, rather than increased, the amount of five instrumental contingent response pairs which
wheel-turning behavior. Hence they concluded that were observed, Bernstein found that simply restricting
the instrumental requirement was a necessary aspect the amount of contingent activity (with no instrumen¬
of their procedure and that contingent response re¬ tal response required) was sufficient to produce in¬
duction alone was not sufficient to increase the proba¬ creases in the response which had previously been
bility of the instrumental response. designated as the instrumental response. Again we
A more recent experiment suggests that the results see that restricting the contingent response in the
obtained in the control experiment reported by Eisen- absence of an instrumental response requirement has
berger et a}, may have limited generality. While work¬ inconsistent effects. In some instances an increase in
ing with a different context, I conducted an experi¬ the nominal instrumental behavior is observed, in
ment in which rats were given free access to a drinking other cases no changes occur.
108 THE NATURE OF REINFORCING STIMULI

Allison and Timberlake (1974) also attempted to estimate of “momentary response probability” and
assess the effects of a restriction in the contingent re¬ consequently of the reward (or punishment) value of
sponse in the absence of an instrumental response a response. The first case described is a situation in
requirement. In Experiment 3 of their series they which the two responses to be used in the contingency
observed that the amount of unconstrained .4% sac¬ have a different rate of decay (habituation or satia¬
charin drinking (the nominal instrumental response) tion) within a session. If, for example, drinking is
was the same in both the presence and the absence of more probable than running during the first 10 min
the .1% solution (the nominal contingent response). of a 1-hr session, but substantially less probable during
In this case, completely suppressing the contingent the last 50 min of the session, a total-duration measure
response appears to have no effect upon instrumental would indicate in some cases that running was the
behavior in the absence of the contingency—and it more probable of the two. Hence in subsequent con¬
might be noted that the topography of the instru¬ tingency sessions a run-to-drink contingency would be
mental and contingent response was identical: licking considered to be a high-to-low contingency based
a drinking tube. upon total-duration measures, but actually should be
The results produced by these approximations to considered a low-to-high contingency for the first 10
the yoked control procedure suggested in earlier dis¬ min of the session and a high-to-low contingency dur¬
cussion are inconsistent. Simple removal of the con¬ ing the last 50 min. At first glance, it does seem rea¬
tingent response from the unconstrained repertoire sonable to stipulate that a fair test of the probability
has been observed to produce increases, decreases, and differential rules require that these within-session
no change in an alternative response designated as the duration-time curves not intersect. In fact, in the run¬
instrumental behavior. If we are going to properly ning and drinking experiments discussed earlier we
assess the relative merits of Premack’s differential have measured changes in the probability of drinking
probability rules and the response deprivation hy¬ and running in consecutive 20-min segments of the
pothesis, it behooves us first to determine whether the session, and in four subjects so observed both re¬
increases in instrumental behavior observed in these sponses decline over the 1-hr session but the curves do
experiments are the product of the reinforcement op¬ not cross over. Eisenberger et al. (1967) also failed to
eration (i.e., the response requirement) or whether observe within-session crossovers of response proba¬
the increase reflects an interaction between uncon¬ bility in their experiments. Presumably, then, these
strained responses which would have occurred in the data would meet the criterion specified by Premack,
absence of any instrumental contingency. As I have and total duration is a good estimate of “momentary
noted in another context (Dunham, 1971), the ques¬ response probabilities.”
tion of what changes will occur in an unconstrained A more basic problem arises, however, when one
multiple-response repertoire when we restrict (or in¬ considers how “momentary” a momentary probability
crease) one or more members has received very little estimate should be. If, for example, one divided an
experimental attention. At present we have little entire session of running and drinking into successive
more than our intuition upon which to base our 5-min segments, a number of reversals in response
predictions about such changes (cf. Bernstein, 1973). probability would be observed over the course of the
session. Using this latter time scale on the abscissa,
our experiments would not be considered a proper
Premack’s Control Requirements
test of Premack’s position. As suggested earlier, the
In addition to the yoked control question discussed concept “momentary probability” needs to be defined
above, Premack (1971) has outlined a number of more precisely if it is to be anything other than a
procedural problems which he suggests will invalidate post hoc analysis of obstreperous results.
tests of the differential probability rules. We shall A second stipulation, if we are to obtain good
now briefly consider each of these problems as they estimates of “momentary probability,” is that the
relate to the negative evidence which has accumulated parameter values used during the contingency ses¬
concerning Premack’s position. sions be the same as those used to measure the inde¬
All three of the procedural problems discussed by pendent probability of responding during base line
Premack revolve around a loosely defined concept of sessions. Premack (1971) suggests:
“momentary response probability.” Specifically, he
suggests that there are three conditions under which For example, if the reinforcement session is to
the total-duration measure of responding obtained use a [variable-interval] 60 second schedule with
during base line sessions is likely to be a distorted a contingent [response] time of 5 seconds, then
 Philip Dunham 109

very frequently, but when they do occur they should

be considered to be at the top of the preference struc¬
ture. With all due respect to the data of introspection,
this stipulation reduces in operational terms to the
suggestion that the duration of some responses is not
a proper estimate of their relative position in the
Probability

preference structure. Again, an experimenter who

wishes to comply with this stipulation must have
some criterion for knowing when he is dealing with a
response like Response A in Figure 3. The unique
property of Response A appears to be that it has a
relatively long interval which separates successive in¬
stances of the behavior and a high position in the
Fig. 3. See text for explanation. (From Premack, 1969, p. 132.) preference structure when an instance of the behavior
is observed. The unique property of Response B is
that it has a relatively short interval which separates
exactly those parameters should be used in the
successive instances of the behavior and a relatively
measurement of response probabilities, (p. 130)
low position in the preference structure when an

Unless this requirement is met, it is suggested that instance of the behavior is observed. Obviously all
the total duration measure will be a distorted esti¬ responses have these two properties in differing de¬
mate of “momentary response probability." Again, grees. Unless it is specified how long an interresponse
however, this requirement poses some problems for interval, for a given session length, is required for
membership in the class of Response A, of how short
the experimenter who wishes to meet the conditions
specified. Unless the experimenter exercises complete an interval is required for membership in the class of
Response B, we cannot know if we have underesti¬
control over the onset and offset of both instrumental
mated or overestimated the “momentary probability''
and contingent responses (a condition which would
eliminate the response requirement), it is the case that of these responses by measuring their total duration.
With reference to our running and drinking experi¬
any instrumental contingency which we arrange will
produce some changes in the manner in which the ments, we typically observe longer interresponse inter¬
vals for running than we do for drinking during the
organism distributes its behavior in time when com¬
pared to base line performance. Consequently, it is 1-hr session, but one cannot determine whether run¬
ning deserves membership in the special class of re¬
impossible strictly to meet the requirement imposed
sponse A from the information provided in Figure 3.
by Premack. The best one can do is keep the base line
To summarize briefly, it appears to be difficult, if
and contingency situations as similar as possible and
depend upon the yoked control discussed earlier to not impossible, to meet the requirements specified by
Premack for a valid test of the differential probability
detect any changes in instrumental response probabil¬
rules. In the experiments cited as evidence against
ity produced by the temporal constraints placed upon
contingent behavior. the differential probability rules, it is possible to sug¬
gest that the total-duration measures are, indeed, a
The third procedural problem discussed by Pre¬
mack (1971) best bo explained by reference to
can
distorted estimate of the “momentary probability” of
Figure 3. Referring to this figure, he says: the contingent responses employed. I would argue
that such an interpretation is necessarily an ad hoc
analysis until such time as the concept “momentary
Response A depicted in the curve on the right
attains an extremely high probability at rela¬ probability" and its relationship to the total duration
tively long intervals, whereas response B shown measure are more precisely defined.
on the left attains half that probability but at
half the interval. The average probability of the
two responses are thus equal; however, their An Optimal-Duration Model as an
momentary reinforcement values will not be Alternative Interpretation
equal, (p. 131)
If we assume that subsequent research continues to
The spirit of this suggestion is that certain re¬ question Premack's position and to support the re¬
sponses like copulation (Response A) do not occur sponse deprivation hypothesis, we are placed in a
no THE NATURE OF REINFORCING STIMULI

curiously counterintuitive position. The evidence +

cited thus far suggests that the organism will, under

certain conditions, increase the probability of a re¬
sponse which is instrumental in placing it in a less
preferred state—when preference is measured in terms
of the total duration of responding during base line
sessions. For example, most people, given the choice,
would prefer eating in a nice restaurant to visiting
the dentist. Taken literally, the response deprivation
hypothesis suggests that it is possible to increase my
patronage of local restaurants by arranging a contin¬
gency in which I must go to a restaurant in order to
visit my dentist, and that this contingency will work
only if it reduces the amount of time I spend at the
dentist below my base line, unconstrained level of
dental care.
Rather than accept such a counterintuitive notion,
it would seem more judicious at present to suggest
that the total-duration measure is a distorted measure
of preference. It may still be possible to develop an
alternative measure which will permit us to maintain Fig. 4. Hypothetical frequency distributions for burst durations
Premack’s fundamental, commonsense assumptions and interburst intervals of paper shredding in the gerbil. As
the burst-duration distribution is displaced to the left, the
about a symmetrical reward-punishment mechanism paper-shredding behavior is a preferred event; as it is displaced
that depends upon the organism’s preference struc¬ right, paper shredding acquires negative properties. As the
ture. interburst interval distribution is displaced to the left, the
paper shredding acquires negative properties; as it is displaced
For the remainder of the discussion in this section to the right, it is a preferred event.
of the chapter, I shall outline, in rudimentary form,
a reinforcement model which maintains many of the
features of Premack’s analysis but suggests a basic is 10 sec, and all instances observed during base line
change in the manner in which we measure the or¬ sessions fall between a minimum duration of 1 sec and
ganism’s preference structure. It is designed to detect a maximum duration of 20 sec. Looking now at the
the momentary changes in preference that are neces¬ frequency distribution of interburst intervals, the
sarily obscured by the total-duration measure. most frequently observed interval between successive
The most convenient way to describe the essential bursts of paper shredding is 60 sec (point A' on the
features of the model is to make reference to the solid curve) with a minimum interval of 1 sec and a
multiple-response repertoire of our gerbil again. As¬ maximum interval of 120 sec.
sume for the moment that our observations of the I suggest that a knowledge of these two distribu¬
gerbil for a 20-min period each day revealed that the tions, and of the variables which alter them, is essen¬
gerbil spent most of its time shredding paper. Given tial for predicting whether a particular contingent
a definition for the onset and offset of the paper- event will function as a reward or as a punishment in
shredding behavior, two properties of the behavior an instrumental contingency. If we consider, for ex¬
are considered important. The first property, called ample, paper shredding as a contingent event for
the burst durationrefers to the amount of time spent instrumental running behavior, the base line observa¬
paper shredding once the animal enters the state. tions we have made of paper shredding tell us how
Second, the interhurst interval, refers to the amount the animal prefers to expose itself to paper shredding
of time observed between successive bursts of paper over the time permitted in the session. These base
shredding. Assume that we have made these tedious line observations should, of course, be made with run¬
observations of the gerbil’s paper-shredding behavior ning also available, since the presence of the running
over a period of several unconstrained base line ses¬ response per se will very likely be one of the variables
sions and the results are those observed in Figure 4. which would alter the burst duration and interburst
Looking first at the frequency distribution for burst interval distributions of paper shredding. Assume that
durations (solid-line curve), the most frequently ob¬ we can regulate the exact amount of paper shredding
served duration represented as point A on the curve which the gerbil is permitted for each completion of
Philip Dunham 111

the instrumental response requirement—specifically, ability rules and the response deprivation hypothesis,
that we can either give the gerbil access to the paper should also be considered in the context of the present
for durations shorter than its preferred (average) base model. It would be interesting, for example, to ob¬
line duration or force the gerbil to shred paper for serve the effects upon other unconstrained responses
durations longer than its preferred (average) base line in the gerbil’s repertoire of placing various temporal
burst duration. constraints upon paper shredding (changing the burst
To the extent that the contingency which we ar¬ duration and interburst interval distributions with or
range between running and paper shredding shortens without changes in total durations). It is possible that
the intervals between successive bursts of paper shred¬ there would be systematic changes in other responses
ding and/or lengthens the duration of a burst (rela¬ similar to those produced by arranging an instru¬
tive to base line durations), I would suggest that mental contingency. These operations make some con¬
paper shredding will function as an aversive event. tact with the recent suggestions made by Baum (1973)
To the extent that the contingency we arrange concerning a correlation-based law of effect.
lengthens the interval between successive bursts of Since the predictions made by this conceptual
paper shredding and/or shortens the bursts of paper scheme are made with reference to the optimal dura¬
shredding (relative to base line durations), I would tions observed during base line sessions, the model
suggest that paper shredding will function as a posi¬ will be called an optimal-duration model in future
tive event. Finally, if the contingency we arrange discussion.
simply feeds paper shredding back to the gerbil in a To conclude the discussion of the optimal-duration
way that does not alter his record of self-exposure (as model, it is instructive to describe one additional
manifested in the two distributions), we should ob¬ hypothetical procedure which would pit the predic¬
serve neutral motivational properties as evidenced by tions made by this model against both Premack's
no change in the probability of instrumental be¬ differential probability rules and the more recent re¬
havior. sponse deprivation hypothesis. Because some of the
These general predictions made by the model can predictions to be discussed require us to “force” the
be made more specific by reference to Figure 4. Using contingent response for durations which exceed the
these hypothetical data for paper shredding, it is sug¬ optimal duration, we shall consider a modification of
gested that given no change in the distribution of in¬ the gerbil environment described earlier in which the
terburst intervals, paper shredding will function as a animal now has access to only two sources of enjoy¬
reward as long as the contingency we arrange dis¬ ment. Assume that the gerbil is placed in a motorized
places the distribution of burst durations in the direc¬ activity wheel which can be activated by simply
tion of durations less than the optimal duration of rotating the wheel 10 deg in either direction. In addi¬
10 sec as observed during base line sessions. Con¬ tion, simply rearing up on the hind legs is sufficient to
versely, assuming that we can force the behavior, produce an intracranial stimulus (ICS) in the lateral
paper shredding will function as an aversive event as hypothalamic area using shock parameters withm the
long as the contingency we arrange displaces the dis¬ “positive” range. Dropping back on all four feet ter¬
tribution of burst durations in the direction of dura¬ minates the ICS,
tions greater than the optimal duration of 10 secs (see Assume also that half-hour unconstrained access to
the broken-line curves). each of these events reveals that the gerbil spends a

Given no change in the distribution of burst dura- total of 10 min per session running in the wheel and a
tionsj paper shredding will function as a reward as total of £ min receiving the ICS, Consider tho prsdic-
long as the contingency we arrange displaces the dis¬ tions of the three models when we arrange some
tribution of interburst intervals in the direction of possible instrumental contingencies between these
intervals greater than the optimal interburst interval two responses.
of 60 sec, as observed in base line sessions. Conversely, The predictions made by Premack's differential
assuming that we can force the behavior, paper shred¬ probability rules are quite clear. Running is the more
ding will function as an aversive event as long as the probable contingent event; hence a self-stimulate-to-
contingency we arrange displaces the distribution of run contingency (low-to-high probability) should pro¬
interburst intervals in the direction of intervals duce an increase in the amount of self-stimulation the
shorter than the optimal interburst interval of 60 sec. animal will take. Conversely, a run-to-self-stimulate
To digress briefly, it might be noted that the yoked contingency (high-to-low) should produce a decrease
control procedure, discussed earlier with reference to in running.
the experimental tests of Premack’s differential prob¬ The predictions made by the response deprivation
 112 THE NATURE OF REINFORCING STIMULI

hypothesis are very different. According to this posi¬ each instance of the contingent response be shorter
tion, either a self-stimulate-to-run or a run-to-self- than the optimal duration of base line bursts; and/or
stimulate contingency will produce increases in instru¬ (b) the intervals between instances of contingent re¬
mental responding if the contingency arranged sponding be longer than the base line optimal dura¬
reduces the amount of the contingent event below the tion of interburst intervals. The total duration of
base line level. Hence, contrary to Premack’s rules, the contingent responding can remain the same as it was
run-to-self-stimulate contingency will, under response during base line sessions, and it is predicted that
deprivation conditions, increase the probability of either of the above constraints on the contingent be¬
running. havior will be sufficient to produce an increase in
According to the optimal-duration model, either of instrumental responding.
the above contingencies (run-to-self-stimulate or self- Although we do not yet have appropriate data to
stimulate-to-run) can be demonstrated to produce support the optimal-duration model, I would argue
either the reward or the punishment outcome or no that it provides a viable alternative interpretation to
change depending upon how we distribute the con¬ the response deprivation hypothesis and has the ad¬
tingent event in time. Consider, for example, a high- vantage of specifying the conditions under which we
to-low contingency in which the animal is required to should observe increases, decreases, and no change in
run in order to self-stimulate. Assume, also, for the instrumental behavior. As stated earlier, the optimal-
moment that our base line observations indicated that duration model is, in some sense, an attempt to trans¬
the optimal duration for self-stimulation was 5 sec late Premack’s notion of “momentary probability”
with an optimal-duration interburst interval of 30 sec into operational terms. Basically, the two distributions
between successive stimuli. If we now arrange the described are assumed to provide a better basis than
contingency in which each ICS is 5 sec in duration, does the total duration of responding for predicting
the ICS will increase the probability of running as when a contingent event will be more or less probable
long as the interburst interval distribution is dis¬ than an instrumental event. In more recent work,
placed in a direction of intervals greater than the Premack and his students appear to recognize the
optimal-duration interburst interval (60 sec). Con¬ possible importance of the organism’s more momen¬
versely, if the interburst interval produced by the tary tendencies to turn a response off or on. In the
contingency is shorter than the 60-sec optimal dura¬ punishment study by Terhune and Premack (1970)
tion, a 5-sec ICS will be aversive. On the other hand, mentioned earlier the amount of suppression pro¬
if the contingency we arrange does not alter the inter¬ duced by a less probable running response was demon¬
burst interval distribution, we can make the ICS strated to be a linear function of the probability that
aversive by displacing the distribution of burst dura¬ the animal would turn off the running response at
tions in a direction greater than the optimal duration fixed times after its initiation. Obviously, the prob¬
of 5 sec, or we can make it positive by displacing the ability that the organism will turn off a particular
distribution of burst durations in a direction less than response some fixed time after its initiation is readily
the 5-sec optimal duration. derived from the burst duration distribution, which is
Finally, if one feeds back the ICS as a tape record¬ one component of the optimal-duration model. Hence
ing of the pattern revealed by the animal during the the results reported by Terhune and Premack can be
base line observations, it is predicted that the event subsumed by both the optimal-duration model and
will be motivationally neutral as indicated by no Premack’s differential probability rules.
change in the probability of the instrumental re¬
sponse.
These predictions differ from those made by Pre¬ BIOLOGICAL CONSTRAINTS ON
mack in specifying that there are certain conditions in REINFORCEMENT
which a high-to-low probability contingency (as de¬
fined by total duration of base line responding) will With the possible exception of autonomically in¬
produce an increase in the probability of the instru¬ nervated responses (cf. Miller, 1969), we have typically
mental response. The predictions also differ from assumed that all responses in an organism’s repertoire
those made by the response deprivation hypothesis. are equally eligible for modification using an instru¬
The optimal-duration model does not require that mental learning procedure. Premack’s differential
the amount of contingent responding permitted dur¬ probability rules, the response deprivation hypothesis,
ing a session be suppressed below the base line total and the optimal-duration model described in the first
duration of that response. It requires instead that (a) part of this chapter all reflect this assumption. These
Philip Dunham 113

positions specify the necessary and sufficient conditions the role of biological constraints in instrumental
for the observation of changes in instrumental be¬ learning situations. I have attempted to restrict the
havior and assume that these conditions will hold, in discussion to the most recent data and to arguments
general, for any pair of responses in any particular specifically concerned with constraints upon instru¬
species. mental response-reinforcer combinations. For a more
Recently, however, evidence has been accumulating extensive discussion of earlier work and a broader
which suggests that certain responses may not be sus¬ perspective, the reader is urged to consult a number
ceptible to such modification while others are highly of different reviews already available in the literature
susceptible. The general explanation which has (cf., Bolles, 1970; Garcia, McGowan, & Green, 1972;
emerged to account for such evidence is the notion Moore, 1971, 1973; Seligman, 1970; Shettleworth,
that the organism brings certain “biological predis¬ 1972, 1973; Staddon 8c Simmelhag, 1971).
positions” into the instrumental learning situation The various attempts to explain why particular
which can either facilitate or inhibit the effectiveness response-reinforcer combinations are effective while
of certain response-reinforcer combinations (cf. Selig- others are not are, in my opinion, all variations on a
man, 1970). common theme. The general theme is that an orga¬
An example of the type of evidence which has pro¬ nism has certain species-typical behavior patterns which
vided the basis for speculation about such biological it exhibits under certain conditions in its natural
constraints is Shettleworth’s (1973) recent work. She habitat. To the extent that a particular response-rein¬
has demonstrated that food is an effective reinforce¬ forcer combination requires behavior which is con¬
ment for a number of responses emitted by the ham sistent with the organism’s natural behavior, the rein¬
ster, Mesocricetus auratus, but will not function as a forcer will be very effective in controlling responding;
reward for various grooming responses such as' face to the extent that the combination is inconsistent with
washing. Data such as these have at least two major the organism’s natural behavior, the reinforcer will be
implications in the context of the present discussion. ineffective. I shall consider several variations on this
First, they directly question the assumption that rein¬ theme which are to varying degrees successful in trans¬
forcers are transituational. If food is an effective re¬ lating this idea into a testable proposition.
ward for rearing responses made by hamsters, it
should be an effective reward for face-washing re¬
The "Functional Relevance" Hypothesis
sponses in the same species. Hence, along with Pre¬
mack's model, the response deprivation hypothesis, The basic notion is that certain responses are func¬
and the optimal-duration model discussed earlier, the tionally relevant in the context of certain rein forcers
research concerned with biological constraints on but not in the context of other reinforcers. For
instrumental learning runs counter (for very different example, in hamsters, digging responses are usually
reasons) to the contemporary tendency to accept the observed as a part of procuring food, but not seen in
weak law of effect as a point of departure in our situations involving sexual behaviors. The functional
analysis of reinforcement. Second, an adequate theory relevance hypothesis suggests that a sexually receptive
of reinforcement will have to take account of any evi¬ female would not be particularly effective as a rein-
dence that there are certain response-rein forcer com¬ forcer for digging, since digging is not particularly
binations which arc not effective (or vice versa) and relevant in the organism's species-typical reactions to
will have to describe the conditions under which we sexual stimuli.
can expect to observe such constraints. Three representative versions of the “functional
Consider, for example. Premack’s notion that a relevance” hypothesis have emerged in the context of;
low-to-high probability contingency will produce an (1) Shettleworth’s work with instrumental reward pro¬
increase in the probability of instrumental respond¬ cedures; (2) work by Walters and Glazer (1971) and
ing. This assumption is directly questioned by Shettle¬ Melvin and Ervey (1973) with instrumental punish¬
worth’s hamster data. Eating was more probable than ment procedures; and (3) Bolles’s (1970) work with
either rearing or face washing. Nevertheless, eating avoidance learning procedures.
would reinforce rearing but not face washing. Pre¬ Shettleworth (1973) observed golden hamsters in a
mack’s position seems at the very least to require an chamber similar to an open field apparatus which
additional assumption to subsume these discrepant had a sand floor, a magazine for delivering food pel¬
cases. lets, and a typical response lever. A number of species-
In the discussion which follows I shall critically typical behavior patterns were measured: (1) face
examine some of the theory and data concerned with washing, (2) digging, (3) open rearing, (4) “scrab-
114 THE NATURE OF REINFORCING STIMULI

bling” (which is a type of digging movement with the

forelegs on the side walls), and (5) bar pressing (shap¬
ing was necessary). After reducing the animals to 80%
of their ad lib weights, different groups were exposed
to five possible instrumental contingencies with food
as the contingent event (e.g., scrabble to eat, face-wash
to eat, etc.). Contingent upon performing the appro¬
priate instrumental response for a period of .5 sec,
the food pellets were delivered on a variable-interval
(VI) 20-sec schedule (following initial sessions of con¬
tinuous reinforcement). The results demonstrated con¬
vincingly that the hamsters increased the probability
of four of the five instrumental responses. Face wash¬
ing was the only instrumental response which could
not be brought under the control of the reinforce¬
ment contingency, even under extended training con¬
ditions with continuous reinforcement.
To account for these results, Shettleworth suggested
that face washing was perhaps the only response of the
five selected which was not a part of the animal’s nat¬
ural behavioral sequence in obtaining food. There
are, however, a number of alternative explanations.
First, as Shettleworth recognizes, the hamster needs a
supply of saliva in order to lick its paws for face wash¬
ing. It is possible that the use of the dry food pellets
acts to decrease the amount of face washing by using
up excess saliva for food mastication. Or the rate of
saliva production per se may simply be inadequate for
the maintenance of high rates of face washing; i.e.,
this may be a trivial instance of requiring an animal
to perform a response which is beyond its physical
capability (like training a rat to fly). As Shettleworth
suggests, it would be interesting to repeat these experi¬ Fig. 5. Paper-shredding and eating response probability as a
ments with water as the reinforcer for thirsty hamsters, function of the presence and absence of a running wheel in
thus eliminating the problem of a “dry mouth.” the environment.
There is another explanation for these data that is
of particular interest because it has general implica¬ The animal spent 80% of the session running in the
tions for research of the type reported by Shettle¬ wheel, a small amount of time eating seeds, and no
worth. One of the several effects of arranging an in¬ time at all shredding the bristol board. In Phase 2
strumental contingency is that the contingent event gerbil A was denied access to the running wheel, and
adds a response to the organism’s repertoire in the the gerbil added paper shredding to the response reper¬
experimental situation. The food used in Shettle- toire and increased the amount of time spent eating.
worth’s experiments produced a certain amount of Subject B was exposed to the same conditions in
eating behavior. The introduction of this response to reverse order. During Phase 1 the wheel was not avail¬
the organism’s repertoire can, in and of itself, facil¬ able, and paper shredding was the most probable
itate or inhibit other instrumental responding. An response in the response repertoire. In Phase 2 the
extreme case of this can be seen in Figure 5, which addition of the wheel to the chamber reduced and
presents the results of a simple experiment conducted eventually eliminated paper shredding, and the ani¬
in our laboratory. mal spent 70-80% of the session running.
Subject A was a hungry gerbil given free access to Thus the addition of running to the response
three items in an experimental chamber for 1 hr each repertoire of the gerbil eliminates paper shredding.
day. The items were a running wheel, a box of sun¬ The results in general suggest that the simple addition
flower seeds, and a strip of bristol board for shredding. or deletion of a response from the repertoire can have
Philip Dunham 115

profound effects upon the probability of other exist¬ expected digging to be relatively easily sup¬
ing responses. Although the simple experiment has pressed by punishment, while alert posturing,
yet to be completed, it would not be surprising to find since it itself is a reaction to sudden or aversive
that running is not a terribly effective reward for stimuli, was expected to show little suppressive
paper shredding, since the former has direct inhib¬ effect, (p. 332)
itory effects upon the latter when the two are observed
together in the complete absence of the instrumental Following their base line observations of digging
contingency operation. and alert posturing, and a series of habituation ses¬
It is possible that the basis for predicting the effec¬ sions during which a tone was made contingent upon
tiveness of particular response-reinforcer combinations either alert posturing (eight gerbils) or digging (eight
is an understanding of the conditions under which gerbils), the gerbils were split into four groups. Two
interactions occur between responses in the absence of groups received daily classical conditioning sessions
the contingency. The implication of these observa¬ in which a tone was paired with a relatively strong
tions is that a control condition should always be em¬ shock (2 mA, 1 sec). These conditioning sessions were
ployed in research of the type reported by Shettle- conducted inside a plexiglass chamber with a grid
worth in which we measure the probability of the floor which was placed inside the larger sand chamber.
instrumental response in the simple presence and Two control groups received identical treatment with¬
absence of the contingent response prior to arranging out the shock being administered. Following each
any contingency between the two. The reader might daily conditioning session, the gerbils were placed
note that the rationale for this control is identical to back in the experimental chamber with the sand floor
the rationale proposed for the use of a yoked control and the tone was made contingent upon the digging
procedure in the context of Premack’s position (p. response (8 gerbils) or upon posturing responses (eight
106). gerbils). When compared to control subjects which
Shettleworth gives some attention to this problem had not received the tone-shock pairings, digging was
of direct inhibitory effects in preliminary work which suppressed by the presentation of the conditioned
assessed the effects of both food deprivation and pellet punisher (tone) contingent upon digging. However,
delivery upon the various behaviors measured in the alert posturing was facilitated by presentation of the
experimental situation. Her observations suggest that conditioned punisher. Walters and Glazer conclude
such inhibitory interactions among responses might that the differential effect of the conditioned punisher
account for the results. For example, she notes that can best be understood in terms of the differential bio¬
food deprivation (which presumably increased the logical functions served by these two responses in the
probability of eating) appeared to inhibit face-wash¬ gerbil’s natural environment. Digging is presumably
ing activity, with little effect upon the other responses. associated with nest building and food gathering, and
It was not clear from her results if the simple addition alert-posturing with the animal's defensive reactions,
of the food pellets per se had any effects upon the There is, however, a plausible alternative explana¬
alternative responses. tion for these results. We have observed, informally*
The counterpart to Shettleworth’s work with “ap¬ that these desert rodents appear to reduce the aversive,
petitive” response-remforcercombinations is some ness of grid shocks by rocking back on the insulating
recent work concerned with the possible biological con¬ fur of th^if hind kgs mto a position which very much
straints upon “aversive” response-reinfo^cr combina¬ resembles alert posturing as described by Walters and
tions, Walters and Glazer (1971), for example, suggest Gla2er. Hence, it seems possible that the animals ex¬
that the organism's species-typical responses to aver posed to the classical conditioning procedure might
sive stimulation may not be susceptible to the usual have learned this type of alert-pogturmg response as
suppressive effects of response-contingent punishment. a preparatory response to the tone in anticipation of
They observed the behavior of Mongolian gerbils in a the impending shock. During subsequent punishment
large chamber with a sand floor. Two responses, alert training, the gerbil’s reaction to the tone would then
posturing and digging in the sand, were selected for be the conditioned “posturing” response. This would
study from the gerbil’s repertoire. The rationale for also account for their observation that animals pun¬
selecting these two behaviors was as follows: ished for digging also increased the amount of alert
posturing during suppression of the digging response.
Given the different biological functions of these Presumably, if digging could have served as a
behaviors, it was felt that they would be differ¬ preparatory response during classical conditioning
entially affected by punishment. Specifically, we sessions, results exactly opposite to those reported by
116 THE NATURE OF REINFORCING STIMULI

Walters and Glazer could have been obtained. Any might be obtained by simply presenting a novel stim¬
reference to the presumed biological function of the ulus to the 7L group instead of the mild electric shock
response in the animal’s natural habitat would be which was assumed to be aversive. Brimer (1970) has
unjustified. demonstrated that the effects of a novel stimulus upon
Until appropriate controls are added to determine an ongoing response are dependent upon the operant
if such preparatory behaviors were acquired during level of that response. For a given response, the pres¬
the classical conditioning phase of the study, one can¬ entation of a novel stimulus will reduce the rate of
not accept the interpretation offered by Walters and responding if that rate is very high or increase (dis-
Glazer. inhibit) the rate of responding if the base line re¬
A second, more recent experiment, which claims to sponse is very low. Since gill extension had habituated
have demonstrated that the organism’s species-typical to about half its original probability when the 7L
reactions to danger or threat are not susceptible to the group received the mild shock, it is possible that the
usual suppressive effects of punishment, was reported increase was an instance of disinhibition similar to
by Melvin and Ervey (1973). These authors punished that described by Brimer. A novel stimulus control
the gill-extension response which is part of the aggres¬ group would presumably answer this question.
sive display of Siamese fighting fish (Betta splendens) In view of the problems with the preceding experi¬
in response to the presence of a conspecific (or mirror ments and with a number of other experiments in the
image). In this experiment each fish was exposed to literature which suggest that elicited defensive or
its mirror image for 60 consecutive trials, each 2 min aggressive reactions can be suppressed by response-
in duration, with an intertrial interval of 40-60 sec. Us¬ contingent punishment (cf. Azrin, 1970; Baenninger
ing this basic procedure the fish were randomly assigned 8c Grossman, 1969), the view that the animal’s defen¬
to one of five different treatment conditions. Group HC sive response repertoire is refractory to the effects of
was a habituation control to monitor the decline in punishment contingencies seems questionable at this
responding which is typically observed over trials point.
with this procedure; two groups, 7E and 7L, received Having considered the recent evidence which ar¬
a 7-V electric shock as the punishing stimulus gues for important biological constraints in the con¬
either early in training (trials 16-30), or late in train¬ text of reward and punishment contingencies, we
ing (trials 45-60). Similarly, two groups, 13E and 13L, can now turn to the third variation on the basic
received a 13-V electric shock either early or late in theme which occurs in the context of avoidance learn¬
training. The results indicated that the intense 13- ing. Bolles (1970) has been most explicit in developing
V shock suppressed gill extension whether pre¬ the notion of biological constraints on avoidance
sented during early or late trials. The 7-V mild learning. His hypothesis, which is referred to as the
shock, however, did not suppress the gill extension species-specific defense reaction (SSDR) hypothesis,
when presented during early trials and facilitated gill states:
extension responses when presented during the late
trials. Suppression or facilitation was always mea¬
For an [avoidance response] to be rapidly
sured relative to the performance of the habituation learned in a given situation, the response must
controls, which revealed a steady decline over the be an effective SSDR in that situation and when
course of training. rapid learning does occur, it is primarily due to
Melvin and Ervey suggest that the facilitation of the suppression of ineffective SSDR’s. (p. 35)
gill extension in the 7L group indicates that species-
typical aggressive behaviors are perhaps not suscepti¬ Bolles uses this argument effectively to explain why
ble to the usual suppressive effects of mild punish¬ certain responses such as lever pressing are not readily
ment. Again, however, a number of problems with the acquired as avoidance responses (cf. Meyer, Cho, &
experiment suggest that the conclusion is premature. Wesemann, 1960) and why, when such responses are
First, a control is required in which a referent re¬ readily trained, they are observed to be modified
sponse (with the same operant rate) is used that does forms of defensive reactions such as freezing (cf. Bolles
not have the unique biological function. It is possible 8c McGillis, 1968).
that Melvin and Ervey would have observed the same The primary evidence cited in support of Bolles’s
results using sexual or feeding behavior, indicating hypothesis was an experiment in which rats were
that the results were not unique to the presumed placed in a running wheel and required to make one
function of the response. of three different responses in order to avoid a sig¬
Second, it seems possible that the same results naled shock (Bolles, 1969). The responses were rearing
Philip Dunham 117

on the hind legs, a 180-deg about-face, or a 90-deg two responses. It would be of some interest to repli¬
revolution of the wheel. The results clearly indicate cate the experiment reported by Grossen and Kelley
that the running response was more rapidly acquired and use a forced-choice procedure in the early acqui¬
than the turning response, and that rearing was not sition trials to make sure operant rates to each plat¬
increased at all by the avoidance contingency. This form were equated, then give the animals a choice of
conclusion held true whether running was also re¬ the center vs. the perimeter to see which is pre¬
quired as the escape response or whether some alterna¬ ferred.
tive behavior was required as the escape response. There are at least two general criticisms of Bolles’s
Bolles concludes from these results that the speed of (1970) position which should be noted. Consider a
learning an avoidance response can be related to the hypothetical case of an animal with a repertoire of
animal’s species-specific defense reactions, and that two responses: A and B. If an aversive stimulus is
running, as a part of the animal’s natural reactions to now introduced, typically a third response (or group
aversive stimuli, will be rapidly acquired relative to of responses) will begin to occur which Bolles would
responses like turning and rearing which are not pri¬ call the animal’s species-specific defense reaction(s).
marily defensive reactions. According to Bolles, the SSDR will be acquired more
More recent experiments reported by Grossen and rapidly as an avoidance response than either response
Kelley (1972) lead to a similar conclusion. These au¬ A or response B. I would suggest that this might often
thors observed that rats placed in a large chamber be observed to be the case, but only because the SSDR
tend to spend more time in contact with the walls of has a unique advantage oyer responses A and B. The
the chamber (thigmotaxis) and more time in a “freez¬ SSDR is sequentially dependent upon the aversive
ing” posture when exposed to intermittent electric stimulus and as such predicts the absence of that aver¬
shock from a grid floor than they do under appeti- sive stimulus for a longer period of time than any
tively motivated conditions. They suggested that this other response in the animal’s repertoire. This im¬
thigmotactic behavior is one of the rat’s species-specific plicit avoidance contingency provides an alternative
defense reactions and should, according to Bolles, be explanation for the observation that shock-elicited
acquired very rapidly as an avoidance response. In a behaviors are often observed to develop rapidly as
subsequent procedure using the same large chamber, avoidance responses—an alternative, that is, to the
two platforms were placed in the chamber; one notion that these responses enjoy a special status as
around the perimeter of the chamber and one in the members of the category known as the animal’s natural
center of the chamber. Three groups of rats were defense reactions. A more extensive analysis of this
assigned to one of three avoidance learning condi¬ implicit contingency and its implications for avoidance
tions; Group C only had access to the center platform learning has already been considered elsewhere and
as a place to avoid or escape the shock; Group P had need not be repeated here (cf, Dunham, 1971).
access to the perimeter platform; and Group PC had A second and perhaps more important criticism of
access to both. In 30 subsequent acquisition trials in Bolles’s position can also be directed at the other vari¬
which the animals were placed in the chamber and ations on the “functional relevance hypothesis” which
had to jump on one of the platforms to escape or I have considered on the preceding pages. As stated at
avoid shock. Group P made 77% avoidance responses. present, my§t versions of the functional relevance
Group G 57%, and Group PG 72% with 92% of the hypothesis are very much ad hoc propositions. There
responses going to the perimeter platform. In a simi¬ is no clearly defined criterion for determining} mjg-
lar control experiment, three groups of rats were pendent of the rate of learning, whether a particular
required to climb either the center or perimeter plat¬ response is or is not “functionally relevant/7 It i§ not
forms to obtain food pellets or given a choice of plat¬ unusual to find various proponents of the notion
forms. No differences in acquisition performance were elaborating at length about how a particular operant
observed, and no systematic preference between plat¬ might be considered a short-term mutation of behav¬
forms was observed. ior in the wild. For example, in discussing the differ¬
Although the results are suggested to support ential rates at which rats acquire an avoidance re¬
Bolles’s SSDR hypothesis, there is an equally obvious sponse in the shuttlebox and in an activity wheel,
alternative which Grossen and Kelly recognize. If one Bolles (1970) states:
simply assumes that shock increases the operant rate
(probability of contact) with the perimeter platform,
It cannot be argued that running in the wheel
the different rates of acquisition can be considered to constitutes an effective SSDR while running in
reflect differences in the initial operant rate of the the shuttlebox is marginally effective merely
118 THE NATURE OF REINFORCING STIMULI

because the former is more rapidly acquired The animals learned the appropriate responses in or¬
than the latter. From the viewpoint of the SSDR der to obtain food and avoid the shocks. Following
hypothesis, both situations are ambiguous in the period of initial training, the rats were satiated
that they permit only limited or compensated with water prior to some training sessions. On these
flight. The running wheel has been recognized as days, the licking avoidance behavior dropped out and
a peculiar piece of apparatus by many investi¬
the animal took a substantial number of shocks, while
gators who have used it in general activity
lever pressing for food was maintained at its previous
studies, however, and perhaps it does permit
level. Black and Young interpret these results in terms
the rat to get away in some meaningful sense.
(p. 38)
of system constraints. Specifically, they suggest that
the normal causal factors of a response (e.g., the wa¬
Until less arbitrary, independent criteria for deter¬ ter regulatory system) may place constraints on the
mining membership in various response categories is extent to which that response can be brought under
proposed, it will not be possible to conduct critical the control of a different set of factors (e.g., the
tests of the suggestion that certain response-reinforcer avoidance contingency). If the normal causal factors
combinations are constrained because the response is for drinking are present (e.g., deprivation), the animal
not “functionally relevant” in the context of the will learn to drink to avoid shocks; if the normal
reinforcer. causal factors are not there, drinking will not be con¬
As stated earlier, the same problem is characteristic trolled by the avoidance contingency. Bar pressing is
of the other versions of the functional relevance assumed not to be under any such system constraints.
hypothesis which have been considered. For example, To quote Black and Young:
Shettleworth suggests that face washing is not a func¬
tionally relevant response in the context of food It would seem, then, that one dimension along
gathering, but that rearing, digging, scrabbling, and which responses might be classified is the degree
lever pressing are. To argue, alternatively, that rear¬ to which they are constrained from being
ing could be considered a part of the animal’s de¬ changed by operant reinforcement by the prop-
perties of the regulatory systems of which they
fense reactions and that bar pressing is not particu¬
are a part. The criterion for classifying responses
larly concerned with food gathering would simply beg
is not so much the conditionability of the re¬
the question. What is needed if the “functional rele¬
sponse or its ease of conditioning under optimal
vance” hypothesis is going to be testable is an inde¬ circumstances, but rather the limitations on such
pendent criterion for defining “functional relevance.’' conditioning. In this sense, bar pressing might
There are, in my opinion, two recent developments be described as a better operant than drinking
in this area of research which escape, to some extent, because it is less constrained by the regulatory
this problem of ad hoc reference to the biological control system of which it is a part then is drink-
significance of the response and provide us with some ing. (p. 44)
independent criteria for predicting whether or not a
particular response-reinforcer combination will be Stated as such, the proposition suggested by Black
effective. Black and Young (1972) working with rats and Young is as ad hoc as some of those discussed
and avoidance responding have suggested what might earlier and differs very little from the functional rele¬
be called a systems constraint hypothesis. Moore (1971, vance hypothesis. There is no criterion specified
1973) working with pigeons and an autoshaping pro¬ which would permit us to know, a priori, whether or
cedure, and Sevens ter (1973) working with stickle¬ not the regulatory system of a response will place
backs, have both proposed what might be called a constraints upon its use as an operant. Consider, for
Pavlovian hypothesis. In the remaining discussion I example, trying to predict whether a gerbil’s running
shall briefly consider each of these two hypotheses and behavior or paper-shredding behavior would be sub¬
the associated data. ject to serious “system constraints” of the sort ob¬
served when drinking was used as an avoidance
behavior. It is not possible, from the preceding com¬
The “Systems Constraint" Hypothesis
ments, to determine if these responses are subject to
Black and Young (1972) recently reported an inter¬ such constraints independent of the situation in which
esting experiment in which one group of thirsty rats the constraints are observed to operate.
was trained to bar-press for food during presentation Black and Young do, however, recognize the prob¬
of one discriminative stimulus (SD) and to lick a lem and offer a partial escape from it by making use
drinking tube to avoid shocks during a second SD. of Vanderwolf’s (1969) observation that some phasic
Philip Dunham 119

skeletal movements are accompanied by dorsal hippo¬ lovian process. It will be possible to consider only the
campal theta waves while others are not. Responses most direct arguments in the present discussion. A
typically labeled “instinctive” or “consummatory" series of well-controlled experiments done in col¬
(eating, drinking, etc.) are not accompanied by theta laboration with Jenkins (cf. Jenkins 8c Moore, 1973)
activity. Black and Young suggest that responses are perhaps most convincing. These authors demon¬
which are not accompanied by theta waves from the strated that birds concurrently autoshaped with grain
dorsal hippocampus will be constrained as operants and water reinforcement “pecked" the key stimulus
by their normal regulatory system; responses which associated with grain and “drank" the key stimulus
are accompanied by theta activity will not be con¬ associated with water reinforcement. These results,
strained. which were observed when birds were both hungry
Hence we are now in a position to make some and thirsty, indicate that an organism develops a
testable predictions and, hopefully, accept or reject conditioned response to the manipulandum which is
the “system constraints” hypothesis. For example, if similar in topography to the response elicited by the
theta waves do not accompany paper-shredding be¬ reinforcer.
havior in the gerbil, its use as an avoidance operant David Rackham, a graduate student in Moore’s
should be constrained in Black and Young's study. laboratory, conducted similar experiments (Rackham,
Specifically, if paper shredding is established as a 1971) in which a light stimulus was paired with access
Sidman avoidance response, satiation of the paper- to a sexually receptive female pigeon. Although the
shredding behavior should reduce avoidance respond¬ controls for nonassociative effects were less extensive
ing. If it does not, we can reject the theta wave cri¬ than in the food and water experiments, the results
terion and the system constraints hypothesis, unless indicated that the topography of the response to the
someone wants to argue about the “normal" causal light was similar to the topography of the response to
factors for paper shredding in gerbils. To my knowl¬ the female. Specifically, the male bird tended to
edge, such an experimental test has not been con¬ “court" the visual stimulus.
ducted. In the context of the autoshaping procedure, per¬
haps the most direct rationale for determining if the
instrumental response emerges from a Pavloyian proc¬
The Pavloyian Hypothesis
ess, as opposed to adventitious reinforcement of
Another formulation which generates a number of arbitrary, emitted operants, is to u$e an omission
testable predictions is the Pavlovian hypothesis. Moore training procedure in which any pecks made during
(1971, 1973), working primarily in the context of the the presentation of the conditioned stimulus actually
“autoshaping phenomenon” reported by Brown and cancel the presentation of grain on that trial. If the
Jenkins (1968), suggests that most instrumental learn¬ pecking actually develops from the Pavlovian con
ing situations also contain the necessary ingredients tingency, the animal should persist in pecking the
for the emergence of classically conditioned responses conditioned stimulus at rates appropriate to a partial-
and that such implicit conditioning can either facili¬ reinforcement schedule. If the pecking develops frym
tate or inhibit the effectiveness of certain response- the adventitious reinforcement of emitted skeletal
reinforcer combinations. movements, the omission training contingency pre¬
The manner in which this Pavlovian conditioning vents such reinforcement, and pecking should not
mechanism is suggested to operate is best explained develop. Results of an omission training experiment
by direct reference to some of the data which argue, conducted by Audrey Kirby (1968), another of Moore’s
quite convincingly, for its existence. Brown and students, and by Williams and Williams (1969) indi¬
Jenkins (1968) reported that repeated pairings of an cate that the omission procedure does not stop the
illuminated response key and the presentation of development of pecking in the autoshaping procedure,
grain were sufficient to cause pigeons to peck the key hence supporting the Pavlovian mechanism.
when it was illuminated. As Moore (1973) suggests, Of several implications of Moore’s work, the one
this phenomenon, called autoshaping, is potentially which is most directly relevant in the context of the
important because it indicates that a standard Pav¬ present discussion is that such Pavlovian conditioning
lovian conditioning procedure “could generate the would be expected to make certain response-reinforcer
key-pecking response so often used in operant condi¬ combinations very effective, while others would not
tioning research" (p. 160). Moore presents a varied work very well at all. Specifically, if the instrumental
array of data which argue collectively that the instru¬ response which we require of the organism is com¬
mental key peck is primarily the product of a Pav¬ patible with the species-typical reaction (uncondi-
120 THE NATURE OF REINFORCING STIMULI

tioned response) elicited by the reinforcer, the rein¬ Peterson, Ackil, Frommer, and Hearst (1972) paired
forcer should very rapidly gain control over the a 15-sec presentation of a retractable lever with either
instrumental behavior. If, alternatively, the instru¬ food reinforcement or intracranial stimulation (ICS)
mental response we require is not compatible with the in two different groups of rats. A second retractable
species-typical reaction elicited by the reinforcer, the lever in the same chamber was presented randomly
reinforcer should not be very effective in gaining con¬ without any relationship to the other events in the
trol over the instrumental behavior. Hence if we can chamber. A measure of the number of contacts with
specify what the unconditioned response to a par¬ the lever indicated that the rats, like the pigeons in
ticular reinforcer will be and determine if that re¬ Brown and Jenkins’s (1968) experiment, learned to
sponse is incompatible with the instrumental response, contact the lever if it signaled food or ICS and paid
it is possible to predict whether a particular response- little attention to the random lever. Of more interest,
reinforcer combination will be effective or not. Moore videotaped records of the rats’ behavior revealed
(1971, 1973) describes a number of examples of effec¬ qualitative differences in the response made to the
tive and ineffective response-reinforcer combinations lever by subjects receiving food and the response made
from both the ethology literature and the operant to the lever by subjects receiving ICS. According to
conditioning literature which support his contention. Peterson et al., in the group receiving the food reward
Pigeons, for example, can easily be trained to peck a ‘Visual observations and the videotaped records re¬
key for food, since the unconditioned response to vealed that contacts of CS+ were almost exclusively
grain (pecking) is perfectly compatible with the in¬ oral and consisted mainly of licking responses and
strumental response (pecking). Alternatively, a num¬ gnawing behavior” (p. 1010). Alternatively, in the
ber of examples indicate that it is very difficult to group receiving the ICS reward the rats typically
train a bird to peck a key in order to avoid shock (cf. showed sniffing and exploratory behavior of the same
Hoffman 8c Fleshier, 1969). When key pecking is de¬ type as elicited by the ICS alone. These results di¬
veloped with any impressive reliability as an avoidance rectly support the main implication of Moore’s Pav¬
response, either elaborate shaping (see Perrari, Todo- lovian analysis—that the unconditioned response to
roff, 8c Graeff, 1973) or an extensive history of food the reinforcer determines the topography of the re¬
reinforced pecking (see Foree 8c LoLordo, 1974) ap¬ sponse to stimuli paired with that reinforcer.
pears to be necessary. In defense of the Pavlovian A second, more recent experiment reported by
hypothesis, Moore (1973, p. 172) has argued per¬ Wasserman (1973) appears, however, to pose some
suasively that many instances of key-pecking avoidance problems for Moore’s Pavlovian mechanism. Wasser¬
might be interpreted as elicited aggressive reactions to man used 3-day-old chicks as subjects and an auto¬
the aversive stimuli. shaping procedure which consisted of 50 daily
Bolles’s SSDR hypothesis, considered earlier, is pairings of an 8-sec green key light with the 4-sec activa¬
similar in spirit to Moore’s Pavlovian mechanism tion of a heat lamp located in the ceiling of a small
when the latter is considered in the context of chamber. The chicks, which were placed in the cold
avoidance learning. Bolles’s SSDRs are, in effect, un¬ chamber, pecked the green response key on 80% of
conditioned responses to aversive stimuli. The essen¬ the trials, as compared to a group presented with the
tial difference between the two positions with refer¬ stimuli using Rescorla’s (1967) random control pro¬
ence to avoidance learning is that Moore’s Pavlovian cedure. When the experimental group was subse¬
mechanism is more precisely specified and testable, quently switched to the random control condition, the
and vulnerable for that reason. To the extent that it pecking slowly extinguished. When the random group
becomes difficult to specify, a priori, in a given situa¬ was switched to paired stimulus conditions, they
tion what the unconditioned response and the rele¬ started to peck and reached an asymptote of pecking
vant conditioned stimuli are, Moore’s Pavlovian mech¬ on 40% of the trials.
anism could also become an ad hoc proposition which In order to determine if the emergence of key
is difficult to disprove. pecking in this situation was the result of Pavlovian
In addition to the various evidence reviewed by as compared to instrumental contingencies, chicks
Moore in support of the Pavlovian analysis, several were trained using an omission procedure in which
more recent experiments have appeared which are pecks on the key would cancel the forthcoming heat
directly relevant to the suggestion that this mechanism lamp. The results, similar to those reported by Kirby
can facilitate or inhibit the effectiveness of certain (1968) and described earlier, indicated that all chicks
response-reinforcer combinations. did peck the key at reduced rates under the omission
Philip Dunham 121

training contingency. The omission training results another male stickleback displayed at the end of an
provide strong support for Moore’s suggestion that aquarium. Both of these instrumental responses were
autoshaping phenomena are under Pavlovian rather performed at very high rates with short interresponse
than instrumental control. A problem arises, however, intervals. When a sexually ripe female was used as the
with the implication of Moore’s analysis, which re¬ reinforcer, however, a male stickleback again learned
quires that the unconditioned response to the rein¬ how to swim through a ring at high stable rates, but
forcer will determine the topography of the response revealed very slow and variable rod-biting perform¬
to the operant manipulandum. In Wasserman’s pro¬ ance. It appeared from these data that some type of
cedure, the response to the heat lamp was suggested constraint was operating which prevented rod-biting
to be a species-typical wing-extension response, yet the behavior from developing when courting a female was
initial autoshaped response was to peck the light. the reinforcer.
Wasserman notes that the pecking response was ob¬ In subsequent experiments, Sevenster analyzed in¬
served during early trials and eventually gave way to terresponse times when the courtship reinforcer was
a response which he described as “snuggling up to the presented on a variable-ratio schedule for rod biting.
response key.” It would appear then that Wasserman The results indicated that the interval between rod-
is dealing with a Pavlovian phenomenon, but one in biting responses got shorter as the interval of time
which the conditioned and unconditioned responses since the last reward got longer. The longest inter¬
are completely different in topography. response interval was observed to be immediately
Some recent work appears to reconcile Wasserman’s after the opportunity to court. Taken together, the
results with Moore’s Pavlovian analysis. Hogan (1974) results indicate that some property of courtship be¬
observed that chicks placed in a cold environment havior actually inhibits (temporarily) the ability of
often peck at their mother, causing the adult bird to the animal to perform the biting response.
move closer, providing a source of heat. Hence peck¬ In order to determine if the actual pairing of the
ing at warm objects is one of a number of uncondi¬ rod biting with the courtship response was necessary
tioned responses to the cold environment employed for such inhibition to be observed, Sevenster con¬
by Wasserman which might be expected to emerge ducted yet another experiment in which a male fish
from the autoshaping procedure. was trained to bite the rod in order to attack another
Although there is a substantial amount of support male. As in earlier experiments, the response was rap¬
for Moore’s suggestion, a potential problem with his idly acquired and performed at a high rate. Once the
analysis is hinted at by the difficulty encountered try¬ Ash was biting the rod at high rates* a sexually ripe
ing to interpret Wasserman’s data. When a bug enters female was introduced into the tank for brief periods
the visual field of a hungry frog, the unconditioned of courtship, with care taken not to associate tli<j rod
response (UR) seems to be the result of some precise with the courtship response. A comparison of the
and hard-wired circuitry between visual input and latency between the opportunity to court and the next
tongue. Hence there is no basis for confusion when biting response and tfl^ opportunity to fight and the
attempting to specify the UR a priori* or in recogniz¬ next biting response indicated that the opportunity to
ing what responses are compatible or incompatible court had little effect upon the latency to the next
with that tongue flip. Higher up the phylogenetic bite. The r^§uh§ y£ til 15 EXporiHlSUt {whldl U5cd
scale, however, that “hard wiring” appears to disap¬ only one subject and should be extended) indicate
pear in a good many cases with the net result that that the inhibitory effect which courtship has upon
the UR to a given $timulu§ i§ much less predictable, biting behavior depends upon the prior temporal
Hence a strict application of Moore’s analysis in the pairings of the operant manipulandum with the sexu¬
case of higher organisms would require, as he has in ally ripe female.
dicated, a more extensive analysis of what factors Of equal interest are Sevenster’s anecdotal observa¬
determine which of several possible URs will emerge tions of what the fish were actually doing during the
in a given situation. long interresponse intervals following presentation of
The final position to be considered was proposed the sexually ripe female. He observed:
by Sevenster (1973) and is very similar to Moore’s
The fish would approach the rod and start cir¬
Pavlovian mechanism. In initial experimentation Sev¬
cling around it with zig-zag like jumps and often
enster observed that a male stickleback (Gasterosteus
with open mouth, sometimes making snapping
aculeatus) would readily learn to swim through a ring
movements at the tip or softly touching it. (p.
or bite the top of a rod in order to be able to attack 277)
122 THE NATURE OF REINFORCING STIMULI

In other words, the fish was treating the rod tip like CONCLUDING COMMENTS
a “dummy female,” and the behavior during the long
interresponse intervals was the species-typical court¬ We started the discussion with the relatively con¬
ship ritual. crete problem of predicting the outcome of any one of
Sevenster’s interpretation of these results is identi¬ 20 possible instrumental contingencies which might
cal, in some respects, to Moore’s suggested Pavlovian be arranged in the hypothetical world of our gerbil.
process. With reference to the courtship experiments, We considered a number of different contemporary
Sevenster suggests that the fish are conditioned to positions, all of which reject the “weak law of effect”
react to the biting rod as a “dummy female” and re¬ as a point of departure for the analysis of reinforce¬
spond to it with courtship behaviors which are in¬ ment phenomena. The six positions considered were:
compatible with the biting behavior required. Alter¬ (a) Premack’s differential probability position; (b) the
natively, swimming through the ring is suggested to response deprivation hypothesis; (c) the optimal-
be compatible with the elicited approach responses duration model; (d) the functional relevance hypothe¬
associated with courtship and aggression; hence this sis; (e) the system constraints hypothesis; and (f) the
response is performed at relatively high rates. It Pavlovian hypothesis. The last three positions empha¬
would, however, be interesting to know if the swim¬ size the importance of the biological predispositions
ming response did reveal the topography appropriate of particular species in attempting to predict the
to the reinforcement. Presumably, the way a fish effectiveness of particular response-reinforcer combi¬
swims through a ring would differ depending upon nations. The first three positions have, in general, ig¬
his intentions to fight or to copulate. nored such biological constraints and concentrated
Up to this point, the results of Sevenster’s experi¬ upon developing a set of predictive rules which are
ments and his analysis are completely compatible with implicitly assumed to hold for all species and response-
Moore’s analysis and results which were developed in reinforcer combinations.
the context of autoshaping. In addition to the Pav¬ Each of the above approaches offers a viable alter¬
lovian mechanism, Sevenster suggests that there may native to the weak law of effect as a point of de¬
parture in our analysis of the reinforcement process.
be a mutually inhibitory interaction between the
They are not, however, without their problems. The
motivational system- which controls courtship and the
major problem with Premack’s differential probability
motivational system which controls aggressive behav¬
position and with the response deprivation hypothesis
ior. Without entering the semantic jungle which at¬
is that much of the research surrounding these posi¬
tempts to distinguish between associative and non-
tions has failed to include control conditions to deter¬
associative effects, it is suggested that the data
mine if the changes in instrumental behavior can be
considered thus far do not seem to justify the intro¬
produced by changing the distribution and/or amount
duction of hypothetical competing “motivational
of contingent responding without a contingency in
states.” It should be difficult enough to determine if
effect. It is possible that many of the increases in in¬
two observable responses are “compatible” or “in¬
strumental responding that have been interpreted as
compatible” using the relatively parsimonious Pav¬
evidence for a reinforcement process in the context of
lovian mechanism outlined by Moore. the Premackian methodology would have occurred in
In summary, there is a growing amount of evidence the absence of the reinforcement operation, i.e., an
to indicate that not all response-reinforcer combina¬ instrumental response requirement.
tions are equally effective in developing control over As a viable alternative to both the response depri¬
instrumental behavior. I have attempted to isolate vation hypothesis and Premack’s differential proba¬
and criticize several variations on the general theme bility rules the optimal-duration model also remains
that the organism brings certain “biological predispo¬ to be investigated before either of the former positions
sitions” into the instrumental learning situation can be unequivocally accepted or rejected.
which either facilitate or inhibit the development of The three positions which have emphasized the
instrumental responding. Of these variations, I would role of biological constraints which operate upon
suggest that Black and Young’s system constraints particular response-reinforcer combinations also suffer
hypothesis and Moore and Sevenster’s Pavlovian hy¬ some serious problems. A number of specific criticisms
pothesis have the advantage of being vulnerable to were offered with reference to specific experimenta¬
relatively precise testing and have generated the most tion in this area in the preceding discussion. These
convincing data. criticisms are, however, overshadowed by the general
Philip Dunham 123

lack of precise definitions and testability which is Millenson (Eds.), Reinforcement: Behavioral analyses.
characteristic of many theoretical notions in this area. New York: Academic Press, 1972.
Bolles, R. C. Avoidance and escape learning: Simultaneous
Before one can test predictions made by the functional
acquisition of different responses. Journal of Compara¬
relevance hypothesis, for example, it is necessary to tive and Physiological Psychology, 1969, 68, 355-358.
have a precise definition of the term functional rele¬ Bolles, R. C. Species-specific defense reactions and avoid¬

vance which can be translated in experimental opera¬ ance learning. Psychological Review, 1970, 77, 32-48.
tions. There is no doubt that the three positions Bolles, R. C., 8c McGillis, D. B. The non-operant nature
of the barpress escape response. Psychonomic Science,
which have emphasized the importance of biological
1968, 11, 261-262.
constraints directly question the generality of alterna¬ Brimer, C. J. Disinhibition of an operant response. Learn¬
tive theoretical accounts of reinforcement such as ing and Motivation, 1970, 2, 346-371.
Premack’s differential probability rules and the re¬ Brown, P. L., 8c Jenkins, H. M. Auto-shaping of the pig¬
sponse deprivation hypothesis. They imply, for exam¬ eon’s key-peck. Journal of the Experimental Analysis of
Behavior, 1968, 11, 1-8.
ple, that the response deprivation rule will not apply
Campbell, B., 8c Lynch, G. S. Influence of hunger and
to particular cases. In some instances substantial re¬ thirst on the relationship between spontaneous activity
ductions in the amount of contingent responding will and body temperature. Journal of Comparative and
not increase instrumental responding, while in oth¬ Physiological Psychology, 1968, 65, 492-498.
ers an increase in instrumental responding might be Dunham, P. J. Contrasted conditions of reinforcement: A
selective critique. Psychological Bulletin, 1968, 69, 295-
observed without a reduction in contingent respond¬
315.
ing. In order, however, to pit the response deprivation Dunham, P. J. Punishment; Method and theory. Psycho¬
hypothesis against such positions as the functional logical Review, 1971, 78> 58-70.
relevance hypothesis, it will be necessary to have some Dunham, P. J. Some effects of punishment upon unpun¬
method of determining if particular response- ished responding. Journal of the Experimental Analysis
of Behavior, 1972, 17, 443-450.
reinforcer combinations are “functionally relevant” Eisenberger, R., Karfman, M., 8c Trattner, J, What is the
prior to conducting the experiment. necessary and sufficient condition for reinforcement in
It seems, at present, reasonable to assume that the¬ the contingency situation? Journal of Experimental Psy¬
oretical schemes such as Premack’s probability rules, chology, 1967, 342-350.
Ferrari, E. A., Tqdorqv, J, G., 8c Graeff, F. G. Nondis-
the response deprivation hypothesis, and the optimal-
cnminated avoidance of shock by pigeons pecking a key.
duration model will have to be modified to account Journal of the Experimental Analysis of Behavior} 1973,
for the biological predispositions which are being dis¬ 79,211-218.
cussed by theorists such as Shettleworth, Moore, and Foree, D. D., 8c LoLordo, V. M. Transfer of the pigeon’s
Sevenster. It will be helpful, when these two con¬ key peck from food reinforcement to avoidance of shock.
Journal of the Experimental Analysis of Behavior* 1974,
temporary approaches to reinforcement are merged in
22, 251-259,
a theoretical scheme, if both approaches are precisely Garcia, Jm McGowan, B. K., & Green, K. F. Sensory
enough formulated to produce testable hypotheses. quality and integration: Constraints on conditioning. In
A. H. Black & W. F. Prokasy (Eds.), Classical condition-
ing. Englewood Gliffs, N.J.: Prentice-Mall, Inc., 1972.
REFERENCES
Grossen, N. E. & Kelley, M. J. Species-specific avoidance
behavior in rats. Journal of Comparative and Physio=
logical Psychology, 1972, 81, 307=310.
& Timberlake, W. Instrumental and contin¬
Ar.r.isyiN, J,, Hoffman, H 5 , 8c Fleshler, m. Aversive control with tfie
gent saggharm licking in rats, Response deprivation pigeon. Journal of the Experimental Analysis of Be*
and reinforcement. Learning and Motivation) 1971, £3 havior, 1959, 2, 213=218.
231-847. Hogan, J. Respondents in Pavlovian conditioning. Science,
Azrin, N. H. Punishment of elicited aggression, Journal of 1974, 186156-157.
the Experimental Analysis of Behavior3 1970, 14 3 7-10. Holstein, S. B., 8c Hundt, A. G. Reinforcement of intra¬
Baenninger, R„ 8c Grossman, J. C. Some effects of punish¬ cranial self-stimulation by licking. Psychonomic Science,
ment on pain elicited aggression. Journal of the Exper¬ 1965, 5, 17-18.
imental Analysis of Behavior, 1969, 12, 1017-1022. Hull, C. L. Principles of behavior: A n introduction to
Baum, W. H. The correlation based law of effect. Journal behavior theory. New York; Appleton-Century, 1943.
of the Experimental Analysis of Behavior, 1973, 20, 137— Hull, G. L. A behavior system. New Haven: Yale Univer¬
153. sity Press, 1952.
Bernstein, D. J. Structure and function in response reper¬ Hundt; A. G., 8c Premack., D, Running as both a positive
toires of humans. Unpublished doctoral dissertation, and negative reinforcer. Science, 1963, 142, 1087-1088.
University of California at San Diego, 1973. Jenkins, H., & Moore, B. R. The form of the auto-shaped
Black, A. H., & Young, G. A. Constraints on the operant response with food or water reinforcers. Journal of the
conditioning of drinking. In R. M. Gilbert 8c J. R. Experimental Analysis of Behavior, 1973, 20, 163-182.
124 THE NATURE OF REINFORCING STIMULI

Kirby, A. J. Explorations of the Brown-Jenkins auto-shap¬ of the instrumental response base rate. Journal of Ex¬
ing phenomenon. Unpublished masters thesis, Dalhousie perimental Psychology, 1965, 69 419-425.
University, 1968. Schaeffer, R. W., Hanna, B., 8c Russo, P. Positive rein¬
Marmaroff, S. Reinforcement: A test of Premack's differ¬ forcement: A test of the Premack theory. Psychonomic
ential probability rules. Unpublished master’s thesis, Science, 1966, 4, 7-8.
Dalhousie University, 1971. Seligman, M. E. P. On the generality of the laws of learn¬
Meehl, P. E. On the circularity of the law of elfect. Psy¬ ing. Psychological Review, 1970, 77, 406-418.
chological Bulletin, 1950, 47, 52-75. Sevenster, P. Incompatibility of response and reward. In
Melvin, K. B., 8c Ervey, D. H. Facilitative and suppressive R. Hinde 8c J. Hinde (Eds.), Constraints on learning.
effects of punishment of species-typical aggressive display New York: Academic Press, 1973.
in Betta splendens. Journal of Comparative and Physio¬ Shettleworth, S. J. Constraints on learning. In D. S.
logical Psychology, 1973, 83, 451-457. Lehrman, R. A. Hinde, 8c E. Shaw (Eds.), Advances in
Meyer, D. R., Cho, C., 8c Wesemann, A. F. On problems the Study of Behavior (Vol. 4). New York: Academic
of conditioning discriminated lever press avoidance re¬ Press, 1972.
sponses. Psychological Review, 1960, 67, 224-228. Shettleworth, S. J. Food reinforcement and the organiza¬
AIiller, N. E. Some reflections on the law of effect produce tion of behavior in golden hamsters. In R. Hinde 8c J.
a new alternative to drive reduction. In M. R. Jones Hinde (Eds.), Constraints on learning. New York: Aca¬
(Ed.), Nebraska Symposium on Motivation. Lincoln: demic Press, 1973.
University of Nebraska Press, 1963. Staddon, J. E. R., 8c Simmelhag, V. L. The “superstition”
Miller, N. E. Learning of visceral and glandular responses. experiment: A re-examination of its implications for the
Science, 1969, 163, 434-445. principles of adaptive behavior. Psychological Review,
Moore, B. R. On directed respondents. Doctoral disserta¬ 1971, 78, 3-43.
tion, Stanford University, 1971. Ann Arbor, Michigan: Terhune, J. G. 8c Premack, D. On the proportionality
University Microfilms No. 72-11, 623. between the probability of not running and the punish¬
Moore, B. R. The role of directed Pavlovian reactions in ment effect of being forced to run. Learning and Motiva¬
simple instrumental learning in the pigeon. In R. tion, 1970, 1, 141-149.
Hinde 8c J. Hinde (Eds.), Constraints on learning. New Thorndike, E. L. Educational psychology: Briefer course.
York: Academic Press, 1973. New York: Columbia University, 1914.
Peterson, G. B., Ackil, J. E., Frommer, G. P., 8c Hearst, Thorndike, E. L. The fundamentals of learning. New
E. L. Conditioned approach and contact behavior to¬ York: Columbia University, 1932.
ward signals for food and brain stimulation reinforce¬ Timberlake, W., 8c Allison, j. Response deprivation: An
ment. Science, 1972, 177, 1009-1011. empirical approach to instrumental performance. Psy¬
Premack, D. Toward empirical behavior laws, I: Positive chological Review, 1974, 81, 146-164.
reinforcement. Psychological Review, 1959, 66, 219-233. Vanderwolf, C. H. Hippocampal electrical activity and
Premack, D. Reversibility of the reinforcement relation. voluntary movement in the rat. Electroencephalography
Science, 1962, 136, 255-257. and Clinical Neurophysiology, 1969, 26, 407-418.
Premack, D. Rate-differential reinforcement in monkey Walker, E. L. Reinforcement: The “one ring.” In J. T.
manipulation. Journal of the Experimental Analysis of Tapp (Ed.), Reinforcement and behavior. New York:
Behavior, 1963, 6, 81-89. Academic Press, 1969.
Premack, D. Reinforcement theory. In D. Levine (Ed.), Walters, G. C., 8c Glazer, R. D. Punishment of instinctive
Nebraska Symposium on Motivation. Lincoln: Univer¬ behavior in the Mongolian gerbil. Journal of Compara¬
sity of Nebraska Press, 1965. tive and Physiological Psychology, 1971, 75, 331-340.
Premack, D. Catching up on common sense, or two sides of Wasserman, E. A. Pavlovian conditioning with heat rein¬
a generalization: Reinforcement and punishment. In R. forcement produces stimulus directed pecking in chicks.
Glaser (Ed.), On the nature of reinforcement. New York: Science, 1973, 181, 875-877.
Academic Press, 1971. Weisman, R. G., 8c Premack, D. Reinforcement and pun¬
Rachlin, H., 8c Herrnstein, R. J. Hedonism revisited: On ishment produced by the same contingent event. Paper
the negative law of effect. In B. A. Campbell 8c R. M. presented at the meetings of the Psychonomic Society,
Church (Eds.), Punishment and aversive behavior. Engle¬ St. Louis, 1966.
wood Cliffs, N.J.: Prentice-Hall, Inc., 1969. Wilcoxon, H. C. Historical introduction to the problem of
Rackham, D. W. Conditioning of the pigeon's courtship reinforcement. In J. T. Tapp (Ed.), Reinforcement and
aggressive behavior. Unpublished masters thesis, Dal¬ behavior. New York: Academic Press, 1969.
housie University, 1971. Williams, D. R., 8c Williams, H. Auto-maintenance in the

Rescorla, R. A. Pavlovian conditioning in its proper con¬ pigeon: Sustained pecking despite contingent non-rein¬
trol procedures. Psychological Review, 1967, 7i, 71-80. forcement. Journal of the Experimental Analysis of Be¬
Schaeffer, R, W. The reinforcement relation as a function havior, 1969, 12, 511-520.
 5

Schedule-Induced Behavior*

J. E. R. Staddon

INTRODUCTION "response-contingent reinforcement" usually embraces

all three of these features.
In most operant conditioning experiments, a single Because of the apparent naturalness of the response-
aspect of behavior such as a key peck or a lever press contingent procedure, it has held the limelight over
is selected as the instrumental response. Although a the years. A mass of experimental literature has devel¬
reinforcer will not ordinarily follow every instance of oped describing results that are in many cases difficult
the response, it follows some, usually immediately, and to interpret. The ubiquitous feedback loop between
does not occur other times. There are departures behavior and its consequences makes it very hard to
from this rule (such as delay of reward procedures, discern the behavioral mechanisms that allow the
mixed classical and operant procedures, and some combined animal-schedule system to settle down into
conjunctive schedules), but it is so common, and seems the classical “schedule performances.” In any response-
so close to the “natural” contingencies of the animal’s contingent procedure, the pattern of responding nec¬
wild environment that it has become the norm. Yet it essarily influences not only the correlation between
embodies a number of arbitrary features: the single responding and reinforcement, but also the correlation
instrumental response, the fixed short delay between between reinforcement, and temporal and stimulus
response and reinforcer, and the absence of reinforce¬ variables. For example, if an animal on a fixed-interval
ment in the absence of the response. The term schedule responds only sporadically, the reinforce¬
ments may occur at variable rather than fixed inter¬
* Research supported by grants from the National Science vals. Yet it is these temporal correlations (termed by
Foundation and the National Institute of Mental Health,
Zeiler “indirect” variables, see Chapter 8) that are
USPHS, to Duke University. I thank the Department of Experi¬
mental Psychology, Oxford University, for facilities during most important in determining the final pattern of
preparation of the chapter. Several colleagues were kind enough performance. If the way they act is to be understood,
to comment on earlier versions. I am especially grateful to Dalbir
the response contingency is an unnecessary complica¬
Bindra, Janice Frank, Werner Honig, Nancy Innis, Peter Killeen,
Jock Millenson, Evelyn Segal, and Michael Zeiler. tion. Consequently this chapter is devoted primarily

125
126 SCHEDULE-INDUCED BEHAVIOR

to behavior induced by response-independent (clas¬ occur, where (in relation to time and stimuli) they
sical conditioning) procedures, and to those aspects occur, and how much (with what ‘strength’) they oc¬
of response-contingent schedules that are not directly cur. The second part deals with the temporal and
dependent on the response contingency. sequential constraints that underlie induced behavior
The term schedule-induced does not yet have a sequences.
widely accepted definition. However, it is clear that
when an animal is exposed to a schedule of periodic
food or electric shock, some activities are facilitated BEHAVIOR INDUCED BY
and others are reduced by this operation. Only those PERIODIC FOOD
activities that are facilitated by the schedule (by com¬
parison with a pre- and post-schedule baseline when The simplest case to consider is the periodic presen¬
no food or shock is delivered) will be termed induced tation of “free” food to a hungry rat or pigeon (FT:
behaviors. The term facultative behavior has been fixed-time reinforcement). At first, the animal is likely
coined to refer to activities that occur on schedules to spend much of its time between food deliveries
but do not appear to be directly affected by schedule exploring around the food site. However, within a few
factors (p. 135). sessions this behavior drops out and is replaced by a
Schwartz and Gamzu, in Chapter 3 of this volume, regular sequence of activities within each interval.
discuss the problem of terminal responses, i.e., in¬ Each activity becomes increasingly well defined and
duced behavior that emerges in the presence of, or is the sequence as a whole becomes increasingly stereo¬
directed toward, stimuli that are highly predictive of typed as training proceeds. Figure 1 shows the patterns
food or some other positive reinforcer; and Hutchin¬ of behavior developed by a pigeon and a rat under
son in Chapter 14 deals with behavior induced by broadly similar conditions of periodic food delivery.
schedules of aversive events. This chapter is mainly Both animals were hungry. The pigeon was in a bare
concerned with interim activities on schedules of posi¬ enclosure and received 3-sec access to food every 12
tive reinforcement; that is, induced behaviors which sec. The rat was in a hexagonal enclosure that allowed
occur at times when a reinforcer is unlikely to be drinking and running in a wheel, as well as other
delivered. Terminal responses are discussed only as activities, and received a food pellet every 30 sec.
much as is necessary to present a comprehensive pic¬ Despite these differences the general features of the
ture of the ways in which induced and facultative behavior are similar. Activities occur in sequence,
behaviors interact under the influence of schedule with some (interim activities) typically occurring early
factors. in the interval (facing the window wall and wing flap¬
The first part of the chapter will be concerned with ping for the pigeon; drinking and running for the
the factors that determine what induced activities will rat). A single terminal response (pecking for the

PIGEON RAT

Fig. 1. The relative frequency

of various activities as a func¬
tion of post-food tftne for a rat
(right panel) and a pigeon (left
panel) on periodic food sched¬
ules. The pigeon data are taken
from Staddon and Simmelhag
(1971); the bird was on a fixed¬
time 12 sec schedule. The rat
data are from Staddon and
Ayres (1975); the rat was on a
fixed-time 30 sec schedule in a
hexagonal enclosure that per¬
mitted drinking, wheel running
and other activities in addition
to eating.
J. E. R. Staddon 127

pigeon; “food anticipation” for the rat) increases in volume attests, there is still considerable controversy
frequency up until the end of the interval. First, I about the correct explanation for the terminal re¬
discuss the types of behavior that have been observed sponses which develop on response-independent sched¬
under conditions of periodic food delivery. In later ules. A confusing factor is the misleading opposition
sections I consider the sequential properties of in¬ between “Pavlovian” and “operant” views, as if these
duced behavior and the possible mechanisms of in¬ two classes were mutually exclusive. Historically this
teraction that these imply. opposition is in part traceable to Skinner’s (1948) par¬
ticular operant account. His “adventitious reinforce¬
ment” hypothesis is that a terminal response occurs
Terminal Responses
first for unspecified reasons, is accidentally contigu¬
In the above examples, time is the variable that ous with food delivery, is strengthened thereby, and is
signals the imminence of food. This is not necessary. thus more likely to occur again. This cycle repeats
Indeed the most popular procedure for studying in¬ and the result is persistent, stereotyped “supersti¬
duced terminal responses (autoshaping: Brown & tious” behavior. This view owes nothing to Pavlovian
Jenkins, 1968) follows the standard classical-condition¬ processes and rests on a contiguity view of the action
ing paradigm: food is intermittently presented, usu¬ of reinforcers for which the evidence is weak. More¬
ally at variable intervals, each delivery being signalled over, its lack of any quantitative content robs it of
by a brief (5-10 sec) stimulus. The conclusions of this predictive power: How often must a given response
work can be summarized quite briefly (see Ch. 3 for be contiguous with reinforcement to become fully
additional references and a fuller account). (1) Some conditioned? How contiguous must it be? Are all
behavior will be induced in the presence of the stimu¬ responses the same in these respects? Without good
lus that signals food. (2) It seems to be necessary that answers to these questions, the hypothesis is essentially
the stimulus predict food, i.e., precede food more re¬ untestable since a response which is increasing in fre¬
liably than any other stimulus precedes food. Simple quency, for whatever reason, is more likely to occur
pairing is not enough by itself (Bilbrey Sc Winokur, in close proximity to reinforcement and thus to fulfill
1973; Gamzu & Williams, 1971, 1973; Rescorla, 1967)* Skinner’s condition for its further increase.
(3) The type of induced behavior depends on a num¬ There are several versions of the Pavlovian view of
ber of factors, (a) The type and strength of the sig¬ terminal responses. Perhaps the simplest is that any
nalled reinforcer (food, water, shock, sex, etc., cf. situation which predicts food is likely to induce food-
Jenkins Sc Moore, 1973). Often the behavior resembles related behavior (ef. Moore, 1973). Hearst and Jenkins
the consummatory response usually made to the rein¬ have developed a refinement of this yiew to deal with
forcer. (b) The nature of the signal stimulus, e.g., the directed nature of much induced activity (sign
whether or not it can be manipulated, its location in tracking; Hearst 8c Jenkins, 1974). Yet another view
relation to the reinforcer site, whether it can be points to the effect of contingency strength (predic-
sensed without the animal orienting towards it, its liyeness) on the range of behavioral yanation {Stad¬
intensity, and its biological “relevance” for the sig¬ don, 1970): Stimuli that are good predictors of food,
naled reinforcer, (c) Past history, e.g., food-related for example, reliably product st£ir£Otypcd behavior.

responses acquired in an instrumental situation may This restriction of variability is taken as the funda¬
reappear in an autoshaping situation, (d) How good a mental property of strong contingencies.
predictor the signal stimulus is. For example, induced
pecking is more likely to develop and is stronger if the
Adventitious Reinforcement
signal stimulus is short than if it is long (Innis Sc
Keehn, personal communication; Ricci, 1973) or if the Induced behavior is a phenomenon worthy of inde¬
inter-stimulus (inter-trial) interval is long rather than pendent study. It cannot convincingly be dismissed as
short (Groves Sc Brownstein, 1973; Terrace, Gibbon, a curious illustration of learning principles better
Farrell, 8c Baldock, 1975; see also Wilton 8c Clements, studied directly in other ways. The latter position is
1971). The induced behavior is more likely to resem¬ most strongly represented by Skinner’s adventitious
ble the consummatory response, and to be more reinforcement hypothesis. The arguments against this
vigorous, the better the relative proximity of the view have been elaborated elsewhere (e.g., Gamzu 8c
signal stimulus to the signalled reinforcer (Jenkins, Schwartz, 1973; Rachlin 8c Baum, 1972; Staddon,
1970; Hearst 8c Jenkins, 1974; Staddon 8c Simmelhag, 1972b, 1976; Staddon 8c Simmelhag, 1971) and need
1971). only be summarized here: (a) During the develop¬
As the chapter by Schwartz and Gamzu in this ment of induced terminal responses a response such
128 SCHEDULE-INDUCED BEHAVIOR

as the animal’s putting its head into the food aperture said to predict A, since one of the terms of the rela¬
may predominate for a while, only to be supplanted tion that defines predictiveness, p(A[B), has vanished.
by another response, such as pecking, despite consis¬ Thus this hypothesis implies a more primitive view of
tent initial pairing of the first response with food. the concept of contingency than can be justified by
Adventitious reinforcement cannot account either for recent experimental work.
the decline in the first activity or the appearance of The adventitious reinforcement hypothesis arose
the second, (b) Terminal responses such as pecking in from a tacit assumption that the effects of response-
a food situation are quite resistant to response contin¬ dependent reinforcement are somehow more funda¬
gencies that make food delivery less likely if the re¬ mental than those of response-independent reinforce¬
sponse occurs (negative automaintenance or omission ment. Skinner explained the response-independent
training: Williams & Williams, 1969). A response that case by means of an account derived from experiments
occurs in spite of a negative contingency is unlikely on response-dependent reinforcement. Even if he is
to require a positive one for its maintenance, (c) Nega¬ right in believing that the two cases share common
tive contingencies do have some suppressive effect on mechanisms, the proper translation may be in the
a response such as pecking. However, much of the opposite direction. Perhaps mechanisms derived from
effect is attributable to effects of the contingency on a study of response-independent procedures can be
the frequency and pattern of food delivery, i.e., on applied to explain the effects of response-dependent
temporal and stimulus (not response) contingencies, reinforcement. This is not a new idea. However, the
(d) There is a logical problem in attributing the failure of the adventitious reinforcement notion, and
maintenance of a response to accidental conjunctions recent advances in our understanding of the concept
between it and food delivery. This problem is not of contingency, make it once again a viable one (cf.
overcome by demonstrating that the imposition of a Schwartz 8c Gamzu, Chapter 3 in this volume; Stad-
negative contingency reduces the level of the behavior, don, 1976).
even if the reduction is below the level that would be
maintained by yoked response-independent food deliv¬
ery. Showing that a response is sensitive to a real
Interim Activities
negative contingency does not force the conclusion
that its prior occurrence was owing to an accidental On periodic food schedules a variety of activities
positive one. (e) Occasional response-independent food occur at times when food delivery cannot occur, for
deliveries, superimposed on a baseline of responding example, early in the interval on fixed-time schedules.
maintained by response-dependent food delivery, often Some are directed at objects in the environment (e.g.,
result in suppression of the instrumental response, pecking in birds, chewing, drinking, or pawing by
even though the absolute number of response-food rats), others have no obvious referent (e.g., head bob¬
conjunctions is increased by this operation (Rachlin bing, beak movement, pacing). Directed activities
& Baum, 1972). The addition of “free” reinforcements come under the control of their own “incentive stim¬
makes the instrumental response less predictive of the uli” (Bindra, 1972). All induced activities appear to
reinforcer. Hence these results support the conclusion depend on motivational variables. Two kinds of moti¬
from experiments on response-independent procedures vational variable are important; variables related to
that a schedule is effective in modifying behavior the scheduled reinforcer (e.g., things that affect hun¬
only to the extent that it arranges a predictive rela¬ ger, if food is scheduled); and variables related to the
tion (i.e., a real contingency) between an event (a particular activity (e.g., to thirst, for induced drink¬
stimulus or a response) and the occurrence of a rein¬ ing).
forcer (Rescorla, 1967; Rescorla 8c Wagner, 1972). Induced behaviors are not all affected in the same
“Prediction” of reinforcer A by response B means, way by schedule variables. Hence there is much need
in this context, that p(A|B) is greater than p(A|B) or, for a “natural history” of induced behaviors. More
in temporal terms, that B precedes A more closely than purely descriptive work needs to be done to map out
A is preceded by any other response (see Gibbon, the sequences of behavior that occur with a variety of
Berryman, 8c Thompson, 1974, for a careful discussion combinations of species, schedule, reinforcer, and sup¬
of quantitative measurement of contingency strength). porting environment. In the absence of the informa¬
The adventitious reinforcement hypothesis, of course, tion such studies could give, generalizations about
attends only to positive pairings, i.e., to p(A|B). underlying mechanisms must be tentative. Only one
Moreover, if, in the ideal case, p(B) approaches zero interim activity, schedule-induced drinking, has been
(the animal does nothing but B), B cannot really be studied in anything like the necessary depth (Falk,
J. E. R. Staddon 129

1969, 1971). Running (Levitsky 8c Collier, 1968; Segal, tions. Some years ago, for example, Skinner (1961)
1969), and schedule-induced aggression (Azrin, Hutch¬ pointed out that on certain schedules the energy ex¬
ison, 8c Hake, 1966; Flory, 1969; Richards 8c Rilling, pended by the animal in responding exceeds that
1972) have also received some attention. Most of this received from the food reinforcement obtained. De¬
work has been done with rats. The following discus¬ spite these failures of energy regulation, response and
sion, therefore, rests more heavily on these behaviors, reinforcement rates usually follow some simple func¬
and particularly on studies of induced drinking, than tional relation. Hence total intake may not be the
is perhaps desirable. most useful measure of induced drinking. How then
should induced behavior be measured?

Induced Drinking
Measurement of Induced Activities
Falk (1961) was the first to draw attention to the
curious fact that hungry rats responding on a schedule The most important factor that determines choice
of intermittent food reinforcement will ingest large of a particular dependent measure is the experi¬
quantities of water (polydipsia). This occurs even menter’s belief about its probable cause. So long as
though the rats are not deprived of water, so that induced drinking was thought to derive from an
their total intake may be several times that necessary altered state of water balance, total amount drunk
to maintain water balance. Since Falk’s original (over the 24 hours) was an appropriate measure. It
paper, he and others have demonstrated induced now seems that this may not be the best way to look
drinking in numerous species (squirrel monkeys, at it. Induced drinking may somehow be related to
chimpanzees, pigeons and doves to some degree), and schedule variables and to the mechanisms through
on a variety of intermittent food schedules (see Falk, which they affect operant behavior. From this point
1969 and 1971, for reviews). of view appropriate measures might be ingestion rate
Falk’s original emphasis was on the excessive (ml/mm), drinking rate (licks/min), or the fraction of
(polydipsic) aspect of schedule-induced drinking. time engaged in drinking.
Hence much of the initial research was an attempt to However, the schedule-induced nature of this effect
reconcile the behavior with known regulatory mecha¬ introduces three complications (Flory, 1971; Staddon
nisms. As is by now well known, none of these at¬ & Simmelhag, 1971). The first concerns the stimulus
tempts was wholly successful. Neither central effects, control of induced drinking. On fixed- and variable-
in the form of an altered water balance caused by interval schedules, drinking typically occurs just after
intermittent food, nor peripheral regulatory media food delivery, and it can easily be shown that once
nisms (the “dry mouth” theory) seem adequate to behavior has stabilized, drinking is directly under the
explain schedule-induced drinking. It would be rash control of each eating bout: each bout of eating pro¬
to dismiss this line of work as unprofitable. More duces a bout of drinking. Since we are interested m
recent efforts of this sort (for example, explorations the relation between frequency and amount of food
of the link between temperature and water regulation delivery and the animal’s tendency to drink, this con¬
systems, e.g., Carlisle, 1973) may yet uncover a regula¬ trolling relation introduces a confounding factor.
tory basis for the effect. However, die relative lack of First, suppose that all the induced drinking; is simply
progress along physiological lines revives interest in post-prandial with each “meal” producing a fixed
the behavioral determinants of the phenomenon. amount of drinking (Lotter, Woods, 8c Vaggelli, 1973*
These are the focus of the present account. Stein, 1964). While the rate of food delivery deter¬
The most striking thing about schedule-induced mines the rate of drinking in such a situation, it
polydipsia is that the total amount of water drunk would be a mistake to conclude that this variable has
each day is so much greater than normal. This ac¬ a direct effect on the tendency to drink. (Similarly, the
counts for Falk’s original emphasis on total intake, as rate at which conditioning trials occur tells us noth¬
a function of various schedule variables. However, in ing about the animal’s tendency to make the condi¬
most studies of operant behavior the total amount of tioned response.)
the behavior is of much less interest than the rate at A second confounding factor is the opportunity for
which it occurs, or the percentage of time that it takes drinking. If, to continue with the post-prandial exam¬
up (see de Villiers, Chapter 9 and Dunham, Chapter 4 ple, the drinking bout produced by each food deliv¬
in this volume). Moreover, the obvious adaptiveness ery is of a fixed duration, scheduling food deliveries
of operant behavior in general has long stood in sharp too frequently could reduce the time available for
contrast to its maladaptiveness in particular situa¬ drinking and thus actually reduce the amount drunk
 130 SCHEDULE-INDUCED BEHAVIOR

Fig. 2. Data replotted from

Flory (1971), and Falk (1969)
relating measures of drinking
(licks/min, ml. drunk/min) to
food rate (pellet deliveries/min).

per food delivery. This would artificially limit the poral placement of schedule-induced drinking (Burks,
function relating food and drink rates. 1970; Segal et al., 1965; Wayner 8c Greenberg, 1973).
A third complication is the limit on the total Three aspects of these data are important for
amount of food that food-deprived animals can be future discussion: (a) The functions are all mo-
permitted to ingest each day. This usually forces a notonically increasing, with higher food rates associ¬
reduction in the total number of daily food deliveries ated with higher rates of drinking, until food deliver¬
if the size of each delivery is increased. Consequently, ies are spaced very close together indeed. The functions
the total amount of water drunk might appear to de¬ for amount drunk in the left hand panel turn down
crease as food portion size is increased, even though only when food deliveries occur once every four sec¬
the rate of drinking is directly related to portion size onds or oftener. (b) The functions for lick rate and
(Falk, 1967; Flory, 1971; Hawkins, Schrot, Githens, & rate of ingestion are generally similar in form, imply¬
Everett, 1972; Staddon & Simmelhag, 1971). These ing that an approximately fixed amount of water is
problems make a number of experimental results diffi¬ ingested with each lick (see Figure 3). (c) Flory's data
cult to interpret. show that at a given food rate, two pellets per food
delivery induce more drinking than one pellet per
Effect of Food Rate
delivery. Moreover, this difference in terms of lick
The two panels of Fig. 2 show data relating rate of rate is greater when food delivery is infrequent. Since
drinking (ml/min, left panel; licks/min, right panel) the ordinate is logarithmic, this means that the pro¬
to frequency of food delivery (opportunities to eat/ portionate (but not the absolute) increase in lick rate
min, here referred to as food rate), replotted from is greater at low food rates.
studies by Flory (1971) and Falk (1969). These re¬
sponse rate vs food rate functions will be referred
Hypotheses to Explain
to as response functions. Food was delivered on fixed-
Schedule-Induced Drinking
interval schedules of different values, and the data
represent stable performance. Similar results have Four simple behavioral hypotheses have been pro¬
been found by Hawkins et al. (1972) with fixed and posed, explicitly or implicitly, as explanations for
variable-time food schedules, and by Segal, Oden, and schedule-induced drinking. These are: (a) the post¬
Deadwyler (1965) and Staddon and Ayres (unpub¬ prandial hypothesis, (b) the opportunity hypothesis,
lished) with fixed-time schedules. As others have (c) the adventitious reinforcement hypothesis, and (d)
shown, the presence or absence of a response contin¬ the motivation hypothesis. These are discussed next,
gency makes little difference to the amount and tem¬ followed by an account of running during food sched-
J. E. R. Sfaddon
131

ules and a summary of the relations between running within the interfood interval depends on interval
and drinking. length—the longer the interval, the later the onset of
drinking (Segal et al., 1965)—and if, after the rat has
Post-Prandial Hypothesis. The simplest hypothe¬ learned to drink in the post-food period, access to the
sis is that the drinking is simply normal post-prandial water bottle is restricted to a time late in the interval,
drinking, so that the more “meals” or “bites” the rat drinking eventually recovers to essentially full strength
takes, the more he drinks (Kissileff, 1969; Lotter et al., (Flory 8c O’Boyle, 1972; Gilbert, 1974). (3) Drinking
1973; Stein, 1964). This idea implies simple propor¬ and eating in rats with free access to food and water
tionality between the rate of drinking and the rate of are linked, but not in the way required by the post¬
food delivery up to a maximum when the animal prandial hypothesis. In a careful series of studies,
drinks all the time except when he is eating. Formally Kissileff (1969) showed that drinking occurs both just
it implies a relation of the form D = KXR + K2, where after and just before eating bouts.
D is the rate of licking (licks/min; or water ingestion, Lotter et al. (1973) have recently revived the post¬
ml/min), R is the frequency of opportunities to eat prandial hypothesis in an attempt to explain schedule-
(food rate), Kx is a constant representing the size of related drinking as an “artifact” of the small “meal”
each post-prandial drinking bout, and K2 is the rate size imposed by intermittent schedules. Much of their
of drinking in the absence of the food schedule. argument rests on demonstrations that increasing re¬
The empirical functions in Figure 2 are not com¬ ward size reduced overall rate of drinking during
patible with this equation because they all show con¬ single test sessions. These tests are not valid because
siderable negative acceleration (in linear coordinates the increased drinking that is associated with increases
as well as the log-log ones of Figure 2) over much of in amount of food takes time to develop, as the an¬
their range. The post-prandial hypothesis can never¬ imals must have time to learn that the reward size
theless be applied to these functions if they are ap¬ (incentive value) in the situation has increased (e.g.,
proximated by two line segments, one with a steep Hawkins et al., 1972). It is known that induced drink¬
slope starting at the origin (Ks — 0) followed by a ing, once developed, is controlled by each meal, as a
shallower segment after a break point in the vicinity discriminative stimulus (e.g., Staddon & Ayres, 1975).
of a food rate of one pellet every two min. However, Hence a reduction in meal rate, such as might occur
these two segments suggest two contradictory inter¬ following an increase in meal size, would automat¬
pretations. The steep segment from the origin is per¬ ically yield a reduction in rate of drinking, at least at
fectly consistent with the post-prandial view with first. It seems likely that a reduction in meal rate did
drinking rate proportional to food rate. However, the occur, since although Lotter et al. do not report the
shallow segment then represents some kind of suppres¬ actual (as opposed to the scheduled) intermeal inter¬
sion of drinking at high food rates, perhaps due to val in their one hour test sessions, the total number of
restricted opportunity to drink. On the other hand, if pellets consumed during tests was disproportionately
the shallow segment is considered to represent the small. For example, in their third experiment reward
post-prandial view, the y-intercept, Ks, is much greater size was increased from one to 12 pellets, Yet the
than the rate of drinking in the absence of the food number of pellets consumed per hour increased from
schedule (which will usually be close to zero since the 70.5 to only 218.5. Hence the number of mcalg must
animals have unlimited access to water in their home have dropped, A reduction in rate of drinking under
cages). Hence K2 must be interpreted as some kind of these conditions says something about the stimulus
"inducing” effect of the food schedule. control of drinking once it has developed, but nothing
In addition to its incompatibility with the func¬ about the reasons for its development.
tions in Figure 2, there are three other difficulties On periodic schedules, the "post-reinforcement
with the post-prandial hypothesis. (1) Schedule- pause is controlled by food delivery as a discrimina¬
induced drinking usually takes a few sessions to tive stimulus. Since the pause* is generally taken up
develop (Hawkins et al., 1972; Reynierse 8c Spanier, with interim activities, their duration must be simi¬
1968; Staddon Sc Ayres, 1975). Yet post-prandial drink¬ larly controlled by food. When the duration or amount
ing is presumably well developed in normal adult of food delivery is increased (as occurred during test
rats. If drinking on schedules of food delivery is sim¬ sessions in the Lotter et al. experiment), post-reinforce¬
ply post-prandial drinking, there seems to be no rea¬ ment pause generally increases, although the increase
son why it should not occur from the start. (2) Drink¬ may be transient (Jensen Sc Fallon, 1973; Staddon,
ing on periodic food schedules is not always restricted 1970, 1974). Thus the first effect of increasing “meal”
to the period just after food delivery. Its location size should be an increase in the amount of drinking
132 SCHEDULE-INDUCED BEHAVIOR

per meal. In a careful reanalysis of the test session weight (Bowen, 1972; Falk, 1969), all point to an effect
data of Lotter et al., Millenson (1975) has recently of food motivation, hunger, and incentive, on induced
shown this to be the case. Thus, their data are con¬ drinking. The more motivated the animal (hunger:
sistent with what is known about schedule-induced deprivation, body weight) and the more motivating
drinking and temporal control on periodic schedules the situation (incentive: frequency, amount, and type
but not, unfortunately, with their conclusion. of food) the greater the tendency to drink. The in¬
centive motivation factor seems to follow the princi¬
Opportunity Hypothesis. Falk (1969) originally
ples now being codified as the quantitative law of
suggested that schedule-induced drinking is related to
effect (see de Villiers, Chapter 9). Thus drinking rate
the intermittency of food delivery. In its simplest
appears to be directly related to amount of food (Fig¬
form this view suggests that the rate of drinking is
ure 2), palatability of food (Falk, 1971), and frequency
more or less constant with the animal drinking for a
of food (Figure 2). Jacquet (1972) has studied the in¬
fixed fraction of the time available between pellets. It
teractions between two components of a multiple VI
implies that the function relating amount drunk per
VI schedule in terms of the relative and absolute rates
interval to size of interval should be monotonically
of both bar pressing and induced drinking in rats. She
increasing; whereas this function is actually bitonic
found that as the relative rate of food reinforcement
(Flory, 1971). Hence this view is not acceptable.
in one component increased (owing to a decrease in
Adventitious Reinforcement Hypothesis. This the rate of reinforcement in the other component), the

hardy perennial has been applied to interim drinking absolute rate of drinking tended to increase. This is
as well as to terminal “superstitious” responses (e.g., positive behavioral contrasty an effect often found
Clark, 1962; Moran, 1974; Segal, 1965). All the objec¬ with food-reinforced behaviors and generally attrib¬
tions to it raised earlier (p. 127) also apply here. In uted to a change in stimulus contingencies (see Chap¬
addition, induced drinking rarely occurs contiguously ter 3).
with food delivery (Segal, 1969 is an exception), and Thus, the evidence appears to support the view
is little affected by lick-contingent delays of food de¬ that induced drinking is related to food motivation,
livery unless these are so extreme as substantially to which is, in turn, affected both by internal factors
reduce food rate. As Figure 2 shows, over most of the (deprivation) and external factors (incentive). In a
range a reduction in food rate results in a decrease in later section I discuss possible mechanisms of interac¬
drinking rate. Unless food rate is controlled, therefore, tion between the motivational states of hunger and
suppressive effects of a negative contingency cannot be thirst that might underlie the empirical relation be¬
interpreted as acting directly on the tendency to tween induced drinking and food motivation.
drink. Induced drinking develops relatively slowly,
in step with the development of temporal discrimina¬ Induction: Terminal and
tion on periodic schedules (Staddon & Ayres, 1975). Interim Activities
Hence activities other than drinking are at first con¬
Since both interim activities and the terminal re¬
tiguous with food delivery. The adventitious rein¬
forcement view neither explains why these drop out, sponse tend to increase with food rate, and since
nor why drinking (rather than some other activity) neither can increase without limit, it seems likely that
these two classes of activity are in competition. This
supplants them in almost every individual rat. When
the water bottle is made available for only a brief section presents the evidence for such competition and
period during the inter-food interval (Flory 8c O’Boyle, looks at some of its effects.
1972), drinking still develops, even though lick-food Functions relating food-reinforcement rate to rate
of pecking or lever pressing (response functions) have
contiguities are specifically excluded and the water
for some years been a standard way of representing
bottle itself is a stimulus signaling the absence of
food (SA). the effects of schedules on behavior. The typical
schedule has been variable-, rather than fixed-interval,
Motivation Hypothesis. The steep fall-off in the and the response an instrumental rather than an in¬
functions of Figure 2 at low food rates, the greater duced one. However, there is every reason to suppose
drinking with 2-pellet versus 1-pellet food deliveries, that terminal responses on both response-contingent
the progressive development of schedule-induced and response-independent schedules are related in
drinking, in step with food-anticipation (Reynierse 8c similar ways to reinforcement and motivational vari¬
Spanier, 1968; Staddon 8c Ayres, 1975), and the inverse ables. Manipulations such as a shift in relative rein¬
relation between schedule-induced drinking and body forcement frequency produce similar contrast effects
J. E. R. Staddon 133

in both (e.g., Gamzu 8c Schwartz, 1973; Redford 8c strumental responding increase with food rate. Hence
Perkins, 1974), and the response contingency seems to it is reasonable to postulate the eventual development
act more to select one terminal response over others of competition between drinking and the terminal re¬
than to affect the “strength” of the response once sponse at high food rates. Moreover, there is some
selected (Staddon 8c Simmelhag, 1971). This is not to evidence that induced drinking (at least) tends to
say that all terminal responses have similar properties grow with food rate rather faster than terminal re¬
—there is by now ample evidence that they do not sponding. For example, the 1-pellet lick rate function
(e.g., Hinde 8c Stevenson-Hinde, 1973)—just that there in Figure 2 has an initial slope greater than one. Yet
seems to be no strong effect of response contingency instrumental (terminal) responding, which is often
per se on the properties of a given terminal behavior well described by Herrnstein’s equation, P = kRx/
once it is established. (Rt + R2), grows linearly at first, since when Ri is
An analysis in terms of terminal and interim peri¬ small the equation reduces to P = kRt/Ro, i.e., a slope
ods associated with different regions of post-food time of one in the log-log coordinates of Figure 2. This
does not seem as applicable to variable-interval as to implies that the fraction of the interval taken up by
fixed-interval schedules. Food does not seem to have an interim activity such as drinking should increase
the same kind of discriminative-stimulus status on with food rate. The eventual flattening out of the re¬
variable-interval schedules as it does on fixed-interval. sponse functions for both interim and terminal activ¬
Nevertheless it can be argued that on VI, as on FI, ities can be taken as one outcome of the competition
time is segmented into interim and terminal periods, between them.
although these are not as simply related to post¬
reinforcement time (Rachlin, 1973). (The factors ac¬ Running. Figure 3 illustrates another effect of
counting for this difference, and for the temporal competition between terminal and interim activities.
location of activities within the inter-food interval, The figure shows rates of wheel running and drink¬
will be taken up in the third section.) Thus it seems ing (in licks/min and ml/min) of five female rats
reasonable to assume that the rate of a terminal exposed to five different fixed-time schedules in a
response such as pecking, once it is established, de¬ hexagonal apparatus that afforded access to a variety
pends much more on variables such as food rate, of activities (Staddon & Ayres, 1975 and unpublished).
palatability of food, size of food portion, and hunger Drinking rate increases with food rate, as in Figure 2,
than on temporal relations between the response and and the function form for licks/min and ml/min is
food delivery. We have just seen that these are the the same. However, the rate of wheel running de¬
variables that determine the level of induced drink¬ creases as food rate increases, suggesting suppression
ing. In a sense, therefore, (instrumental) terminal as by competition from drinking and the terminal re¬
well as interim responses can be regarded as schedule- sponse (which in this ease was waiting m the feeder
induced behavior. area accompanied by pawing and chewing at the
If each terminal or interim response (e*g., lick, feeder opening: ‘food anticipation”). Thus, it ap¬
peck, bar press) is assumed to require an approxi¬ pears as if running is not schedule induced, but rather
mately constant time, then any increase in rate of “fits in” at times when the tendency to engage in the
responding with food rate implies that the responding two dominant classes of activity is weak. This view is
will take up an increasing fraction of the interfood in agreement with the temporal distribution of the
interval as interval length decreases.1 As we have al¬ three classes of activity, with drinking occurring first
ready seen, the rates of both induced drinking and in- in the interval, followed by running and then food
anticipation (cf. Figure 1). It is also consistent with
the finding of Staddon and Ayres (1975) that during
i This can easily be shown algebraically. For an interim
activity A, where each response takes up a fixed time tA, then if acquisition, the temporal pattern of food anticipation
A takes up a fixed fraction, kr of the interfood interval, T: and drinking developed to essentially its final form
NAtA — kjT, where NA is the average number of occurrences of
before much running had occurred.
A per interval. The response function for A is then given by
NA/T = k1/tA, which is a constant. Hence any growth in the Other evidence suggesting that running is not
rate of A with food rate means that A takes up an increasing schedule induced is that when food delivery is dis¬
fraction of the interval as food rate increases. If the rate of A is
continued (extinction), running rate and the fraction
proportional to food rate, NA/T — k2/T, then the fraction of the
interval taken up by A is equal to tAk2/T, i.e., also proportional of total time the animal spends in the running wheel
to food rate. Similar calculations can be carried out for the area increases (Staddon 8c Ayres, 1975, and unpub¬
terminal response, B, and it is clear that when tAk9 -j- tBk3 = T
lished). Even if running is required for the produc¬
(where k2 and k3 are the constants of proportionality) the entire
interval is taken up. tion of food, its overall frequency may not be in-
134 SCHEDULE-INDUCED BEHAVIOR

Fig. 4. Schematic representation of the relation between inter¬

food interval and the proportion of the interval taken up by
interim, “facultative,” and terminal behaviors.

enclosed in a running wheel in their apparatus, so

that running may have been confounded with general
activity, which, as Killeen (1975) has shown, increases
with food rate. Similarly, Staddon and Ayres (1975)
report a decrease in overall activity, measured as area
changes per min, in extinction, but wheel running in
their apparatus increased in extinction. Smith and
Clark (1974), using an apparatus similar to that of
Levitsky and Collier and a multiple spaced-responding
schedule, obtained mixed results: one rat showed less
running at low food rates, two others showed a bitonic
relation. Thus, apart from exceptions that may reflect
peculiarities of some kinds of running-wheel ap¬
paratus, it appears that running is suppressed by a
food schedule.
The picture that emerges from this account is sum¬
marized in Figure 4. The figure shows the inter-food
interval divided into three periods; an interim2 pe¬
Fig. 3. Drinking rate flicks/min and ml/min) and running riod, devoted to drinking (if water is available) and
rate (turns of a 27cm dia running wheel/min) vs. food rate perhaps other activities, such as aggression (see be¬
on various fixed time food schedules for four female rats (Stad- low); a terminal period, devoted to food anticipation
don and Ayres, unpublished data.) Points show individual rats,
lines are means. Each rat was exposed to four of the five or the instrumental response; and a third period,
food rates; hence each point is the average of four animals.

2 Until this point the term “interim” has been applied to all
those activities which precede the terminal response within the
creased, although its temporal distribution adapts to interfood interval. In Figure 4, however, only activities induced
the schedule (Skinner 8c Morse, 1958). by the schedule, such as drinking, are so labelled; “neutral”
Levitsky and Collier (1968) have reported an in¬ activities, such as running, fall into the “facultative” category.
There seems to be no reason to settle on either usage as defini¬
crease in running under schedule conditions as op¬
tive, providing that whenever the term is used it is clear which
posed to extinction. However, the rat was entirely is intended.
J. E. R. Sfaddon 135

when activities not induced by the schedule, faculta¬ Hence even on a variable schedule, if water is avail¬
tive activities, can occur. Running appears to be a able it is possible that food will not be eaten in the
facultative activity in this sense. Other possibilities period just after a previous food delivery. This period
are comfort activities, such as preening and grooming. therefore qualifies as an interim period in terms of the
The diagram has two critical features: (a) that the obtained, if not the scheduled, distribution of food
percentage of time devoted to both terminal and deliveries. The temporal locus of induced drinking
interim activities increases as interval size decreases, on variable schedules may thus be traced to the way
and (b) that this progressive increase is limited at in which the animal’s initial behavior interacts with
short intervals, when the entire interval is taken up the properties of the schedule; by drinking after eat¬
by the two classes of induced activities. ing, the animal can produce an obtained distribution
The diagram reveals several uncertainties. First, of intereating times that is quite different from the
the exact form of the area boundaries is not known. programmed distribution. The obtained distribution
In particular, it is necessary to know the function then maintains the behavior that led to it. This kind
relating overall rate of an activity such as licking or of “self-fulfilling” schedule-behavior interaction is not
key pecking to the percentage of time taken up. The an uncommon occurrence in situations that allow for
simple proportionality assumed for illustrative pur¬ the expression of induced behavior (see Staddon Sc
poses in footnote 1 may not hold generally. Such func¬ Ayres, 1975, for other examples). Any irregularity in
tions are not available at present. It is also likely that the function relating probability of food to post-food
the boundaries of the interim and terminal areas are time (i.e., any deviation from a constant probability)
not fixed, but depend on the strength of behavior in should accentuate this “self-fulfilling” tendency,
the “other” category. For example, in the experiment especially if the probability is relatively lower (not
by Staddon and Ayres (1975), one rat showed anom¬ necessarily zero) in the immediate post-food period.
alous behavior traceable to a very strong tendency to Millenson (personal communication) has data in sup¬
run. It failed to show the usual temporal distribution port of this inference, since he finds that schedule-in¬
of activities within the inter-pellet interval until duced drinking is less reliably obtained on a random-
either running was prevented, or more time was made interval (i.e., constant probability) schedule than on
available for it by preventing drinking (see p. 147). a variable-interval schedule with an arithmetic pro¬
The diagram is also restricted to periodic food sched¬ gression of intervals (in which the probability of food
ules in which opportunities for all activities are con¬ increases with post-food time).
tinuously available; it does not deal with either vari¬ This interpretation recognizes a link between eat¬
able schedules or limited—availability schedules. ing and subsequent drinking, although such a link
Finally, the possible mechanisms underlying the dis¬ need not be either strong or immodifiable. For
tribution of activities remain to be considered. example, FIQry and O'Boyle (1972) have shown that if
drinking is prevented in the period just following
food delivery, but is permitted later in the interfood
Interim Activities and Sa Periods
interval, the rat drinks almost as much as when water is
Drinking and other schedule-induced interim ac¬ continuously available. Gilbert (1974) has reported a
tivities always seem to occur (in the steady state) at similar result. Drinking neeurs after each response on
times, or in the presence of stimuli, that signal the spaced-responding schedules (Segal Sc Holloway, 1963)
absence of food (SA or interim periods). On fixed- and has been reported to occur after brief stimuli on
interval schedules, food is not available early in the second-order schedules (Porter 8c Kenshalo, 1974;
interval. Hence the occurrence of schedule-induced Rosenblith, 1970) although there are some conflicting
drinking or attack at those times is explainable. How¬ results (Allen, Porter Sc Arazie, 1975; Porter, Arazie,
ever, food can occur at any time on variable interval Holbrook, Cheek 8c Allen, 1975). These results all
schedules: food delivery is as likely just after food as suggest that although the link between eating and sub¬
it is at other times. Yet even on variable schedules, sequent drinking is a factor in the temporal location
induced drinking still tends to be restricted to the of drinking in the interfood interval, it is not essential
period just after food delivery. How can this be ex¬ to its induction.
plained? In limited-availability procedures, presentation of
Under free conditions, rats tend to drink just be¬ the water bottle signals a period when food will not
fore and just after meals (e.g., Kissileff, 1969). It seems be delivered. The fact that rats drink with undimin¬
likely that post-prandial drinking, at least, is main¬ ished vigor during these periods conforms to the
tained when animals are exposed to intermittent food. general conclusion that induced drinking is charac-
 136 SCHEDULE-INDUCED BEHAVIOR

teristic of interim (SA) periods (Falk, 1969, 1971; Stad- induced biting attack on a rubber hose (Hutchinson,
don 8c Simmelhag, 1971). It also indicates that such Azrin, 8c Hunt, 1968) and rats on schedules of food
periods need not be defined temporally (cf. Wiittke 8c or water reward will attack another animal (Gentry 8c
Innis, 1972). If interim activities have something to do Schaeffer, 1969; Thompson 8c Bloom, 1966).
with the animal’s ability to “time,” i.e., to refrain Figure 5 shows data replotted (in log-log coordi¬
from making the terminal (food-related) response at nates) from three experiments that have studied
times when food is not available, one might expect schedule-induced attack in pigeons as a function of
them to be reduced in strength when an external cue food rate (Cherek, Thompson, 8c Heistad, 1973; Cohen
is available. Unfortunately the effect of stimuli on the 8c Looney, 1973; Flory, 1969). Cherek et al. measured
strength and locus of interim activities has not been the rate at which pigeons pecked against the front of
systematically assessed. a transparent box containing a live target bird, on a
response-initiated fixed-interval schedule. Flory used a
stuffed target pigeon and a fixed-time food schedule;
Schedule-Induced Attack and Cohen and Looney used a mirror as the target
and a multiple fixed-ratio 25 fixed-ratio N (N varied
Rats show strong induced drinking on periodic
from 25 to 150) schedule. Cherek et al. and Flory both
food schedules, but pigeons and doves show weaker
used a “delay” contingency which prevented food
effects (Shanab 8c Peterson, 1969). However, several
delivery within 15 sec of an attack response—a com¬
experiments have shown that these birds on food
mon protection against “adventitious” reinforcement
schedules will attack another bird, a stuffed model, a
of attack by food delivery. The target bird was avail¬
mirror, or even a color slide of a pigeon (e.g., Azrin,
able on a fixed-ratio 2 schedule for 15 sec at a time in
Hutchinson, 8c Hake, 1966; Cohen 8c Looney, 1973;
the Cherek et al. study, but continuously available in
Flory 8c Ellis, 1973). Squirrel monkeys show schedule-
the others. Despite these differences of target, sched¬
ule, and target availability, the attack rate data in
Figure 5 show considerable overall agreement. Rate of
aggressive pecking peaks at a food rate between .30
and 1.0 per min and declines sharply at lower and
higher rates. In linear coordinates the falloff at high
food rates is more gradual than the falloff at low
rates. The absolute rate of target pecking in the Cohen
and Looney study was higher than in the others be¬
cause they report “local” rate (i.e., rate in the post-food
pause) rather than overall rate. The decline in re¬
sponding at higher food rates in their study was more
gradual (in linear coordinates), but it is not clear
whether this was because of the ratio food schedule,
the mirror-image target, or other features of their
situation.
The general agreement among the functions in
Figure 5 suggests that food rate (deliveries/min) is the
determining factor in all these experiments and that
the way in which this food rate comes about, whether
via an interval or a ratio schedule, is much less im¬
portant. Certainly periodic food is essential to the
attack responding, since attack declines to a negligible
level in extinction (Cherek et al., 1973). In an explicit
ABAB comparison of response-dependent and re¬
sponse-independent interval schedules of food delivery,
Cherek et al. found little consistent effect of the re¬
sponse contingency on rate of attack. There are as yet
no data on yoked-control comparison of time versus
Fig. 5. Data replotted from Cohen and Looney (1973), Cherek
et al. (1973) and Flory (1969) relating measures of attack rate to ratio procedures, so factors in addition to food rate
food rate on various intermittent schedules (see text for details). cannot be entirely excluded.
J. E. R. Staddon 137

Drinking and Attack attack rate on FI 2 min would suggest that the decline
in attack rate at higher food rates reflects a time-
Comparison of the attack rate data in Figure 5
course limitation, rather than a non-monotonic rela¬
with the drinking rate data in Figure 2 shows that al¬
tion between food motivation and tendency to attack.
though both types of behavior fall off drastically at
food rates less than about 0.5 per min, attack respond¬
Other Interim Activities
ing also shows a decline at higher food rates. Induced
drinking only shows such a falloff at very high food Data on interim activities other than drinking,
rates (in excess of about one per 4 sec, Flory, 1971). running, and attack are sparse (see Falk, 1971, for a
Staddon and Ayres (unpublished) find that this de¬ review). Pica (eating of non-food objects) has been
cline in the average function occurs because some observed in rats and monkeys (e.g., Villareal, 1967);
individuals fail to drink at all when food is delivered and pigeons, humans, and sometimes rats, show a
very frequently. When pellets were delivered every 5 variety of stereotyped motor patterns, such as pacing,
sec perhaps half the rats in the Staddon and Ayres neck stretching, wing flapping and preening or groom¬
study failed to drink in most intervals (cf. Segal et al., ing (Kachanoff, Leveille, McLelland & Wagner, 1973;
1965). The most obvious explanation for this is that Keehn, 1972; Staddon 8c Simmelhag, 1971). In unpub¬
the time available between food pellets is simply in¬ lished observations I have noticed that schedule-in¬
sufficient, although since some rats continued to drink duced wing flapping in pigeons is a strikingly auto¬
the limitation seems not to be the purely mechanical nomous behavior. Although all the necessary tests
one of getting to the water bottle and back again be¬ have not been carried out (see pp. 145 et seq.), it
fore the next pellet delivery—especially as rats on appears as if each bout is internally timed (see pp.
some variable schedules will drink so much that they 145) so that wing flapping persists even if food is
postpone food delivery or fail to pick up the pellet made available during a bout. Pigeons will sometimes
when it is delivered (Clark, 1962; Falk, 1961). The repeatedly miss food deliveries in this way. Gilbertson
decline in the attack rate function occurs at relatively (personal communication) has obtained reliable in¬
low food rates and cannot be explained by any kind duced preening by attaching a piece of solder wire to
of mechanical limitation. the pigeon’s wing as a minor irritant. There is also
Induced attack, like induced drinking, is an interim one report in which pigeons, trained to key peck for
activity and occurs in the period just after food de¬ food on a fixed-ratio schedule, pecked a bolt head as
livery on fixed-interval and fixed-ratio schedules an interim activity (Miller 8c Gollub, 1974).
(Richards & Rilling, 1972). Like induced drinking, it There are also few data on schedules using positive
occurs after each response on spaced-responding reinforcers other than food or water. Gilbertson (per¬
(DRL) schedules (Knutson & Kleinknecht, 1970). sonal communication) has trained male pigeons to
However, data are lacking on its temporal position peck on a ratio schedule for the sight of a female.
relative to other induced activities, such as drinking, These birds show courtship behavior, bow-cooing and
preening, etc., when opportunities for several are wing flapping, as interim activities, as an after effect
available. Comparison is made more difficult because of the sight of the female (see also Nelson, 1905;
most attack studies have been done with pigeons, most Sfivenster, 1973}.
studies of induced drinking with rats. It would also be
useful to find a facultatiy^ aciiYVy analogous to run¬
Induced States
ning in rats, that could be used in similar fashion to
help clarify th$ interactions among induced activities The regions of post-food time identified in Figure
in pigeons.
4 as “interim,” “facultative,” and “terminal,” and their
The early falloff in attack rate as food rate in¬ associated behaviors are more properly considered as
creases beyond one delivery per 2 min suggests a states or “moods” of the behaving animal (rather than
difference in the time courses of drinking and attack. simply as “behaviors")- This is because they represent
Perhaps there is a limit to the speed with which a different kinds of behavioral potential. The same
tendency to post-eating attack can build up, no matter stimulus, water for example, has different effects dur¬
how strong the inducing factors, i.e., the hunger and ing the interim period from those it has during the
incentive motivation associated with the food sched¬ terminal period. The rat drinks in the one but not in
ule. Data on the effects of hunger and of food type the other; hence his state must have changed, and this
and amount on attack rate would help to sort this out. change can be traced to the different temporal cues
For example, a direct relation between hunger and effective during these two periods. Similarly, brief test
138 SCHEDULE-INDUCED BEHAVIOR

presentations of food during the interim period may the induced state. For example, one could see whether
fail to elicit eating (Konorski, 1967; Staddon, unpub¬ a rat on a periodic food schedule will respond in the
lished observations), at least the first couple of times. interim period to stimuli that in its past have been
Eventually the animal will eat, but by then the dura¬ associated with access to water. The second method is
tion of the interim period (which is defined by the concerned with the motivational properties of the
animal’s history of exposure to food opportunities) state, with for example, the reinforcing effectiveness of
will have changed to allow for eating earlier in the water during a food schedule as compared with its
interval, as it does when an animal is shifted from a effectiveness in the absence of the schedule.
fixed-to a variable-interval schedule (e.g., Innis Sc Stad- The cognitive properties of induced interim states
don, 1971; Rachlin, 1973). have not been adequately explored; much more has
The states that fill up the interfood interval differ been done on their motivational properties. I first con¬
in their motivational properties. Not only does the rat sider the similarities between induced drinking and
drink during the interim period, but its state resem¬ thirst, and then the properties of interim states gen¬
bles thirst, just as the terminal state resembles hunger. erally.
For example, rats and monkeys will learn to press a
lever to obtain access to water during the interim
Induced Drinking and Thirst
period (Falk, 1971). Whether “thirst” in this sense is
identical to “thirst” that follows water deprivation is Falk (1969, 1971) has identified several factors that
hard to say. They certainly share many properties, as point to a similarity between thirst and the rat’s state
will become clear in a moment. when induced drinking is observed: (a) The acquisi¬
Perhaps because of a preoccupation with stimulus- tion of induced drinking is impaired by pre-loading
response notions of behavioral causality, the proper¬ with water; (b) rats and squirrel monkeys will learn
ties even of terminal states have been little explored to press a lever during the interim period to obtain
until recently. Pavlov (1927) induced such states by access to water; (c) rats drink less if the terminal food
means of stimulus-reinforcer contingencies, but reinforcer contains water; (d) rate of induced drinking
studied only a fraction of the animal’s potential be¬ is directly related to the palatability of the available
havior: the “conditioned response” of salivation. liquid. To these can be added the finding that water-
Occasional anecdotes filtered out of his laboratory deprived rats will lick a stream of air (e.g., Mendelson,
suggesting that the conditioning operations produced Zielke, Slangen Sc Weijnen, 1972; Werner Sc Freed,
much more extensive changes than this. For example, 1973), and this activity can be induced by food sched¬
Liddell (recounted in Lorenz, 1969) noticed that a dog ules in the same way as drinking. On the other hand,
released from its harness would approach and jump some physiological studies report differences between
upon the metronome CS. Zener (1937) in a classic schedule-induced and “normal” drinking (e.g., Carlisle,
paper described a variety of other behaviors in the 1971, 1973).
presence of the conditioned stimulus, suggesting that Water acts as a reinforcer for water-satiated rats on
animals develop “expectations” about the imminence periodic food schedules, as shown by its effectiveness
of food. Bolles (1972) has recently defended a revival in maintaining lever pressing. Along the same lines, a
of this position. Recent work on auto shaping (e.g., study by Allen and Porter (personal communication)
Browne, 1973) lends it some support. using a multiple FI 1 min FI 1 min food schedule,
Labelling induced states with terms such as showed positive contrast effects with a water-reinforced
“hunger” or “food expectancy” is convenient, and re¬ response. Water was at first available in both com¬
minds us of moods familiar from introspection. Un¬ ponents of the multiple schedule on a FI 0.75 sec
fortunately a feeling of familiarity is not the same schedule. Later, response rate on the water lever was
thing as exact knowledge, and may even hinder the recorded in one component as a function of whether
search for it. Induced states can be explored in two or not water was available in the other. When water
main ways: (a) By looking at the effect of various test was removed in one component, response rate on the
stimuli, when the animal is in the state, as compared water lever in the other component increased (posi¬
to when it is not; (b) by looking at the effectiveness of tive contrast). Thus, the tendency to drink on
various reinforcers when it is in the state as compared periodic food schedules is determined both by food
to when it is not. The first is equivalent to the method rate variables and by variables related to water avail¬
of transfer tests, used to discover “what is learned” in ability. Since the effectiveness of water as a reinforcer
a learning situation. It is concerned with what might is presumably also related to food rate, these effects
be termed the cognitive and perceptual properties of point to quite complex interactions between food rate
J. E. R. Staddon 139

(the fundamental instigating variable), the tendency duced drinking and the food-related terminal response
to drink, and the capacity of water to act as a rein¬ are related to food motivation in a similar way; hence
forcer. the “hungrier” the animal during the terminal period,
These data on the effects of water motivation on the “thirstier” he is during the interim period. These
induced drinking support the earlier generalization data, together with physiological links between hypo¬
that the strength of schedule-induced drinking is thalamic structures involved in eating and drinking
jointly determined by both food and water motiva¬ (Akerman, Andersson, Fabricius, 8c Svensson, 1960;
tion. Its sensitivity to water-motivation factors under¬ Wayner, 1970), and at least one demonstration, with
lines both its “state” character and its similarity to doves, of adjunctive eating on a water schedule (Mc¬
normal thirst. The evidence is not sufficient to assert Farland, 1965), make the idea of a reciprocal interac¬
that deprivation-induced and schedule-induced tion between these two motivational states a plausible
“thirst” are identical, however, and it would be sur¬ one. There are difficulties in testing the idea of com¬
prising if they were in view of the different temporal plete reciprocality, however, because severely water-
properties of the two. deprived animals will not eat. This may underlie the
failure of Carlisle, Shanab, and Simpson (1972) to in¬
duce eating in thirsty rats by means of a periodic
Terminal-Interim Interaction
water schedule.
Several lines of evidence suggest that for many The notion that during the interim period on a
species the interim period on periodic food schedules food schedule animals are motivated in ways antag¬
is aversive: (a) Many of the interim activities devel¬ onistic to food motivation explains the apparent
oped by pigeons on fixed-time schedules are suggestive aversiveness of the interim period. A thirsty animal
of flight; neck-stretching, hopping, wing flapping, and might well try to escape from a food situation. Indeed,
even brief hopping flights (Staddon Sc Simmelhag, Pliskoff and Tolliver (1960) have shown that hungry
1971; unpublished observations). Many of their move¬ rats maintained on a fixed-ratio food schedule will
ments resemble the intention movements made by respond more on a second lever that removes them
wild pigeons just before they take flight (e.g., Davis, from the food schedule (by producing a 5-min time¬
1973). (b) Pigeons will learn to peck a key during the out) when deprived of water for three days than when
interim period on FI to produce a timeout (house- not water-deprived. There are several uncertainties,
lights off and different stimuli on the response keys), however. For example, even in large enclosures
and their tendency to do so is bitonically related to pigeons do not stray far from the response key during
interval value, although the peak is at about FI 4 the interim period (Staddon, unpublished observa¬
rather than FI 2, the value for maximum attack tions), although careful studies of the effects of sched¬
(Brown 8c Flory, 1972). Numerous other studies have ule variables on spatial position have not been carried
shown that both rats and pigeons find interim periods out, Hence the induced aversiyeneas of the food site
aversive (e.g., Appel, 1963; Azrin, 1961; Thompson, (or response key—the data do not distinguish between
1964). (c) As we have already seen, pigeons, rats, and the two) must decline faster with distance than its
monkeys will attack other animals, objects, or repre¬ attractiveness. What determines the form of such
sentations during the interim period, and pigeons will gradients and how might they be measured? Another
peck a key for the opportunity to do so (Cherek et al., difficulty is that most of the data on the aversiveness
1973). of the interim period come from pigeons, most data
Staddon and Simmelhag (1971) tentatively pro¬ on induced drinking from rats. How legitimate is
posed that there is a reciprocal interaction between generalization from one species to another?
the terminal and interim states on periodic schedules; Schedule-induced aggression poses a related prob¬
and that this mechanism serves the adaptive function lem. Is it induced by the schedule in the same way as
of removing animals from food situations at times drinking? The falloff in the response function at high
when food delivery is unlikely. There is some evidence food rates suggests not, but there may be other ex¬
that enforced proximity to the food site enhances planations for this. Or is aggression one outcome of
schedule-induced drinking. Clark (1962) found less the conflict between the tendencies to approach and
drinking when the water bottle was moved away from retreat from the food site? This kind of explanation—
the food tray, and Staddon and Ayres found much less disinhibition of activity A owing to conflict between
drinking in their hexagonal apparatus (in which food strongly excited incompatible activities B and C—has
and water sites were separated) than that reported by been proposed for the displacement activities studied
Flory (1971) and others (compare Figures 2 and 3). In¬ by ethologists (cf. Hinde, 1970). However, the hypoth-
140 SCHEDULE-INDUCED BEHAVIOR

esis has not been presented in a quantitative way that to the temporal pattern of reinforcer delivery. Indeed,
allows for a convincing test. the temporal pattern of the instrumental response was
It is also clear that the division into terminal and the aspect of reinforcement-schedule performance that
interim periods is an oversimplification. At interfood first attracted attention (Ferster 8c Skinner, 1957),
intervals greater than 20 or 30 sec the middle of the and it continues to provide a topic for dozens of re¬
interval seems to be a period when non-induced (fac¬ search reports each year. This temporal regularity
ultative) activities, such as running, can occur. The raises two obvious questions: (a) What determines the
growth of this period with interval duration parallels temporal locus of the various activities? (b) What is
the shift from “break and run” to “scallop” on fixed- the nature of the “clock” that times these activities? I
interval schedules as interval value increases (Schnei¬ take up these two questions first. The third section
der, 1969). The “scallop” period, when the cumulative deals more generally with types of sequential inter¬
record shows a gradual transition to a high, steady action.
rate, may correspond to the “facultative” period in
Figure 4. Possibly activities such as preening and
Factors Affecting the Temporal Locus of
grooming, which are not sensitive to food reinforce¬
Terminal and Interim Periods
ment contingencies (Shettleworth, 1973) and do not
compete with other activities (“disinhibited” activ¬ Most studies of the relation between the temporal
ities; McFarland, 1970), can occur at this time. sequence of reinforcers and the temporal sequence of
There are other puzzling facts that emphasize how behaviors have been concerned only with the pattern
little we understand the mechanisms underlying in¬ of the instrumental response. As we have already seen,
duced behavior. For example, pigeons trained to peck periodic reinforcement generally produces a corre¬
for food on a fixed-interval schedule will continue to sponding periodicity in behavior, with the instru¬
do so if periodic food delivery is maintained indepen¬ mental response occupying the last third or so of the
dently of responding. However, if the interfood inter¬ inter-reinforcement interval. There are some excep¬
val is relatively long, pecking often becomes confined tions, in situations with weak reinforcers or labile re¬
to the middle of the interval (Shull, 1970; Staddon 8c sponses (e.g., Weiner, 1969), and many species (fish,
Frank, 1975a). Is this pecking different from food- octopus) apparently fail to show this kind of temporal
related pecking, such as that induced in auto shaping adaptation. However, it is sufficiently common that
situations, which tends to occur with highest prob¬ the search for rules to describe it seems justified.
ability close to food delivery? Or is it simply induced The rule seems to be that the local rate of respond¬
by the food situation in a similar way to induced ing (i.e., rate over some short time interval) is directly
drinking, where, at low food rates, each food delivery related to the relative proximity to reinforcement or,
seems to produce a more or less constant amount of on aperiodic schedules, the relative density (i.e., rate
drinking? A final problem is the role of stimulus con¬ over some brief time interval) of reinforcement
trol and “conditioning.” A behavior induced orig¬ (Catania 8c Reynolds, 1968; Jenkins, 1970; Staddon,
inally for one reason may be maintained for another. 1972b; Zeiler, Chapter 8 in this volume). In short,
Thus an interim activity that develops in the post¬ pigeons peck more at times when food is more likely.
food period because this period signals the absence of No one has yet succeeded in reducing this rather
food may subsequently come under partial control by common-sensical principle to a mathematical form
features of the environment (as drinking comes under that is universally satisfactory (see de Villiers, Chapter
the control of the water-bottle, for example). Conse¬ 9 in this volume, on the quantitative law of effect),
quently, a change in the animal’s environment may but the details are not important for present purposes.
affect terminal and interim behaviors differently. This This relative proximity rule should not be thought
differential effect may explain the effects of restraint of as a causal law about the effect of independent
on “temporal discrimination” described below. reinforcement variables on dependent behavioral
ones. It is an equilibrium principle, that describes the
steady-state relation between reinforcement and be¬
TEMPORAL AND SEQUENTIAL havior, once behavior has settled down. On response-
STRUCTURE OF INDUCED ACTIVITIES contingent schedules reinforcement rate is affected by
behavior (as well as vice versa) and often many
All the species commonly used in operant condi¬ equilibria are possible. For example, a pigeon on a
tioning experiments, rats, pigeons, monkeys, and high-ratio schedule may cease to respond because its
people, adapt the temporal pattern of their behavior initial response rate is too low to produce sufficient
J. E. R. Staddon 141

reinforcement to sustain pecking. This outcome, with 1973). Because interim periods must be short under
zero reinforcement supporting zero responding, is just these conditions, there is no opportunity for full¬
as consistent with the law of effect as an equilibrium blown interim activities, and thus no easy verification
in which a high response rate is maintained by a high of the hypothesis.
reinforcement rate. By itself the law of effect does not With the exception of a number of experiments on
predict which will occur. Other factors, involving the quantitative aspects such as duration (Schwartz 8c Wil¬
historical development of the final equilibrium, and liams, 1972) and force (Chung, 1965; Cole, 1965),
including factors that may facilitate or inhibit in¬ study of the topographic details of pecking on various
duced behavior, must be understood for a full ac¬ schedules has not been extensive. In informal observa¬
count. tions I have noticed that on variable-interval sched¬
We have already seen that even when no instru¬ ules pigeons generally turn away from the key between
mental response is required, behavior can usually be pecks, whereas pecking on ratio schedules, or during
subdivided into terminal and interim classes, with the the terminal “run” on fixed-interval, is much more
(induced) terminal response apparently following the single-minded, with little turning away. It does not
same set of rules as if it were an instrumental re¬ seem far-fetched to interpret this turning away (and
sponse. As in other cases discussed in this chapter, thus the gap between pecks associated with it) as an
most of the evidence comes from the study of a few interim period, with properties similar to interim
behaviors in a few species. Pecking in pigeons seems to periods on periodic schedules.
follow the density of food delivery in the same way on
non-contingent as on response-contingent schedules, at
least under the restricted conditions studied by Stad¬ Timing of Induced Sequences
don and Simmelhag (1971). However, under other
The reliable association between induced interim
conditions, particularly when (free) food delivery is
activities and periodicity of the terminal, generally
relatively infrequent (less than one per 2 min or so),
instrumental response, has led to a number of at¬
this relation breaks down. Instead of occurring in
tempts to find a causal link between the two (e.g.,
anticipation of food delivery, pecking may occur in a
Glazer & Singh, 1971; Hodos, Ross, & Brady, 1962;
burst in the middle of each interval (Shull, 1970;
Nevin 8c Berryman, 1963; see Harzem, 1969, and
Staddon 8c Frank, 1975a). Unfortunately the observa¬
Kramer & Rilling, 1970, for reviews). Is the periodicity
tional work necessary to decide whether or not there is
of the terminal response on, say, a fixed-interval sched¬
another response, other than pecking, that anticipates
ule caused in some way by the regular sequence of
food in these cases (i.e., a terminal response) has not
interim activities that typically precedes it? This ques¬
been done, although it is clear that pecking is not in¬
tion cannot be answered until the mechanism by
duced on long fixed-time schedules as it is on short
which interim activities might serve to “time” the
(Simmelhag, unpublished observations).
terminal response is made more explicit. Suggestions
Interim activities occur at times when the terminal
in previous published work are of two general kinds:
response is not occurring. There seem to be two con¬
chaining explanations, and “behavioral clock” ex¬
ditions under which a terminal response fails to occur
planations.
(or occurs at a reduced rate): (a) When relative rein¬
forcement rate is low, but there is the opportunity for
Chaining
reinforcement at any time (e.g., variable-interval
schedules); (b) when there is no reinforcement oppor¬ Chaining is the simplest possibility. The model is
tunity (overall reinforcement rate may be high or low; the “domino theory,” according to which each be¬
the period immediately following food on fixed-inter¬ havior in the temporal sequence A-B-C-D- etc. directly
val schedules, and following a response on spaced-re- produces the next: the offset of activity A produces
sponding schedules, are examples). In case (b), the the onset of B, the offset of B the onset of C, and so
terminal response does not occur at all during SA on. In the usual form of this explanation, each indi¬
periods, which are the occasion for observable interim vidual activity is assumed to take a characteristic time,
activities. However, in case (a), if interim periods can i.e., the distribution of bout durations will show a
be said to exist at all they must be brief and inter¬ mode at a “preferred” duration (see McGill, 1963, for
spersed between occurrences of the terminal response: a review of stochastic processes and “temporal dis¬
variations in rate then correspond to variations in the crimination”). This is not necessary, however. If the
percentages of time taken up by the terminal and number of links in the chain is fixed, the time from
interim periods (see footnote 4, p. 144, and Rachlin, the beginning of the chain to the onset of a given
 142
SCHEDULE-INDUCED BEHAVIOR

later member will not be random (i.e., exponentially It is perhaps worth noting that most attempts to
distributed) even if each link has a random duration. explain wholly endogenous behavior sequences by
This follows directly from the central limit theorem, means of behavior or reflex chains have been unsuc¬
since the time of onset of chain link M is equal to the cessful. For example, insect flight and walking pat¬
sum of the durations of links 1 through M-l. The terns, once thought to depend on chain reflexes, have
more intervening links, the more sharply peaked will been shown to involve central programming (e.g.,
be the distribution of times of onset of an activity late Wilson, 1961, 1966). However, chain accounts have
in the chain. Thus, whether or not each activity is been quite successful in explaining behavior sequences
intrinsically timed, the chaining mechanism can incorporating extrinsic stimuli, such as the “lock and
nevertheless result in “temporal discrimination/’ de¬ key” courtship sequences described by Tinbergen
fined as a peaked, nonrandom distribution of starting (1951), and the hunting behavior of the wasp Philan-
times, for an activity late in the chain. thus triangulum and many other invertebrate preda¬
Unfortunately, chaining explanations do not fit the tors (see Hinde, 1970, for a review). In the context of
facts for behavior sequences induced by periodic operant behavior, the chaining concept arose in con¬
schedules. There are three kinds of evidence that pose nection with chained schedules. These parallel the
problems;
ethological examples just mentioned, in that responses
(a) Interim activities such as drinking are often produce external stimuli that in turn produce other
repetitive. Thus the kind of chain actually observed is responses. It seems prudent to reserve chaining ac¬
closer to A-A-A- * * . -Q (a homogeneous chain) counts specifically for situations in which the succes¬
than to the A~B—C— . . . — Q (heterogeneous chain) of sive stimuli are provided by the environment, with
the model. What determines run length in the homo¬ only the response elements of the chain being con¬
geneous chain? No merely probabilistic process will tributed by the animal.
suffice to make the length of the chain other than (c) Induced sequences show both variability of
random (i.e., independent of time); some kind of succession (A is not always followed by B) and
counting mechanism is required. Yet there is no temporal variability (B does not always occur at, or
evidence that animals can count better than they can for, the same time). Both these features are incom¬
time, so that an explanation of timing in terms of patible with simple chaining, but might be modeled
counting is unsupported.
by a Markov process (Cane, 1959, 1961; Staddon,
(b) In a simple chain, the necessary and sufficient 1972a). However, analyses of behavior sequences in
cause of activity N is the occurrence of activity N-l, both pigeons (Staddon, 1972a) and rats (Staddon &
Hence prevention of activity N-l should eliminate Ayres, 1975) show that even a Markov account is not
activity N. This is not what happens in temporal be¬ adequate, at least in a simple form. The essential
havior sequences, For example, elimination of the property of a Markov process is that each state (activ¬
water bottle normally present during a fixed-time food ity) is dependent only on the preceding one. There¬
schedule usually causes rats to make the terminal re¬ fore there should be no dependence of the onset (or
sponse earlier in the interval (Staddon 8c Ayres, 1975). offset) of an activity on time, other than the time
Many experiments, using both spaced-responding and elapsed since the preceding activity (or since the be¬
periodic schedules, have shown that prevention of ginning of the activity). Yet on fixed-time schedules
interim activities disrupts temporal discrimination by both pigeons and rats show such dependencies. The
causing the instrumental response to occur too soon time between two successive activities tends to be
(e.g., Frank & Staddon, 1974; Glazer & Singh, 1971; shorter the later the first activity ends within the inter¬
Laties, Weiss, Clark, & Reynolds, 1965; Laties, Weiss, val; and the duration of a given activity tends to be
8c Weiss, 1969).3 Since the terminal response is usually shorter the later it begins.
preceded by interim activities such as drinking, and Davey, Harzem, and Lowe (personal communica¬
elimination of these activities if anything facilitates tion) report that “running” rate on fixed-interval
the terminal response, they cannot be links in a chain schedules (i.e., rate of lever pressing following the first
that ends with that response. press in each interval) is directly related to pause
(time to the first lever press): the later the rats begin
3 This discussion considers spaced-responding (DRL) schedules
on the same basis as interval schedules. In the spaced-responding to press, the faster they go. Pigeons showed no effect
case, the terminal response is timed from the previous terminal of pause on overall running rate. In subsequent ex¬
response, whereas in interval schedules it is timed from rein¬
periments Staddon and Frank (1975b) have found
forcement, but performance on both schedules seems to be
similarly affected by (for example) prevention of interim that the rate at which many pigeons accelerate to their
activities. fixed terminal rate depends on pause: the longer the
J. E. R. Staddon 143

pause, the more rapidly they accelerate. The low rate etc. This relation can be looked at either within or
at which they start, and the high rate that they finish across individuals. Within individuals it is plausible:
up with, is more or less constant, but the time they suppression of interim activities disrupts a developed
take to get from one to the other decreases as the time temporal discrimination (Frank 8c Staddon, 1974;
available for responding before food delivery de¬ Schwartz 8c Williams, 1971), and interim activities
creases. such as drinking tend to develop in step with a ter¬
All these observations underline the dependence of minal response such as food anticipation during train¬
induced activities on the time elapsed since the begin¬ ing (e.g., Pouthas 8c Cave, 1972; Staddon 8c Ayres,
ning of the fixed interval. Chaining cannot easily ac¬ 1975). There is less evidence for a correlation across
count for this dependency. individuals. For example. Smith and Clark (1974)
found no correlation between rates of running or in¬
duced licking and efficiency of performance on spaced-
Behavioral Clocks
responding schedules. However, Glazer 8c Singh (1971)
“Behavioral clock” interpretations are less explicit found that temporal discrimination was inversely re¬
than the chaining account. They are based on the fre¬ lated to degree of restraint in three groups of rats that
quent observation that prevention of interim (“col¬ were either unrestrained, partially restrained, or
lateral”) activities disrupts “temporal discrimination” severely restrained. In informal observations we have
(i.e., pausing) on fixed-interval and spaced-responding noticed that pigeons trained in small Skinner boxes
(DRL) schedules. The idea that the animal “uses” the sometimes fail to show the typical fixed-interval “scal¬
interim activities to suppress the terminal response for lop” and respond more or less continuously; animals
a time is little more than a restatement of this obser¬ trained in the usual large boxes rarely show this
vation. One refinement is to attribute to the collateral pattern. These various experiments cannot be rigor¬
activities an intrinsic periodicity, so that they serve the ously compared, because of species differences and be¬
function of a behavioral clock. In this form the cause amount of training obviously interacts with
hypothesis resembles chaining, The difference is that these differences. Nevertheless, taken together there is
the stimuli (“causal factors”) for the terminal re¬ much evidence that temporal discrimination is fav¬
sponse are assumed to be present all the time; the ored by an environment that affords animals oppor¬
response fails to occur early in the interval only be¬ tunities for interim activities.
cause it is suppressed by the “collateral” behaviors that Although there is evidence for some relation be¬
constitute the clock. When they have run their course, tween temporal discrimination and interim activities,
the terminal response occurs. The mechanism in this no particular interim activity i§ necessary for appro¬
case is a type of disinhibition, whereas in chaining the priate timing. While many authors report vigorous
terminal response is directly produced (elicited, con¬ “collateral” behaviors on temporal schedules (e.g.,
trolled) by the penultimate behavior in the chain. Hendry & Dillow, 1956; Labes et ah, 1965; Znriff,
However, the arguments against chaining apply also 1969), others report none (e.g,. Anger, 1956; Kclleher,
to this form of behavioral clock. In particular, the Fry, & Cook, 1959; Reynolds & Catania, 1962). And
negative correlation between the offset of the last in¬ although pigeons trained under unrestrained condi¬

terim activity and the onset of the terminal response tions show the expected disruption when shifted to
shows that the terminal response is directly affected conditions of bodily restraint, the disruption is tran¬
by post-food time. Although evidence from prevention sient and after protracted training there is little steady-
experiments shows that interim activities do exert state difference (Frank & Staddon, 1974). Presumably
some suppressive effect on the terminal response (since the transient disruption occurs because interim activ¬
their elimination causes the terminal response to ities possible under free conditions are prevented
occur earlier), the correlation data show that this dis- when the pigeon is restrained. Evidently other interim
inhibiting effect is not the sole determiner of the activities soon develop, however, since behavior re¬
temporal locus of the terminal response; some kind of covers to almost the same level as before the shift.
“internal clock” is also involved. Frank and Staddon also found a disruption when
Although inhibition due to interim activities is not birds trained under restrained conditions were shifted
the only factor affecting the timing of the terminal to free conditions. This disruption is harder to ex¬
response, it may be a factor. A further refinement of plain in terms of prevention of previously available
the behavioral clock view is to suppose that some interim activities. However, it can be understood in
measure of temporal discrimination is a function of terms of a wider scheme for classifying sequential
some property of the interim activities—vigor, rate, interactions, to which I now turn.
144 SCHEDULE-INDUCED BEHAVIOR

Types of Sequential Interaction that inhibition is due to the activation of an incom¬

patible activity follows directly from the hypothesis
It has not yet proved possible to explain in detail
that interim and terminal states are incompatible
the mechanisms underlying induced, or indeed any
and has gained some currency in studies of condi¬
other, behavior sequences. Short of such a complete
tioned inhibition in Pavlovian situations (e.g., Anok¬
explanation, terms such as inhibition9 disinhibition,
hin, 1974; Konorsky, 1967). The “strength” of a state
elicitation, and causal factors have become current in
(activity) is equivalent to the strength of its causal
the animal behavior literature as a way of classifying
factors. Examples of causal factors are discriminative
types of sequential interaction. These terms can be
and eliciting stimuli, time, and antecedent activities
employed in several ways. The present section devel¬
(as in chaining).
ops an approach that is consistent with the facts
already discussed and suggests questions that can be
Competition Assumption
answered empirically. This approach is closely related
to the more formal state-space approach recently elab¬ For simplicity, I assume that the animal can be in
orated by McFarland (McFarland, 1974; McFarland 8c only one state at a time. Thus, states compete for ac¬
Sibly, 1975; Sibly & McFarland, 1974} and to the cess to what might be termed the behavioral final
theoretical system o£ Atkinson 8c Birch (1970}. common path.4 In the following discussion it is as¬
sumed that this competition is all at one level, every
Definitions state (activity) competing directly with all the others
that are possible in the situation. However, the
Behavioral State. Evidence already discussed shows
scheme can easily be generalized to allow for hierar¬
that the overt activity that is actually observed (per¬
chical or other multilevel interactions, with a given
formance) is only one aspect of an underlying be¬
state competing directly only with states at its own
havioral state. Terms that convey aspects o£ the term
level.
state, in the sense used here, are “mood,” “expectation,”
This scheme suggests a taxonomy of simple be¬
“motivational state,” and even “operant/7 in the sense
havioral interactions. I first develop such a taxonomy,
that an operant is a class of behaviors with common
and then apply it to some of the data and concepts
controlling factors. Bindra’s (1969) “central motive
discussed earlier in the chapter.
state” is also close to the present meaning. States are
mutually incompatible; they are the basic interacting
Simple Sequential Interactions
elements in this scheme.
This view allows for two kinds of simple inter¬
Activity. This is an observed class of motor pat¬
action between successive behaviors. It is assumed
terns; it has both stimulus and response components.
that the shift from one behavior to another is owing
These motor patterns (e.g., pecking, drinking, pacing
to a change in only one causal factor, which either in¬
in a particular place, etc.) are necessarily defined sub¬
creases (inhibition, elicitation) or decreases (disinhibi¬
jectively, but little practical difficulty is usually en¬
tion, subduction) with time. These two types of inter¬
countered in settling on reliable categories. A state
action define four terms, as follows:
exists independently of any particular activity, but the
performance of an activity may act back on the Inhibition. This occurs when activity A ceases to
strength of a state (i.e., on the level of its causal fac¬ occur because of an increase in the causal factors
tors), either increasing it (“momentum” effects; posi¬
tive feedback) or decreasing it (self-inhibition: nega¬ 4 Quantitative variations in response rate can be handled
within this scheme by assuming that repetitive activities occur at
tive feedback). Providing the environment is constant, a more or less fixed, maximum rate, so long as the animal is in
activities are assumed to be generally reliable (one-to- the appropriate state, and that variations in rate occur because
of switching between states. There is evidence for this kind of
one) indicators of their associated states, and the
fixity in the case of drinking: licks occur at a more or less fixed
terms activity and state are treated as equivalent in rate within each bout (e.g., Marowitz & Halpern, 1973). Varia¬
the following discussion. tions in overall lick rate are therefore associated with a propor¬
tional increase in the percentage of time spent in the licking
Causal Factors. These are environmental factors state. There is some evidence that even “operant” behaviors
such as pecking are similarly constrained, although perhaps not
affecting the strength of states. Causal factors are as¬
to the same degree (Blough, 1963; Gilbert, 1958). The notion is
sumed always to be facilitatory, so that suppression of a hard to test unless typical bout lengths are considerably longer
given activity by a stimulus is assumed to be due to a than the modal interbehavior interval. The problem of defining
the length of an activity bout, discussed by ethologists (e.g., Isaac
decrease in its causal factors and/or an increase in the
& Marler, 1963; Nelson, 1973), confronts essentially the same
causal factors for an incompatible activity. The idea issue.
J. E. R. St addon 145

Performance (and this differential persists for a sufficient time):

dA dB dA d
A I B —^ ^ , where—j^- =— (CFA). It is assumed that

the CF functions are continuous.

A.
Experimental Analysis. In the examples illus¬
trated in Figure 6 all that is actually observed is a
switch from one activity to another. The diagrams
showing changes in CF strength with time constitute
hypotheses about the underlying causation and must
be tested by manipulating the putative causal factors.
In each example, only one CF changes with time; if it
is under direct experimental control, verification of
the causal hypothesis is a trivial matter. However, in
B.
the cases discussed in this chapter, the changing CF
is usually an inferred “internal dock” that is only
under indirect experimental control. For example,
Time suppose (for the sake of illustration) that behavior A
is an interim activity such as drinking, and B a ter¬
Fig. 6. Simple dyadic interactions.
minal response such as lever pressing, with the switch
from A to B timed from food delivery. Then the
(CFs) for some other activity, B (the next-in-priority changing CF (GF& in Figure 6A, CFA in Figure 6B)
activity). When CFB > CFA, A is displaced by B. In¬ must also be assumed to be timed from food delivery,
hibition is illustrated in Figure 6A. The top line so that the absolute time of the origin of the curves

shows the observed sequence of behaviors: A followed in Figure 6 can be controlled* but not their form.
by B. The curves below show the changes in stimulus How might the two cases in Figure 6 he distin¬
factors (CFs) hypothesized to underlie this change. guished experimentally? The answer to this question
They show that the level of CFs for A remains con¬ depends critically on quantitative issue?; the ^b^olut^
stant, but that A is supplanted by B when the CFs for values of the strengths of the CFs for the two activ¬
B increase beyond the level of those for A. ities. With the values shown in Figure 6 it is apparent
that complete elimination of activity A (e.g.? by re¬
Disinhibition. This is simply the reverse of inhi¬
moving a constant CF not shown in the diagrams: the
bition: activity B occurs (and activity A ceases to occur)
water bottle) will cause activity B to occur earlier in
because the CFs for antagonistic activity A decrease the interval under case B than under case A. Similarly,
below the level of those for B. This is illustrated in
removal of a constant CF for activity B (the lever)
Figure 6B.
should cause activity A to occur throughout the in¬
Elicitation. The causation of behavior B in Figure terval, under case A, but prolong A only slightly
6A illustrates elicitation: B occurs because its CFs in¬ under case B. In this simple case, therefore, it is rela¬
crease, all other CFs remaining constant. tively easy to distinguish between these two hypoth¬
eses. However, if the CFs for both activities change
Subduction. This is a neologism to describe the
with time, or if neither CF ever decreases to zero, dis¬
opposite of elicitation: a behavior ceases because its crimination is much more difficult. One possible tech¬
CFs decrease in strength, all other CFs remaining con¬
nique in that case is to introduce a third competing
stant. It is illustrated by the offset of behavior A in activity, such as running, whose CFs can be assumed
Figure 6B.
to be constant during the test period. By manipulat¬
It is clear that the two kinds of interaction illus¬ ing its strength, an estimate of the relative strengths of
trated in Figure 6 are simply extreme cases on a con¬ A and B, as a function of time, might be arrived at.
tinuum of dyadic interactions between successive
behaviors. For two such behaviors, A and B, the CFs Internal Feedback. The diagrams in Figure 6 im¬
for A can either decrease, increase, or remain constant ply that the CFs for a given activity do not depend on
with time, and similarly for B. If A is occurring whether or not that activity is actually occurring. Yet
initially, a shift to B will occur only if the maintained this kind of independence is unlikely to be general.
rate of change in CFB is greater than that for CFA Once begun, an activity may have a certain momen-
 146 SCHEDULE-INDUCED BEHAVIOR

turn and resist competition from other activities that those due to the onset of extrinsic stimuli), variation
might have been sufficient to prevent its initial occur¬ in relative levels will cause variation in the time of
rence. Conversely, even in the absence of competition, switching from one behavior to another and, if the
most activities cease to occur after a time, presumably added random variation is large, may even cause re¬
because of some kind of self-inhibition (Hull’s, 1943, versals of order.
“reactive inhibition”; the “consummatory force” of
Atkinson Sc Birch, 1970). The circumstantial evidence Applications

for these kinds of internal feedback interaction is Variability, internal feedback, and the possibility of
strong. Unfortunately, they are hard to measure competition at several levels, can obviously combine
directly just because the loops are internal ones. to produce sequences of behavior that defy analysis by
For example, self-inhibition cannot immediately be means of simple experimental tests. Bearing these
distinguished from the hypothesis of an internal complex possibilities in mind, it may nevertheless be
clock. When first exposed to a running wheel, a rat useful to see to what extent relatively simple interac¬
may tend to run for a more or less fixed time; is this tions, such as those illustrated in Figure 6, can ex¬
because of a fixed internal clock that times the run plain the experimental results discussed earlier. Three
bout, or because of negative feedback from the re¬ cases will be considered: interactions between running
sponse? The second possibility can be evaluated by and drinking, the effect of interfood interval, and an
surgical intervention (e.g., deafferentation), or by vary¬ anomalous experimental result owing to the persis¬
ing the resistance of the running wheel. If these tence of running.
operations change the duration of running, some role
Running I Drinking Interactions. Figure 7A shows
for feedback is demonstrated. In any particular case,
the postulated time course of CFs for eating (E),
tests of this sort can usually be devised. However, the drinking (D), running (R), and food anticipation
possibility of self-feedback greatly complicates the
(FA), underlying the maintained temporal distribu¬
task of experimental analysis.
tion of these activities shown in Figure IB, which
This general problem is a familiar one to students
shows data from a rat on a fixed-time 30 sec schedule.
of the major homeostatic systems such as food and
The vertical lines marked “P” in Figure 7 show the
water regulation (see Satinoff & Henderson, Chapter 6).
times of pellet delivery. The CFs for running are as¬
However, much more is known about the internal
sumed to be essentially constant, so that running con¬
feedbacks that affect the duration of an eating bout, stitutes a disinhibited activity which occurs only when
for example, than about the comparable factors affect¬
the strengths of drinking and food anticipation are
ing running, preening or lever pressing. The fact that low. One prediction from this hypothesis is that the
the mechanisms underlying such responses are unlikely
elimination of drinking (e.g., by removing the water
to be fixed, but may depend on the situation in which bottle) should cause running to occur earlier in the in¬
the response occurs (e.g., whether the response is terval, but should have little effect on the time of onset
schedule-induced, or occurs under free conditions), is
of food anticipation. Elimination of running (by re¬
a further complication. moving the running wheel) should cause food anticipa¬
tion to begin earlier and drinking to persist later in the
Variability. Although schedule-induced behavior
interval. Elimination of drinking and running should
is characteristically highly stereotyped, there is never¬
theless variation in its form and, particularly, its
temporal pattern from one interfood interval to the > Performance p Performance
next. Temporal variability (variation in the temporal
E| D | R | FA 1 |E| D [ FA |
location of an activity as a function of post-food time)
is always found, but variability of succession (variation
in the order of activities) is less common. Thus, the
function relating the strength of causal factors to
post-food time must be assumed to vary from interval
to interval. The simplest form of variation is a
stochastic process with zero mean superimposed on the
A. B.
average CF functions (e.g., the curves in Figure 6).
Since it is assumed that the CF functions change at a Fig. 7. Hypothesis for interactions among behaviors induced
finite rate (i.e., there are no step-functions other than by periodic food. A: Low to intermediate food rate. B: High
food rate.
J. E. R. Staddon 147

cause food anticipation to begin sooner than elimina¬

tion of running alone. These predictions assume that
the pattern of CFs illustrated in Figure 7 remains
more or less constant during the test period, i.e., they
are predictions about transfer effects.
In general these predictions are borne out. Elimina¬
tion of running does appear to cause drinking to per¬
sist longer and food anticipation to begin earlier in
the interval, and elimination of drinking does seem to
have more effect on running than on food anticipa¬
tion (Segal, 1969; Staddon 8c Ayres, 1975). However,
the Staddon and Ayres data come from steady-state
adjustments to the manipulations rather than transfer
(first-day) measures, the Segal data do not show
temporal location of activities, and exact data do not
seem to be available elsewhere. Hence these predic¬
tions cannot yet be precisely evaluated.

Effects of Interfood Interval. Figure 7B shows the

postulated interactions on a short (5-15 sec) fixed-time
food schedule. The CFs for running are the same as in
Figure 7A, but the CFs for food anticipation rise to
their asymptote more quickly, because of the shorter
interval. Consequently, the crossover of the curves for
drinking and food anticipation occurs at a level
higher than the level of the CFs for running, which
cannot therefore occur, apart from the effects of ran¬
dom variation. Thug the decrease in the frequency of
running with interfood interval shown in Figure 3 15 Post-eating time (sec)
explained by assuming that the CFs for drinking and
Fig. §. An unusual pattern of behavior induced in a female
food anticipation reach the same asymptote in short rat by a 30-sec periodic food schedule,

intervals as in long ones, and therefore cross over at a

level that is inversely related to interval length; the
lion of time spent in various activities (activity areas)
shorter the interval, the higher the crossover point.
by a female rat in a hexagonal apparatus m which it
Frank and Staddon (1974) found that pigeons
trained on a periodic schedule under conditions of received a food pellet every 30 sec (Staddon k Ayres,
1975). The behavior of this animal was anomalous, as
bodily restraint showed disrupted temporal discrim¬
can be seen by comparing Figure 8 with Figure IB.
ination (i.e., the terminal response, key pecking, oc¬
curred earlier in the interval) when shifted to unre¬ The rat in Figure 8 showed running as an apparent
terminal response, since the activity increased in fre¬
strained conditions. Figure 7B sheds some light on
this result if it is assumed that the shift had more quency up until the time of the next food delivery. A
naive reinforcement theory interpretation might con¬
effect on the CFs (controlling stimuli) for whatever
interim activity was occurring under restrained condi¬ clude that running in this case was adventitiously
reinforced by food delivery—with which it was almost
tions (analogous to drinking in Figure 7B) than on
the terminal response. This assumption seems reason¬ invariably contiguous. However, this interpretation is
able since the main CF for pecking was the response contradicted by the results of extinction tests, in
which the first effect of food omission was an increase
key, which was not affected by the shift. Figure 7B
makes it clear that any reduction in the CFs for the in food anticipation, followed later by an increase in
interim activity, relative to those for the terminal re¬ running above the level observed under the food
schedule.
sponse, will cause the terminal response to occur
earlier in the interval, as Frank and Staddon found. Figure 9 shows what may be a more accurate repre¬
sentation. The curves are identified as in Figures 6
Persistence of Running. Figure 8 shows the frac- and 7. However, the abscissa is post-eating (rather
 148 SCHEDULE-INDUCED BEHAVIOR

s Performance temporal sequences, or to interpret the results of

e1 D 1 R_Ie experimental manipulations.

CONCLUDING COMMENTS

The experimental literature on schedule-induced

behavior, though extensive, is too unsystematic to
point clearly to any particular theoretical integration.
This chapter is an attempt to provide an organizing
Fig. 9. Hypothesis for interactions underlying behavior shown
framework to guide both experimentation and inter¬
in Figure 8.
pretation.
The traditional emphasis on a single, instrumental
response is misplaced. The work reviewed here shows
than post-pellet-delivery) time, since this animal did that the temporal pattern of any activity depends on
not always eat the pellet as soon as it was delivered. its interactions with other activities that are induced
The main difference between Figure 9 and Figure 7 A by the situation. These induced behaviors must be
is in the postulated GF function for running, which considered on the same footing as the instrumental
shows positive acceleration once the response is ex¬ response. They are often just as vigorous (sometimes
pressed (a positive feedback, Fb+, ‘'momentum” even more vigorous), are as reliably produced, and
effect), and declines with the continued occurrence of share some of the same causal factors. The “laws”
running (a negative feedback, Fb-, “self-inhibition” of operant behavior are not a property of isolated re¬
effect). A consequence of the momentum effect is that flexes, but emergent properties of a set of interactions
when the food pellet is delivered, the CFs for running among induced states and their associated behaviors.
are above those for eating. The pellet is not eaten Each behavior has its own controlling (causal) factors,
until the CFs for running decrease (self-inhibition) be¬ both stimuli and time. Any environmental change
low those for food anticipation, the animal enters the affects the instrumental response both directly, and
feeder area, sees the food, and eats. indirectly through its effects on other causally related
The model predicts the initial perseveration of activities. The effects of such changes cannot be fully
food anticipation in extinction as well as the eventual understood until these interactions have been teased
dominance of running. It also predicts that the out.
temporal pattern of food anticipation will approach
that shown by the other rats (e.g., the animal in
Figure IB) if drinking is eliminated (allowing running
to cease earlier in the interval), and should be essen¬ REFERENCES

tially the same as the pattern shown by the others if

running is eliminated. The data support both these Akerman, B., Andersson, E., Fabricius, E., 8c Syensson, L.
Observations on central regulation of body temperature
predictions. Other features of the model for this
and of food and water intake in the pigeon (Columba
animal have not been tested in detail (fQr example, livia). Acta Physiologica Scandanavica, 1960, 50, 328-336.
the proper tests to distinguish response-produced Allen, J. D., Porter, J. H., 8c Arazie, R. Schedule-induced
feedbacks from response-produced clocks were not drinking as a function of percentage reinforcement.
carried out). However, the basic hypothesis that the Journal of the Experimental Analysis of Behavior, 1975,
25, 223-232.
anomalous behavior of this individual is largely, per¬
Anger, D. The dependence of interresponse times upon the
haps entirely, attributable to its tendency to run seems relative reinforcement of different interresponse times.
quite well supported. Journal of Experimental Psychology, 1956, 52, 145-161.
Once the mechanisms underlying behavioral se¬ Anokhin, P. K. Biology and neurophysiology of the condi¬
tioned reflex and its role in adaptive behavior. S. S. Cor¬
quences are fully understood, the representations in
son. Trans. New York: Pergamon, 1974.
Figures 6, 7, and 9 will undoubtedly appear cumber¬
Appel, J. B. Aversive aspects of a schedule of positive rein¬
some and redundant. However, these diagrams force forcement. Journal of the Experimental Analysis of
one to be precise about often loosely applied terms Behavior, 1963, 6, 423-428.
such as inhibition. They also suggest clear experi¬ Atkinson, J. W., 8c Birch, D. The dynamics of action. New
York: Wiley, 1970.
mental tests. Without such a framework it is often
Azrin, N. H. Time-out from positive reinforcement.
difficult to ask good experimental questions about Science, 1961, 133, 382-383.
J. E. R. Staddon 149

Azrin, N. H., Hutchinson, R. R., 8c Hake, D. F. Extinc¬ nal of the Experimental Analysis of Behavior, 1962, 5,
tion-induced aggression. Journal of the Experimental 61-63.
Analysis of Behavior, 1966, 9, 191-204. Cohen, P. S., 8c Looney, T. A. Schedule-induced mirror
Bilbrey, J., 8c Winokur, S. Controls for and constraints on responding in the pigeon. Journal of the Experimental
auto-shaping. Journal of the Experimental Analysis of Analysis of Behavior, 1973, 19, 395-408.
Behavior, 1973, 20, 323-332. Cole, J. L. Force gradients in stimulus generalization.
Bindra, D. A unified interpretation of emotion and motiva¬ Journal of the Experimental Analysis of Behavior, 1965,
tion. Annals of the New York Academy of Sciences, 1969, 8, 231-241.
159, 1071-1083. Davis, J. M. Socially-induced flight reactions in pigeons.
Bindra, D. A unified account of classical conditioning and Unpublished doctoral dissertation, Duke University,
operant training. In A. H. Black and W. F. Prokasy 1973.
(Eds.), Classical conditioning II: Current research and Falk, J. L. Production of polydipsia in normal rats by an
theory. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1972. intermittent food schedule. Science, 1961, 133, 195-196.
Blough, D. S. Interresponse time as a function of continu¬ Falk, J. L. Control of schedule-induced polydipsia: Type,
ous variables: A new method and some data. Journal of size, and spacing of meals. Journal of the Experimental
the Experimental Analysis of Behavior, 1963, 6, 237-246. Analysis of Behavior, 1967, 10, 199-206.
Bolles, R. C. Reinforcement, expectancy, and learning. Falk, J. L. Conditions producing psychogenic polydipsia in
Psychological Review, 1972, 79, 394-409. animals. Annals of the New York Academy of Sciences,
Bowen, C. The how, when and where of polydipsia. Un¬ 1969, 157, 569-593.
published M. A. thesis, Duke University, 1972. Falk, J. L. The nature and determinants of adjunctive
Brown, P. L., 8c Jenkins, H. M. Auto-shaping of the pi¬ behavior. Physiology and Behavior, 1971, 6, 577-588.
geon’s keypeck. Journal of the Experimental Analysis of Ferster, C. B., 8c Skinner, B. F. Schedules of reinforcement.
Behavior, 1968, 11, 1-8. New York: Appleton-Century-Crofts, 1957.
Brown, T. G., 8c Flory, R. K. Schedule-induced escape Flory, R. K. Attack behavior as a function of minimum
from fixed-interval reinforcement. Journal of the Ex- inter-food interval. Journal of the Experimental Analysis
perimental Analysis of Behavior, 1972, 17, 395-403. of Behavior, 1969, 12, 825-828.
Browne, M. Latent learning revisited: The role of reward Flory, R, K. The control of schedule-induced polydipsia:
and overt behavior changes in the acquisition of stimu¬ Frequency and magnitude of reinforcement. Learning
lus relations. Unpublished doctoral dissertation, Indiana and Motivation, 1971, 2, 215-227.
University, 1973. Flory, R. K., 8c Ellis, B. B. Schedule-induced aggression
Burks, C. D. Schedule-induced polydipsia: Are response- against a slide-image target. Bulletin of the Psychonomic
dependent schedules a limiting condition? Journal of Society, 1973, 2, 287-290.
the Experimental Analysis of Behavior, 1970, 19, 351- Flory, R. K., 82 O’Boyle, M. K. The effect of limited water
358. availability on schedule-induced polydipsia. Physiology
Cane, V. Behaviour sequences as semi-Markov chains. and Behavior, 1972, 8, 147-149.
Journal of the Royal Statistical Society, B, 1959, 21, Frank, J., & Staddon, J. E. R. Effects of restraint on tem¬
36-58. poral discrimination behavior. Psychological Record,
Cane, V. Some ways of describing behaviour. In W. H. 1974, 21, 123-130.
Thorpe and O. L. Zangwill (Eds.), Current problems Gamzu, E., & Schwartz, B. The maintenance of key peck¬
in animal behaviour London: Cambridge University ing by stimulus-contingent and response-independent
Press, 1961. food presentation. Journal of the Experimental Analysis
Carlisle, H. J. Fixed-ratio polydipsia: Thermal effects of of Behavior, 1973, 19, 65-72.
drinking, pausing, and responding. Journal of Compara¬ Gamzu, E., 8c Williams, D. R. Classical conditioning of a
tive and Physiological Psychology, 1971, 75, 10-22. complex skeletal response. Science, 1971, 171, 923-925.
Gamzu, E. R., 8c Williams, D. R. Associative factors under¬
Carlisle, H. J. Schedule-induced polydipsia: Blockade by
lying the pigeon’s key pecking in auto-shaping proce¬
intrahypothalamic atropine. Physiology and Behavior,
dures. Journal of the Experimental Analysis of Behavior,
1973, 11, 139-143.
1973, 19, 225-232.
Carlisle, H. J., Shanab, M. E., 8c Simpson, C. W. Schedule-
Gentry, W. D., 8c Schaeffer, R. W. The effect of FR
induced behaviors: Effect of intermittent water rein¬
response requirement on aggressive behavior in rats.
forcement on food intake and body temperature. Psy-
Psychonomic Science, 1969, 14, 236-238.
chonomic Science, 1972, 26, 35-36.
Gibbon, J., Berryman, R., 8c Thompson, R. L. Contingency
Catania, A. C., 8c Reynolds, G. S. A quantitative analysis
spaces and measures in classical and instrumental condi¬
of the responding maintained by interval schedules of
tioning. Journal of the Experimental Analysis of Be¬
reinforcement. Journal of the Experimental Analysis of havior, 1974, 21, 585-605.
Behavior, 1968, 11, 327-383.
Gilbert, R. M. Ubiquity of schedule-induced polydipsia.
Cherek, D. R., Thompson, T., 8c Heistad, G. T. Respond¬ Journal of the Experimental Analysis of Behavior, 1974,
ing maintained by the opportunity to attack during an 21, 277-284.
interval food reinforcement schedule. Journal of the Gilbert, T. F. Fundamental dimensional properties of the
Experimental Analysis of Behavior, 1973, 19, 113-123. operant. Psychological Review, 1958, 65, 272-282.
Chung, Shin-Ho. Effects of effort on response rate. Journal Glazer, H., 8c Singh, D. Role of collateral behavior in
of the Experimental Analysis of Behavior, 1965, 8, 1-7. temporal discrimination performance and learning in
Clark, F. C. Some observations on the adventitious rein¬ rats. Journal of Experimental Psychology, 1971, 91, 78-
forcement of drinking under food reinforcement. Jour¬ 84.
150 SCHEDULE-INDUCED BEHAVIOR

Groves, L. C., 8c Brownstein, A. J. Effects of trial and cycle of temporally spaced responses. Journal of the Experi¬
duration on automaintenance. Paper presented at South mental Analysis of Behavior, 1959, 2, 91-106.
Eastern Psychological Association Meeting, New Orleans, Killeen, P. On the temporal control of behavior. Psy¬

1973. chological Review, 1975, 82, 89-115.

Harzem, P. Temporal discrimination. In R. M. Gilbert 8c Kissileff, H. R. Food-associated drinking in the rat. Jour¬

N. S. Sutherland (Eds.), Animal discrimination learning. nal of Comparative and Physiological Psychology, 1969,
New York: Academic Press, 1969. 67, 284-300.
Hawkins, T. D., Schrot, S. H., Giteiens, S. H., 8c Everett, Knutson, J. F., 8c Kleinknecht, R. A. Attack during differ¬

P. B. Schedule-induced polydipsia: An analysis of water ential reinforcement of low rate of responding. Psy¬
and alcohol ingestion. In R. M. Gilbert 8c J. D. Keehn chonomic Science, 1970, 19, 289-290.
(Eds.), Schedule effects: Drugs, drinking and aggression. Konorski, J. Integrative activity of the brain. Chicago: Uni¬

Toronto: University of Toronto Press, 1972. versity of Chicago Press, 1967.

Hearst, E., 8c Jenkins, H. M. Sign-tracking: The stimulus- Kramer, T. J., 8c Rilling, M. Differential reinforcement of
reinforcer relation and directed action. Psychonomic low rates: A selective critique. Psychological Bulletin,
Society Monograph, 1974. 1970, 74, 225-254.
Hendry, D. P., & Dillow, P. V. Observing behavior during Laties, V. G., Weiss, B., Clark, R. L., 8c Reynolds, M. D.
interval schedules. Journal of the Experimental Analysis Overt “mediating” behavior during temporally spaced
of Behavior, 1966, 9, 837-349. responding. Journal of the Experimental Analysis of
Hinde, R, A, Animal behaviour; A synthesis of ethology Behavior/1965, 8, 107-116.
and comparative psychology, Second edition. New York: Laties, V. G., Weiss, B., 8c Weiss, A. B. Further observa¬

McGraw-Hill, 1970. tions on overt “mediating” behavior and the discrimina¬

Hinde, R., 8c Stevenson-Hinde, J. (Eds.) Constraints on tion of time. Journal of the Experimental Analysis of
learning. London: Academic Press, 1973. Behavior, 1969, 12, 43-57.
Hodos, W., Ross, G. S., 8c Brady, J. Complex response Levitsky, D., 8c Collier, G. Schedule-induced wheel run¬

patterns during temporally spaced responding. Journal ning. Physiology and Behavior, 1968, 3, 571-573.
of the Experimental Analysis of Behavior, 1962, 5, 473- Lorenz, K. Z. Innate bases of learning. In K. H. Pribram

479. (Ed.), On the biology of learning. New York: Harcourt,

Hutchinson, R. R., Azrin, N. H., 8c Hunt, G. M. Attack Brace and World, 1969.
produced by intermittent reinforcement of a concurrent Lotter, E. C., Woods, S. C., 8c Vasselli, J. R. Schedule-
operant response. Journal of the Experimental Analysis induced polydipsia: An artifact. Journal of Comparative
of Behavior, 1968, 11; 489-496. and Physiological Psychology, 1973, 83, 478-484.
Hull, C. L. Principles of Behavior. New York: Appleton- Marowitz, L. A., 8c Halpern, B. P. The effects of environ¬

Century, 1943. mental constraints on licking patterns. Physiology and

Isaac, D., 8c Marler, P. Ordering of sequences of singing Behavior, 1973, 11, 259-267.
behaviour in Mistle Thrushes in relationship to timing. McFarland, D. J. Hunger, thirst and displacement pecking

Animal Behaviour, 1963, 11, 179—188. in the barbary dove. Animal Behaviour, 1965, 13, 293-
Innis, N, K., 8c Staddon, J. E. R. Temporal tracking on 300.
cyclic-interval reinforcement schedules. Journal of the McFarland, D. J. Adjunctive behaviour in feeding and

Experimental Analysis of Behavior, 1971? 163 411—423. drinking situations. Revue du Comportement Animal,
Jac^uet, Y. F. Schedule-induced licking during multiple 1970, 4, 64-73.
schedules. Journal of the Experimental Analysis of Be¬ McFarland, I). J. Time-sharing as a behavioural pheno¬

havior, 1972, 19, 413-423. menon. In D. Lehrman, R. Hinde, 8c E. Shaw (Eds.),

Jenkins, H. M. Sequential organization in schedules of Advances in the study of behavior. Vol. 5., New York;
reinforcement. In W. N. Schoenfeld (Ed.), The theory of Academic Press, 1974.
McFarland, D. J., 8c Sibly, R. The behavioural final com¬
reinforcement schedules. Englewood Cliffs, N.J.: Pren¬
mon path. Philosophical Transactions of the Royal
tice-Hall, Inc., 1970.
Society of London, Series B, 1975, 270, 265-293.
Jenkins, H. M., & Moore, B. R. The form of the auto¬
McGill, W. J. Stochastic latency mechanisms. In R. D.
shaped response with food or water reinforcers. Journal
Luce, R. R. Bush, 8c E. Galanter (Eds.), Handbook of
of the Experimental Analysis of Behavior, 1973, 20, 163-
mathematical psychology. Vol. I. New York; Wiley, 1963.
181. Mendelson, J., Zielke, S.7 Werner, J. S., 8c Freed, L. M.
Jensen, C., 8c Fallon, D. Behavioral aftereffects of rein¬
Effects of airstream accessibility on airlicking in the rat.
forcement and its omission as a function of reinforce¬ Physiology and Behavior, 1973, 11, 125-130.
ment magnitude. Journal of the Experimental Analysis
Millenson, J. R. The facts of schedule-induced polydipsia.
of Behavior, 1973, 19, 459-468. Behavior Research Methods and Instrumentation, 1975,
Kachanoff, R., Leveille, R., McLelland, J. P., 8c Wagner, 7, 257-259.
M. J. Schedule-induced behavior in humans. Physiology Miller, J. S., 8c Gollub, L. R. Adjunctive and operant bolt
and Behavior, 1973, 11, 395-398. pecking in the pigeon. Psychological Record, 1974, 24,
Keehn, J. D. Schedule-dependence, schedule-induction, 203-208.
and the Law of Effect. In R. M. Gilbert 8c J. D. Keehn Moore, B. The role of directed Pavlovian reactions in
(Eds.), Schedule effects: Drugs, drinking, and aggression. simple instrumental learning in the pigeon. In R. Hinde
Toronto: University of Toronto Press, 1972, Pp. 65-94. 8c J. Stevenson-Hinde (Eds.), Constraints on learning.
Kelleher, R. T., Fry, W., 8c Cook, L. Interresponse time London: Academic Press, 1973.
distribution as a function of differential reinforcement Moran, G. The effects of lick-contingent delays of rein-
J. E. R. Staddon 151

forcement on the development of schedule-induced poly¬ eral pecking. Journal of the Experimental Analysis of
dipsia. M.A. Thesis, Dalhousie University, 1974. Behavior, 1971, 16, 155-160.
Nelson, K. After-effects of courtship in the male three Schwartz, B., 8c Williams, D. R. The role of the response-
spined stickleback. Zietschrift fur vergleichende Physi¬ reinforcer contingency in negative automaintenance.
ologic, 1965, 50, 569-597. Journal of the Experimental Analysis of Behavior, 1972,
Nelson, K. Does the holistic study of behavior have a 17, 351-357.
future? In P. P. G. Bateson 8c P. H. Klopfer (Eds.), Segal, E. F. The development of water drinking on a dry-
Perspectives in ethology. New York: Plenum Press, 1973. food free-reinforcement schedule. Psychonomic Science,
Nevin, J. A., 8c Berryman, R. A note on chaining and 1965, 2, 29-30.
temporal discrimination. Journal of the Experimental Segal, E. F. The interaction of psychogenic polydipsia
Analysis of Behavior, 1963, 6, 109-113. with wheel running in rats. Psychonomic Science, 1969,
Pavlov, I. P. Conditioned reflexes. Trans. G. V. Anrep. 11, 141-144.
London: Oxford University Press, 1927. Segal, E., 8c Holloway, S. M. Timing behavior in rats
Pliskoff, S., 8c Tolliver, G. Water-deprivation-produced with water drinking as a mediator. Science, 1963, 140,
sign reversal of a conditioned reinforcer based upon 888-889.
dry food. Journal of the Experimental Analysis of Be¬ Segal, E. F., Oden, D. L., 8c Deadwyler, S. A. Determinants
havior, 1960, 3, 323-329. of polydipsia: IV. Free-reinforcement schedules. Psy¬
Porter, J. H., 8c Kenshalo, D. R. Schedule-induced drink¬ chonomic Science, 1965, 3, 11-12.
ing following omission of reinforcement in the Rhesus Sevenster, P. Incompatibility of response and reward. In
monkey. Physiology and Behavior, 1974, 12, 1075-1077. R. Hinde 8c J. Stevenson-Hinde (Eds.), Constraints on
Porter, J. H., Arazie, R., Holbrook, J. W., Cheek, M. S., learning. London: Academic Press, 1973.
8c Allen, J. D. Effects of variable and fixed second-order Slianab, M. E., 8c Peterson, J. L. Polydipsia in the pigeon.
schedules on schedule-induced polydipsia in the rat. Psychonomic Science, 1969, 15, 51-52.
Physiology and Behavior, 1975, 14, 143-149. Shettleworth, S. J. Food reinforcement and the organiza¬
Pouthas, V., 8c Cave, C. Evolution de deux conduites col- tion of behavior in Golden Hamsters. In R. Hinde & J.
laterales au cours d’un conditionnement au temps chez le Stevenson-Hinde (Eds.), Constraints on learning. Lon¬
rat. L}Annee Psychologique, 1972? 72, 17—24, don; Academic Press, 1973.
Rachlin, H. Contrast and matching. Psychological Review, Shull, R. L. The response-reinforcement dependency in
1973, 80, 217-234. fixed-interval schedules of reinforcement. Journal of the
Rachlin, H., 8c Baum, W. M. Effects of alternative rein¬ Experimental Analysts of Behavior, 1970, 11, 55—60.
forcement: Does the source matter? Journal of the Ex¬ SlRLY, R., 8c McFarland, D. J. A state-space approach to
perimental Analysis of Behavior, 1972, 18, 231-241. motivation. In D. J. McFarland (Ed.), Motivational
Redford, M. E., 8c Perrins, C. C., Jr. The role of auto-
control systems. London: Academic Press, 1974.
pecking in behavioral contrast. Journal of the Experi¬ Skinner, B. F. “Superstition” m the pigeon. Journal of
mental Analysis of Behavior, 1974, 21, 145-150. Experimental Psychology, 1948, 38, 168-172-
Rescorla, R. A. Pavlovian conditioning and its proper Skinner, B. F. Cumulative record. Englewood Cliffs, N-J-*
control procedures. Psychological Review, 1967, 74, 71- Prentice-Hall, Inc., 1961.
BO. Skinner, B. F.,
8c Morse, C. W. Fixed -interval reinforce¬
Rescorla, r. A., 8c Wagner, A. R. A theory of Pavlovian ment Of Funning in a wheel. / ournal of the Expert-
conditioning; Variations in the effectiveness of rein¬ mental Analysis of Behavior, 1958, 1, 371“379*
forcement and nonreinforcement. In A. Black 8c W. F.
Slangen, J. L., 8c Weijnen, J. A, The reinforcing effect of
Prokasy (Eds,), Classical Conditioning 1L Englewood
electrical stimulation of the tongue in thirsty rats.
Cliffs, N.J.; Prentice-Hall, Inc., 1972.
Physiology and Behavior, 1972, 6, 505-568.
Reynierse, J. H., & Seanier, D, Excessive drinking in rats'
Smith, J, B„ & Clark, F. C. Intercurrent and reinforced
adaptation to the schedule of feeding. Psychonomic
Science, 1968, 10, 95-96, behavior under multiple spaced-responding schedules.
Reynolds, G. S., 8c Catania, A, C, Temporal discrimination
Journal of the Experimental Analysis of Behavior, 1974,
21, 445-454.
in pigeons, Science, 1962, 135) 314-315,
Staddon, J. E. R. Effect of reinforcement duration on fixed-
Ricci, J. A, Key pecking under response-independent food
interval responding. Journal of the Experimental Analy¬
presentation after long simple and compound stimuli.
sis of Behavior, 1970, 13, 9-11.
Journal of the Experimental Analysis of Behavior, 1973,
19, 509-516. Staddon, J. E. R. A note on the analysis of behavioral
Richards, R. W., & Rilling, M. Aversive aspects of a fixed- sequences in Columba Livia. Animal Behaviour, 1972,
interval schedule of food reinforcement. Journal of the 20, 284-292. (a)
Experimental Analysis of Behavior, 1972, 17, 405-411. Staddon, J. E. R. Temporal control and the theory of rein¬

Rosenblith, J. Z. Polydipsia induced in the rat by a second- forcement schedules. In R. M. Gilbert 8c J. R. Millenson
order schedule. Journal of the Experimental Analysis of (Eds.), Reinforcement: Behavioral analyses. New York:
Behavior, 1970, 14, 139-144. Academic Press, 1972. (b)
Schneider, B. A. A two-state analysis of fixed-interval Staddon, J. E. R. Temporal control, attention, and mem¬

responding in the pigeon. Journal of the 'Experimental ory. Psychological Review, 1974, 81, 375-391.
Analysis of Behavior, 1969, 12, 677-687. Staddon, J. E. R. Learning as adaptation. In W. K. Estes
Schwartz, B., 8c Williams, D. R. Discrete-trials spaced (Ed.), Handbook of learning and cognitive processes.
responding in the pigeon: The dependence of efficient Vol. II. New York: Erlbaum Associates, 1976.
performance on the availability of a stimulus for collat¬ Staddon, J. E. R., 8c Ayres, S. L. Sequential and temporal
152 SCHEDULE-INDUCED BEHAVIOR

properties of behavior induced by a schedule of periodic Wayner, M. J., 8c Greenberg, I. Schedule dependence of
food delivery. Behaviour, 1975, 54, 26-49. schedule-induced polydipsia and lever pressing. Physi¬
Staddon, J. E. R., 8c Frank, J. The role of the peck-food ology and Behavior, 1973, 10, 965-966.
contingency on fixed-interval schedules. Journal of the Weiner, H. Controlling human fixed-interval performance.
Experimental Analysis of Behavior, 1975, 23, 17-23. (a) Journal of the Experimental Analysis of Behavior, 1969,
Staddon, J. E. R., 8c Frank, J. A. Temporal control on 12, 349-373.
periodic schedules: Fine structure. Bulletin of the Psy- Williams, D. R., 8c Williams, H. Auto-maintenance in the
chonomic Society, 1975, 6, 536-538. (b) pigeon: Sustained pecking despite contingent non-rein¬
Staddon, J. E. R., 8c Simmelhag, V. L. The “superstition” forcement. Journal of the Experimental Analysis of
experiment: A re-examination of its implications for Behavior, 1969, 12, 511-520.
the principles of adaptive behavior. Psychological Re¬ Wilson, D. M. The central nervous control of flight in a
view, 1971, 78, 3-43. locust. Journal of Experimental Biology, 1961, 38, 471-
Stein, T. S. Excessive drinking in the rat: superstition or 490.
thirst? Journal of Comparative and Physiological Psy¬ Wilson, D. M. Insect walking. Annual Review of Ento¬
chology, 1964, 58j 237-242. mology, 1966, 11, 103-122.
Terrace, H, S., Gibbon, J., Farrell, L., 8c Baldock, M. D, Wilton, R. N., 8c Clements, R, O. Behavioral contrast as

Temporal factors influencing the acquisition and main¬ a function of the duration of an immediately preceding
tenance of an autoshaped keypeck. Animal Learning and period of extinction. Journal of the Experimental Analy¬
Behavior, 1975, 3, 53-62. sis of Behavior, 1971, 16, 425-428.
Thompson, D. M. Escape from SD associated with fixed- Wuttke, W., 8c Innis, N. K. Drug effects upon behaviour

ratio reinforcement. Journal of the Experimental Analy¬ induced by second-order schedules of reinforcement: The
sis of Behavior, 1964, 7, 1-8. relevance of ethological analyses. In R. M. Gilbert 8c
Thompson, T., & Bloom, W. Aggressive behavior and ex¬ J. D. Keehn (Eds.), Schedule effects: Drugs, drinking,
tinction-induced response rate increase. Psychonomic and aggression. Toronto: University o£ Toronto Press,
Science, 1966, 5, 335-336. 1972.
Tinbergen, N. A study of instinct. London: Oxford Uni¬ Zener, K. The significance of behavior accompanying
versity Press, 1951. conditioned salivary secretion for theories of the con¬
Villareal, J. Schedule-induced pica. Paper read at ditioned response. American Journal of Psychology, 1937,
Eastern Psychological Association Meeting, Boston, 1967- 50, 384-403.
Wayner, M. J. Motor control function of the lateral Zuriff, G. E. Collateral responding during differential rein¬
hypothalamus and adjunctive behavior. Physiology and forcement of low rates. Journal of the Experimental
Behavior, 1970, 5, 1319-1325. Analysis of Behavior, 1969, 12, 971-976.
 6

Thermoregulatory Behavior*

Bvelyn Satinoff
and
Robert Hendersen

INTRODUCTION Second, it was assumed that when sufficient informa¬

tion .tinmt the neural control of reflexive thermoreg-
Temperature regulation ig a homeogtatic procegg ulatory regpongeg wag obtained, it would account far
which is mainly behavioral, hut for many year§ be¬ operant behavior ft§ WsIL Ths fir§t purpygg tllig
havior was largely ignored in its analysis. There are chapter is to show that the neural controls of thermo¬
two probable reasons for this omission. First, no con¬ regulatory reflexes and operantg are functionally and
venient method for quantifying thermoregulatory be¬ neuroanatomically separate and that ws can nsvsr
havior was available until 195?. In that year, Carlton fully understand thermal homeostasis without under¬
and Marx in one study and Weisg in another demon¬ standing its operant aspects.
strated that operant behavior was precisely attuned to Operant behavior provides an okgant means of
regulating body temperature. They showed that rats tapping important features of thermoregulation. A
in the cold would press a bar that turned a heat lamp second purpose of this chapter is to demonstrate the
on, thereby preventing a fall in internal temperature.1 utility of behavior in interpreting the effects of vari¬
ous drugs on body temperature, in analyzing thermal
* The preparation of this chapter and some of the un¬
published research described in it was supported by Research preferences, and in studying phylogenetic and onto¬
Grant #NS 12033 from the National Institute of Neurological genetic differences in thermoregulatory functioning.
Diseases and Stroke and Grant #CRR Psychology from the Uni¬
versity of Illinois Research Board to the first author. We thank
Thermoregulation, because it is an exemplary nega¬
Drs. R. D. Luce, H. Rachlin, B. Schwartz, and especially W. tive feedback system, is usually discussed within the
Honig and J. Staddon for their helpful comments on previous framework of control theory. One of the most impor-
versions of this manuscript.

1 The effectiveness of thermal reinforcement has been demon¬

strated in various species, including baboons (Gale, Mathews, & Lipton, 1968; Matthews, 1969; Weiss & Laties, 1961), mice
Young, 1970), macaques (Carlisle, 1970), squirrel monkeys (Adair, (Baldwin, 1968; Revusky, 1966), Barbary doves (Budgell, 1971),
1970; Carlisle, 1966), dogs (Cabanac, Duclaux, & Gillet, 1970), chicks (Zolman, 1968), lizards (Hammel, Caldwell, & Abrams,
cats (Clark & Lipton, 1974; Weiss, Laties, & Weiss, 1967), pigs 1967), alligators (Davidson, 1966), and goldfish (Rozin & Mayer,
(Baldwin & Ingram, 1967), rats (Epstein & Milestone, 1968; 1961).

153
154 THERMOREGULATORY BEHAVIOR

tant concepts in control theory is the set point—that These facts imply that the preoptic area is impor¬
value of the input at which the output is zero. This tant for maintaining body temperature by activating
chapter will show how behavior is an invaluable tool reflexive responses. In most of the experiments lead¬
in determining, when body temperature changes, ing to this conclusion, the animals had no opportu¬
whether the change is due to a shift in set point. nity for operant control of their temperature. Except
for occasional observations on postural changes, such
as huddling or sprawling (Freeman 8c Davis, 1959;
SEPARATION OF OPERANT AND Hellstrom 8c Hammel, 1967), measurements were
RESPONDENT TEMPERATURE REGULATION made only of body temperature and of such reflexes as
shivering, panting, and changes in vasomotor tone.
For many years integrated control of body tempera¬ In 1964, Satinoff combined the neurophysiological
ture was thought to depend on the integrity of two technique of cooling the brain with the operant
areas of the brain, the preoptic/anterior hypothalamic measure of bar pressing for radiant heat. She found
area (for brevity, we shall call this the preoptic area) that cooling the preoptic region of rats elicited not
and the posterior hypothalamus. Damage to the pre¬ only shivering and an increase in body temperature,
optic area produced animals that could not reflex- but operant responding for heat as well. As with re¬
ively maintain their body temperatures within normal flexes, skin and brain temperatures interact in con¬
limits when placed in hot environments (Teague & trolling operant responding. Brain cooling increases
Ranson, 1936); after posterior hypothalamic lesions the rate of working for heat much more in cold than
animals were able to regulate normally in the heat, in neutral environments. Conversely, when their hypo¬
hut became hypothermic in the cold (Keller, 1963; thalamus was warmed, rats decreased responding for
Pachomov, 1962). This concept of two equal and external heat in the cold (Carlisle, 1966; Corbit, 1970;
opposing centers, one in the posterior hypothalamus Murgatroyd 8c Hardy, 1970).
controlling heat production and the other in the pre¬ Because preoptic thermal stimulation produces
optic area controlling heat loss, gradually gave way to both reflexive and operant responding, it is reasonable
a theory in which the preoptic area was preeminent in to expect that damage in that area would eliminate
temperature regulation. The change occurred for a both types of controls. Although such damage impairs
number of reasons: reflexive responses, it does not impair thermally moti¬
vated instrumental responding. Lipton (1968) demon¬
1. Thermally sensitive units (that is, neurons whose strated that rats with preoptic lesions would, when
firing rates are greatly influenced by their temper¬ placed in a hot environment, turn a heat lamp off and
ature) were found in much greater abundance in a cooling fan on, thereby avoiding death from over¬
the preoptic area than in the posterior hypothala¬ heating. Carlisle (1969) later showed that such
mus (Edinger 8c Eisenman, 1970; Eisenman 8c Jack- lesioned rats pressed at a much higher than normal
son, 1967).
rate for heat reinforcement in the cold and were able
2. Heating the preoptic area caused sweating, pant¬
to prevent severe hypothermia. Satinoff and Rutstein
ing, vasodilation, and all other autonomic correlates
(1970) tested rats with preoptic lesions in a 5°C
of heat loss with a concomitant fall in body temper¬
chamber twice a week. In one of the weekly sessions
ature (Magoun, Harrison, Brobeck, 8c Ranson,
no bar was available; body temperatures fell an aver¬
1938; Proppe 8c Gale, 1970). This would be ex¬
pected from a heat loss center. But cooling the age of 2.4°C in a 1-hr session for at least two months
preoptic region caused the heat-producing responses postoperatively. In the other session, which lasted for
of shivering and increased metabolic rate, the heat 2 hr, holding a bar down kept a heat lamp on, and
loss response of vasoconstriction, and a rise in body the rats depressed the bar 32% of the time, main¬
temperature (Hammel, Hardy, 8c Fusco, 1960; taining their temperatures within .7°C of normal.
Morishima 8c Gale, 1972; also see Satinoff, 1974, Controls kept the bar depressed only .05% of the
for review). This would not be expected. Heating time (Figure 1).
or cooling the posterior hypothalamus did not
These experiments demonstrate that operant be¬
elicit any of these responses (Adair 8c Hardy, 1971;
havior can compensate for reflexive deficits. In the
Freeman 8c Davis, 1959).
examples cited above, the reflexive deficits were pro
3. After lesions in the preoptic area, animals were
unable to regulate their body temperatures reflex- duced by hypothalamic lesions. In other experiments,
ively in either warm or cool environments (Carlisle, reflexive deficits were caused by thyroidectomy (Laties
1969; Satinoff, 1974; Satinoff 8c Rutstein, 1970; 8c Weiss, 1959), vitamin deficiency (Weiss, 1957; Yeh 8c
Squires 8c Jacobson, 1968). Weiss, 1963), or various drug treatments (see page
Evelyn Satinoff and Robert Hendersen 155

Fig. 1. Duration of heat-on time for the first eight trials in the
cold of rats with preoptic lesions and of normal rats maintained,
at 80% normal body weight. (From Satinoff 8c Rutstein, 1970. <g>
1970 by the American Psychological Association. Reprinted by
permission.)

167), and rats also learned to compensate through

instrumental behavior.
It further appears that the neural networks con¬
trolling reflexive and operant thermoregulatory re¬
sponses are to a large degree independent of one
another. Operant responses are not integrated solely
in the preoptic area because they continue to appear
when that region is largely destroyed. Additional DAYS
evidence for this independence is that lateral hypo¬
Fig. 2. Effect of lateral hypothalamic lesions on body weight,
thalamic lesions can disrupt thermoregulatory oper¬ core temperature, responding for heat, and shock avoidance.
ants without affecting reflexive regulation (Satinoff 8c Shaded area indicates period of tube feeding; TR pre—rectal
temperature immediately before the 1-hr test; T-p post—rectal
Shan, 1971). Well-trained rats that had pressed a lever
temperature at the end of the test. (From Satinoff sc Shan, 1971.
that turned on a heat lamp in the cold no longer did (c) 1971 by the American Psychological Association. Reprinted by
so after lateral hypothalamic lesions. Most of the permission.)
animals were nonetheless able to maintain their body
temperature reflexively. The operant deficit was not or any of these in combination. In this case, we can
always accompanied by impairments in feeding or rule out debilitation, forgetting, and motor problems.
drinking. When it was, the behaviors recovered at On some tests in the cold, rats were injected with
different rates. For instance, in Figure 2, rat SY18 re¬ quinine HC1, a drug that lowered their internal
gained its preoperative body weight in 10 days, yet it temperature by interfering with shivering (Satinoff,
did not bar-press for heat at preoperative levels until unpublished research). On those tests the rats re¬
over 40 days had elapsed. sponded at preoperative levels, whereas their response
Of course, there are several ways in which a treat¬ rates returned to near zero on nondrug days. This
ment may result in a loss of (or decrement in) respond¬ demonstrates that the rats were able to make the re¬
ing. These include general debilitation, forgetting, sponse and had not forgotten how. Sensory deficits
impairments in arousal, motor, or sensory processes. possibly contributed to the loss of responding. Rats
156 THERMOREGULATORY BEHAVIOR

with lateral hypothalamic lesions may not be as sensi¬

tive as normals to skin temperature changes (or skin RESPONDENTS
temperature pathways may be damaged by the lesion
so the signal is inaccurate), and it may require the
addition of a fall in internal temperature to get them
working for heat. There may well also have been a
deficit in arousal, and the lesions may have elevated
the threshold for operant behavior. However, since
thermoregulatory reflex adjustments are made during OPERANTS
sleep, arousal level presumably would not have
affected those components, Thus it appears that Fig. 3, Diagramatic representation of the mechanisms for oper¬
thermally motivated operant behavior depends on a ant and respondent temperature regulation. T = temperature;
C = core} LPH = lateral posterior hypothalamus; PO/AH =
distinct neural system passing through the lateral preoptic anterior hypothalamus.
hypothalamus.
Operant and reflexive thermoregulation appear to
tant facts: (1) Preoptic thermal stimulation leads to
be uncoupled in the posterior as well as the lateral
hypothalamus. Although local heating or cooling m
coordinated operant and reflexive responses. This is
because the temperature of the preoptic area affects
that area docs not elicit reflexive thermoregulatory
neural activity in both preoptic tissue and the lateral
responses (Adair, 1974; Freeman & Davis, 1959), these
posterior hypothalamus, (2) Preoptic lesions lead to a
treatments do alter operant thermoregulation. When
loss of thermoregulatory respondents while leaving
squirrel monkeys were given control over their am¬
operants intact. It is assumed here that operant
bient temperature, they selected higher air tempera¬
thermoregulation after such lesions depends mainly
tures when the posterior hypothalamus was cooled
on extrapreoptic temperature receptors. (3) Posterior
and lower air temperatures when it was warmed. This
hypothalamic thermal stimulation leads to operant
operant regulation was just as precise as when the pre¬
responses, but not to reflexive ones. (4) Lateral hypo¬
optic region was heated or cooled (Adair, 1974). These
thalamic lesions eliminate operant responding only.
results are compatible with the decreased operant re¬
In thermoregulation, then, reflexive and operant
sponding after lateral hypothalamic lesions reported
responses to thermal stresses are functionally and
above. The lateral and posterior hypothalamus ap¬
anatomically separate, and animals can compensate
pear to be part of the same pathway, which is involved
for deficits in one system through the mechanisms of
in the control of thermoregulatory operants. Lesions
the other system.
in the posterior hypothalamus generally lead to more
drastic deficits, including somnolence and complete and
possibly permanent adipsia and aphagia (McGinty,
USING BEHAVIOR TO ASSESS
1969). Lateral hypothalamic lesions cause less severe
effects: the rats are drowsy instead of totally somno¬ REGULATION

lent (Wampler, 1970), and later their adipsia and

aphagia recover through stages to relatively normal The separation of reflexive and nonreflexive
eating and drinking (Teitelbaum 8c Epstein, 1962). thermoregulation appears phylogenetically. Fish, am¬
The medial forebrain bundle, possibly part of a mech¬ phibians, and reptiles have highly sophisticated non¬
anism which facilitates operant behavior (Stein, 1964), reflexive means of regulating their body temperature,
includes both the lateral and posterior hypothalamus, whereas automatic mechanisms are either nonexistent
so it is reasonable that lesions in those areas should or few and inefficient (see Templeton, 1970, for a re¬
eliminate and stimulation excite the same sorts of be¬ view). For this reason, ectotherms2 are excellent prep¬
havior. arations for studying homeostasis. One need not dam¬
Even though the preoptic area appears to be in¬ age the brain to isolate its systems. Instead, we can
volved primarily in respondent thermoregulation, pre¬
optic thermodetectors also affect the operant system. 2 The familiar terms for “cold-blooded” and “warm-blooded”
In fact, both systems can be conceptualized as in animals are poikilotherm (from the Greek poikilos, “varied,
changing,” and therme, “heat”) and homeotherm (Greek
Figure 3, which admittedly is a tremendous over¬ homoios, “like”). Since all of these animals thermoregulate, more
simplification with respect to reflexive controls and appropriate words describe them on the basis of whether the
heat source is external or internal. Hence we are using the more
probably with respect to operant controls as well.
precise terms ectotherm (Greek ektos, “outside”) and endotherm
Nevertheless, it adequately accounts for four impor- (Greek endon, “within”) (Cowles, 1962).
Evelyn Satinoff and Robert Hendersen 157
PER CENT TIME

Th T,

Fig. 5. A lizard moves from direct sun (left) into shade (middle)
at body temperature T^, and it moves from shade to sunlight
A.M. P.M.
at body temperature T^. The difference in temperature, T^—Tj,
is a refractory range in which the lizard can operate without
Fig. 4. The percentage of time spent in the sun (open circles)
and the shade (solid circles) by a horned lizard, Phrynosoma
resorting to thermoregulatory activities. The effector output
Y is here equated with the availability of sun. Y = 0 in the
coronatum, during August. The values given are midpoints for
shade and Y = maximum in direct sunlight. (From Heath, 1970.)
1-hr intervals. (From Heath, 1965. Reprinted by permission of
the University of California Press.)

Because shuttling is a common thermoregulatory

study organisms with a simpler thermoregulatory behavior of lizards in their natural environment it is
organization (Roiin, 1968). relatively easy to study in the laboratory, Hammel,
Caldwell, and Abrams (1967) demonstrated that blue-
Th^rmor^gvlotion in IstQthvrms tongued lizards regulated their internal temperature
between SO and %1 °C by shuttling between 15 and
For many years it was assumed that vertebrates 45°C chamber?. In ft different gitnation, the lizards
other than mammals and birds could not control their were placed in a hot compartment and allowed to
body temperature;? at all. When lizards, for example, escape to a cooler one. Increasing the temperature of
were trapped and put into a cage in the laboratory, the hot compartment caused the animals to escape at
their temperatures fluctuated with that of the sur= lower colonic temperatures {Myhre & Hammel, 1969);
rounding medium. However, m their natural environ¬ Lizards also learned to go to a platform which, when
ments these reptiles regulate with a variety of be¬ depressed by the weight of the animal, turned on an
havioral mechanisms (Cowles & Bogert, 1944).9 Figure overhead heat lamp, Response frequencies increased
4 illustrates one common behavior—shuttling back and response duration decreased with increasing in=
and forth between sun and shade. This enables the tensity of the heat reinforeer. As the intensity of the
animal to maintain its body temperature within a heat changed, the lizards compensated behaviorally,
fairly narrow range, generally no more than § or 4°C receiving a roughly constant amount of heat per hour
(Figure 5; Heath, 1965). Once within this range it can (Garrick, 1973).
attend to business other than thermoregulating. If the Frogs and fish also clearly show a thermoregulatory
regulated range were narrower, the lizard would have component m their behavior, Frogs selected temper¬
to spend all of its time shuttling back and forth. As atures from 25-28°C in a thermocline (a long, ther¬
Heath (1970) points out, such an animal would be a mally graded alley) ranging from 0-40 °G (Cabanac 8c
very good thermoregulator but a very inefficient Jeddi, 1971). Different species of fish aggregated at
lizard. different points in a thermal gradient (Fry & Hoch-
3 This may not be characteristic of all vertebrates, however. achka, 1970). Six species of fish were trained to
Bogert (1959) noted that when several green iguanas, the largest
regulate the temperature in their tanks by their
lizards on the American continent, were exposed to direct sun¬
light in the desert in summer, they did not seek shade but sat in spatial movements (Neill, Magnuson, 8c Chipman,
the sun until they died. One specimen at the San Diego zoo never 1972). Swimming into warmer water caused the entire
went indoors on cool evenings and had to be taken inside. One
tank to heat up, whereas swimming into cooler water
night it was inadvertently overlooked by its keeper and was
found the next morning in a state of cold narcosis. caused a drop in tank temperature. The fish all kept
158 THERMOREGULATORY BEHAVIOR

the tank temperature within a 4-7 °C range, the mean cord (Duclaux, Fantino, & Cabanac, 1973) heating
varying for different species, although a range of 22°G caused the animals to move toward colder water, indi¬
was available. cating that amphibians also are responsive to changes
Such experiments do not guarantee that the be¬ in both internal and skin temperature.4
havior is under the control of operant contingencies. If fish, amphibians, and reptiles prefer some
The behavior may not even be temperature-related, temperatures to others, and these preferences can be
but may instead depend upon other stimulus features altered by thermally stimulating the brain, there must
of the environment. For example, if a heat lamp is be temperature-sensitive neurons in the brain. Both
used to generate the thermal gradient, the resulting cold- and warm-sensitive cells have been found in the
behavior may be controlled by the light, rather than diencephalon of Australian lizards (Cabanac, Ham¬
the heat, produced by the lamp. To demonstrate mel, 8c Hardy, 1967) and brook trout (Greer 8c Gard¬
preference, behavior must shift appropriately when ner, 1970).
the relationship between the thermal stimuli and In summary, ectothermic temperature regulation is
other environmental cues is reversed. Furthermore, determined by a combination of skin, brain, and other
the behavior may be a form of kinesis or taxis body temperatures, just as it is in mammals and birds.
(Fraenkel 8c Gunn, 1961). What appears to be choice These conclusions could only have been drawn on the
among thermal stimuli may instead be behavior basis of behavioral experiments, because nonreflexive
elicited by them. In a thermal gradient these stimulus behavior is the sole or predominant means of thermo¬
functions are confounded, Nevertheless, there are a regulation in these organisms.
few experiments wdiich unambiguously demonstrate
operant regulation, Iguanas learned to press a disc to
Thermoregulation in Infants
escape from heat. The responses were independent of
substrate temperature or heating rate, and appeared Many newborn mammals and birds have great diffi¬
to depend solely on internal temperature (Kemp, culty maintaining their body temperatures in the
1969). Goldfish can press levers and keep the tempera¬ cold. Because they are so small, they have a large
tures of their aquariums between 33.5 and 36.5°C. In surface-to-volume ratio and they lose heat very rap¬
a thermal titration situation they pressed the lever idly. Under natural conditions there are a variety of
both to decrease water temperature when it was too solutions to this problem—staying in a nest, bassinet,
high and to prevent its rising above the desired levels or marsupial pouch, clinging to the mother, or hud¬
(Rozin & Mayer, 1961). dling with siblings (Dawes, 1968). However, if such
a newborn is unfortunate enough to stray from the
What Controls the Regulation? mother or nest, it will die at air temperatures that
would not bother an adult. Is there a regulated body
Thermoregulatory behavior in ectotherms is con¬ temperature even in newborns, and is their problem
trolled by a combination of brain and other body simply that they do not have the mechanisms to main¬
temperatures, just as it is in mammals and birds. tain it? Or is a temperature control system lacking at
When the brainstem was heated to 41 °C, lizards birth, developing only later in life? We can answer
exited from the hot side of a shuttle box at colonic this question by providing behavioral opportunities
temperatures 1 to 2°C lower than normal. When the for temperature selection.
bram was cooled to 25°C; the lizards exited at colonic Piglets less than 1 day old chose thermal environ¬
temperatures 1 to 2°G higher than normal (Hammel, ments that allowed them to maintain their body tem¬
Caldwell, 8c Abrams, 1967). Arctic sculpins were placed peratures within .03°C of what it was when they were
in warm water from which they could escape by swim¬
4 In these experiments, large deviations from normal brain
ming back to water at 5°C to which they had been temperature (at least 5°C) produced relatively small change in
adapted. Heating the forebrain lessened the time the deep body temperature threshold at which behavioral
spent in warm water, while cooling the brain some¬ responses appeared (only 1— 2°C). Similar effects have been seen
in mammals. Gorbit (1970) reported a number of experiments on
times suppressed the escape response (Hammel, rats in which he examined the effects of changes in hypothalamic
Stromme, 8c Myhre, 1969). These results have been re¬ temperature on lever pressing for convective cooling. The
peated in several species of fish, from both warm and hypothalamic temperature threshold for the behavioral heat loss
response was very high (40.3-43 °C). However, reflexive heat loss
cool waters, and in every case altering brainstem responses (which lizards lack) were activated at the much lower
temperature affected the tank temperature the fish brain temperatures of 38°C. It may simply be that the thresholds
selected (Crawshaw 8c Hammel, 1971, 1973, 1974). In for activating behavioral and reflexive thermoregulatory re¬
sponses are very different, although see Corbit (1970) for alter¬
frogs, abdominal (Cabanac 8c Jeddi, 1971) or spinal native explanations.
Evelyn Satinoff and Robert Hendersen 159

29-6° 30-8°
—I- —i—

3= £

n ♦

>4
-

*»■ ^
N\

_ r
36-4° 37-9°
_1_1_
-

19 21 23 25 27
P05ITI0N IN APPARATUS

Tig. 6. Response o£ 1-day-old laboratory mice taken from the

mother and placed together at a moderately warm position in
the temperature gradient. All animals initially faced down the
gradient. (From Ogilyie Sc Stinson, 1966. Reprinted by permission
of the National Research Council of Canada.)

with the sow (Mount, 1963), This indicates a very

good behavioral thermoregulatory system. Neonatal
hamster pups were also very sensitive to thermal Fig. 7. A; in a neutral environment puppies prefer the fur
motber and avoid the metallic one. B: Tbe fur mother bas
gradients and moved quickly away from cooler areas bsen cooled to 14°C and the metallic mother warmed to 33°C.
toward a heat source, where they became quiescent. The puppy was fed before the beginning of the test. (From
However, if they were placed directly under the heat Jeddi. 1970.)

source, they did not move away and died from over¬
heating (Leonard, 1974). Similarly, Lday-old mice contact was established (jeddi, 1971). Thus the search
moved from a cooler to a warmer environment. Figure for contact comfort in very young animals may have a
6 shows the behavior of six newborn mice placed in a thermoregulatory component. In fact, Harlow (1971,
thermocline. Within 2 hr they were all in positions at p. 70) reports that infant macaques given a ehuic^ be¬
substrate temperatures at least 5QC higher than at tween warm wire surrogate mothers and cool doth
their initial positions (Ogilyie 8c Stinson, 1966), surrogate^ showed a preference for the warm surrogate
Neonatal puppies and rabbits also show thermo¬ during the first 20 days of life.
regulatory behavior. Puppies aged 12 hr to 10 days Generally, young mammals select temperatures
were placed in the presence of two artificial mothers higher than those chosen by adults. As physiological
constructed of metal coils, one of which was covered and hormonal capabilities develop and physical char¬
with fur. In a warm environment (30°C) the puppies acteristics change (fur appears and surface-to-mass
preferred the furred coil (Figure 7a). However, when ratio declines), selected temperatures become lower.
the metallic surrogate was warmed and the fur mother Young birds also thermoregulate behaviorally.
cooled, the puppies spent almost 100% of the time Hogan (1974) observed that when the hen did not
with the metallic mother (Figure 7b; Jeddi, 1970). initiate brooding (which warms the chicks), 3- to 8-
Rabbits oriented toward a furred artificial mother in day-old chicks became cool and stimulated the hen to
a cold environment but not in a warm one. This be¬ sit by rubbing against her and pecking her feathers.
havior was very efficient in regaining normal body In an experimental situation, two breeds of 1- and 2-
temperature, which had dropped precipitously before day-old chicks quickly learned to peck a key when that
160 THERMOREGULATORY BEHAVIOR

DISTURBANCE
CIRCADIAN RHYTHMS'
FEVER L OUTPUT
HORMONAL STATE-
►(REGULATED
OTHER
TEMPERATURE)

CONTROLLING SYSTEM CONTROLLED SYSTEM

Fig. 8. Control diagram of the relation between set point (or reference input), actual
body temperature, and a reflexive response. The comparators (circles) are mixing points.
Whenever the combination of pluses and minuses do not cancel one another, an error
signal is generated. When this occurs (that is, whenever there is a disturbance such that
heat gain and heat loss mechanisms are not at minimum levels), a response is activated
which alters the regulated body temperature. Information from temperature receptors
is then fed back to the comparator and the error signal is adjusted. Several points must
be clarified; (1) There is no single regulated body temperature. That term is a convenient
fiction for some mathematical combination of all the temperatures that contribute to
effector output (Brown 8c Brengelmann, 1970). (2) The reference input variables leading
into the set point indicate that the set point is not constant but fluctuates because of
the influence of a variety of nonthermal inputs. (3) This diagram is not sufficient to
describe the control of operant responses. For that, additional loops feeding back to
the response controller are required for both response effectiveness and response cost (Van
Sommers, 1972).

response was reinforced with heat and light (Zolman, corrective measures (effector responses) which reduce
1968). the error are activated. If the system is working
Is the thermoregulatory behavior of neonatal mam¬ optimally, temperature is maintained as closely as
mals and birds under the control of operant con¬ possible to the set point (Figure 8; see Milsum, 1966,
tingencies? The interpretation of existing behavioral for a comprehensive discussion of biological control
experiments remains ambiguous. Of the experiments systems). Set point is the value of the input of a con¬
discussed above, only the behavior of the young chicks trol system at which the output is zero. It is neither a
that pecked a key for heat is clearly under the control theory nor an explanation; it is merely a descriptive
of an operant contingency, and even this may have a device which is useful in describing the operation of
large respondent component (Wasserman, 1973). homeostatic systems. Clearly, there would be no set
In summary, many neonatal animals that do not point without a nervous system, but for our purposes
possess reflexive mechanisms sufficient for maintaining it does not matter where or how the set point is
body temperatures nevertheless demonstrate thermo¬ achieved. It is irrelevant whether the reference tem¬
regulatory capabilities when provided with behavioral perature is a function of the difference in firing rates
opportunities to do so. In this respect, young mam¬ between temperature-independent and temperature-
mals and birds are much like ectotherms. sensitive neurons, or whether it is the point at which
warm- and cool-sensitive neurons are minimally active
(to enumerate just two of the ways it could be repre¬
THERMOREGULATION AND THE sented). This does not imply that the same set point
CONCEPT OF SET POINT will be adequate for describing characteristics of the
system at more fine-grained levels of analysis. Thermo¬
So far we have been describing how animals lack¬ regulatory functioning may be described in terms of
ing reflex mechanisms are nonetheless able to thermo- one, two, or many set points. The formulation we
regulate behaviorally. Normally, of course, in mam¬ choose depends on the aspect of the system being
mals and birds, both reflexes and operants maintain studied. For example, for a physician concerned about
a constant body temperature. Control theory provides a feverish patient, the deviation of the patient's body
a useful framework for describing this thermal inte¬ temperature from normal is all he needs to know in
gration. A control system maintains its output (actual order to decide whether or not to institute corrective
body temperature) at or near some reference value measures. The physician can operate as if there is a
(thermal set point). If there is a discrepancy (error single set point, as illustrated in Figure 9a. We have
signal) between the set point and the achieved output. already seen that although reflexive and operant
Evelyn Satinoff and Robert Hendersen 161

CONTROLLER Circadian Rhythms

One might assume that diurnal reptiles become

cold at night because they have no choice; they derive
a. their heat primarily from the sun, so they necessarily
cool down and become inactive when the sun sets.
However, the decrease in body temperature which
occurs at night is more than the passive result of
cooler surroundings. Four species of lizards living in a
thermocline for several days maintained a high body
temperature when the lights were on, whether they
were active or not, but they selected the cooler part of
the gradient during the lights-off part of the cycle
b. (Regal, 1967). Thus the lizards moved into ambient
temperatures at which they became sluggish and
uncoordinated.
Because lizards cool down at night, they have prob¬
lems in the morning. If they were to remain under¬
ground until their burrows warmed up sufficiently to
allow them to begin daily activity, they would lose a
lot of valuable activity time. Fortunately for them, a
warm burrow is not necessary for activity. In the
c. laboratory, horned lizards emerged from their bur¬
rows before sunrise at body temperatures about 15°C
below their normal activity levels. Two groups of
lizards were kept at constant conditions of 18 to 27°C,
Tig. 9. Schematization of three possible controlling systems for
thermoregulation, For clarity, feedback loops are not shown. except for 8 hr a day when they had access to heat
The controller equation relating error signal and response lamps. Both groups emerged from the sand at about
determines both the threshold and the size of the response.
the same time prior to the onset of the heat lamps, al¬
Note that models (b) and (c) are equivalent to model (a) if we
assume that the thresholds for elicitation of the various re¬ though, of course, each group’s internal temperature
sponses need not be identical, (a) One central thermostat whose was dose to that of its environment (Heath, 1962).
output activates all available operant and reflexive thermo¬
Thus reptiles appear to show a circadian variation
regulatory responses, (b) Two central thermostats, one con¬
trolling all reflexive, the other all operant responses, (e) Each in their thermal set points.
thermoregulatory response is independent of any other. Mammals also show circadian fluctuations in body
temperature. If we extrapolated from reptiles, we
would say that these diurnal oscillations are rhythmic
mechanisms normally work in concert, they are in fact
shifts in set point. However, peaks in body tempera¬
functionally distinct. Therefore, the behavioral scien¬
ture normally coincide with periods of activity, Are
tist must use a more detailed level of analysis. He can
animals active because their temperatures are high, or
operate within the framework of Figure 9b. For the
are their temperatures high beeau§e they are active?
physiologist interested in a particular thermoregula¬
One way of answering this question would be to
tory response, such as vasomotor tone, the level of
measure all reflexive thermoregulatory responses
analysis must be finer still, allowing a detailed*
(metabolic rate, vasomotion, etc.). If different body
quantitative analysis of the vasomotor controller. He
temperatures are observed* and yet the reflexive
might use the control diagram outlined in Figure 9c.
thermoregulatory responses do not vary so as to
For most purposes in this chapter, a single set point
change them, then one could assume a shift in set
notion is sufficient to characterize general features of
point (Hensel, 1974). Another, less cumbersome way
thermoregulation.
would be to use operant behavior to determine
The thermoregulatory set point is affected by a
thermal preferences. If there are rhythmic shifts in set
variety of internal and environmental variables and
point, preferred temperature should reflect them.
fluctuates from time to time. Because ectotherms have
only nonreflexive regulatory mechanisms, we can see
Hormonal State
many of the determinants of the set point more clearly
in them than in mammals. Regulated temperature varies with reproductive
162 THERMOREGULATORY BEHAVIOR

condition. Pregnant blue spiny lizards regulated at by sewing a thermocouple into one of the mice that
lower temperatures than did males or postparturient the snake ate. The snake moved so as to keep warm
females. They tolerated lower minimum internal that part of the body containing the bolus. As soon as
temperatures and stopped basking at lower maxi- the snake defecated, basking abruptly ended. Thus in
mums (Garrick, 1974). Plasma progesterone levels cool-active reptiles, digestive requirements determine
were twice as high in pregnant lizards of this species thermoregulatory activity.
as in postparturients (Callard, Chan, & Potts, 1972), Reptiles need more heat after eating so that they
which suggests that this hormone may be involved in can digest their meals. Mammals need less external
the change in behavior. When Garrick injected heat after eating because of increased metabolic heat
progesterone into postparturient lizards, it lowered production. Rats bar-pressed more for heat when they
the temperature around which the lizards regulated. were fed immediately after a 1-hr test session in the
Progesterone caused hyperthermia in female rats cold than when they were fed before the session
(Freeman, Crissman, Louw, Butcher, & Inskeep, 1970) (Hamilton 8c Sheriff, 1959). Internal temperatures
and in humans (Gordon, 1972). This appears to be an were elevated alter feeding and did not return to pre¬
upward shift in set point, because human females pre¬ feeding base lines for an average of 2 hr (Grossman &
ferred warmer stimuli during the luteal phase of the Rechtschaffen, 1967). Decreased rates of pressing for
menstrual cycle, when progesterone levels are high, heat following a meal thus help maintain a constant
than during the follicular phase, when levels are low thermal balance. Varying the quality of the diet also
(Cunningham 8c Cabanac, 1971). Of course, thermo¬ affects response rate for heat, Rats fed a high-fat or
regulatory changes during the menstrual cycle may high-carbohydrate diet gained weight, and worked less
not be caused by progesterone per se. Levels of other for heat than they did when they were fed a high-
gonadal hormones, particularly estrogen, wax and protem or powdered chow diet (Hamilton, 1963).
wane simultaneously, and nongonadal hormones affect Conversely, rats fed a high-fat diet or made hyper-
temperature as well. Regardless of which hormones phagic by hypothalamic lesions worked more than did
are involved, however, it is clear that the set point controls to escape heat (Lipton, 1969). The increases
changes with reproductive condition in lizards and in responding were related to higher body weight.
mammals, although in opposite directions. These changes in response rate for both heat escape
Thermoregulatory effector mechanisms serve non- and heat reinforcement can be interpreted as be¬
thermoregulatory functions as well. Interpreting be¬ havioral compensation for the lessened ability to lose
havior changes is difficult, because nonthermal heat reflexively (fat animals have more insulation).
factors may produce such changes. For example, Mc¬ There does not appear to be any shift in thermal
Lean and Coleman (1971) noted that female rats set point.
housed in large cages showed less of a drop in body
temperature during estrus than did restricted rats.
They concluded that the increased activity commonly THERMAL PREFERENCE
seen in estrual rats was a response to a lower body
temperature. This conclusion is not warranted. In¬ Control theory is a way of systematically describ¬
creased activity may be a sign of sexual agitation, and ing how preferences for thermal stimuli vary under
sexual responses may take precedence over thermo¬ different conditions. Stimuli that decrease the devia¬
regulatory responses during estrus. If the activity were tion from set point are desired, whereas those that in¬
truly a response to a lower body temperature, one crease the error signal are aversive. Factors such as
would expect to see increases in other heat-producing fever, circadian rhythms, and hormonal changes which
and heat-conserving behaviors at the same time, but shift the set point do not change the basic relationship
food intake and nest building actually decrease during between error signal and stimulus preference. Thus
estrus (Wade, 1972). the same thermal stimulus may be positively reinforc¬
ing in one condition and aversive in another. The
value of a given stimulus is determined by the context
Food Intake
within which it is applied.
Digestion is another activity which affects thermo¬
regulation. Regal (1966) noted that certain reptiles,
Preference and Subjective Pleasure
active at low temperatures, moved toward the warmer
parts of their terraria after they had eaten. He Preferences for thermal stimuli are related to re¬
measured the body temperature of a boa constrictor ports of subjective comfort. Just as people can judge
Evelyn Satinoff and Robert Hendersen 163

brightness, loudness, and warmth, so can they judge very pleasant +2 r

the pleasure a particular stimulus provides. Pleasure .*
# •
judgments are assessments of affective quality, whereas
preference is a description of choice behavior. Noth¬ pleasant + 1 L • • •• M(« a qa co

ing in their respective definitions demands that the + • • — — aa 0

two measures be associated. Nevertheless, it is prob¬
• « AAA A O A
ably reasonable to assume that stimuli judged as neutral o \- • AA •• •
AAA A a
pleasant would also be desirable in preference tests, * A a •
AOOAA* tv • A
while those judged as unpleasant would be aversive.
• cS ‘
Cabanac (1971) has beautifully demonstrated how unpleasant - 1 • O M AV

judgments of pleasure depend on context. Each of his

subjects was immersed up to his chin in a tub of water
whose temperature was controlled by the experimen¬ very unpleasant - 2 L **
ter. With heat loss controlled this way, the subject’s i_L
10 20 30 40 SO
core temperature could be maintained above or below (a) STIMULI °C
his set point. (Note that this procedure does not
THERMAL SENSATION
change the set point, but instead increases the error very pleasant +2r
signal by altering the regulated variable—actual body
temperature.) While sitting in the bath, each subject
dipped his left hand for 30 sec into a container filled
pleasant +1
with water at a particular temperature and judged
the pleasure provided by this thermal stimulus. Then
the subject put his hand back into the bath until the
neutral 0
sensation disappeared, then dipped his hand once M 0
more into the container, which was now filled with 0
water of yet a different temperature, and judged it. A
Unpleasant - 1
series of such ratings is shown in Figure 10a. The
subjects perceived stimuli at the extreme ends of the
scale as pleasant or unpleasant depending upon
very unpleasant 2
whether they were hypothermic or hyperthermic.
Thus when they were hypothermic (internal tempera¬ IQ 20 3Q SO 30
stimuli qe
ture below 37°C) they reported that warm or hot
stimuli to the hand were pleasant. When they were
(b> Fever 9 HyperthermictBatlr306G T:38.5»39°C

hyperthermic (internal temperature above 37°C) Fig. 10. (a) Judgments of thermal pleasure given by a single
they perceived cool or cold hand stimuli as pleasant. subject when hypothermic (open symbols) and hyperthermic
(closed symbols), (b) Similar judgments made when the subject
The change in hand temperature was not sufficient to was feverish (open circles) or hyperthermic (closed circles), The
change deep body temperature. However, in every bath temperature was 36°C. The subject’s internal temperature
case, the pleasant stimulus was one that, had it been was 38.5-39.0°C. (From Cabanac, 1969.)

extended over the entire body, would have decreased

the difference between the set point and the actual tered. The error signal can also be manipulated by
body temperature. changing the set point while maintaining a constant
Similar results were obtained in an experiment in body temperature. Fever can be described as an up¬
which the dependent variable was a measure of pref¬ ward shift in set point (see page 166). Cabanac (1969)
erence (Cabanac, Massonnet, 8c Belaiche, 1972). Sub¬ tested a subject when he had a fever (because of in¬
jects sitting in a water bath had to manipulate a valve fluenza) and when he was well. The temperature of
to change the temperature of another bath in which the water bath, and hence the subject’s internal
an arm was immersed. Results paralleled those from temperature, was identical in both series of tests.
the experiments using judgments of thermal pleasure. Judgments of pleasure, however, were very different
Thermal preference systematically changed when the (Figure 10b). The subject liked warmer stimuli when
temperature of the bath in which the subjects sat was he had a fever and cooler stimuli when he was well
varied. but hyperthermic. These experiments were later re¬
In the experiments discussed above, the set point peated in four other subjects (Cabanac 8c Massonnet,
remained constant while body temperature was al- 1974), and the results strongly support the theory that
164 THERMOREGULATORY BEHAVIOR

reported pleasure is determined by deviations from keys pulled a chain that warmed their chamber. When
set point. They further imply that when body tempera¬ the force requirement was increased, the monkeys
ture is abnormal, preference measures can determine tolerated wider air temperature fluctuations and their
whether or not the abnormal level is caused by a interresponse times lengthened. Eventually they
change in set point. We shall develop this idea more stopped working completely and just sat and shivered
fully when we discuss drug administration. (Adair 8c Wright, 1976). Response cost also deter¬
Cabanac’s experiments demonstrate that the same mines behavioral thermoregulation in the tropical
peripheral stimulus is perceived as pleasant or nox¬ lizard Anolis cristellus (Huey, 1974). In an open park,
ious depending upon the person’s internal state. where basking sites were readily available, the lizards
Placing a subject in a temperature-controlled bath kept their body temperatures within fairly narrow
affects peripheral, core, and brain temperatures. In¬ limits. In an adjacent forest where shuttling from
formation from all three normally covaries. Are these shade to sun required much more movement, the
signals functionally interchangeable, or can organisms animals tolerated lower and more variable body
distinguish among them? When Corbit and Emits temperatures.
(1974) warmed the hypothalamus, rats pressed a bar
that cooled their hypothalamus rather than a bar
that lowered the air temperature. When the skin was OPERANT CONTINGENCIES IN
warmed, the opposite preference appeared. When the THERMAL HOMEOSTASIS
animal has no choice (e.g., when its hypothalamus is
being warmed and all it has available to it is a bar Thermoregulation differs in important ways from
that cools the air), it will use any opportunity which other regulatory systems. It should come as no sur¬
is available to decrease the error signal. However, the prise, then, that there are substantial differences be¬
animal can detect the site of the disturbance and, tween the effects of thermal reinforcers and the effects
given the choice, will direct its behavior toward chang¬ of more traditional reinforcers such as food and
ing the temperature at that site. water. For instance, one feature of thermally rein¬
forced behavior that may seem curious is that response
rate varies inversely with magnitude and duration of
Nonthermal Determinants of Thermal Choice
heat reinforcement (Carlisle, 1966; Weiss & Laties,
Operant selection of thermal remforcers depends 1961). This would be expected if we consider how the
upon what other sorts of reinforcers are concurrently reinforcement is tied to thermal homeostasis. The an¬
available. One may forego the opportunity to thermo- imals do not respond so as to produce a maximal
regulate efficiently in order to engage in a more highly amount of heat; rather, they produce an optimal
preferred activity, as when an avid football fan sits amount. In most experiments with heat reinforce¬
shivering in the cold to watch an exciting game. ment, the animal can reach “satiation” (i.e., set point)
Carlisle and Snyder (1970) demonstrated this effect within a single session or even within a fraction of a
very dramatically. Rats bar-pressed for heat and main¬ session. Because the animal stops responding whenever
tained their body temperatures very well in the cold. set point is reached, an inverse relation between rein¬
However, when a lever press which produced elec¬ forcement rate and response rate is to be expected
trical stimulation in the posterior hypothalamus was under these conditions.
concurrently available, the rats worked for the brain Although animals perform very well under inter¬
stimulation exclusively, allowing their body tempera¬ mittent schedules with ingestive reinforcers, this is not
tures to fall to the point of death. In another experi¬ the case with thermal reinforcers. Carlisle (1969) had
ment (Weiss & Laties, 1963), injections of d-ampheta- difficulty obtaining stable fixed-ratio (FR) responding
mine increased the rate of heat-reinforced bar pressing m rats with schedules as low as FR 5 or FR 10. How¬
of rats in the cold. The opposite result, a decrease in ever, when bar pressing was reinforced intermittently
rate, was produced when a food-reinforced fixed-ratio by access to heat on a continuous reinforcement (CRF)
schedule was concurrently available (Laties, 1971). schedule with a second bar (an FR-CRF chained
Thus effects of thermal reinforcement (or any type of schedule), good performance was obtained with sched¬
reinforcement, for that matter) must be considered in ules as high as FR 128 (Carlisle, 1970). Pliskoff, Wright,
a broad context which includes other available rein¬ and Hawkins (1965) obtained similar results with
forcers. rewarding brain stimulation, another case in which
Thermoregulatory behavior also depends on the performance on intermittent schedules is often quite
amount of effort involved in making a response. Mon¬ poor. Thus thermal reinforcers resemble brain stimu-
Evelyn Satinoff and Robert Hendersen 165

lation more than they resemble ingestive reinforcers. of ways in which drugs alter body temperature. For
This is not surprising considering the similarities be¬ instance, many general anesthetics depress central
tween the two. Both are direct and prompt, no con- nervous system function, including most thermo¬
summatory response is made, and neither can be regulatory reflexes (Lomax, 1970), and body tempera¬
stored the way food and water can. ture then varies with ambient temperature.5 Drugs
like amphetamine generally stimulate behavior, mak¬
Avoidance of Thermal Change ing an animal active and emotional, and body temper¬
ature may passively rise. Other drugs act directly on
People anticipate temperature changes. Someone effector mechanisms involved in thermoregulation. For
who goes outside on a bitterly cold day equips him¬ example, cholinesterase inhibitors cause profuse sweat¬
self with coat, hat, and gloves before leaving the ing and salivation (Koelle, 1970) and so lower body
warmth of a heated building, or, knowing that it is temperature. Adrenergic blocking agents may pro¬
very hot outside, chooses not to leave his air-condi¬ duce the same outcome by causing peripheral vaso¬
tioned home. Can animals similarly anticipate and dilation (Nickerson, 1970). Some compounds shift the
avoid thermal change? set point. Pyrogens, for example, displace it upward.
Rats can avoid heat. Matthews, Morin, and Church This phenomenon is called fever.
(1971) placed animals in a temperature-controlled What we want to know about any drug that affects
chamber and programmed a version of a Sidman body temperature is whether it changes the set point
(1953) avoidance schedule. A 5-sec exposure to heat or merely alters effector activity. One of the easiest
occurred every 5 sec if a bar was not pressed. Each ways to determine whether a body temperature change
response delayed the onset of the next heat exposure represents a change in set point is to measure the
for 15 sec. When ambient temperature was manipu¬ thermally reinforced responses that accompany it. Op¬
lated, behavior which avoided very hot air occurred erant behavior reflects the error signal—the differ¬
at different rates. Response rates were higher in the ence between the set point and the achieved tempera¬
hotter environments. Thermal avoidance thus de¬ ture. Because of this we can make the following two
pends on the relationship between the ambient tem¬ assertions;
perature and the thermal effects of responding rather
than on the absolute values of the stimuli. This is 1. If an animal’s body temperature changes to a new
similar to the context dependencies we noted previ¬ level, and if the animal selectively performs oper¬
ously in Cabanac's work. ants which defend the new level against deviations
Analyses of avoidance learning that emphasize m either direction, then the temperature change
species-specific defense reactions (Bolles, 1970) have represents a set point displacement.
been based almost entirely on data collected with elec¬ 2. If an animal’s body temperature changes to a new
tric shock as the aversive stimulus. By using heat as an level, and if the animal selectively performs oper¬
aversive stimulus, one can study avoidance learning in ants which counteract the change, then the temper¬
a situation where reactions other than those charac* ature change represents something other than a
teristically produced by shock may be prepotent. shift in set point.
Overheated rats are initially active, but later spread
out and sprawl (Roberts, Mooney, & Martin, 1974). To illustrate these points, let us consider the effects
That rats learn to bar-press to avoid and to escape of raising the body temperature in two different ways
heat suggests they can learn responses which are quite in a situation in which an animal has the opportunity
different from their unconditioned reactions to the to alter its thermal environment:
aversive stimulus.
1- Make the animal febrile by injecting a pyrogen, If
pyrogens raise the set point, the animal will re¬
THE OPERANT AS A MEASURE OF SET spond for warmth, but not for cooling, until its
body temperature approaches the new, elevated
POINT AFTER DRUG ADMINISTRATION
set point. A commonplace example of this is that
people in the first stages of a fever report feeling
Operant methods are extremely useful in analyzing cold and try to warm themselves.
drug effects on body temperature. If all we know
about a drug is that it changes body temperature, we 5 In the days before air conditioning a serious problem with
cannot assign it any particular role in thermoregula¬ surgery on hot summer days was keeping the patient’s tempera¬
ture down. Now, with operating rooms maintained at about
tion. This problem arises because there are a number 24°C, the problem is to keep it up at normal levels.
166 THERMOREGULATORY BEHAVIOR

2. Make the animal hyperthermic by placing it in a available, the fever of cats in the cold was substan¬
hot environment. In this case, body temperature tially higher than when it was not available, and in
rises not because the set point is elevated, but be¬ fact was very close to the levels attained in a neutral
cause reflexive heat loss mechanisms are inadequate environment. It thus appears that reflexive mech¬
to the task of removing excess body heat. The ani¬
anisms are simply inadequate to accomplish the full
mal should work to cool itself, whereas it should
rise in temperature when the environment is cold.
not work for warmth.
Can reptiles develop fevers? Despite an almost com¬
plete lack of reflexive thermoregulatory mechanisms,
Compounds That Alter Set Point ectotherms, as we have seen, do indeed regulate their
temperatures when they have the behavioral oppor¬
Pyrogens cause fever. The current concept of fever tunity to do so.
is that injection or natural entry of bacterial pyrogens In an ingenious experiment, Bernheim, Vaughn,
into the body causes the release of an endogenous and Kluger (1974) allowed iguanas to adjust their
pyrogenic substance from blood leucocytes. This leu¬ body temperatures by shuttling between cold and
cocytic pyrogen then travels to the brain and acts on warm chambers. Then they injected the iguanas with
thermosensitive cells, raising the set point. This leads a pyrogen (one that caused fever in rabbits). Follow¬
to increased heat production and decreased heat loss ing the injection the iguanas spent more time in the
until body temperature rises to the new set level. warmer side of the chamber and developed an average
Fever was accompanied by heat-producing instru¬ fever of 2°C. Another group of lizards were given the
mental responding in baboons (Gale, Mathews, 8c same dose of pyrogen but kept in a constant ambient
Young; 1970), cats (Weiss, Laties, & Weiss, 1967), and temperature (below the febrile level). There was no
dogs (Cabanac, Duclaux, & Gillet, 1970). Febrile dogs, change in body temperature, indicating that the fever
for example, worked more for heat in the cold and was produced solely behaviorally.
less for cool air in hot environments than did normal Other sorts of data can be combined with operant
animals (Figure 11). measures to assess changes in set point. Clark and
The height of a fever is determined to some extent Coldwell (1973) reported that after intraventricular
by the ambient temperature; it is lower in the cold injection of tetroclotoxin—the puffer fish poison-cats
(Fekety, 1963; Weiss et ah, 1967). Does this imply that had lowered body temperatures even though regula¬
fever represents an alteration in the sensitivity of tory mechanisms appeared to be intact. While the
peripheral cold receptors? Operant measures suggest animals were recovering but still hypothermic, they
a different interpretation. In the Weiss et al. experi¬ were exposed to infrared heat lamps, which raised
ments, the cats were also tested in a chamber which their temperature sharply. When the lamps were
had a heat-producing lever, When the operant was turned off, their body temperature returned to where
it would have been had the heat load not been im¬
RESPONSES IN 30 mn posed. The cats did not shiver while they were recov¬
ering from the hypothermia caused by the poison, but
when body temperature was further lowered with ice
packs, they did shiver. This is good evidence that
tetrodotoxin lowers the thermoregulatory set point.
Clark and Lipton (1974) then showed that patterns of
instrumental responding for thermal reinforcement
were compatible with this interpretation. After tetro¬
dotoxin, while body temperature was falling rapidly,
the cats increased lever pressing to escape heat in a
warm environment and decreased pressing for heat in
the cold. Declines in body temperature were very
similar whether the cats were in the cold without a
bar, or whether they were responding for heat rein¬
forcement or escaping from heat. The lowered body
Fig. 11. Operant responding for infrared heat and for cool air temperature represents a downward shift in set point.
in a dog with and without a fever. Responding varies systema¬ Many drugs lower body temperature, but their
tically with ambient temperature. When feverish, the dog
mechanisms of action may be dissimilar. Behavior
responds more for heat and less for cool air than when not
feverish. (From Cabanac et al., 1970.) differentiates among them. To illustrate this, we shall
 Evelyn Satinoff and Robert Hendersen 167

consider three compounds, all of which, when injected cautious in making this interpretation because the
systemically, lower body temperature in rats, but increase in responding may represent a general activa¬
which have very different effects on operant respond¬ tion. If salicylate is. in fact lowering the set point, not
ing: chlorpromazine, quinine, and sodium salicylate. only should responding to escape heat increase, but
Chlorpromazine depresses operant behavior in a dose- responding for heat should decrease.
related manner regardless of whether the response pro¬
duces heat in the cold (Weiss 8c Laties, 1963) or
Neurochemical Basis of Thermoregulation
reduces heat in a warm environment (Polk 8c Lipton,
1975). Thus the hypothermic effect of chlorpromazine In recent years one of the major efforts in the study
is probably not due to its specific action on tempera¬ of temperature regulation has been to identify the
ture pathways, but rather is a side effect of the general neurotransmitters released at the synapses of neurons
central depressant properties of this drug. Quinine, on in thermoregulatory pathways. The protocol in many
the other hand, increases lever pressing for heat in the of these studies is simple; inject the transmitter and
cold (Satinoff, unpublished research). This is not a measure pre- and postinjection body temperatures.
general excitation but rather a specific thermoregula¬ From these sorts of data pharmacological models of
tory effect; in food-deprived rats, responding for food heat loss and heat production are constructed.
decreases after quinine injection (Figure 12). Behav¬ Not surprisingly, pharmacological models abound.
ior compensated for the fall in body temperature, The data on which they are based vary with the
suggesting that quinine was simply acting on one or species tested, the dose of drug used, and the route of
several effector mechanisms. In fact, when we mea¬ injection. We shall first list some of the conflicting
sured the physiological responses of quinine-treated results in this field to give an idea of the magnitude
rats in the cold, we found that all reflexive responses of the problem, and then we shall demonstrate why
were normal except shivering, which was greatly re¬ operant measures can provide the right sorts o£ data
duced. Sodium salicylate has a quite different effect— on which to build a pharmacological model.
the rats increased responding to escape heat even In the first experiments studying neurotransmitters
while their body temperature was dropping (Polk 8c and body temperature, Feldberg and Myers (1963) re¬
Lipton, 1975). Thus there was a coordinated change ported that intraventricular injections g£ norepmephr
in body temperature and instrumental behavior. rinc (NE) abolished shivering and lowered pyrogen*
In summary, all of these compounds lower body induced fever in cats. Serotonin (5*HT), on the other
temperature, but chlorpromazine depresses operant hand, caused shivering and a rise in rectal tempera¬
behavior regardless of whether it is hcat-producing or ture in afebrile cats. When smaller doses were injected
heat-reducing, quinine causes a compensatory change into the preoptic/anterior hypothalamus, the results
in behavior, and sodium salicylate elicits parallel be¬ were similar. Injections into other areas of the brain
havioral and physiological responses. did not affect body temperature. On the bads of these

Of these, the results with sodium salicylate are results, Feldberg and Myers suggested that NE activates
potentially the most interesting from our point of heat loss effectors (sweating, panting, vasodilation) and
view. They imply that this drug is shifting the thermo¬ 5-HT activates heat-production pathways (shivering).
regulatory set point downward. One must be extremely This was the first neurochemical model of tempera¬
ture regulation. Similar results with dogs (Feldberg,
Hellon, 8c Lotti, 1967) and monkeys (Myers & Yaksh,
1969) led to the hope that this model might be
applicable to all mammalian species. However, differ¬
ent results soon appeared in other species. With intra¬
ventricular administration, rabbits became hyperther¬
mic after NE and hypothermic after 5-HT (Cooper,
Cranston, 8c Honour, 1965; Jacob 8c Peindaries, 1973).
Mice responded to both of these amines with a drop
IQO

in body temperature (Brittain 8c Handley, 1967;

Handley 8c Spencer, 1972). Rats became hypothermic
-owe hour- after 5-HT (Feldberg 8c Lotti, 1967; Myers 8c Yaksh,
1968), but it was found that NE can lead to either a
Fig. 12. Cumulative record showing response rate for heat and
food of one rat before and after injection of quinine HC1 (50
fall in temperature (Bruinvels, 1970, 1973; Satinoff 8c
mg/kg) and saline. The pen resets to base line every 15 min. Cantor, 1975), a rise (Myers 8c Yaksh, 1963), or a fall
168 THERMOREGULATORY BEHAVIOR

followed by a rise (Feldberg 8c Lotti, 1967). To com¬ creased responding to escape heat. In this case behavior
pound the problem, intrahypothalamic and intra¬ compensated for the lowering of internal temperature.
ventricular injections sometimes lead to opposite The drug effect is therefore not a change in the set
effects. Furthermore, acetylcholine (ACh) has potent point.6 We would not expect the fall in body tempera¬
thermoactive properties which vary with the species ture to be the result of an integrated thermoregula¬
(Brimblecombe, 1973). Histamine also affects body tory process. Although there were no measures of
temperature (Brezenoff 8c Lomax, 1970), as does dopa¬ reflexive responses in these experiments, Satinoff and
mine (Hansen 8c Whishaw, 1973; Yehuda 8c Wurtman, Cantor (1975) reported similar results after intraven¬
1972). As things stand now, no one can predict how a tricular injections of NE: body temperature fell and
particular transmitter will affect thermoregulation in the rats compensated in the cold by increasing lever
an untested species. pressing for radiant heat. This was not a general ac¬
Most of the experiments on transmitter substances, tivating effect of the NE, because when it was injected
temperature regulation, and instrumental behavior in a warm environment and body temperature fell,
have been done with rats. We shall use this work to the rats did not increase responding to escape heat.
illustrate how operant measures can clear up confu¬ When these authors later examined how the fall was
sion about pharmacological mechanisms. Our analysis brought about, they found that at normal room tem¬
rests on the assertions made on page 165: when set perature it was caused by an immediate and intense
point is displaced, animals will work to bring actual peripheral vasodilation. Metabolic rate actually in¬
body temperature as close as possible to the new level; creased quickly thereafter as if to compensate for the
when body temperature is altered without a set point fall (Cantor 8c Satinoff, 1976). Wherever intraventricu¬
change, behavior will be compensatory. lar NE is acting, it is not shifting the set point.
We can tentatively conclude that, in rats, NE in¬
Norepinephrine jected in the preoptic area shifts the set point upward
by activating both operant and reflexive mechanisms
In a neutral environment (25°C) preoptic in¬ leading to augmented heat production and decreased
jection of low doses of NE (.05-15 fig) raised heat loss. Any hypothermic effects of NE, either after
hypothalamic temperature. The same effect occurred intraventricular injection or after injection of agents
in the cold, and the rats increased lever press¬ such as 6-hydroxydopamine, which causes release of
ing for heat. Both of these effects were monotonic endogenous NE (Breese, Moore, 8c Howard, 1972;
functions of the concentration of NE (Beckman, 1970). Hansen 8c Whishaw, 1973; Nakamura 8c Thoenen,
The operant augmented the rise in internal tempera¬ 1971; Simmonds 8c Uretsky, 1970), are apparently
ture, which suggests that the set point had been raised. caused by action either on a controller that activates
(However, before this conclusion can be reached it effector pathways or on the effector pathways them¬
must be shown that the increased responding for heat selves. We predict that the hypothermia seen after
is a specific thermoregulatory effect and not just a 6-hydroxydopamine injections will be accompanied by
general increase in activity.) If low doses of NE in the increases in heat-reinforced behavior.
preoptic area shift the thermoregulatory set point up¬
ward, then we would expect that all reflexive re¬
Acetylcholine
sponses would be integrated so as to increase heat
production and decrease heat loss. In fact, this is The same reasoning can be applied in interpreting
exactly what happens. The rise in temperature is the effects of other transmitter substances implicated
caused by increased shivering, increased metabolic in thermoregulation. ACh and other cholinomimetic
rate, and vasoconstriction (Satinoff 8c Hackett, 1975). agents lowered body temperature in rats when in¬
Avery (1971; see also Avery 8c Penn, 1973) has re¬ jected intrahypothalamically (Beckman 8c Carlisle,
ported quite different results. In room air (23°C), 1969; Crawshaw, 1973; Kirkpatrick 8c Lomax, 1970).
preoptic injection of high doses of NE (25 fig) lowered
body temperature. Heat escape was the behavioral 6 Similar considerations apply to the analysis of the effects of
measure: as long as the bar was held down, the heat brain stimulation on body temperature. Electrical stimulation of
lamp was off and a cooling fan on. The rats held the the preoptic area while the rats were working for heat in the
cold produced changes in body temperature but did not cause
bar down less postinjection; that is, they allowed the appropriate shifts in operant behavior (Crawshaw & Carlisle,
chamber to stay hot. The body temperature change 1974). Thus although electrical stimulation produced changes in
and the operant behavior moved in opposite direc¬ body temperature, it did not affect motivational aspects of
thermoregulation, and therefore such changes in body tempera¬
tions: body temperature dropped, yet the rats de¬ ture need not be interpreted as changes in thermal set point.
Evelyn SatinofF and Robert Hendersen 169

ACh (50 fig) caused both a fall in brain temperature REFERENCES

and a decrease in rate of working for heat in the cold
(Beckman 8c Carlisle, 1969). Here the behavior and the Adair, E. Control of thermoregulatory behavior by brief
physiological responses are complementary, and if it displacements of hypothalamic temperature. Psycho-

has no general depressant effect on operant respond¬

,
nomic Science 1970, 20, 11-13.
Adair, E. Hypothalamic control of thermoregulatory be¬
ing, we can assume that ACh lowers the set point. havior: Preoptic-posterior hypothalamic interaction. In
On the other hand, carbachol (a cholinomimetic Recent studies of hypothalamic function (international
drug) (8 /xg) increased body temperature and also symposium in Calgary, Alberta.) Basel: Karger, 1974.
increased bar holding to escape heat (Avery 8c Penn, Adair, E. R., & Hardy, J. D. Posterior hypothalamic
thermal stimulation can alter behavioral, but not physio¬
1973). In this case the behavior compensated for the
logical, temperature regulation. Paper presented at In¬
change in body temperature, and therefore this dose ternational Congress of Physiology, Munich, 1971.
of carbachol did not change the set point. Like high Adair, E. R., 8c Wright, B. A. Behavioral thermoregulation
doses of NE, it might have acted directly on effector in the squirrel monkey when response effort is varied.
Journal of Comparative and Physiological Psychology,
pathways (in this case to promote heat loss) or blocked
1976, 90, 179-184.
the appropriate synapses linking the central controller
Avery, D. D. Intrahypothalamic adrenergic and cholinergic
to those pathways. This latter possibility is likely be¬ injection effects on temperature and ingestive behavior in
cause atropine, a cholinergic blocking agent, led to a ,
the rat. Neuropharmacology 1971, 10, 753-763.
rise in body temperature in rats when injected into Avery, D. D., 8c Penn, P. E. Effects of intrahypothalamic
the preoptic area (Kirkpatrick 8c Lomax, 1967). injections of adrenergic and cholinergic substances on
behavioral thermoregulation and associated skin tem¬
perature levels in rats. Pharmacology, Biochemistry, and
Serotonin ,
Behavior, 1973, 1 159-165.
Baldwin, B. A. Behavioural thermoregulation in mice.
This indoleamine lowered body temperature in rats ,
Physiology and Behavior 1968, 2, 401-407.
Baldwin, B. A., 8c Ingram, D. L. The effect of heating and
when injected intraventricularly (Bruinvels, 1970;
cooling the hypothalamus on behavioural thermoregula¬
Feldberg 8c Lotti, 1967; Myers 8c Yaksh, 1968). As with tion in the pig. Journal of Physiology (London), 1967,
NE, intrahypothalamic injections of 5-HT had the ,
191 375-392.
opposite effect, but produced little change in the rate Beckman, A. L. Effect of intrahypothalamic norepineph¬
of bar pressing for heat in the cold (Crawshaw, 1972). rine on thermoregulatory responses in the rat. Amer¬
Crawshaw concluded that 5-HT does not act in the , ,
ican Journal of Physiology 1970, 218 1596—1604.
Beckman, A. L., 8c Carlisle, H. Effect of intrahypothalamic
preoptic area to shift the set point, but rather that its infusion of acetylcholine on behavioral and physiologi¬
effect on body temperature is unspecific. Bruinvels cal thermoregulation m the rat. Nature, 1969, 221 561— ,
(1970), on the basis of pharmacological data, also con¬ 562.
cluded that 5-HT does not act on thermosensitive Bernheim, H. A., Vaughn, L. K., 8c Kluger, M. J. Induc¬
receptors, but instead its hypothermic effect is caused tion of fever in lizards in response to gram-negative
bacteria, Federation Proceedings, 197L ??j 457*
by some unspecific action.
Bogert, C, M, How reptiles regulate their body tempera¬
From all of these experiments we can tentatively ture. Scientific American, 1959, 200, 105-120.
conclude that in rats, NE raises the set point, ACh Bolles, R, C, Species speeihc defense reactions and avoid¬
lowers it, and 5-HT does not affect it. The number of ance learning. Psychological Review, 1970, 77, 32-48,
experiments investigating the action of transmitter Breese, G. R., Moore, R. A., 8c Howard, J, L, Central
actions of 6-hydroxydopamine and other phenylethyla-
substances on both body temperature and operant
mine derivatives on body temperature in the rat. Journal
behavior is still small, and even when such experi¬ ,
of Pharmacology and Experimental Therapeutics 1972,
ments become more numerous everything will not 180, 591-602.
magically fall into place. Some of the species differ¬ Brezenoff, H. E., 8c Lomax, P. Temperature changes fol¬
ences may indeed be real. However, many of the con¬ lowing microinjection of histamine into the thermo¬
regulatory centers of the rat. Experientia, 1970, 26,
flicting results about the action of transmitter sub¬
51-52.
stances on body temperature could be cleared up if Brimblecombe, R. W. Effects of cholinomimetic and choli-
operant measures were used more often to specify nolytic drugs on body temperature. In E. Schonbaum &
whether the set point was being altered. It is fruitless P. Lomax (Eds.), The pharmacology of thermoregula¬
to build neurochemical models of thermoregulation tion. Basil: Karger, 1973.
without knowing if a compound is really changing the Brittain, R. T., 8c Handley, S. L. Temperature changes
produced by the injection of catecholamines and 5-
set point. This can best be done by a careful analysis
hydroxytryptamine into the cerebral ventricles of the
not only of physiological responses, but of operant conscious mouse. Journal of Physiology (London), 1967,
responding as well. 192, 805-813.
170 THERMOREGULATORY BEHAVIOR

Brown, A. C., & Brengelmann, G. L. The interaction of Clark, W. G., 8c Lipton, J. M. Complementary lowering of
peripheral and central inputs in the temperature regula¬ the behavioural and physiological thermoregulatory set-
tion system. In J. D. Hardy, A. P. Gagge, 8c J. A. J. points by tetrodotoxin and saxitoxin in the cat. Journal
Stolwijk (Eds.), Physiological and behavioral temper¬ of Physiology (London), 1974, 238, 181-191.
ature regulation. Springfield, Ill.: C. C. Thomas, 1970. Cooper, K. E., Cranston, W. I., 8c Honour, A. J. Effects of
Bruinvels, J. Effect of noradrenaline, dopamine, and 5- intraventricular and intrahypothalamic injection of nor¬
hydroxytryptamine on body temperature in the rat after adrenaline and 5-hydroxytryptamine on body tempera¬
intracisternal administration. Neuropharmacology, 1970, ture in conscious rabbits. Journal of Physiology (Lon¬
9, 277-282. don), 1965, 181, 852-864.
Bruinvels, J. Effects of intracisternal administration of Corbit, J. D. Behavioral regulation of body temperature.
metaraminol and noradrenaline on body temperature in In J. D. Hardy, A. P. Gagge, 8c J. A. J. Stolwijk (Eds.),
the rat. In E. Schonbaum & P. Lomax (Eds.), The Phar¬ Physiological and behavioral temperature regulation.
macology of thermoregulation. Basil: Karger, 1973. Springfield, Ill.: C. C. Thomas, 1970.
Budgell, P. Behaviour thermoregulation in the Barbary Corbit, J. D., 8c Ernits, T. Specific preference for hypo¬
dove (Streptopelia mom). Animal Behaviour, 1971, 19, thalamic cooling. Journal of Comparative and Physio¬
524—531. logical Psychology, 1974, 86, 24-27.
Cabanac, M. Plaisir ou deplaisir de la sensation thermique Cowles, R. B. Semantics in biothermal studies. Science,
et homeothermie. Physiology and Behavior, 1969, 4, 1962, 135, 670.
359-364. Cowles, R. B., & Bogert, C. M. A preliminary study of the
Cabanac, M. Physiological role of pleasure. Science, 1971. thermal requirements of desert reptiles. Bulletin of the
175, 1103-1107. American Museum of Natural History, 1944, 83, 265-
Cabanac, M., Duclaux, R., & Gillet, A. Thermoregula¬ 296.
tion comportementale chez le chien: Effets de la fievre Crawshaw, L. I. Effects of intracerebral 5-hydroxytrypta-
et de la thyroxine. Physiology and Behavior, 1970, 5, mme injection on thermoregulation in the rat. Physi¬
697-704. ology and Behavior, 1972, 9, 133-140.
Cabanac, M., Hammel, H. T., 8c Hardy, J. D. Tiliqua Crawshaw, L. I. Effect of intracranial acetylcholine injec¬
scincoides: Temperature-sensitive units in lizard brain. tion on thermoregulatory responses in the rat. Journal
Science, 1967, 158, 1050-1051. of Comparative and Physiological Psychology, 1973, 83,
Cabanac, M., 8c Jeddi,E. Thermopreferendum et thermo¬ 32-35.
regulation comportementale chez trois poikilothermes. Crawshaw, L. I., 8c Carlisle, H. J. Thermoregulatory
Physiology and Behavior, 1971, 7, 375-380. effects of electrical brain stimulation. Journal of Com¬
Cabanac, M., & Massonnet, B. Temperature regulation parative and Physiological Psychology, 1974, 87, 440-448.
during fever: Change of set point or change of gain? A Crawshaw, L, I., 8c Hammel, H. T. Behavioral thermo¬
tentative answer from a behavioural study in man. Jour¬ regulation in two species of antarctic fish. Life Sciences,
nal of Physiology (London), 1974, 218, 561-568. 1971, 10, 1009-1020.
Cabanac, M., Massonnet, B.? 8c Belaiche, R, Preferred Crawshaw, L. I„ 8c Hammel, H. T. Behavioral temperature
skin temperature as a function of internal and mean regulation in the California horn shark, Heterodontus
skin temperature. Journal of Applied Physiology, 1972, francisci. Brain, Behavior, and Evolution, 1973, 7, 447-
33, 699-703.
^ 452.
CALLARD, I. P„ Chan, S. W. C., 8c Potts, M. A. The control Crawshaw, L. I., 8c Hammel, H. T. Behavioral regulation
of the reptilian gonad. American Zoologist, 1972, 12, of internal temperature in the brown bullhead, Ictalurus
273-287. nebulosus. Comparative Biochemistry and Physiology,
Cantor, A., & Satinoff, E. Thermoregulatory responses to 1974, 47A, 51-60.
intraventricular norepinephrine in normal and hypo¬ Cunningham, D. J., 8c Cabanac, M. Evidence from be¬
thalamic rats. Brain Research, April 1976, Vol. 106. havioral thermoregulatory responses of a shift in set-
Carlisle, H. J. Behavioural significance of hypothalamic point temperature related to the menstrual cycle. Jour¬
temperature-sensitive cells. Nature, 1966, 209, 1324-1325. nal of Physiology (Paris), 1971, 63, 236-238.
Carlisle, H. J. The effects of preoptic and anterior hypo¬ Davidson, R. S. Operant stimulus control applied to maze
thalamic lesions on behavioral thermoregulation in the behavior: Heat escape conditioning and discrimination
cold. Journal of Comparative and Physiological Psychol¬ reversal in Alligator mississippiensis. Journal of the
ogy, 1969, 69, 391-402. Experimental Analysis of Behavior, 1966, 9, 671-676.
Carlisle, H. J. Thermal reinforcement and temperature DAWES, G. S. Foetal and neonatal physiology. Chicago:
regulation. In W. G. Stebbins (Ed.), Animal psychophys¬ Year Book Medical Publishers, 1968.
ics; The design and conduct of sensory experiments. Duclaux, R., Fantino, M., 8c Cabanac, M. Comportement
Englewood Cliffs, N. J.: Prentice-Hall, Inc., 1970. thermoregulateur chez Rana esculenta. Pfliigefs Ar¬
Carlisle, H. J., & Snyder, E. The interaction of hypothal¬ chives, 1973, 342, 347-358.
amic self-stimulation and temperature regulation. Experi- Edinger, H. M., 8c Eisenman, J. S. Thermosensitive neurons
entia, 1970, 26, 1092-1093. in tuberal and posterior hypothalamus of cats. American
Carlton, P. L., 8c Marx, R. A. Heat as a reinforcement for Journal of Physiology, 1970, 219, 1098-1103.
operant behavior. United States Army Medical Research Eisenman, J. S., 8c Jackson, D. C. Thermal response pat¬
Laboratory Technical Report No. 229. Fort Knox, 1957. terns of septal and preoptic neurons in cats. Experi¬
Clark, W. G., 8c Coldwell, B. A. The hypothermic effect mental Neurology, 1967, 19, 33-45.
of tetrodotoxin in the unanaesthetized cat. Journal of Epstein, A. N., 8c Milestone, R. Showering as a coolant for
Physiology (London), 1973, 230, 477-492. rats exposed to heat. Science, 1968, 160, 895-896.
Evelyn Satinoff and Robert Hendersen 171

Fekety, F. R., Jr. Heat balance and reactivity to endotoxin. Hansen, M. G., 8c Whishaw, I. Q. The effects of 6-hydroxy-
American Journal of Physiology, 1963, 204, 719-722. dopamine, dopamine, and ^/-norepinephrine on food
Feldberg, W., Hellon, R. F., 8c Lotti, V. J. Temperature intake and water consumption, self-stimulation, temper¬
effects produced in dogs and monkeys by injection of ature, and electroencephalographic activity in the rat.
monoamines and related substances into the third ven¬ Psychopharmacology, 1973, 29, 33-44.
tricle. Journal of Physiology (London), 1967, 191, 501— Harlow, H. Learning to love. San Francisco: Albion, 1971.
515. Heath, J. E. Temperature-independent morning emer¬
Feldberg, W., 8c Lotti, V. J. Temperature responses to gence in lizards of the genus Phrynosoma. Science, 1962,
monoamines and an inhibitor of MAO injected into the 138, 891-892.
cerebral ventricles of rats. British Journal of Pharma¬ Heath, J. E. Temperature regulation and diurnal activity
cology and Chemotherapy, 1967, 31, 152-161. in horned lizards. University of California Publications
Feldberg, W., 8c Myers, R. D. A new concept of tempera¬ in Zoology, 64, No. 3, 97-136. Published in 1965 by The
ture regulation by amines in the hypothalamus. Nature, Regents of the University of California.
1963, 200, 1325. Heath, J. E. Behavioral regulation of body temperature in
Fraenkel, G. S., 8c Gunn, D. L. The orientation of animals. poikilotherms. Physiologist, 1970, 13, 399-410.
New York: Dover, 1961. Hellstrom, B., 8c Hammel, H. T. Some characteristics of
Freeman, M. E., Crissman, J. K., Louw, G. N., Butcher, temperature regulation in the unanesthetized dog. Amer¬
R. L., 8c Inskeep, E. K. Thermogenic action of pro¬ ican Journal of Physiology, 1967, 213, 547-556.
gesterone in the rat. Endocrinology, 1970, 86, 717-720. Hensel, H. Neural processes in thermoregulation. Physio¬
Freeman, W. J., 8c Davis, D. D. Effects on cats of conduc¬ logical Review, 1974, 53, 948-1017.
tive hypothalamic cooling. American Journal of Physi¬ Hogan, J. A. Responses in Pavlovian conditioning studies.
ology, 1959, 197, 145-148. Science, 1974, 186, 156-157.
Fry, F. E. J., 8c Hochachka, P. W. Fish. In G. C. Whittow Huey, R. B. Behavioral thermoregulation in lizards: Impor¬
(Ed.), Comparative physiology of thermoregulation, Vol. tance of associated costs. Science, 1974, 184, 1001-1003.
1: Invertebrates and nonmammalian vertebrates. New Jacob, J& Peindaries, R. Central effects of monoamines
York: Academic Press, 1970. on the temperature of the conscious rabbit. In E. Schon-
Gale, C. C., Mathews, M., 8c Young, J. Behavioral thermo¬ baum 8c P. Lomax (Eds.), The pharmacology of thermo¬
regulatory responses to hypothalamic cooling and warm¬ regulation. Basel; Karger, 1973.
ing in baboons. Physiology and Behavior, 1970, 5, 1—6. Jeddi, E. Confort du contact et thermoregulation compor-
Garrick, L. D. Internal and external environmental in¬ tementale. Physiology and Behavior, 1970, 5, 1487-1493.
fluences on thermoregulation in two Ikard species. Un¬ Jeddi, E. Thermoregulatory efficiency of neonatal rabbit
published doctoral dissertation, Rutgers University, 1973. search for fur comfort contact. International Journal of
Garrick, L. D. Reproductive influences on behavioral Biometeovology, 1971, 15, 337-341.
thermoregulation in the lizard, Sceloporus cyanogenys. KELLER, A. D. Temperature regulation disturbances m dogs
Physiology and Behavior, 1974, 12, 85-91. following hypothalamic ablations. In J. D. Hardy (Ed.),
Gordon, M. S. Animal physiology: Principles and adapta¬ Temperature: Its measurement and control in science
tions (2nd ed.). New York: Macmillan, 1972. and industry (Vol. 3). New York: Reinhold, 1963.
Greer, G. L., 8c Gardner, D. R. Temperature-sensitive Kemp, F. D. Thermoregulatory operant behavior in the
neurons in the brain of brook trout. Science, 1970, 169. lizard Dipsosaurus dorsalis as a function of body temper¬
1220-1222. ature, substrate temperature, and heating rate. Journal
Grossman, S. P., 8c Reghtschaffen, A. Variations in brain of Biological Psychology, 1969, 11, 36-39.
temperature in relation to food intake. Physiology and Kirkpatrick, W. E., k Lomax, P. The effect of atropine
Behavior, 1967, 2, 379—383. on the body temperature of the rat following systemic
Hamilton, C. L. Interactions of food intake and tempera¬ and intracerebral injection. Life Sciences, 1967, 6, 2273-
ture regulation in the rat. Journal of Comparative and 2278.
Physiological Psychology, 1963, 36, 476-488. Kirkpatrick, w. E., & Lomax, p. Temperature changes
Hamilton, C. L., 8c Sheriff, W„ Jr, Thermal behavior of following iontophoretic injection of acetylcholine into
the rat before and after feeding. Proceedings of the the rostral hypothalamus of the rat. Neuropharmacology,
Society of Experimental Biology and Medicine, 1959, 1970, 9, 195-202.
102, 746-748. Koelle, G. B. Anticholinesterase agents. In L. S. Good¬
Hammel, H. T., Caldwell, F. T., Jr., 8c Abrams, R. M. man & A. Gilman (Eds.), The Pharmacological basis of
Regulation of body temperature in the blue-tongued therapeutics. New York: Macmillan, 1970.
lizard. Science, 1967, 156, 1260-1262. Laties, V. G. Effects of d-amphetamine on concurrent
Hammel, H. T„ Hardy, J. D., 8c Fusco, M. M. Thermo¬ schedules of heat and food reinforcement. Journal of
regulatory responses to hypothalamic cooling in unan¬ Physiology (Paris), 1971, 63, 315-318.
esthetized dogs. American Journal of Physiology, 1960, Laties, V. G., 8c Weiss, B. Thyroid state and working for
198, 481-486. heat in the cold. American Journal of Physiology, 1959,
Hammel, H. T., Stromme, S. B., 8c Myhre, K. Forebrain 197, 1028-1034.
temperature activates behavioral thermoregulatory re¬ Leonard, C. M. Thermotaxis in golden hamster pups.
sponses in arctic sculpins. Science, 1969, 165, 83-85. Journal of Comparative and Physiological Psychology,
Handley, S. L., 8c Spencer, P. S. J. Thermoregulatory 1974, 86, 458-469.
effects of intraventricular injection of noradrenaline in Lipton, J. M. Effects of preoptic lesions on heat-escape
the mouse and the influence of ambient temperature. responding and colonic temperature in the rat. Physi¬
Journal of Physiology (London), 1972, 223, 619-631. ology and Behavior, 1968, 3, 165-169.
172 THERMOREGULATORY BEHAVIOR

Lipton, J. M. Effects of high-fat diets on caloric intake, Pliskoff, S. S., Wright, J. E., 8c Hawkins, T. D. Brain
body weight, and heat-escape responses in normal and stimulation as a reinforcer: Intermittent schedules. Jour¬
hyperphagic rats. Journal of Comparative and Physio¬ nal of the Experimental Analysis of Behavior, 1965, 8,
logical Psychology, 1969, 68, 507-515. 75-88.
Lomax, P. Drugs and body temperature. International Re¬ Polk, D. L., 8c Lipton, J. M. Effects of sodium salicylate,
view of Neurobiology, 1970, 12, 1-43. aminopyrine, and chlorpromazine on behavioral tem¬
Magoun, H. W., Harrison, F., Brobeck, J. R., 8c Ranson, perature regulation. Pharmacology, Biochemistry, and
S. W. Activation of heat loss mechanisms by local heat¬ Behavior, 1975, 3, 167-172.
ing of the brain. Journal of Neurophysiology, 1938, 1, Proppe, D. W., 8c Gale, C. C. Endocrine thermoregulatory
101-114. responses to local hypothalamic warming in unanesthe¬
Matthews, T. J. A convective thermal controller for be¬ tized baboons. American Journal of Physiology, 1970,
havioral experiments. Behavioral Research and Method¬ 219, 202-207.
ological Instrumentation, 1969,1, 126-128. Regal, P. J. Thermophilic response following feeding in
Matthews, T. J., Morin, L. P., & Church, R. M. Avoid¬ certain reptiles. Copeia, 1966, 3, 588-590.
ance of thermal stimuli in the rat. Psychonomic Science, Regal, P. J. Voluntary hypothermia in reptiles. Science,
1971, 22, 59-60. 1967,133, 1551-1553.
McGinty, D. J. Somnolence, recovery, and hyposomnia Revusky, S. H. Cold acclimatization in hairless mice mea¬
following ventro medial diencephalic lesions in the rat. sured by behavioral thermoregulation. Psychonomic
Electroencephalography and Clinical Neurophysiology, Science, 1966, 6, 209-210.
1969, 26, 70-79, Roberts, W. W., Mooney, R. D. 8c Martin, J. R. Thermo¬
McLean, J. H., 8c Coleman, W. P. Temperature variation regulatory behaviors of laboratory rodents. Journal of
during the estrous cycle; Active vs. restricted rats. Psy¬ Comparative and Physiological Psychology, 1974, 84,
chonomic Science, 1971, 22, 179-180. 693-699.
Milsum, J. H. Biological control systems analysis. New ROZIN, P. The use of poikilothermy in the analysis of
York. McGraw-Hill, 1966. behavior. In D. Ingle (Ed.), The central nervous system
Morishima, M. S., & Gale, C. C. Relationship of blood and psh behavior. Chicago: University of Chicago
pressure and heart rate to body temperature in baboons. Press, 1968.
American Journal of Physiology, 1972, 223, 387-395. Rozin, P.,8c Mayer, J. Thermal reinforcement and thermo¬
Mount. L. E. Environmental temperature preferred by the regulatory behavior in the goldfish, Garassius auratus.
young pig. Nature, 1963, 199, 1212-1213- Science, 1961, 134, 942-943.
MURGATROYD, D., & Hariw, J. D. Central and peripheral Satinoff, E, Behavioral thermoregulation in response to
temperatures in behavioral thermoregulation of the rat. local cooling of the rat brain. American Journal of
In J. D. Hardy, A. P. Gagge, 8c J. A. J. Stolwijk (Eds,), Physiology, 1964, 206, 1389-1394.
Physiological and behavioral temperature regulation, Satinoff, E. Neural integration of thermoregulatory re¬
Springfield, Ill.! C. C. Thomas, 1970, sponses. In L. V. DiCara (Ed.), Limbic and autonomic
Myers, R. D.? 8c Yaksh, T, L. Feeding and temperature nervous system: Advances in research. New York: Plenum
responses in the unrestrained rat after injections of Press, 1974.
cholinergic and aminergic substances into the cerebral Satinoff, E., & Cantor, A. Intraventricular norepinephrine
ventricles. Physiology and Behavior, 1968, 3, 917-928. and thermoregulation in rats. In P. Lomax, E. Schon-
Myers, R. D., 8c Yaksh, T. L. Control of body temperature baum, 8c J. Jacob (Eds.), Temperature regulation and
in the unanesthetized monkey by cholinergic and ami¬ drug action. Basil: Karger, 1975.
nergic systems in the hypothalamus. Journal of Physi¬ Satinoff, E„ 8c Hackett, E. R. Reflexive thermoregulatory
ology (London), 1969, 202, 483-500.
responses after intrahypothalamic injection of norep¬
MYHRE, K., &! Hammel, H. T. Behavioral regulation of in¬ inephrine in rats. Neuroscience Abstracts, 1975, 1, 416.
ternal temperature in the lizard Tiliqua scincoides. Satinoff, E., 8c Rutstein, J. Behavioral thermoregulation
American Journal of Physiology, 1969, 217, 1490-1495. in rats with anterior hypothalamic lesions. Journal of
Nakamura, K., 8c Thoenen, H. Hypothermia induced by Comparative and Physiological Pyschology, 1970, 71, 77-
intraventricular administration of 6 hydroxydopamine 82.
in rats. European Journal of Pharmacology, 1971, 16, Satinoff, E., 8c Shan, S. Y. Y. Loss of behavioral thermo¬
46-54.
regulation after lateral hypothalamic lesions in rats.
Neill, W. H., Magnuson, J. J., & Chipman, G. D. Be¬ Journal of Comparative and Physiological Psychology,
havioral thermoregulation by fishes: A new experimental 1971, 77, 302-312.
approach, Science, 1972, 176, 1443-1445. Sidman, M. Avoidance conditioning with brief shock and
Nickerson, M. Drugs inhibiting adrenergic nerves and no exteroceptive warning signal. Science, 1953, 118,
structures innervated by them. In L. S. Goodman 8c A. 157-158.
Gilman (Eds.), The pharmacological basis of therapeu¬ Simmonds, M. A., 8c Uretsky, N. J. Central effects of 6-
tics. New York: Macmillan, 1970. hydroxydopamine on the body temperature of the rat.
Ogilvie, D. M., 8c Stinson, R. H. The effect of age on British Journal of Pharmacology, 1970, 40, 630-638.
temperature selection by laboratory mice (Mus muscu- Squires, R. D., 8c Jacobson, F. H. Chronic deficits of tem¬
lus). Canadian Journal of Zoology, 1966, 44, 511-517. perature regulation produced in cats by preoptic lesions.
Pachomov, N. The effects of posterior and anterior hypo¬ American Journal of Physiology, 1968, 214, 549-560.
thalamic lesions on the maintenance of body tempera¬ Stein, L. Reciprocal action of reward and punishment
ture in the rat. Journal of Neuropathology and Experi¬ mechanisms. In R. B. Heath (Ed.), The role of pleasure
mental Neurology, 1962, 21, 450-460. in behavior. New York: Harper 8c Row, 1964.
Evelyn Satinoff and Robert Hendersen 173

Teague, R. S., 8c Ranson, S. W. The role of the anterior Weiss, B. Thermal behavior of the subnourished and panto¬
hypothalamus in temperature regulation. American thenic acid-deprived rat. Journal of Comparative and
Journal of Physiology, 1936, 117, 562-570. Physiological Psychology, 1957, 50, 481-485.
Teitelbaum, P., 8c Epstein, A. N. The lateral hypothalamic Weiss, B., 8c Laties, V. G. Behavioral thermoregulation.

syndrome: Recovery of feeding and drinking after lat¬ Science, 1961, 133, 1338-1344.
eral hypothalamic lesions. Psychological Review, 1962, Weiss, B., 8c Laties, V. G. Effects of amphetamine, chlor-
69, 74-90. promazine, and pentobarbital on behavioral thermo¬
Templeton, J. R. Reptiles. In G. C. Whittow (Ed.), Com¬ regulation. Journal of Pharmacology and Experimental
parative physiology of thermoregulation, Vol. 1: Inverte¬ Therapeutics, 1963, 140, 1-7.
brates and nonmammalian vertebrates. New York: Weiss, B., Laties, V. G., 8c Weiss, A. B. Behavioral thermo¬
Academic Press, 1970. regulation by cats with pyrogen-induced fever. Archives
Van Sommers, P. The biology of behaviour. New York: of International Pharmacodynamics, 1967, 165, 467-475.
Wiley, 1972. Yeh, S. D. J., 8c Weiss, B. Behavioral thermoregulation dur¬

Wade, G. N. Gonadal hormones and behavioral regulation ing vitamin Be deficiency. American Journal of Physi¬
of body weight. Physiology and Behavior, 1972, 8, 523- ology, 1963, 205, 857-862.
534. Yehuda, S., 8c Wurtman, R. J. The effects of d-ampheta-
Wampler, R. S. Changes in sleep and arousal accompany¬ mine and related drugs on colonic temperatures of rats
ing the lateral hypothalamic syndrome in rats. Unpub¬ kept at various ambient temperatures. Life Sciences,
lished doctoral dissertation. University of Pennsylvania, 1972, 11, 851-859.
1970. Zolman, J. F. Discrimination learning in the young chick
Wasserman, E. A. Pavlovian conditioning with heat rein¬ with heat reinforcement. Psychological Record, 1968, 18,
forcement produces stimulus-directed pecking in chicks. 303-309.
Science, 1973, 181, 875-877.
 7

Determinants of
Reinforcement
and Punishment*

W. H. Morse
and
R. T. Kelleher

REPRODUCIBLE BEHAVIORAL PROCESSES In the early 1930s B. F. Skinner developed tech¬

niques for the experimental study of behavior. An es¬
The scientific study of behavior poses many diffi- sential feature in his approach to behavior was the
culties. One difficulty results, paradoxically, from our emphasis on the rate of occurrence of some identi¬
familiarity with numerous isolated facts about the be¬ fiable “response” as a significant property of behavior.
havior of ourselves, other people, and animals. The Techniques for studying reflexly elicited behavior had
interpretations customarily given to these facts lead already been developed, but it is not possible to
to preconceived opinions, which frequently interfere identify an eliciting stimulus for much of the be¬
with the unbiased study of behavior. Moreover, be¬ havior of an individual that can be predicted and
havior is essentially dynamic in the sense that behav¬ controlled. To say that behavior occurs in the absence
ioral processes reflect changes in the interactions be¬ of an identifiable eliciting stimulus does not imply
tween an individual and his environment which take that the behavior is not determined, but simply that
place in time. Even the simplest relationships may not it does not have the functional properties of reflexly
be readily apparent to casual observations at any elicited behavior. For example, a food-deprived rat
moment. Finally, a pattern of behavior is the result of given access to a supply of small food pellets will eat
many interrelated factors, including environmental for a period of time and then cease. If the rate of in¬
circumstances that have long since ceased to exist, gesting pellets is recorded, a simple and reproducible
thus posing special problems for identification and curve of eating is obtained that describes the ingestion
study. of food under these conditions (Skinner, 1930, 1938).
Although the rate of eating following food depriva¬
* The preparation of this chapter was supported by U.S.
tion is a reproducible temporal process, it is not pos¬
Public Health Service Grants MH 07658, MH 02094, DA 00499,
and RR 00168 and by Research Career Program Award sible to analyze this behavior simply in terms of
1-K5-MH22589 (RTK). We thank Mrs. Patricia Lavin for help momentary eliciting stimuli. Because the presentation
in preparation of the manuscript and Drs. P. B. Dews, J. W.
McKearney, and J. B. Smith for helpful comments about the
of food is often immediately followed by ingestion, it
manuscript. may seem that the ingestion of food is elicited by the

174
W. H. Morse and R. T. Kelleher 175

presence of food itself (presumably its sight and smell). the likelihood that a similar response will occur again
But the rate of eating declines in time; the presence of after the food is eaten is increased, and further re¬
food does not continue to have the same preemptive¬ sponses will occur with a characteristic temporal pat¬
ness. Thus it is necessary to invoke some other factors, terning. If behavior is so altered by the presentation
such as habituation, fatigue, adaptation, or depriva¬ of food, the conditions under which food presentation
tion, operating in conjunction with the sight and smell occurs—in this example, the depression of the lever—
of food. The deprivation of food critically determines define the property with respect to which responses
rate of eating and also changes other classes of re¬ are called similar. Skinner (1937; 1938; 1953, p. 66)
sponses in a reproducible way, but it is not an elicit¬ uses the term operant to describe this functionally
ing stimulus in the sense in which the term is used in identifiable class and calls the change in the frequency
reflex physiology. of the operant the process of operant conditioning.
The occurrence of emitted behavior generally bears Because subsequent behavior is altered under these
a temporal relation to the deprivation and presenta¬ conditions, the food is said to be a reinforcer and pre¬
tion of particular environmental conditions, whether senting food in a specified relation to an operant is
or not the deprivation produces any conspicuous reinforcement.1
physiological change. For example, if a rat is confined If only depressions of the lever exceeding a certain
in a small space and then given access to a revolving force are followed by food presentation (differen¬
wheel, it will run for some time and then gradually tial reinforcement), weaker responses diminish and
cease. As in the example of the rat eating, a record in stronger responses become more frequent. Even
time of the running behavior will reveal that the stronger responses can be selected through further
running has a characteristic temporal pattern. After progressive differential reinforcement. It should be
the rat is deprived of access to the wheel, the avail¬ noted that merely specifying relations between re¬
ability of the wheel is closely followed by running, sponses and consequent stimuli may not specify a
yet it is not useful to regard the wheel as an eliciting functional class of responses that could be called op¬
stimulus for running. In studying the occurrence of erant. “No property is a valid defining property of a
such behavior in time, it is clearly desirable that par¬ class until its experimental reality has been demon¬
ticular instances be easily identified, reproducible, strated, and this rule excludes a great many terms
and functionally significant. The criterion for specify¬ commonly brought into the description of behavior''
ing functionally significant emitted responses as op (Skinner, 1938, p. 41; 1969), Yet it is important to
erants will be taken up later in this section. recognize that these broad principles do apply beyond
Behavioral phenomena that have an identifiable experimental situations that can be precisely de¬
temporal pattern under specified conditions and which scribed. In his recent writings, Skinner has used the
are reproducible in different individuals may be de¬ term contingencies of reinforcement to refer to the
scribed as reproducible behavioral processes (Zimmer¬ interrelations between antecedent behavior and con¬
man, 1963). An understanding of such reproducible sequent events that define operants (Skinner, 1969,
behavioral processes is to be found in the exact charac¬ pp. 7, 127).
terization of the temporal relations among the wents The examples above are important because basic
comprising such processes and in the specification of concepts applicable to the formulation of behavior as
the conditions under which they occur. The present a scientific system were developed in this situation, It
chapter will discuss reinforcement and punishment in is clear from Skinner’s experimental reports that the
the context of reproducible behavioral processes. In basic data m the example of “conditioning” and in
many respects, reinforcement and punishment are the earlier one of “changes in hunger” (food depriva¬
analogous if not equivalent processes; therefore, con¬ tion) were the orderly changes in rate of responding.
siderations pertaining to the one will usually apply to
the other. “Conditioning” . . . and “a change in hunger”
Future behavior is mainly determined by the conse¬ differ as processes only with respect to the con¬
quences of past behavior. How behavior is changed ditions under which they are observed. The
by experience can be demonstrated as a reproducible thing changing (the observed aspect of behavior)
behavioral process with a food-deprived rat in an is the same in both. (Skinner, 1932, p. 276)
apparatus containing a food dispenser and a lever i Skinner uses reinforcement to refer to the specifiable con¬
projecting from the wall. Any response of the rat that ditions in the environment that give rise to operant conditioning.
depresses the lever will be followed by the presenta¬ In this chapter the single term reinforcement will be used to
refer to both the orderly change in behavior and its environ¬
tion and ingestion of food. Under such circumstances. mental determinants.
176 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

The change in behavior under the specified condi¬ effects of consequent events and response-produced
tions that identifies the food as a reinforcer and identi¬ shocks as consequent events maintaining behavior).
fies lever pressing as an operant is a reproducible Thus the status of consequent events defined as “rein-
behavioral process. forcers” or “punishers” in one context may be changed
A distinction has been made between operations as when they are scheduled differently. This raises im¬
experimental procedures that are imposed by the portant questions about the fundamental concepts
environment and processes as the behavioral efiects of applicable to a scientific formulation of behavior and
these procedures (Catania, 1973, p. 33; Ferster & Skin¬ about the generality of the concepts of “reinforcers”
ner, 1957, p. 730). Although reinforcement is often and “punishers.” The defining characteristics of rein¬
described as a relation or operation (the presentation forcers and punishers do not encompass all the effects
of a reinforcer in a specified temporal relation to an of such stimuli on behavior. How consequent events
operant), it is clear that the operation of reinforce¬ modify behavior is to be understood in both the
ment (or punishment) has a behavioral effect implicit development and the maintenance of subsequent be¬
in its meaning. Behavioral processes are best viewed havior.
as orderly changes in time and need not imply inter¬ The description of reinforcement and punishment
vening mechanistic principles. The terms reinforce¬ as reproducible behavioral processes differs from the
ment and punishment are used here to refer to the usual description of these terms as operations (see
reproducible changes in behavior resulting from the especially Catania, 1968, 1969). A discussion of the
experience of the individual under certain specified differences is instructive in clarifying the precise
conditions. The connotations of these terms include usage of terms. In common usage the terms reinforcer
both a temporal sequence of behavior and the condi¬ and punisher are emphasized as basic terms, while
tions under which this behavior occurs. reinforcement and punishment are defined as the
Since the time of the first reports by Skinner, many presentation of a reinforcer or punisher in a specified
orderly changes in rates of responding under other temporal relation to an operant. The increased occur¬
specified conditions have been described (see especially rence of responses similar to one that immediately
Ferster & Skinner, 1957; Skinner, 1938). Among the preceded some event identifies that event as a rein¬
most important specifications are the schedules de¬ forcer. A punisher is defined in an analogous way: the
scribing the arrangements for initiating and termi¬ decreased occurrence of responses similar to one that
nating stimuli in time and in relation to specified immediately preceded some event identifies that event
responses. Such schedules engender changes in be¬ as a punisher. Reinforcers and punishers, as environ¬
havior with characteristic temporal properties and mental “things,” appear to have a greater reality than
rates of responding that are consistent in different orderly temporal changes in ongoing behavior. Such
individuals. Hence it is appropriate to view schedule- a view is deceptive. There is no concept that pre¬
controlled performances as reproducible behavioral dicts reliably when events will be reinforcers or
processes.2 This conception differs from the usual ap¬ punishers; the defining characteristics of reinforcers
proach of analyzing schedule performances as special and punishers are how they change behavior. Events
consequences of reinforcement (or punishment). that increase or decrease the subsequent occurrence
While the latter approach has the advantage of limit¬ of one response may not modify other responses in the
ing the number of basic behavioral processes to two same way. The modification of behavior by a rein¬
fundamental cases, some schedule-controlled patterns forcer or by a punisher depends not only upon the
are more sensitive dependent variables in revealing occurrence of a certain kind of consequent environ¬
how behavior is modified by environmental conditions mental event but also upon the qualitative and quan¬
than the changes in level of responding that are titative properties of the ongoing behavior preceding
usually used to define reinforcement or punishment, the event and upon the schedule under which the
In particular, the effects of consequent stimuli can event is presented.
be greatly changed, depending upon how they are In characterizing reinforcement as the presentation
scheduled (see this chapter’s sections on disparate of a reinforcer contingent upon a response, the ten¬
dency is to emphasize the event and to ignore the im¬
2 Tfie characterization of different schedule performances as
behavioral processes and the specifications of the conditions portance of both the contingent relations and the
under which these performances occur are beyond the scope of antecedent and subsequent behavior. It is how they
this chapter (see Ferster & Skinner, 1957; Morse, 1966; Skinner, change behavior that defines the terms reinforcer and
1966; Zeiler, chapter 8 in this volume). The present chapter
emphasizes the importance of basing behavioral concepts on punisher; thus it is the orderly change in behavior
orderly reproducible changes in behavior. that is the key to these definitions. It is not appro-
W. H. Morse and R. T. Kelleher 177

priate to presume that particular environmental differential reinforcement, it is useful to consider the
events such as the presentation of food or electric shaping of responses commonly studied in experi¬
shock are reinforcers or punishers until a change in mental situations. Knowing the appropriate condi¬
rate of responding has occurred when the event is tions for using a reinforcer, one can give a general
scheduled in relation to specified responses. There is specification for shaping operant behavior with it:
little value in naming only one of the conditions select a response with vector properties, follow the oc¬
necessary for the change. Identifying an event as a currence of a particular magnitude of this response
reinforcer or punisher independently of the conditions one or more times with the reinforcer, then withhold
of use has limited predictive utility. On the other it until the response magnitude exceeds the value
hand, prior identification of the suitable conditions previously reinforced, and reinforce this greater mag¬
that will result in the same behavioral process does nitude. Thus by making the presentation of rein¬
provide generality. The purpose of the present chap¬ forcers intermittent and dependent upon some pro¬
ter is to give perspective to basic concepts used in the gressively changing property of behavior, one can
experimental study of behavior by discussing some of shape behavior toward some ultimate specification
the determinants of reinforcement and punishment. through successive approximations. By the shaping of
behavior, one can develop new forms of behavior that
could not exist without an explicit history of differ¬
THE CONTINUITY OF BEHAVIOR IN ential reinforcement. Nevertheless, important aspects
TIME (SHAPING) of the shaping process are still unknown. Different
responses vary in stereotypy, rate of occurrence, dis¬
Because reinforcement and punishment are be¬ creteness of identification, and the extent to which
havioral processes occurring in time, they can only be they are changed by consequent events. A consequent
understood in the temporal context of sequential event can be more easily used to increase the fre¬
interactions of behavior with the environment.3 Em¬ quency of occurrence of an operant with a low initial
phasizing “reinforcers” and “punishers” as primary frequency than one with a high initial frequency. Be¬
events neglects the importance of both antecedent and cause the presentation of a reinforcer tends to enhance
subsequent behavior. Operant behavior is determined behavior, it is easier to shape a response involving
mainly by the consequences of past behavior—not so some discrete activity than a response involving sus¬
much the particular consequences, but their sequence tained immobility. In fact, it may be difficult to shape
in time and in relation to the individual’s behavior. an operant involving little or no movement, such as
The scheduling of events is critically important. The “holding” or “standing still” (Blough, 1958). Yet
outstanding characteristic of operant behavior is that operants come under the schedule control of conse¬
it can be differentiated in form and in temporal pat¬ quent events even when their average rate is refractory
terning by consequent events. Conditioned operant to change (Skinner 8c Morse, 1958).
behavior emerges from existing behavior through suc¬ The importance of the shaping sequence is recog¬
cessive approximations to new and more complex nized m cases where the final form of behavior did not
forms of behavior by the process of successive differen¬ occur initially, which is the situation usually described
tial reinforcement (shaping). Behavior that has be¬ to illustrate the principle of shaping. Under such
come highly differentiated can be understood and ac¬ circumstances it is clear that transitional behavior is
counted for only in terms of the history under which essential in developing the final behavior. It is less
the behavior was shaped by different consequences. often recognized that an individual’s past experience-
This schedule gives the exact historical specification how his behavior has been shaped—is usually a de¬
of the temporal and sequential relations between terminant of his subsequent behavior. Even in dealing
environmental events and behavior. with repetitive responses such as a rat pressing a lever
One purpose of this chapter is to emphasize how or a pigeon pecking at a disc, the quantitative effect
present and future behavior depends upon the sequen¬ of presenting a particular event after an instance of
tial ordering of behavior. Because it is generally such a response depends on the subject’s history. For
understood that behavior can be shaped by successive example, when food is presented after every 100 re¬
sponses under a fixed-ratio schedule, responding may
3 Perhaps shaping the suppression of behavior with punish¬ be well maintained in a subject with a history of
ment is not exactly analogous to shaping with reinforcement; responding under this or other schedules but not in a
little is known about the former. The work of Azrin (1960)
clearly indicates that punishment depends upon sequential re¬
subject without an appropriate history. The effective¬
lations between behavior and the environment. ness of an event in maintaining a sequential pattern
178 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

of responding depends on the ongoing pattern of re¬ followed by food presentation. Under these conditions
sponding itself, which in turn depends on the subject’s the subject’s behavior changes in a predictable way.
experimental history. These topics will be taken up The development of standardized equipment and the
in following sections. use of standardized procedures has increased the like¬
lihood that any experimenter can reproduce such re¬
sults. Unfortunately, this success in engendering be¬
DISPARATE EFFECTS OF havior under what are actually special conditions has
CONSEQUENT EVENTS led to uncritical beliefs about reinforcers and
punishers.
A remarkable diversity exists in the physical char¬ Of particular significance are instances in which the
acteristics of events that can reinforce behavior. In¬ same event can under different conditions reliably
cluded among these events are: food, water, sex, elec¬ produce opposite effects on behavior. With events,
tric shock; and intracranial stimulation; changes in such as presentation of food or water, that charac¬
lights, sounds, temperature, and gravity: opportunity teristically lead to further behavior on the part of the
to explore, run, groom, lick a stream of air, play, or individual, such changes in the direction of effect are
fight: and the injection of various drugs. It has been easily missed. If an individual fails to consume food
assumed wrongly that the reinforcing or punishing or water presented under the usual conditions, for
effect of an event is a consistent property of the event example, the possibly suppressing effects of these
itself; the presentation of food after a response has events may not be apparent, Disparate effects of the
been considered an inherently positive event that will same event are most likely to be observed when the
enhance subsequent responding, while the presenta¬ presentation of the event directly affects the indi¬
tion of electric shock after a response has been con¬ vidual. Such changes in the reinforcing (or punishing)
sidered an inherently negative event that will suppress effectiveness of intracranial stimulation, electric shock,
subsequent responding. That food presentation may and drug injections will be discussed below.
not affect responding in an animal that has not been Opposite effects of intracranial stimulation on
deprived of food is usually considered a quantitative lever-pressing responses in the rat have been shown by
variation in the effect of food presentation rather than Steiner, Beer, and Shaffer (1969). In the initial phase
evidence against food presentation having inherent of their study, each lever-pressing response on one of
properties as a reinforcer. But the effects of reinforc¬ two levers (lever S) resulted in electrical stimulation
ing events are not invariant. Even under a given of an area of the hypothalamus; at appropriate
degree of deprivation, the presentation of food to an stimulus parameters, rapid and reliable responding
individual may not have a consistent reinforcing was engendered and maintained on lever S. The pat¬
quality. One may have an aversion to a particular terns of responding and intracranial stimulation were
food as a child, be indifferent to it as an adolescent, tape-recorded. In subsequent phases of the study,
and eat it readily as an adult, It is less widely recog¬ intracranial stimulations of the same intensity were
nized that under appropriate conditions the suppres¬ presented to each rat according to the pattern previ¬
sive effects of electric shock presentation can be re¬ ously recorded for that rat. Responses on lever S were
duced or even converted to an enhancing effect. Such recorded but had no programmed consequences,
disparate effects of consequent events are most likely whereas responses on lever E postponed the scheduling
to occur when there is a history of schedule-controlled of further intracranial stimulations for 20 sec. Under
responding and when there are multiple determinants these conditions, the rates of responding on lever S
of behavior. decreased to near zero, while rates of responding on
In common practice experimenters normally use lever E increased and were stably maintained in sub¬
consequent events that do reliably modify the re¬ sequent sessions. These results show that depending
sponse classes that are being studied and do have upon the circumstances, responding could be main¬
generality from individual to individual. In most tained by either the presentation or the postponement
experiments in which food is used as the reinforcing of the same intracranial stimulation.
consequence, the subject is initially deprived severely Opposite effects of presenting electric shock under
(65 to 80% of free-feeding body weight) and its re¬ different schedules have been studied by Kelleher and
sponse to the presentation of food preempts other Morse (1968a). Squirrel monkeys were trained initially
activity. The presentation of food is made contingent under a variable-interval (VI) schedule of food presen¬
upon a simple response, such as pressing a key for a tation that maintained a steady rate of responding.
rat or monkey, that occurs infrequently before it is Then a 10-min fixed-interval (FI) schedule of electric
W. H. Morse and R. T. Kelleher 179

S-43

Fig. 1. Alternate periods of maintenance and suppression of responding by different

schedules of electric shock presentation in a squirrel monkey. Ordinate—cumulative
number of responses; abscissa—time. Electric shock presentations (12.6 mA) are marked
by short diagonal strokes on the cumulative record and the event record. The recording
pen reset to the base line at the end of each 11-min cycle. The paper did not move during
the 1-min time-out period at the end of each cycle. During the first 10 min of each cycle,
positively accelerated responding, characteristic of performance under fixed-interval (FI)
schedules, was maintained; during the last min of each cycle, in which each response
produced an electric shock, responding was suppressed. (From Morse & Kelleher, 1970.)

shock presentation was superimposed upon the sched¬ well maintained with nalorphine injections but not
ule of food presentation. When the schedule of food with saline injections. These results with intravenous
presentation was eliminated, responding characteristic injections of nalorphine, like those obtained with
of FI schedules could be maintained by the schedule intracranial stimulation and with electric shocks, indi¬
of electric shock alone. In one experiment, when an cate that factors such as the controlling schedule can
electric shock was produced by each response during determine the effect of an event on behavior. Inter¬
the last minute of an 11-min cycle ending with a time¬ estingly, the experiments on electric shock are more
out period, responding was positively accelerated dur¬ puzzling to many people than those on intracranial
ing the first 10 min (FI schedule) but suppressed dur¬ stimulation or drug injections. This undoubtedly
ing the last minute of each cycle (Figure 1). Thus occurs because the latter two situations are relatively
electric shocks of the same intensity that maintained unfamiliarj suggesting that tacit eommonsensc notions
responding under the FI schedule suppressed respond¬ pervade scientific thinking more than is usually
ing during the part of the cycle in which each re¬ realized.
sponse produced electric shock. With quantitative variations in the magnitudes of
Another study suggests that intravenous injections consequent events, the possibilities of varied effects
of nalorphine, a drug that antagonizes the actions of are increased. Simply altering some parameter of a
morphine, can function in seemingly opposite ways in consequent event can completely change the subse¬
rhesus monkeys (Goldberg, Hoffmeister, Schlichting, quent frequency of occurrence of responses which
8c Wuttke, 1971). The administration of nalorphine to produce the event.* Responses that produce intracra¬
a morphine-dependent monkey precipitates an imme¬ nial stimulation, for example, will usually increase in
diate and severe withdrawal syndrome. In one phase of frequency as the intensity or duration of stimulation
the study, morphine-dependent monkeys were trained is increased over some range of values; however, at
under a schedule in which key pressing produced higher values responding will decrease and eventually
intravenous injections of morphine. After stable per¬ cease. When responding is maintained by response-
formance had developed, injections of either saline or
nalorphine were substituted for morphine. Although 4 When suppression of responding occurs only at certain
response-produced nalorphine injections did precipi¬ parameter values of a consequent event, the suppression may be
lasting or transitory. In most instances in which responding is
tate a severe withdrawal syndrome, response rates
suppressed by intense electric shock, the effect has been shown to
were higher than those maintained by morphine or be lasting and is appropriately described as operant punishment.
saline. In a subsequent phase of this study with mor¬ Responding will also be suppressed just after it has resulted in
the presentation of a large amount of food or the injection of a
phine-dependent monkeys, intravenous injections of
high dose of drug. Yet under certain circumstances, it may be
nalorphine were automatically delivered in the pres¬ possible to show that rate of responding generally increases as
ence of a stimulus; responding terminated the stim¬ the amount of food or dose of drug increases even if responding
just after the event is suppressed temporarily. (For example, see
ulus and the associated injections. Under this schedule
the description below of the experiment by Hawkins & Pliskoff,
of stimulus-injection termination, responding was 1964.)
180 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

produced electric shocks, similar increases and de¬ consequent event is often effective in modifying the
creases in rate of responding would be expected with behavior of different individuals, it becomes common
increases in intensity or duration of electric shock. In practice to refer to reinforcers without specifying the
some circumstances, responding is not initiated even behavior that is being modified. These common prac¬
under conditions in which it has previously been well tices have unfortunate consequences. They lead to the
maintained by an event. With intracranial stimula¬ erroneous views that responses are arbitrary and that
tion, for example, responding may not occur unless the reinforcing or punishing effect of an event is a
each daily session begins with an automatically pre¬ specific property of the event itself.
sented stimulation. A similar phenomenon is de¬ The commonly used contingency table describing
scribed in the section headed “Characteristics of Re¬ relations between the presentation and withdrawal of
sponses77 under conditions in which electric shock stimuli and their behavioral effects (Skinner, 1953,
both elicits and modulates responding. Such results pp. 73, 185; Rachlin, 1970, p. 79) provides an example
further indicate the varied effects of environmental of the tendency to categorize stimuli in terms of in¬
events in controlling responding. herent properties. When the borders of the table are
When an event that occurs after a response in¬ designated in terms of stimulus classes (positive-nega¬
creases the subsequent frequency of occurrence of that tive; pleasant-noxious) and experimental operations
response, the presentation of the same event after a (stimulus presentation-stimulus withdrawal), the cells
different response or according to a different schedule of the table are, by definition, varieties of reinforce¬
may not affect behavior in the same way. The condi¬ ment and punishment. One problem is that the
tions required for the suitability of various events in processes indicated in the cells have already been as¬
modifying behavior can differ markedly, but under sumed in categorizing stimuli as positive or negative;
suitable conditions even different events can function a second is that there is a tacit assumption that the
similarly. Thus it is imperative to study factors m presentation or withdrawal of a particular stimulus
addition to the events themselves which are involved will have an invariant effect. These relations are
in the processes of operant reinforcement and punish¬ clearer if empirical operations are used to designate
ment, the border conditions, as shown in Table 1. In this
case the cells of the table are unambiguously related
to the designated conditions; the top row indicates the
0NOOINO BEHAVIOR process of reinforcement and the bottom row the
process of punishment. If the presentation of a par¬
Both the qualitative and quantitative properties of ticular stimulus increases behavior under one condi¬
ongoing behavior are important aspects of reinforce¬ tion and decreases behavior under another condition,
ment and punishment. Emphasizing the sequential there is no need for a category of paradoxical rein¬
patterning of behavior as a determinant of subsequent forcement or punishment. In trying to understand
behavior shifts the focus of the interaction between why the same stimulus event can have different effects
behavior and environmental events toward behavior on behavior it is no help to consider reinforcement or
itself. Historically the situation has always been just punishment as paradoxical. The characterization of
the opposite. A stimulus paired with a reinforcer is behavioral processes depends upon empirical observa¬
said to have become a conditioned remforcer, but tions. Tbe same stimulus event, under different condi¬
actually it is the behaving subject that has changed, tions, may increase behavior or decrease behavior. In
not the stimulus. Similarly, the physical properties of the former case the process is called reinforcement
a discriminative stimulus are the same before and and in the latter the process is called punishment.
after it controls behavior; it is the subject that has The work of Premack (1959, 1965, 1971) has empha¬
become discriminative, not the stimulus. It is, of sized that reinforcers and punishers are not discrete
course, useful shorthand to speak of conditioned rein¬
forcers or discriminative stimuli, just as it is conve¬ Table 1
nient to speak about a reinforcer rather than speaking
about an event that has followed an instance of a Present Stimulus Withdraw Stimulus
specific response and resulted in a subsequent increase
Increase ^
in the occurrence of similar responses. The latter may Behavior f Reinforcement Reinforcement
be cumbersome, but it has the advantage of empirical
referents. Because many different responses can be Deci ease "i Punishment Punishment
Behavior f
shaped by consequent events, and because a given
W. H. Morse and R. T. Kelleher 181

fixed classes of events. He has made the intriguing be well maintained by the presentation of the second
proposal that reinforcement and punishment are component despite a low rate of responding in the
based on the probabilities of responses associated with second component (see Gollub, Chapter 10 of this
different events. He suggests that when an event volume). For example, relations between rates of re¬
associated with a high response probability follows an sponding controlled by various intensities of intra¬
event associated with a low response probability, rein¬ cranial stimulation and the effectiveness of such stim¬
forcement will occur; however, when an event asso¬ ulation in controlling behavior have been studied by
ciated with a low response probability follows an Hawkins and Pliskoff (1964). Each response on one of
event associated with a high response probability, two response keys resulted in electrical stimulation of
punishment will occur. As noted previously, the an area of the hypothalamus; this response key (key B)
process of reinforcement can be demonstrated in the was retracted from the apparatus after every fifth
situation in which the lever-pressing responses of a response. Responses on the other response key (key A)
food-deprived rat result in the delivery of food pellets. were maintained under a VI schedule by the reintro¬
Under the conditions of this demonstration, the duction of key B. As the intensity of the intracranial
initial probability that the rat will press the lever is stimulation was increased over a range of parameter
low, whereas the initial probability that it will eat the values, the rate of responding on key B (computed
food pellets is high; thus reinforcement is easily from the latency of the first of five responses) in¬
demonstrated. creased and then decreased (or simply decreased).
A notable aspect of Premack’s formulation is its Over the same range of intensities, however, the rates
recognition of the relativity of events as reinforcers or of responding on key A increased. Thus under this
punishers. This relativity has been demonstrated two-component chained schedule, the effectiveness of
under conditions in which the access of rats to a drink¬ the second component in maintaining responding in
ing tube or an activity wheel could be controlled the first component was directly related to the inten¬
experimentally (Premack, 1971). Licking on the drink¬ sity of intracranial stimulation but wa§ not related m
ing tube was sensed automatically by means of an any simple way to rate of responding in the second
electronic circuit and recorded on a counter (drinkom- component. This type of experiment, like those de¬
eter). The activity wheel revolved at a preset rate for scribed in the preceding section, indicates the impor¬
5 sec whenever the rat pressed a retractable lever. The tance of the schedule of presentation of any event in
initial relative probabilities of drinking and running determining how it affects behavior.
responses were assessed in daily lS-min control sessions. Premack’s promising theoretical account of the con¬
The rats spent more time drinking than running when ditions under which an event will function as a rein
they were water-deprived, but spent more time running forcer or a punisher is being refined and extended,
than drinking when they were not water-deprived. In but it is still difficult to apply in some situations. One
subsequent experimental sessions, the activity wheel problem is how to assign an initial probability of re¬
was operated only after the rat made a specified num¬ sponse to certain types of events—for example, events
ber of licks on the drinking tube; that is, drinking re¬ such as intracranial stimulation or intravenous drug
sulted in brief periods of forced running. Drinking injection that are delivered directly to the animal.
responses were increased above control levels (rein¬ Premack notes that it should be possible to develop
forcement) by operation of the activity wheel in rats indirect ways of assessing the initial response probabil¬
which were not water-deprived but were decreased be¬ ities of such events. It seems likely, however, that this
low control levels (punishment) by operation of the indirect approach would entail the same difficulties as
activity wheel in rats which were water-deprived. the discriminative stimulus hypothesis of condi¬
Moreover, the degree of suppression was inversely re¬ tioned reinforcement (see Gollub, Chapter 10 of this
lated to the probability of operating the activity wheel volume). Although the point of view in the present
in the control sessions. Thus operation of the activity chapter is similar to that of Premack in stressing the
wheel could be either a reinforcer or a punisher de¬ relativity of events as reinforcers and punishers, our
pending on whether the initial relative probability of emphasis is on the ongoing rate of responding at the
running was high or low. time an event occurs and on the way in which the
The results of some experiments with two-compo¬ event is scheduled. The role of schedules will be con¬
nent chained schedules seem inconsistent with the no¬ sidered in more detail at the end of this chapter.
tion that the reinforcing effectiveness of an event is It is clear that the effect of a given consequent
directly related to the response probability associated event on rate of responding is likely to be different
with it; that is, responding in the first component can when it follows responding occurring at different fre-
182 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

quencies. Depending on the frequency of ongoing re¬ lows each response under FR (fixed-ratio) and FI
sponding, behavior may be modulated more than schedules of food presentation in the pigeon, a pattern
changed in absolute level. For example, Skinner and of suppression develops that is different under the two
Morse (1958) studied rats running in an activity schedules (Azrin, 1959; Holz 8c Azrin, 1962). Under
wheel under conditions in which running resulted in a single type of schedule, the effects of response-con¬
the presentation of a food pellet under a 5-min FI tingent electric shocks may be different when they are
schedule. The rats characteristically paused for a rela¬ introduced in different temporal relations to the con¬
tively long period of time after each food presentation sequent event (Holz 8c Azrin, 1962). Besides the type
and then ran until food was presented again. Whether of maintenance schedule, other parameters are also
or not the overall rate of running was increased or important in determining the effects of noxious
decreased from the level of running that prevailed stimuli as consequent events. When behavior is main¬
when the schedule was not in effect, the pattern of tained under VI or FR schedules of food presentation,
running became orderly with respect to the schedule the suppressive effect of response-produced electric
of food presentation. shocks is critically dependent upon the degree of food
Reinforcement depends upon the quantitative deprivation (Azrin, 1960; Azrin, Holz, 8c Hake, 1963).
properties of behavior, so that different responses are For example, the suppression produced by an intense
modified differently* It is usually easier to increase the shock delivered every 100 responses became progres¬
frequency of an operant that is occurring infrequently sively greater as the maintenance body weight of the
than that of an operant that is occurring frequently. subject was increased from 60 to 85% (Azrin et al.,
Although interactions between the levels of ongoing 1963). This finding shows clearly that the effect of a
behavior and consequent events have tended to be response-produced electric shock depends on the pre¬
ignored in §tudie§ on reinforcement, these considera¬ vailing conditions. In this case, whether the same in¬
tions have become increasingly important with the tense electric shock suppressed behavior or not de¬
development of techniques for engendering strong, pended on the degree of food deprivation.
reproducible patterns of behavior. Because the suppressive effects of response-pro¬
Much information is available on the importance duced electric shocks do depend upon the exact main¬
of ongoing maintenance conditions in determining tenance conditions, experiments on the effects of in¬
the effects of consequent noxious stimuli. As noted troducing response-produced electric shocks have
previously, the defining operation of a punisher is a often yielded different results. At one time such differ¬
subsequent decrease in the frequency of responses ences were interpreted as indicating that punishment
similar to one that immediately preceded the was a less reliable behavioral process than was rein¬
punisher. There is no fundamental logical difference forcement. Such differences are due entirely to a lack
between the punishment and reinforcement situa¬ of comparability in other features of the situation be¬
tions; in both there is an assumption that the level of ing studied; when the maintenance conditions of
behavior before the presentation of the event is experiments are comparable, the effects of response-
measurable and sufficiently reproducible to permit produced electric shocks are comparable and repro¬
identification of changes in its rate. In dealing with ducible from experiment to experiment.
reinforcement, the level is usually low or is developed The process of reinforcement is as dependent upon
through shaping, and experimenters may easily con¬ variations in environmental conditions as is the
duct experiments without forcing themselves to pon¬ process of punishment. The historical difference be¬
der the determinants of behavior in the absence of tween reinforcement and punishment is that a greater
reinforcement (cf. Segal, 1972). In dealing with range of maintenance conditions has been studied
punishment, the practical situation is entirely differ¬ with punishment, which explains why punishment
ent. Measurable levels of some behavior are required may have appeared to be variable. In actual fact,
both before and after the introduction of the punish¬ studies on reinforcement have dealt mostly with re¬
ing event; in practice this is usually accomplished by strictive, idealized cases, which may have given the
using some schedule of reinforcement to engender a false impression that the effects of known reinforcers
sustained rate of responding. Here the experimenter are not critically dependent upon conditions under
is forced to use some explicit condition, and therefore which they operate. The important point is not that
evidence has accumulated on the effects of using punishment is a variable process, but that both
noxious stimuli as consequent events under different punishment and reinforcement depend upon the
maintenance conditions (Azrin 8c Holz, 1966; Fantino, quantitative conditions of the environment. When
1975). For example, when a brief electric shock fol- one considers the potential range of environmental
W. H. Morse and R. T. Kelleher 183

conditions under which behavior can be studied, per¬ S-55

haps more is known about punishment than about
reinforcement.

CHARACTERISTICS OF RESPONSES

Although the range of behaviors that can be con¬

trolled by operant conditioning is vast, the type of
response selected for measurement can be critical.
Responses of the classes most commonly used have
the following characteristics: they are easily identifi¬
able so that repeated instances can be reliably
counted; they are easily recorded with automatic
equipment; they have short durations; and they are
readily repeatable. Some operants are easily estab¬
lished and “well behaved." In contrast, certain types
of species-specific responses, especially elicited re¬
sponses, are difficult to control directly by reinforce¬
ment or punishment.
Responses elicited by electric shock in the squirrel
monkey are of interest because their temporal pattern¬
ing can be modulated by consequent events. These
stereotyped patterns of behavior include attacks on
other members of the same species or on certain other
nearby objects (Azrin, Hutchinson, & Hake, 1967;
Hake & Campbell, 1972; Hutchinson, A2rm, & Hake,
1966; Hutchinson, Azrin, 8c Renfrew, 1968; Hutehin
son, Renfrew, 8c Young, 1971). If the monkey i§ par¬

tially restrained in a chair, for example, electric shocks

to the monkey's tail will cause it to pull and bite a
Fig. 2. Different patterns of responding (switch closures related
leash attached to its collar. In one study, electric shock to pulling and biting a leash) in a squirrel monkey under fixed*
was used both to elicit and to modulate !ca§h"pulliiig time {FT) (A) arid PI m schedules of electric shock presenta¬
tion, 2nd rapid cessation of responding when §hs«k§ nst
responses (Morse, Mead, 8c Kelleher, lOffr). The leash
presented (G); ordinate—cumulative numDei of responses;
was fastened to a lever so that biting and pulling on abscissa—time. Electric shock presentations (f mA) are |nd icalyd
the leash repeatedly dosed a switch attached to a by short diagonal §troke§ en the Cumulative i;eeoi*d* strokes on
lever, Two of the three monkeys were studied inidally
the event record indicate shocks delivered under the FT 60-sec
schedule, The retarding pen reset to the haac line whenever 250
under an FT (fixed-time) schedule in which an electric responses accumulated and at end of session. A—Session 18, FT
shock was delivered automatically every 60 sec. Each 60-gec schedule; B—Sessions 5, 9, and 99, shocks s^hoduUd under
a FI 30-sec schedule; C—Session 104, no shocks scheduled. In
electric shock elicited pulling and biting the leash,
part A responding occurred predominantly after shocks, pro¬
which caused a burst of switch closures temporally ducing a pattern of deceleration. With continued exposure
related to the biting and pulling. The burst of switch to the FI schedule (B), responding occurred predominantly
before the shock, producing a pattern of acceleration. When
closures just after shock usually ceased abruptly after
shocks were omitted {C), few responses occurred, (From Morse
a few seconds; however, a few more switch closures & Kelleher, 1970.)
often occurred just before the next electric shock was
delivered. As the session proceeded, the number of
switch closures just after shock tended to decrease, matically at 60 sec after the previous shock. Under
while the number occurring just before shock tended this FI 30-sec schedule, the switch closure was con¬
to increase (Figure 2A). sidered a response, defined by its relation to the shock.
Subsequently, the schedule was changed so that the Initially, this response occurred predominantly after
first closure of the switch 30 sec after an electric shock an electric shock; however, most shocks were produced
produced the next shock; if no switch closure occurred by a response occurring between 30 and 60 sec after
between 30 and 60 sec, the shock was delivered auto- the preceding shock (left of Figure 2B). With further
 184 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT
200 RESPONSES

Fig. 3. Development of posi¬

tively accelerated responding
(switch closures related to pull¬
ing and biting leash) in a squir¬
rel monkey under a 5-min FI
schedule of electric shock pre¬
sentation (7 mA). Recording as
in Figure 2. Top—Session 117,
initial performance under 5-min
FI schedule after 30-sec FI
schedule; bottom—Session 153.
30 MINUTES (From Morse 8c Kelleher, 1970.)

exposure to the FI 30-sec schedule (right of Figure any particular behavior. For example, if a food pellet
2B), responding declined soon after an electric shock is delivered to a rat according to an FT schedule
was delivered and then increased until the first re¬ under suitable conditions (appropriate type of food,
sponse after 30 sec produced the next shock. Only the degree of deprivation, and temporal parameter of the
first electric shock m most sessions was delivered auto¬ schedule), some identifiable sequence of behavior will
matically. When electric shocks were not delivered develop. There is no correct response or problem
(Figure 2C), this monkey seldom responded. solution m this situation, but the rat's behavior is
The rapid loss of responding in the absence of elec¬ changed. The delivery of the pellets inevitably follows
tric shocks distinguishes this FI pattern of responding some operant feature of the rat's behavior. This
from other FI patterns of key pressing engendered m feature then becomes more prominent and more likely
the squirrel monkey by the presentation of electric to be followed by a subsequent pellet delivery and
shock (Kelleher & Morse, 1968a) or by the termination thus becomes a maintained response. A positively
of a stimulus-shock complex (Morse & Kelleher, 1966). accelerated pattern of responding can be developed
Although the performances were developed and main¬ and maintained under such an FT schedule of food
tained by the shock, two of the monkeys usually began presentation (Skinner, 1948).
responding only after a shock occurred. In another Adventitious punishment differs from adventitious
monkey, responding was maintained under only the reinforcement only in the direction of the effect (cf.
FI 30-sec schedule (no shocks delivered automatically) Fantino, 1973), but in this case it is necessary to study
and then under a FI 5-min schedule of electric shock some prominent feature of behavior that can be de¬
presentation (Figure 3). A positively accelerated re¬ creased in frequency. For example, Azrin (1956) main¬
sponding characteristic of FI schedules subsequently tained key pecking in pigeons under a VI schedule of
developed, most responding occurring before the re¬ food presentation and then presented an intense elec¬
sponse-produced electric shock. The leash pulling and tric shock according to an FT schedule. A negatively
biting controlled by the electric shock appear to have accelerated pattern of responding was engendered and
characteristics of both elicited and operant behavior maintained under this FT schedule.
(see also the section on response-produced electric In determining the effects of events presented inde¬
shocks near the end of the chapter). pendently of responding, the quantitative properties
of ongoing behavior are especially important. These
properties, in turn, depend on such factors as the
ADVENTITIOUS REINFORCEMENT AND history of the individual and the tendency of the
PUNISHMENT: IMPORTANCE OF HISTORY event to elicit responding. It has become accepted that
an event is more likely to change ensuing behavior
Adventitious relations between behavior and the when it coincides with certain features of behavior
occurrence of some event are especially useful in than when it coincides with other features. Thus
understanding reinforcement and punishment. In this different behaviors vary in their frequency of occur¬
case, the event is presented in time independently of rence and in their susceptibility to modification by
W. H. Morse and R. T. Kelleher 185

response-independent environmental events (Staddon

8c Simmelhag, 1971; also see Schwartz 8c Gamzu, Chap¬
ter 3 of this volume). In pigeons with a long history
of responding under schedules of response-dependent
food presentation, the presentation of food under FT
schedules can maintain responding indefinitely. Al¬
though the rates of responding under such FT sched¬
ules are characteristically lower than those under com¬
parable FI schedules, similar patterns of responding
can be maintained (Zeiler, 1968). In pigeons which
have had only three response-dependent food presen¬
tations, however, FT schedules maintain low rates and
erratic patterns of responding (Neuringer, 1970). It is
frequently asserted that FI responding maintained by
response-produced electric shocks is basically different
from other instances of FI responding; however,
10 MINUTES
phenomena involved in studies of electric shock are
analogous to those involved in studies of food presen¬ Fig. 4. Patterns of responding under a 10-min FT schedule of
electric shock presentation (3 mAj in squirrel monkeys with a
tation. As noted in the preceding section, electric
history of responding under schedules of electric shock post¬
shocks do elicit various responses, but similar phenom¬ ponement. Ordinate—cumulative number of responses; abscissa-
ena are reported in studies of autoshaping with food, time. Short diagonal strokes on the cumulative record indicate
shock presentations. The records of monkeys K5 and K31 have
Moreover, studies comparing FT and FI schedules of
been broken into 30-min segments and displaced along the
electric shock presentation have produced results abscissa; those of monkey K28 have been broken into 10-min
similar to those obtained with schedules of food segments. Note that the pattern of responding in many of the
individual segments were S-shaped because responding decreased
presentation; that is, under the FT schedules, rates of
near the end. (From Kelleher, Riddle, Sc Cook, 1063. (g) 1063 by
responding were relatively lower and responding was the Society for the Experimental Analysis of Behavior, Inc.)
not always positively accelerated (Kelleher, Riddle, &
Cook, 1963; McKearney, 1974; Morse 8c Kelleher, well Established, and then extinguished ill squiiTEl

1970). monkeys, for example, responding recovered when

Experimental history can be critical in determining electric shocks were presented independently of re¬
whether an event occurring independently of respond¬ sponses, but ceased again when no shocks were de¬
ing will result in adventitious punishment or adven¬ livered. When electric shocks were delivered under a
titious reinforcement. For example, responding main¬ 10-min FT schedule, substantial levels of responding
tained under a schedule of food presentation is were maintained, as shown in Figure 4. The patterns
suppressed under many conditions in which response- of responding were comparable to those that have
independent electric shocks are delivered intermit¬ been described under FT schedules of food presen¬
tently (Azrin, 1956; Estes 8c Skinner, 1941). When re¬ tation.
sponding in the rhesus monkey was maintained under Circumstances in which adventitious punishment
a schedule of electric shock postponement, however, it could be changed to adventitious reinforcement were
was found that superimposing an FT schedule of first described by Herrnstein and Sidman (1958).
electric shock delivery markedly increased responding Initially, responding of rhesus monkeys under a sched¬
(Sidman, Herrnstein, 8c Conrad, 1957). Moreover, ule of food presentation was suppressed in the pres¬
when electric shocks were no longer scheduled under ence of a clicking sound by intermittent electric
the avoidance procedure, responding not only per¬ shocks delivered under an FT schedule. Then the
sisted under the FT schedule but became positively monkeys were trained to respond under a schedule in
accelerated as “the animal lever-pressed right ‘into’ the which responses postponed electric shocks. Finally, the
shock” (Sidman, Herrnstein, 8c Conrad, 1957, p. 53). schedule of food presentation was reinstated, but
Many subsequent studies have shown that the delivery when the clicking sound associated with the FT
of electric shocks independently of responding can en¬ schedule was presented, responding was enhanced
hance responding in animals that have responded rather than suppressed. Whether response-indepen¬
under schedules of electric shock postponement (for dent electric shocks suppressed or enhanced respond¬
example, Kelleher, Riddle, 8c Cook, 1963; Waller 8c ing depended on the experimental history of the
Waller, 1963). After avoidance responding had been monkey.
186 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

CRITERIA FOR COMPARING

CONSEQUENT EVENTS

The fundamental importance of orderly changes in

rate of responding to the study of behavior was dis¬
cussed in the first section of this chapter. Operants
were defined as functionally identifiable, reproducible
classes of responses. The behavioral changes associated
with increases in responding were considered rein¬
forcementj and those associated with decreases in re¬
sponding were considered punishment. It was noted
that various schedule conditions gave rise to charac¬
teristic reproducible patterns of responding in time,
yet little consideration was given to the criteria for
identifying these different reproducible behavioral
processes. These criteria are not absolute. They de¬ 15 MINUTES

pend very much on the “state of the art” and the

Fig. 5. Generality of characteristic FI performance (no respond¬
consensus of contemporaries. ing, then acceleration to a maintained steady rate of responding).
In comparing performances in the earliest experi¬ Ordinate—cumulative number of responses; abscissa—time. An
FI schedule of presentation of food or water was in operation
ments on schedule-controlled behavior maintained by
in all examples shown in this figure. Upper frame—individual
food presentation with performances in more recent pigeon (F-4) pecking plastic key (food). Three different dura¬
experiments, it is clear that progress has been made in tions of the fixed interval are shown; the general pattern
persists despite the hundredfold change in the schedule param¬
achieving reproducibility and control of behavior. In
eter. Food presentations, ending each fixed interval, are
general, this improvement in control of behavior is marked by short diagonal strokes on the cumulative record.
not because of any change in the properties of the Lower left frame—performances under a 10-min FI schedule.
food used to maintain behavior, although it might be Food or water presentations, ending each interval, are marked
by the resetting of the recording pen to the base line. Lower
said that there had been a change in the effect of the right frame—performances under a 5-min FI schedule. The
food. The change has come about because optimal species, the type of switch recording the response, and the
reinforcer presented are indicated above the records. The pigeon
parameters of various features in the situation have
pecked a plastic key with its beak: the rat and chimpanzee
been COmbmedj including the parameters d 1 the con¬ pressed a horizontal lever with their paws; the cat depressed
sequent event, the location and nature of external a rounded knob with its paw. Tbe rat turned tbe wheel by
running: only a turn of 180° is reinforced, but the cumulative
stimuli, the types of keys used, the reliability of the
distance the wheel turns is recorded directly. (From Kelleher
controlling equipment, the conditions of deprivation, Sc Morse, !96Sb.)
the training conditions, and the experience of the sub¬
ject, Because various combinations of conditions will
suffice and because no single feature is likely to be es¬ soon found it possible to develop discriminative per¬
sential, it may not be always possible to explain the formances very quickly. Given an explicit description
reasons for technical advances. In some instances, spe¬ of what behavior is to be achieved, the conditions
cific changes m current practices have been shown to sufficient to realize the result can usually be found.
be important. For example, if food-deprived rats are Schedule-controlled patterns of responding give a
maintained at 60-65% of ad lib body weight rather meaningful way of comparing different species, differ¬
than at 80%, characteristic schedule-controlled perfor¬ ent maintenance events, or other interventions. Sched¬
mances are more easily obtained. In the development ule-controlled patterns appear to have great general¬
of stimulus control, the location, intensity, and dura¬ ity; they occur in diverse species with a variety of
tion of the controlling stimuli and the schedule un¬ different maintenance events (Figure 5). Different
der which they are presented can result in “errorless” schedule performances depend upon the particular
discriminations, whereas only slightly different condi¬ maintenance conditions. Usually, subjects with similar
tions can result in a much slower development of con¬ past experience exposed to the same parameter values
trol. Perhaps the most important ingredient of ad¬ can be expected to respond comparably, although the
vances in experimental control is the explicit attempt actual rates of responding may differ somewhat (Wal¬
by investigators to achieve greater control. After the ler 8c Morse, 1963). To produce the same response rate,
initial work showing the possibility of errorless dis¬ the parameters of the schedule may have to be differ¬
criminations (Terrace, 1963), many other investigators ent for different individuals. The value of producing
W. H. Morse and R. T. Kelleher 187

10 MINUTES

Fig. 7. Characteristic FI performance in the squirrel monkey

under a multiple schedule of stimulus-shock termination and
food presentation (Monkey S-50). The arrow indicates the
change from the schedule of stimulus-shock termination to the
schedule of food presentation. Left of the arrow—in the presence
of a white light, electric shocks were scheduled to occur at 3-sec
intervals starting after 5 min; the first response after 5 min
Fig. 6. Patterns of responding of three species (pigeon, rat, and
terminated the stimulus-shock complex for 1 min. No shocks
were delivered in the record segment shown. Right of the
monkey) under mult (multiple) FI FR schedules of reinforce¬
ment. (From Skinner, 1956. © 1956 by the American Psycho¬ arrow—in the presence of a red light, the first response after 5
logical Association. Reprinted by permission.)
min was followed by food presentation and terminated the light
for 1 min. Food presentations are indicated by short diagonal
strokes on the cumulative record. The recording pen reset to
the base line at the end of each fixed interval, The recorder did
not run during the minute of darkness following each fixed
such comparable rates and patterns of schedule per¬ interval. (From Kelleher & Morse, 1968b.)
formances is that these reproducible temporal patterns
represent an invariant behavioral process. Stevens
(1951, pp. 20-21), discussing the importance of in¬ One might note that the type of maintenance event,
variance as a tool of thought, concludes by saying; not specified in this example, is not critical. Figure 7
"The scientist is usually looking for invariance shows, for example, FI performances in a squirrel
whether he knows it or not. . . . The delineation of monkey maintained by the presentation of food and
the conditions of invariance for any phenomenon by the termination of a stimulus complex comprising
would tell us all we want to know about the matter,” a visual stimulus and an associated shock schedule
This same point of view is expressed by Skinner (1956, (Kelleher & Morse, 1968b; Morse & Kelleher, I960).
pp. 230-231) in commenting on Figure 6, which shows Schedule performances are invariant in part be¬
performances of a pigeon, rat, and monkey under a cause techniques have been devised that produce in¬
mult (multiple) FR FI schedule: variance. By “delineating” the conditions of invari¬
ance for different species or for different maintenance
Pigeon, rat, monkey, which is which? It doesn’t events, there is a meaningful behavioral basis for com¬
matter. Of course, these three species have be¬ paring the effects of other independent variables. Of
havioral repertoires which are as different as course, there are various bases, both formal and
their anatomies. But once you have allowed for
empirical, for making comparisons among different
differences in the ways in which they make con¬
conditions, but a compelling argument can be made
tact with the environment, and in the ways in
which they act upon the environment, what that comparisons among different events should be
remains of their behavior shows astonishingly made on the basis of their similarities rather than
similar properties. Mice, cats, dogs and human their differences. The many different events that have
children could have added other curves to this been used to maintain or suppress behavior function
figure. And when organisms which differ as similarly under appropriate conditions, but the con¬
widely as this nevertheless show similar prop¬ ditions required for their suitability as reinforcers or
erties of behavior, differences between members punishers are different. Thus the essential aspect in
of the same species may be viewed more hope¬ studying events as reinforcers and punishers is not any
fully. Difficult problems of idiosyncrasy or indi¬
inherent property of the events but rather the speci¬
viduality will always arise as products of bio¬
fication of the conditions under which events modify
logical and cultural processes, but it is the very
business of the experimental analysis of behavior behavior. As we have noted, the experience of the
to devise techniques which reduce their effects individual and the schedule under which events are
except when they are explicitly under investiga¬ scheduled have often been neglected in favor of the
tion. more static properties of events. The value of dealing
188 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

with reproducible behavioral processes has already vironmental events, such as deprivation of food or
been described. The next sections will consider some presentation of electric shock, the generality of pre¬
actual instances involving comparisons between differ¬ dictions about drugs affecting underlying motivational
ent maintenance events that developed from practical or emotional states can be evaluated in objective ex¬
applications in the field of behavioral pharmacology. periments. If drugs directly affect motivational states,
the kind of effect a drug has on different behaviors
should then depend on similarities or differences in
COMPARISONS OF THE EFFECTS OF the events controlling the behavior. Relevant experi¬
DRUGS ON PERFORMANCES MAINTAINED mental studies refute this view.
BY DIFFERENT CONSEQUENCES First, different patterns of responding maintained
by the same event are selectively affected by drugs
Although there was once considerable work com¬ even when these patterns repeatedly alternate under
paring strengths of different drive states, in recent a multiple schedule during the same session (see
times interest in this topic has diminished. This Kelleher 8c Morse, 1968b). In such instances, the differ¬
change has resulted partly from the repeated find¬ ential effects of drugs cannot be attributed to the
ing that different schedule-controlled patterns o£ re¬ consequent event. The direction of the dependency of
sponding can be engendered in individual subjects drug effects on schedule performance can differ among
with multiple schedules. Schedule performances em¬ drugs. Barbiturates decrease responding under many
body a great deal of what traditionally has been called parameter values of FI schedules at doses that do not
motivation (see the section on response-produced decrease responding under FR schedules (Dews, 1955;
electric shocks near the end of the chapter). For other Morse, 1962). Other drugs have the opposite effect:
reasons, however, behavioral pharmacologists have responding under FR schedules can be decreased by
long been interested in determining whether drugs doses o£ amphetamines that increase responding under
have selective and specific effects on behavior con¬ FI schedules (Kelleher 8c Morse, 1964; Smith, 1964).
trolled by noxious stimuli as compared with other Because the schedule can profoundly modify the
events. Many investigators have compared the effects effects of drugs, comparable schedules and comparable
of drugs on behavior maintained by presentation of schedule-controlled patterns of responding must be
food with their effects on behavior maintained by established with different events (for example, food
termination (or postponement) of electric shock. and electric shock) before there can be meaningful
Much of the interest in such comparisons derives comparisons of the effects of drugs on responding con¬
from motivational interpretations of the clinical uses trolled by these events. When schedule conditions and
of drugs. After the development of the major and the performances are comparable, many drugs have simi¬
minor tranquilizers, these drugs were soon used widely lar effects on behaviors controlled by different events.
m the clinical treatment of psychiatric disorders in¬ For example, in the rat responding under a FR 1
volving agitation, apprehension, tension, or anxiety schedule of reinforcement, chlorpromazine decreases
states. Consequently, it was generally accepted that the responding maintained by the presentation of food,
effects of these drugs on behavior would be under¬ intracranial stimulation, or heat. Appropriate doses of
stood in terms of their direct effects on underlying amphetamine increase responding maintained by the
motivational states or drives, presentation of food, intracranial stimulation, or
Such motivational interpretations of the clinical heat, while higher doses decrease responding (for de¬
uses of drugs promoted interest in experimental study tails see Kelleher 8c Morse, 1968b).
of how drugs affect behavior controlled by noxious Experiments by Weiss and Laties (1963) using heat
stimuli. It has been assumed that noxious stimuli as a reinforcer are particularly significant for the
control behavior by engendering an emotional state of present discussion because they studied the effects of
fear or anxiety; changes in behavior after a drug have several drugs on skin and body temperature, as well
been explained as changes in this emotional state. as on frequency of responding maintained by heat
Motivational interpretations have also been applied presentation. The experiments were conducted with
to the effects of drugs on behavior maintained by food individual shaved rats in a small chamber in a re¬
presentation or water presentation; changes in be¬ frigerated room; whenever the rat pressed a lever
havior after drugs have been explained as changes in within the chamber, a lamp above the chamber de¬
hunger or thirst. livered 2 sec of infrared heat. At certain temperatures,
Because hypothetical drive states, such as hunger or chlorpromazine decreased rates of responding even
anxiety, are assumed to depend upon controlling en- though it enhanced the rate at which temperature fell
W. H. Morse and R. T. Kelleher 189

in the cold, and amphetamine increased rates of re¬ keys. A pulsating electric shock of low intensity was
sponding even though it caused the skin temperature continuously delivered, and the first response after 10
to rise significantly. Noting that these effects are simi¬ min terminated the shock; under this schedule, the
lar to those obtained when food or water is used to rate of responding depended upon the intensity of the
maintain behavior, Weiss and Laties (1963, p. 7) con¬ pulsating shock. Monkeys of another group were food-
cluded that “the behavioral properties of these drugs deprived and studied under an FI 10-min schedule of
are largely independent of the reinforcer that main¬ food presentation. The parameters of the schedules
tains the behavior, or, put another way, of the motiva¬ were selected to give comparable rates of responding,
tional state that supports it.” and response patterns characteristic of FI schedules
Some investigators have reported that chlorproma- were maintained under both food presentation and
zine and reserpine have more marked effects on be¬ shock termination. Chlorpromazine and imipramine
havior maintained by electric shock than on behavior decreased rates of responding under both types of FI
maintained by the presentation of food. Other in¬ schedules, while selected doses of amphetamine,
vestigators have reported that chlorpromazine has meprobamate, and chlordiazepoxide increased rates
more marked effects on behavior maintained by pre¬ of responding. These results support the view that the
sentation of food or intracranial stimulation than on behavioral effects of these drugs depend mainly upon
behavior maintained by avoidance of electric shock. schedule-controlled patterns of responding.
Still other studies comparing behaviors maintained by The other study directly compared the importance
food and by electric shock found no difference in of type of reinforcer and schedule of reinforcement as
sensitivity to reserpine (for details see Kelleher 8c determinants of the behavioral effects of drugs (Kelle¬
Morse, 1964, 1968b, 1968c). These results reflect on her & Morse, 1964). Under some conditions the termi-
the difficulties involved in comparing behavioral
effects of drugs on performances maintained with
different reinforcers, In most of the studies the types
and parameters of the schedules differed as well as
the consequent events. When different reinforcers are
presented according to different schedules, the effects
of a drug may be largely determined by the schedule-
controlled patterns of responding, For comparing the
effects of drugs on behaviors maintained by different
reinforcers, it is useful to start with similar schedules
of reinforcement; however, there is still no a priori
basis for equating such parameters as amounts of
food and intensity of electric shock. It is unreasonable
to presume that certain parameter values of one arbi¬
trarily chosen schedule of food presentation will be
comparable to the same parameters of an arbitrarily
chosen schedule of electric shock termination.
The most satisfactory way to attack these problems Fig. 8. Characteristic mult FI FR schedule performance main¬
is to obtain as nearly as possible identical patterns of tained in squirrel monkeys by food presentation (upper record,
monkey S-l) and by stimulus-shock termination (lower record,
responding maintained by different events and then to monkey S-26). The sequence of visual stimuli and corresponding
establish dose-effect relations for drugs on these pat¬ schedules is the same in the upper and lower records. At the
terns. Functional relations between drugs and be¬ beginning of the records, the 10-min FI schedule was in effect
in the presence of a white stimulus. At the termination of the
havior maintained by different schedules with each FI component the recording pen reset to the bottom of the
event can then be compared. Earlier it was noted that record, and a pattern of horizontal lines was present for 2.5
the conditions sufficient to realize a desired behavioral min; during this time-out (TO) period, responses had no pro¬
grammed consequences. The next short diagonal stroke on the
performance can usually be found. It is noteworthy cumulative record indicates that the 30-response FR component
that two different procedures for establishing com¬ was in effect in the presence of a red stimulus. Again, the
parable patterns of responding with formally com¬ cumulative recording pen reset to the bottom of the record
at the termination of the FR component and was followed by
parable schedules of food presentation and electric the 2.5-min time-out component. This cycle was repeated
shock termination have been developed. throughout each session. At the bottom of the record for
monkey S-26, the short diagonal strokes on the event line
Cook and Catania (1964) studied an FI schedule of
indicate electric shock (6.2 mA) presentation. (Modified from
electric shock termination in a group of squirrel mon¬ Kelleher 8c Morse, 1964.)
190 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

nation of a schedule complex, comprising a visual but decreased rates of responding under both FR
stimulus and an associated schedule of shock presenta¬ schedules. Note that .3 mg/kg of d-amphetamine,
tion, can maintain schedule-controlled patterns of which produced the maximum increase in rates of
responding characteristic of FI, FR, and mult FI FR responding on both FI schedules (relatively low con¬
schedules in the squirrel monkey (Morse 8c Kelleher, trol rates), decreased rates of responding on both FR
1966). In one series of experiments, responding under schedules (relatively high control rates). Many in¬
such a mult FI FR schedule was compared with re¬ vestigators have found that amphetamines tend to
sponding under a mult FR FI schedule of food presen¬ increase response output under schedules that main¬
tation (Kelleher & Morse, 1964). Although maintained tain low rates of responding but tend to decrease re¬
by different events, performances under the two multi¬ sponse output under schedules that maintain high
ple schedules were similar. Representative records for rates of responding. It is often assumed that decreases
two monkeys are shown in Figure 8. The FR compo¬ in responding maintained by food presentation are
nent of each multiple schedule sustained a high rate caused by anorexic effects of amphetamine even
(about 2.3 responses per sec). The FI component of though such decreases occur under a variety of condi¬
each multiple schedule was characterized by a pause tions. The similarity of the pairs of dose-effect curves
(period of no responding) followed by acceleration of in Figure 9 indicates that this interpretation is wrong.
responding to a steady rate; the average rate in the A mere decrease in responding after amphetamine, or
interval was about .6 response per sec. any other drug, is not sufficient evidence of anorexia.
The effects of d-amphetamine on responding under Figure 9 shows that the effects of d-amphetamine de¬
each of the component schedules are shown in Figure pend more upon the type of schedule than upon the
9. Except at the highest dose, d-amphetamine in¬ scheduled event (Kelleher 8c Morse, 1968b).
creased rates of responding under both FI schedules

d-AMPHETAMINE (mq/kg)
CONTROL RESPONSE RATE
Fig. 9. Effects of d-amphetamine sulfate on rates of responding (RESPONSES/SEC)
under multiple FI FR schedules of food presentation and
stimulus-shock terminations. Three squirrel monkeys were Fig. 10. Dependence of effect of d-amphetamine on predrug
studied on each multiple schedule. Each drug was given intra¬ rate of responding in a squirrel monkey. Abscissa—average rate
muscularly immediately before the beginning of a 2.5-hr session. of responding in successive minutes of a 10-min FI schedule
At least duplicate observations were made on each monkey at (circles) and under a 30-response FR schedule (triangles); ordi¬
each dose level. Summary dose-effect curves for the four nate-relative rate of responding after .3 mg/kg d-amphetamine,
component schedules were obtained by computing the means intramuscularly. Rates of responding were recorded separately
of the percentage changes in average response rates from control during the FR component and during successive minutes of
to drug sessions. The dashed line at 100% indicates the mean the FI component. Open and filled symbols indicate data from
control level for each component. The vertical lines on the two different sessions. The line through the points was fitted by
left of the figure indicate the ranges of control observations inspection. Based on data of a single monkey used in
expressed as a percentage of the mean control value. Note the computing the averaged data under FI and FR schedules of
general similarity of the pairs of dose-effect curves for FI and for stimulus-shock termination in Figure 9. (From Kelleher &
FR components. (Modified from Kelleher & Morse, 1964. © 1964 Morse, 1968b. © 1968 by the Society for the Experimental
by the Society for the Experimental Analysis of Behavior, Inc.) Analysis of Behavior, Inc.)
% OF CONTROL RATE OF RESPONDING

Fig. 11. Dependence of effect of

chlordiazepoxide and mepro¬
bamate on intensity of electric
shock or on predrug rate of re¬
sponding in the squirrel mon¬
key. Ordinate—relative rate of
responding after oral doses of
each drug; abscissa—shock in¬
tensity (left frame) and predrug
rate of responding correspond¬
ing to each of the three shock
intensities (right panel). (From
Cook & Catania, 1964. Reprinted
from Federation Proceedings 2);
(me) (resp./r 832, 1964.)

There is a graded relation between the increase in In considering Figure 11 again, on the left side the
low rates o£ responding and the decrease in high rates drug effects are shown to depend upon electric shock
of responding after amphetamines. As shown in Figure intensity; on the right side they are shown to depend
10, the relative response rate is an inverse linear func¬ upon the control rate. The two functions are similar
tion of the control rate. The two sets of data points because variations in shock intensity changed the con¬
are derived from the rates during complete sessions trol rate. The function on the left has limited predic¬
after d-amphetamine (.3 mg/kg, intramuscularly) and tive generality, however, while that on the right fits
the corresponding rates during the previous control these data imo the broader context of rate-dependent
sessions under the multiple schedule of stimulus-shock effects. The advantage of describing these results in
termination. This same functional relation has been terms of rate dependencies is that they take on an ap¬
found in several different species under conditions in plicability beyond the situation in which they were
which different predrug rates of responding were en¬ observed.
gendered by different schedules of reinforcement, or Although the actions of many drugs on behavior
by sampling different temporal periods of a single can be quantitatively related to the predrug rate of
schedule. This model of amphetamine action sug¬ responding, this does not imply that all the behavioral
gests that observed increases and decreases in respond¬ effects of drugs can be interpreted as rate dependen¬
ing do not reflect qualitatively different processes. cies (see Kelleher 8c Morse, 1968b). Nevertheless, rate
In the experiments by Cook and Catania (1964) in dependencies do operate widely and with profound
which squirrel monkeys responded under an FI 10- effects. In any experiments in behavioral pharmacol¬
min schedule of termination of electric shock, the ogy, it is necessary to take into account the predrug
effects of meprobamate and chlordiazepoxide de¬ rate in order to make valid predictions. Many of the
pended upon the average predrug rate of responding, seemingly qualitative differences in the effects of
which in turn depended upon the intensity of the drugs on different performances result from a quanti¬
electric shock. The proportional increases in rate of tative difference in predrug rates of responding.
responding were inversely related to predrug rates of The strong dependence of the effects of drugs on
responding for both drugs, except that the highest schedule-controlled behavior has implications that go
predrug rates were slightly decreased by both drugs beyond behavioral pharmacology. It indicates that be¬
(Figure 11). The rate-dependent effects of meproba¬ havioral processes such as reinforcement or punish¬
mate and chlordiazepoxide appear similar to those of ment must be viewed in the context of ongoing
the amphetamines and barbiturates, but have not behavior. Schedule-controlled behavior not only gives
been as thoroughly studied. rise to organized, integrated performances but deter-

191
 192
DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

mines how other interventions will further modify that will not maintain responding initially may do so
behavior. Rates and patterns of schedule-controlled in individuals with well-developed behavior (Gold¬
responding are, therefore, fundamental properties of berg, 1973). There are many examples of drug-taking
behavior. behavior being modified by a subject’s history. For ex¬
ample, Schlichting, Goldberg, Wuttke, and Hoffmei-
ster (1971) found that the rate and pattern of respond¬
DRUG INJECTIONS AS CONSEQUENT
ing maintained under FR schedules of d-amphetamine
EVENTS MAINTAINING BEHAVIOR
injections depended on whether rhesus monkeys had
a history of responding maintained by cocaine, co¬
Experiments on the use of drugs as consequent or deine, or pentobarbital. Thus rates and patterns of
discriminative stimuli provide other instances in responding maintained by drug injections are a com¬
which the solution of practical problems in behavioral posite result of the history of the individual, the
pharmacology has contributed to the study of behav¬ schedule of drug injection, and the dose of drug in¬
ior generally. During the past decade it has been jected.
demonstrated repeatedly that responding in experi¬ Previously we have used the term metastable to
mental animals can be maintained by the intravenous refer to two different stable patterns of responding
injection of drugs from several different classes (for maintained under the same schedule parameters, one
example, see Deneau, Yanagita, & Seevers, I960). The before and one after an intervening treatment (Morse
injection of the drug thus functions as a reinforcer in 8c Kelleher, 1966, 1970; Staddon, 1965). Instances of
these situations. As with any environmental event, opposite effects of consequent events might be viewed
drug injections maintain operant behavior only under as extreme cases of. metastability. In an earlier section
certain conditions; as more information accumulates, Of this chapter it was noted that the drug nalorphine
the control over behavior improves. For example, can both enhance behavior leading to its presentation
rates of responding maintained by injections of co¬ and enhance behavior associated with its postpone¬
caine or d-amphetamine are of the order of 50 times ment (see Goldberg, Hoffmeister, & Schlichting, 1972).
greater in current experiments than in gome of the Intravenously injected drugs, like electric shocks, are
earliest ones (see Goldberg, 1973)* Better control comes presented directly to the subject, which makes it easier
about through better specification of the relevant to use the same event in different ways. It may be of
conditions. no fundamental significance that mainly “noxious
Much of the research on the self-administration of events” have been shown to function in a variety of
drugs has been motivated by practical problems of modes. The converse situation for drugs that are gen¬
drug abuse in man, One interest has been in develop¬ erally used as "positive” reinforcers has not been
ing animal models that would predict the abuse po¬ studied, but Smith and Clark (1972) have shown that
tential of drugs in man. The most commonly used there are conditions under which food delivery will
procedure is to allow a subject to inject a given dose be postponed by food-deprived subjects. Thus it seems
of a drug with each response for an extended period clear that the maintenance of behavior by self-injected
of time each day (3 to 24 hr). Such procedures are drugs is determined by various conditions, only one
capable of distinguishing between many drugs that of which is the intrinsic properties of the drug itself.
are likely to be abused in man and certain drugs that Some conditions have been determined under which
are unlikely to be abused. Comparing the levels of patterns of responding maintained by FR and FI
behavior maintained by different drugs provides prac¬ schedules of drug injection are comparable to per¬
tical information of limited generality—as, for exam¬ formances maintained by similar schedules of food
ple, in comparing the amounts eaten of oatmeal and presentation (Goldberg, 1973; Goldberg, Kelleher, 8c
Cream of Wheat. Under some conditions neither Morse, 1975). An important parameter in any ex¬
would be taken, and a starving person would take periment on drug self-administration is the dose,
both. It is a mistake to consider drugs as having inher¬ which may critically determine the level of responding
ent reinforcing efficacies. In determining the charac¬ under certain schedules. The amount of food has not
teristics of a drug as a reinforcer or punisher, the seemed important in many studies with schedules of
conditions that are sufficient to develop the same food presentation, but it is because the amount of
operant behavior should be determined. As with food food presented has been relatively constant and ap¬
or electric shock the capacity of a particular dose of propriate to the schedule parameters. When extreme
drug to maintain behavior depends upon various con¬ amounts of food are presented or food delivery very
ditions and may change over time. Doses of cocaine intermittent, the amount of food presented becomes im-
W. H. Morse and R. T. Kelleher 193

portant (Morse, 1966; see Collier, Hirsch, 8c Kanarek, shock (discussed in earlier sections). Until recent years,
Chapter 2 in this volume). response-produced electric shocks were seldom used
Variations in drug dose and amount of food can under conditions in which they increased subsequent
have similar effects (Goldberg, 1973). For example, responding, yet numerous studies have shown diverse
average rate of responding under 10- or 30-response conditions under which key pressing is reliably main¬
FR schedules first increased and then decreased as the tained by response-produced electric shocks. Many of
dose of cocaine injected was increased or as the amount the features mentioned in earlier sections (“The Con¬
of food presented was increased. Increasing the dose tinuity of Behavior in Time” and “Ongoing Behav¬
of cocaine or the amount of food resulted in a high ior”) are important in developing such behavior; by
rate of responding at the beginning of each session, having an existing level of ongoing responding and by
but rates of responding decreased as the session pro¬ scheduling the electric shocks intermittently, the
gressed. The effects of varying the amount of drug or schedules of shock presentation may come to modu¬
food were also studied under a second-order FI late responding and develop schedule control.
schedule of FR components, each terminating with a Various studies have shown that responding can be
briefly presented visual stimulus. (A second-order maintained under FI schedules of electric shock de¬
schedule is one in which the behavior specified by a livery in squirrel monkeys trained under schedules of
schedule contingency is treated as a unitary response electric shock postponement that engender steady rates
that is itself reinforced according to some schedule— of responding. For example, McKearney (1968, 1969)

Kelleher, 1966; see Gollub, Chapter 10, and Zeiler, studied squirrel monkeys trained under such an avoid¬
Chapter 8 in this volume.) Under the second-order ance schedule. A 10-min FI schedule of response-
schedule, response rates remained constant as the produced electric shock was then introduced concur¬
parameter value of the reinforcer was varied over a rently. Subsequently, when the schedule of electric

wide range. Again, the functions relating response shock postponement was eliminated and only the
rate to amount of drug or food were similar to one an¬ 10-min FI schedule of response-produced electric
other, although they differed from the functions un¬ shock was in effect, a pattern of positively accelerated
der simple FR schedules. The lower frequency of responding developed and was well maintained. Simi¬

drug injection or food presentation under the second- larly, Byrd (1969) has shown in the cat that after a
order schedules limits cumulative effects that may de¬ history of postponement of electric shocks, responding
crease rates of responding (see also footnote 4). Once can be well maintained under an FI schedule of elec¬
again, the way behavior is controlled by consequent tric shock presentation. McKearney (1Q69) also studied

events depends more upon the schedule than the type a range of electric shock intensities and FI durations.
of scheduled event. Although injections of cocaine As the lived interval was decreased from 10 to 1 min,
and presentation of food have very different proper¬ patterns of responding (as indicated by quarter life

ties, striking parallels between drug-maintained and values) were little affected, but rates of responding
food-maintained behavior can be obtained when they were inversely related to the FI duration. Responding
are studied under comparable schedules. Indeed, we ceased, however, when electric shocks were no longer
may ask whether studies of this nature have implica¬ scheduled and redeveloped when shocks were again
tions for the abuse of food. presented under the FI schedule.
A study by Byrd (1972) has shown that responding
can be established and maintained under second-order
RESPONSE-PRODUCED ELECTRIC SHOCKS schedules of electric shock presentation. Again, in
AS CONSEQUENT EVENTS squirrel monkeys with a history of responding under
MAINTAINING BEHAVIOR schedules of electric shock postponement, characteris¬
tic FI patterns of responding were maintained under
The evidence is overwhelming that behavior is an FI 8-min schedule of electric shock presentation; a
more controlled by the nature of the prevailing sched¬ brief (1-sec) visual stimulus immediately preceded
ule than by the nature of the scheduled events. As each electric shock. Performance was subsequently
noted earlier, compelling support for this view comes maintained under a second-order schedule in which
from experiments in which the same event has dis¬ the brief stimulus was presented under an FI 4-min
parate or opposite effects on behavior when scheduled schedule component; electric shock was delivered only
differently. The most thoroughly studied examples of after the completion of four FI components. Charac¬
such opposite effects are the maintenance and sup¬ teristic positively accelerated responding was engen¬
pression of behavior by response-produced electric dered in the individual FI components. Patterns of
 194 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

responding maintained by presentation of the brief

stimulus intermittently associated with delivery of an
electric shock were similar to those maintained by
brief stimuli intermittently associated with food pre¬
sentation (see Gollub, Chapter 10 of this volume).
Under some conditions, the introduction of the FI
schedule of shock presentation can be abrupt. For ex¬
ample, in one study, squirrel monkeys were trained to
postpone electric shocks under schedules in which the
period of time by which shock was postponed de¬
creased with successive responses until a shock was
delivered automatically (Kelleher & Morse, 1969). Cer¬
tain parameters of this interlocking schedule of elec¬
tric shock postponement engendered a stable pattern
of positively accelerated responding between electric
shocks (Figure 12, ripper frame). A monkey trained
under this schedule was then maintained under an
FI 5-mm schedule of electric shock presentation; the
patterns of positively accelerated responding were
more marked than they had been under the schedule
of shock postponement (Figure 12, center and bottom
frames).
An experiment was described earlier (see Figure 1)
in which responding was both maintained and sup¬
pressed by the same response-produced electric shock
(Kelleher & Morse, 1968a). This experiment is signifi¬
cant in showing that schedule conditions other than
electric shock postponement can be used to develop FI
performances with response-produced electric shock.
Two monkeys were trained initially under a VI sched¬
ule of food presentation, and then FI schedules of
electric shock presentation were superimposed on the 20 MINUTES
schedule of food presentation. In one monkey, re¬
Tig. 12. Performances under an interlocking schedule of post¬
sponding was initially suppressed under the combined
ponement of electric shocks (upper frame) and a FI 5-min
schedule but subsequently recovered. Recovery from Schedule Of presentation of electric shock (middle and bottom
punishment has been frequently observed (Azrin & frames) (monkey 5-07). Short diagonal strokes on both cumula¬
tive and event records indicate 3-mA shock presentations. The
HoU, 1966). The rate of responding of the other mon¬
pattern Of positively accelerated responding became more
key wa§ more suppressed but later recovered after marked when response-produced shocks occurred under the FI
schedule. (Modified from Morse & Kelleher, 1970.)
numerous changes in the schedules of shock presenta¬
tion and after prolonged exposure to low shock in¬
tensities followed by gradually increasing shock in¬
tensities. Eventually, responding was enhanced in first 10 min of each cycle, whereas responding during
both monkeys under the combined schedules of food the 11th minute of each cycle remained almost com¬
presentation and electric shock presentation and con¬ pletely suppressed (see Figure 1). Studies of variations
tinued to be maintained when the food schedule was in shock intensity showed that the mean number of
eliminated (for details see Kelleher & Morse, 1968a). responses per session increased from 1,548 at a shock
As noted earlier (under “Disparate Effects of Conse¬ intensity of 1 mA to 4,227 at a shock intensity of 12.7
quent Events”) in one experiment, the first response mA. During the entire study, responding in the 11th
occurring after 10 min produced an electric shock, and minute of each cycle was completely suppressed.
each subsequent response during the 11th minute also When the time-out period was eliminated so that
produced a shock. A 1-min time-out period occurred each 11-min cycle was followed immediately by the
at the end of the 11th minute. Clear patterns of posi¬ start of the next cycle, performance was affected (Fig¬
tively accelerated responding developed during the ure 13). An increase in responding during the early
W. H. Morse and R. T. Kelleher 195

S-43

Fig. 13. Performance under a

two-component FI 10-min FR 1
schedule of electric shock pre¬
sentation without a time-out
period separating 11-min cycles.
Shock presentations are indi¬
cated by a diagonal stroke on
cumulative and event records;
the termination of one cycle
(and the beginning of the next
cycle) is indicated by the record¬
ing pen resetting to the base
line. A-C—Sessions 186, 187, and
194. Note that there was less
suppression during the FR 1
component when it was not
followed by a time-out period
than when it was (Figure 1).
(From Kelleher & Morse, 1968a.
© 1968 by the Society for the
Experimental Analysis of Be¬
10 MINUTES havior, Inc.)

part of some cycles was a transient effect. Responding presence of a visual stimulus. Then an FR 1 schedule
under the 1-response FR component in the 11th min¬ of electric shock presentation was in effect during
ute increased gradually and stabilized at a higher rate certain 1- or 3-min periods associated with a different
than had been maintained with the time-out; this stimulus. Although the parameters of electric shock
resulted in a three- to fourfold increase in the num¬ were identical in the two components of the multiple
ber of shocks delivered. Thus the effects of electric schedule, rates of responding were well maintained
shock in suppressing responding during the FR com¬ under the VI schedule but suppressed under the FR
ponent were more pronounced when a time-out period schedule.
followed that component. When scheduled shocks The experiments by Kelleher and Morse (1968a)
were omitted, responding gradually decreased to near and by McKearney (1972) emphasize the importance
zero; when electric shocks were scheduled again, the of the schedule of electric shock presentation because
previous performance was gradually recovered (Figure identical electric shocks had opposite effects on re¬
14). The extinction of performance under the two- sponding under two different schedules. Responding
component schedule appears to be similar to that was maintained by electric shocks presented under an
occurring during extinction after FI schedules. The FI 10-min schedule or a VI 3-min schedule and sup¬
persistence of key pressing under this schedule con¬ pressed by electric shocks presented under an FR 1
trasts with the rapid cessation of leash pulling de¬ schedule. The schedule of electric shock delivery de¬
scribed earlier (in the section headed “Characteristics termined whether its effects were characteristic of rein¬
of Responses”). forcement or of punishment.
Both maintenance and suppression of responding The effects of events that modulate behavior de¬
with response-produced electric shocks have also been pend not only on the nature of the events and the
observed under a multiple schedule (McKearney, schedule under which they are presented but also
1972). In squirrel monkeys previously trained under a upon the experimental history of the individual. The
schedule of electric shock postponement, characteristic historical determination of behavior does not neces¬
steady rates of responding were maintained under a sarily imply any lack of modifiability. Although the
VI 3-min schedule of electric shock presentation in the conditions under which electric shocks came to con-
196 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

Fig. 14. The extinction and. re¬

development of performance
under the two-component mult
FI 10-min FR 1 schedule of
electric shock presentation with¬
out time-out periods. Recording
as in Figure 13. A, B—Sessions
195 and 196, on extinction; C,
D—Sessions 197 and 199, on a
two-component shock schedule.
(From Kelleher & Morse, 1998a.
(e) 1968 by the Society for the
Experimental Analysis of Be¬
havior, Inc.)

trol behavior in the examples above were complex sentation was in effect on key L and responding on key
and depended on history, the maintained performances R had no programmed consequences, a period of
were under the control of the prevailing schedule of transition followed. Responding on key R declined
shock presentation. while responding on key L increased and became posi¬
A final example showing how historically deter¬ tively accelerated. Eventually, a characteristic FI pat¬
mined behavior is modulated by current conditions is tern of responding was maintained on key L while
provided by a study in which a squirrel monkey had low levels of responding occurred on key R (Figure
been trained to press one of two keys (key R) under a 15). The changed contingency “shaped” a pattern of
schedule of electric shock postponement, and respond¬ responding on key L. This example is important in
ing on this key was then maintained under an FI showing that historically determined performances
5-min schedule of electric shock presentation. Occa¬ maintained by a schedule of electric shock presenta¬
sional responses occurred on the other key (key L) in tion are not simply the temporal modulation of a
every experimental session, although they had no pro¬ highly stereotyped response pattern. The FI pattern of
grammed consequences. When the contingencies were responding occurred on the key associated with the
reversed so that the FI schedule of electric shock pre- schedule of electric shock presentation.
 W. H. Morse and R. T. Kelleher 197

S - 125
Schedule control is the single most important prop¬
erty of operant behavior. Partly because of the con¬
ception of schedules as variations of a basic process of
reinforcement rather than as the actual determinants
of behavior, it has only slowly been appreciated that
schedule-controlled behavior can determine the effects
of consequent events. That the schedule of presenta¬
tion of an event should determine the effect of the
event is unexpected from traditional formulations;
that it occurs suggests that traditional terms and time
scales may be inappropriate. Diverse conditions will
each result in characteristic reproducible and orderly
behavioral performances. Even though it may not
seem so at a superficial level, the discovery of the de¬
terminants of these diverse conditions gives a strong
basis for generalizing about behavior. Reinforcement
and punishment are best considered as reproducible
behavioral processes.
Some consequent events that maintain behavior are
especially forcing at particular parameter values, so
that past experience is of little consequence; except
when there is some ongoing behavior, certain bland
events are relatively ineffective and other snappy
events are likely to suppress behavior. Many of the
activities that people engage in, such as growing
Fig. 15. Effects of changing the response key on which presen¬
peonies, sailing boats, and riding motorcycles, may
tations of electric shock are scheduled under FI 5-min. Ordinate
—cumulative number of key presses on key that produced tell more about the history of the individual (or his
electric shocks; abscissa—time. The recording pen reset to the affluence) than can anything inherent in the activities
base line with the presentation of electric shock and the be¬
themselves. Some ongoing behavior or past experience
ginning of a 1-min time-out period. Short diagonal strokes on
the cumulative record indicate the end of the time-out; short may be important in the development of behavior but
diagonal strokes on the event record indicate key presses on the not in its continued maintenance. For example, teach¬
key that did not produce electric shocks. A—stable performance
ing programs shape behavior in a graded way; how¬
under FI schedule of shock presentation programmed on key R;
B, C, D—Sessions 3, 18, and 55 under the FI schedule of shock ever, when the final level of competence is reached,
presentation programmed on key L. The average rate of re¬ the behavior at that time is no longer so critically
sponding on key L gradually increased, while that on key R
decreased after the contingency was changed. (Kelleher & Morse, dependent on slight gradations. Various devices and
unpublished observations.) techniques are often used in initial development of
lever pressing that are usually of no consequence after
performances are well maintained. Traditionally, most
CONCLUSIONS
experimental studies of reinforcement or punishment
have used preemptive consequent events under con¬
Valid concepts applicable to the scientific study of ditions in which prior experience was not a critical
behavior evolved from discovering and controlling the determinant. Food presented to a highly deprived an¬
determinants of orderly changes in responding. Im¬ imal or a strong electric shock are immediately pre¬
portant determinants of reinforcement and punish¬ emptive. Such consequent events are no better rein¬
ment are the parameters of consequent events, the forcers than those that do depend upon history. In
quantitative properties of ongoing behavior, and the nonexperimental situations most behavior is main¬
ways consequent events are scheduled. The scheduling tained under conditions where history is important.
of relations between behavior and consequent events One man is a lawyer, another a doctor; each “likes”
brings diverse factors into operation in time as a his work and each is maintained by the environment.
dynamic coherent complex. The notions of schedule There is nothing about torts or about warts that is
and of schedule-controlled behavior conveniently char¬ interesting to everyone. It is only under certain special
acterize the sequential interaction between behavior circumstances that environmental consequences are
and environment. especially forcing in engendering behavior. Even then.
198 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

the conditions necessary for the development of be¬ vary research in operant behavior. Glenview, Ill.: Scott,
havior are more critical than the conditions necessary Foresman, 1968.
Catania, A. C. On the vocabulary and the grammar of
for its maintenance, implying that as soon as behavior
behavior. Journal of the Experimental Analysis of Be¬
develops, its history becomes important. In experi¬ havior, 1969, 12, 845-846.
mental situations behavior is often developed under Catania, A. C. The nature of learning. In J. A. Nevin
forcing conditions where history is of little impor¬ (Ed.), The study of behavior. Glenview, Ill.: Scott, Fores¬
tance, but in nonexperimental situations most be¬ man, 1973.
Cook, L., 8c Catania, A. C. Effects of drugs on avoidance
havior is shaped from already existing behavior under
and escape behavior. Federation Proceedings, 1964, 23,
conditions where the shaping sequence is important. 818-835.
How could it be otherwise? People are behaving all Deneau, G, A,, Yanagita, T„ 8c Seevers, M. H. Self-admin¬

the time, and their behavior blends with the condi¬ istration of psychoactive substances by the monkey: A
measure of psychological dependence. Psychopharma-
tions of the current environment, In contrast, in the
cologia, 1969, 16, 30-48.
laboratory it is usually arranged so that there is no
Dews, P. B. Studies on behavior, I: Differential sensitivity
strong ongoing behavior before it i§ developed, to pentobarbital of pecking performance in pigeons
In recent years, it has become possible experimen¬ depending on the schedule of reward. Journal of Phar¬
tally to study sx tuatioiis of increasing complexity and macology and Experimental Therapeutics, 1955, 113,
393-401.
to extend the range of conditions sufficient to engen¬
Estes, W. K, k Skinner, B. F. Some quantitative properties
der behavior into the domain of historical determi¬
of anxiety. Journal of Experimental Psychology, 1941,
nants. While some may view thi§ as a messy pollution 23, 390-400.
of well-established logical definitions, it brings the Famtino, F. Aversive control. In J. A. Nevin (Ed.), The
range of phenomena studied in laboratory settings study of behavior. Glenview, Ill.: Scott, Foresman, 1973.
Ferster, C. B., & Skinner, B. F. Schedules of reinforcement.
closer to those of ordinary behavior. The concepts of
New York: Appleton-Century-Crofts, 1957.
historically determined and schedule-controlled be¬
Goldberg, S. R. Comparable behavior maintained under
havior may seem unfamiliar and less precise than tra¬ fixed-ratio and second-order schedules of food presenta¬
ditional formulations, but they are more valid. tion, cocaine injection, or d-amphetamine injection in
the squirrel monkey. Journal of Pharmacology and Ex¬
perimental Therapeutics, 1973, 186, 18-30.
Goldberg, S. R., Hoffmeister, F., & Schlichting, U. U.
REFERENCES Morphine antagonists: Modification of behavioral effects
by morphine dependence. In J. M. Singh, L. Miller, 8c
Azrin, N, H, Some effects of two intermittent schedules of H. Lai (Eds.), Drug addiction, Vol. 1: Experimental
immediate and non-immediate punishment. Journal of pharmacology. Mount Kisco, N.Y.: Futura, 1972.
Psychology, 1956, 42, 3-2 L Goldberg, S. R., Hoffmeister, F., Schlichting, U. U., &
Azrin, N. H. Punishment and. recovery during fixed-ratio Wuttke, W. Aversive properties of nalorphine and
performance. Journal of the Experimental Analysis of naloxone in morphine-dependent rhesus monkeys. Jour¬
Behavior, 1959, 2, 301-305. nal of Pharmacology and Experimental Therapeutics,
Azrin, N. H. Effects of punishment intensity during vari¬ 1971, 179, 268-276.
able-interval reinforcement. Journal of the Experimental Goldberg, S. R., Kelleher, R. T., 8c Morse, W. H. Second-
Analysis of Behavior> I960, 2, 123-142. order schedules of drug injection, Federation Proceed¬
Azrin, N. H,, 8c Holz, W- c. Punishment. In W. L Honig ings, 1975, 34, 1771-1776.
(Ed.) , Operant behavior: Areas of research and applica¬ Hare, D. F., & Campbell, R. L. Characteristics and re¬
tion. Englewood Cliffs, N.J.! Prentice-Hall, Inc., 1966. sponse-displacement effects of shock-generated respond¬
Azrin, N. H., Holz, W. C., & Hake, d. F. Fixed-ratio pun¬ ing during negative reinforcement procedures: Pre-shock
ishment. Journal of the Experimental Analysis of Be¬ responding and post-shock aggressive responding. Journal
havior, 196$, 6, 141—149. of the Experimental Analysis of Behavior, 1972, 12, 303—
Azrin, N. H., Hutchinson, R. R., & Hare, D. F. Attack, 323.
avoidance, and escape reactions to aversive shock. Jour¬ Hawkins, T. D., 8c Pliskoff, S. S. Brain-stimulation inten¬
nal of the Experimental Analysis of Behavior, 1967, 10, sity, rate of self-stimulation, and reinforcement strength:
131-148. An analysis through chaining. Journal of the Experi¬
Blough, D. S. New test for tranquilizers. Science, 1958, 121, mental Analysis of Behavior, 1964, 7, 285-288.
586-587. Herrnstein, R. J., 8c Sidman, M. Avoidance conditioning as
Byrd, L. D. Responding in the cat maintained under re¬ a factor in the effects of unavoidable shocks on food-
sponse-independent electric shock and response-produced reinforced behavior. Journal of Comparative and Physi¬
electric shock. Journal of the Experimental Analysis of ological Psychology, 1958, 51, 380-385.
Behavior, 1969, 12, 1-10. Holz, W. C., 8c Azrin, N. H. Interactions between the dis¬
Byrd, L. D. Responding in the squirrel monkey under criminative and aversive properties of punishment. Jour¬
second-order schedules of shock delivery. Journal of the nal of the Experimental Analysis of Behavior, 1962, 5,
Experimental Analysis of Behavior, 1972, 18, 155-167. 229-234.
Catania, A. C. Glossary. In A. C. Catania (Ed.), Contempo- Hutchinson, R. R., Azrin, N. H., 8c Hake, D. F. An auto-
W. H. Morse and R. T. Kelleher 199

matic method for the study of aggression in squirrel The theory of reinforcement schedules. Englewood Cliffs,
monkeys. Journal of the Experimental Analysis of Be¬ N.J.: Prentice-Hall, Inc., 1970.
havior, 1966, 9, 233-237. Morse, W. H., Mead, R. N., 8c Kelleher, R. T. Modulation
Hutchinson, R. R., Azrin, N. H., 8c Renfrew, J. W. Effects of elicited behavior by a fixed-interval schedule of elec¬
of shock intensity and duration on the frequency of tric shock presentation. Science, 1967, 157, 215-217.
biting attack by squirrel monkeys. Journal of the Experi¬ Neuringer, A. J. Superstitious key pecking after three
mental Analysis of Behavior, 1968, 11, 83-88. peck-produced reinforcements. Journal of the Experi¬
Hutchinson, R. R., Renfrew, J. W., 8c Young, G. A. Effects mental Analysis of Behavior, 1970, 13, 127-134.
of long-term shock and associated stimuli on aggressive Premack, D. Toward empirical behavior laws, I: Positive
and manual responses. Journal of the Experimental reinforcement. Psychological Review, 1959, 66, 219-233.
Analysis of Behavior, 1971, 15, 141-166. Premack, D. Reinforcement theory. In D. Levine (Ed.),
Kelleher, R. T. Conditioned reinforcement in second- Nebraska symposium on motivation. Lincoln: University
order schedules. Journal of the Experimental Analysis of of Nebraska Press, 1965.
Behavior, 1966, 9, 475-485. Premack, D. Catching up with common sense, or two sides
Kelleher, R. T., 8c Morse, W. H. Escape behavior and of a generalization: Reinforcement and punishment. In
punished behavior. Reprinted from Federation Proceed¬ R. Glaser (Ed.), The nature of reinforcement. New York:
ings, 1964, 23, 808-817. Academic Press, 1971.
Kelleher, R. T., 8c Morse, W. H. Schedules using noxious Rachlin, H. Introduction to modern behaviorism. San
stimuli. III: Responding maintained with response- Francisco: W. H. Freeman, 1970.
produced electric shocks. Journal of the Experimental Schlichting, U. U., Goldberg, S. R., Wuttke, W., 8c Hoff-
Analysis of Behavior, 1968, 11, 819-838. (a) meister, F. d-Amphetamine self-administration by rhesus
Kelleher, R. T., 8c Morse, W. H. Determinants of the monkeys with different self-administration histories.
specificity of behavioral effects of drugs. Ergebnisse der Proceedings of the European Society for the Study of
Physiologic, 1968, 60, 1-56. (b) Drug Toxicity, 1970. Excerpta Medicine International
Kelleher, R. T., 8c Morse, W. H. Determinants of the be¬ Congress, Series 220, 1971, pp. 62-69.
havioral effects of drugs. In D. H. Tedeschi 8c R. E. Segal, E. F. Induction and the provenance of operants. In
Tedeschi (Eds.), Importance of fundamental principles R. M. Gilbert & J. R. Millenson (Eds.), Reinforcement:
in drug evaluation. New York: Raven Press, 1968. (c) Behavioral analyses. New York: Academic Press, 1972.
Kelleher, R. T., 8c Morse, W. H. Schedules using noxious Sidman, M., Herrnstein, R. J., 8c Conrad, D. G. Mainte¬
stimuli, IV: An interlocking shock-postponement sched¬ nance of avoidance behavior by unavoidable shocks.
ule in the squirrel monkey. Journal of the Experimental Journal of Comparative and Physiological Psychology,
Analysis of Behavior, 1969,12, 1063-1079. 1957; 50} 553-557.
Kelleher, R. T., Riddle, . W. C., 8c Cook, L. Persistent SKINNER, B. F. On the conditions of elicitation of certain
behavior maintained by unavoidable shocks. Journal of eating reflexes. Proceedings of the National Academy of
the Experimental Analysis of Behavior, 1963, 6, 507-517. Sciences, 1930, 16, 433-438.
McKearney, J. W. Maintenance of responding under a Skinner, B. F. On the rate of formation of a conditioned
fixed-interval schedule of electric shock-presentation. reflex. Journal of General Psychology, 1932, 7, 274-286.
Science, 1968, 160, 1249-1251. Skinner, B. F. Two types of conditioned reflex: A reply to
McKearney, J. W, Fixed-interval schedules of electric shock
presentation: Extinction and reggyery of performance
under different shock intensities and fixed-interval dura¬ Skinner,
,
Konorski and Miller, Journal of General Psychology,
1937,16 272-279.
B, F, The behavior of organisms, New York; Ap-
tions. Journal of the Experimental Analysts of Behavior; pleton-Century-Croft5, 1938-
1969, 12, 301-313. Skinner, B. F. “Superstition” in the pigeon, Journal of
McKearney, J. W. Maintenance and suppression of re¬ Experimental Psychologyy 1948, 38j 160—172.
sponding under schedules of electric shock presentation. Skinner, B. E. Science and human behavior. New York:
Journal of the Experimental Analysis of Behavior, 1972, Macmillan, 1953.
17, 425-432. Skinner, B. F. A case history in scientific method. American
McKearney, J. W. Differences in responding under fixed¬ Psychologist, 1956, 11, 221-233.
time and fixed-interval schedules of electric shock presen¬ Skinner, B. F. Operant conditioning. In W. K. Honig (Ed.),
tation. Psychological Reports, 1974, 34, 907-914. Operant behavior: areas of research and application.
Morse, W. H. Use of operant conditioning techniques for Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966.
evaluating the effects of barbiturates on behavior. In Skinner, B. F. Contingencies of Reinforcement. Englewood
J. H. Nodine 8c J. W. Moyer (Eds.), Psychosomatic medi¬ Cliffs, N.J.: Prentice-Hall, Inc., 1969.
cine: The first Hahnemann Symposium. Philadelphia: Skinner, B. F., 8c Morse, W. H. Fixed-interval reinforce¬
Lea and Febiger, 1962. ment of running in a wheel. Journal of the Experi¬
Morse, W. H. Intermittent reinforcement. In W. K. Honig mental Analysis of Behavior, 1958, 1, 371-379.
(Ed.), Operant behavior: Areas of research and applica¬ Smith, C. B. Effects of d-amphetamine upon operant be¬
tion. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966. havior of pigeons: Enhancement by reserpine. Journal
Morse, W. H., 8c Kelleher, R. T. Schedules using noxious of Pharmacology and Experimental Therapeutics, 1964,
stimuli, I: Multiple fixed-ratio and fixed-interval termi¬ 146, 167-174.
nation of schedule complexes. Journal of the Experi¬ Smith, J. B., 8c Clark, F. C. Two temporal parameters of
mental Analysis of Behavior, 1966, 9, 267-290. food postponement. Journal of the Experimental
Morse, W. H., 8c Kelleher, R. T. Schedules as fundamental Analysis of Behavior, 1972, 18, 1-12.
determinants of behavior. In W. N. Schoenfeld (Ed.), Staddon, J. E. R. Some properties of spaced responding in
200 DETERMINANTS OF REINFORCEMENT AND PUNISHMENT

pigeons. Journal of the Experimental Analysis of Be¬ on fixed-ratio reinforcement. Journal of the Experi¬
havior, 1965, 8, 19-27. mental Analysis of Behavior, 1963, 6, 125-130.
Staddon, J. E. R., 8c Simmelhag, V. L. The “superstition” Waller, M. B., 8c Waller, P. F. The effects of unavoidable
experiment: a reexamination of its implications for the shocks on a multiple schedule having an avoidance
principles of adaptive behavior. Psychological Review, component. Journal of the Experimental Analysis of
1971, 78, 3-43. Behavior, 1963, 6, 29-37.
Steiner, S. S., Beer, B., 8c Shaffer, M. M. Escape from self- Weiss, B., 8c Laties, V. G. Effects of amphetamine, chlor-
produced rates of brain stimulation. Science, 1969, 163, promazine, and pentobarbital on behavioral thermoreg¬
90-91. ulation. Journal of Pharmacology and Experimental
Stevens, S. S. Mathematics, measurement, and psycho¬ Therapeutics, 1963, 140, 1-7.
physics. In S. S. Stevens (Ed.), Handbook of experi¬ Zeiler, M. D. Fixed and variable schedules of response-
mental psychology. New York: Wiley, 1951. independent reinforcement. Journal of the Experimental
Terrace, H. S. Discrimination learning with and without Analysis of Behavior, 1968, 11, 405-414.
“errors.” Journal of the Experimental Analysts of Be¬ Zimmerman, D. W. Functional laws and reproducible pro¬
havior, 1963, 6, 1-27. cesses in behavior. Psychological Record, 1963, 13, 163-
Waller, M. B., 8c Morse, W. H. Effects of pentobarbital 173.
 8

Schedules of Reinforcement
the controlling variables*

Michael Zei/er

INTRODUCTION because the data have been so powerful that they de¬
manded and received major attention, while the more
Schedules of reinforcement are among the most conjectural theoretical efforts assumed secondary
powerful determinants of behavior. The effects of status. Typically, theory has not been formal or quan¬
each type of schedule are systematic and orderly in titative; it has been at a lower level, consisting of
individual organisms, and they are replicable within hypotheses about the essential controlling relations
and across species (for an example see Skinner, 1959, (but see Schoenfeld, Cole, Blaustein, Lachter, Martin,
p. 374, Figure 14). The particular performance gen¬ 8c Vickery, 1972, for a more formal taxonomic ap¬
erated depends on the schedule used, but each sched¬ proach). The present purpose is to offer another such
ule has characteristic effects. In fact, one way to evalu¬ analysis.
ate the adequacy of experimental control is by seeing The obvious origins of schedule research appear
if the behavior typical of specific schedules is repro¬ in Skinner’s (1938) demonstration that a reinforcer
duced (Sidman, 1960). Failure to obtain the expected does not have to follow every response in order to
performances indicates deficiency in the experimental maintain responding, but that it need only occur
laboratory. intermittently. The importance of intermittent rein¬
Some psychologists have considered research on forcement eventually might have become apparent
reinforcement schedules to be atheoretical, yet start¬ wtih discrete-trial procedures, but it happened im¬
ing with Skinner (1938) and continuing with Ferster mediately within Skinner’s free operant paradigm. In
and Skinner (1957) and Morse (1966) there has been research in which each response terminated a trial the
concern with theoretical analysis. Perhaps the perva¬ focus tended to be on resistance to extinction under
siveness of theory has not been generally recognized low-valued ratio schedules (the partial-reinforcement
effect); in contrast, the free operant experiments em¬
* Preparation of this chapter and several of the experiments phasized the nature of performance under maintained
reported were supported by Research Grant GB-25959 from the
reinforcement. The publication in 1957 of Ferster and
National Science Foundation. I would like to thank M. J. Marr,
W. H. Morse, and E. Davis for their comments. Skinner’s Schedules of Reinforcement represents the

201
202 SCHEDULES OF REINFORCEMENT

beginning of the modern era of schedule research. in responding as a function of a stimulus event fol¬
This encyclopedia of schedules not only describes the lowing the response. The stimuli having these effects
performances occurring under many simple and com¬ are reinforcing stimuli or reinforcers. Schedules of
pound schedules, but also pioneered in treating sched¬ reinforcement are the rules used to present reinforcing
ules as a distinct subject matter. stimuli.
It became possible to use schedule performance to
study the effects of other variables; for example, in
behavioral pharmacology schedules provided a foun¬ Time and Response Schedules
dation for assessing the actions of drugs (see Harvey,
1971). But it soon became evident that schedules did The most widely used schedules are defined in
more than establish reliable and recoverable base terms of time and responses. They may or may not
lines. The schedule itself played an important role in require a particular response. All response-indepen¬
determining how the variables of primary interest op¬ dent schedules are time schedules, and they are re¬
erated. It served not just as a convenient vehicle for ferred to as fixed-time (.FT) or variable-time (VT)
observing other processes at work but, for example, schedules depending on whether the interreinforcer
could determine whether a certain dosage of a given time is fixed or changes from one reinforcer presen¬
drug increased, decreased, or had no effect on the rate tation to the next, Other schedules are response-
of responding (cf* Kelleher Sc Morse, I960). Dews dependent. Of these, the ones that only require re¬
(1903) concluded; sponses are ratio schedules; and they are fixed-ratio
(FR) or variable-ratio (VR) schedules depending on
Schedule-controlled behavior does not merely whether a fixed or variable number of responses is
provide a baseline for convenient study of other required. Interval schedules involve both response and
variables) it is itself close to the heart of the temporal requirements, but they will be treated here
matter, This emphasis on the importance of as simple schedules. Interval schedules combine time
schedules is not intended to imply that all of schedules and a fixed-ratio schedule (FR 1); the first
psychology should be reduced to a study of them.
response emitted after a specific time has elapsed pro¬
An influence can be all-pervading without being
duces the reinforcer, and earlier responses have no
all-embracing. ... It is suggested that schedule
scheduled consequences. In fixed-interval (FI) sched¬
influences operate generally in psychology; that
when these influences can operate, they will; and ules the time is constant; in variable-interval (VI)
that a student of any problem in psychology—in schedules it varies. In VT, VI, and VR schedules the
motivation, generalization, discrimination, or experimenter determines the precise sequence of in
the functions of the frontal lobes—ignores the terreinforcer times (VT, VI) or responses per rein¬
consequences of the precise scheduling arrange¬ forcer (VR). There also are schedules that provide
ments of his experiments at his peril, (p. M8) reinforcer presentation after irregular time periods or
irregular numbers of responses, but the precise se¬
The ubiquity of schedule effects means that an un¬ quences are not prespecified. Instead, each time period
derstanding of how the scheduling of reinforcers de¬ or response is equally eligible for reinforcement ac¬
termines performance is of fundamental significance. cording to some probability. These schedules are
Intermittent reinforcement organizes and maintains known as random-time (RT), random-ratio (RR), and
highly predictable extended sequences of behavior, random-interval (RI) schedules depending on whether
and it also determines the effects of many other vari¬ the probability of a remforcer occurring refers to time
ables. The present chapter is an effort to describe how alone, to responses alone, or to a response occurring at
intermittent reinforcement operates to control be¬ a certain time.
havior. Response-independent schedules are here referred
to as time schedules, despite earlier references to them
as interval schedules preceded by an appropriate
TYPES OF SCHEDULES qualifying adjective (e.g., “free,” “response-indepen¬
dent,” “noncontingent”). Since interval schedules by
The word reinforcement refers to the effect of an definition require a response, to use them to refer to
operation; it does not describe an independent vari¬ a response-independent arrangement is internally in¬
able but is the interaction of an independent variable consistent and misleading. The time schedule designa¬
with behavior. By reinforcement is meant an increase tion avoids this ambiguity.
Michael Zeiler 203

Differentiation Schedules sponse can be manipulated to generate numerous

schedules.
In differentiation schedules reinforcers are pre¬
Although Schoenfeld et al. (1972) have proposed
sented when a response or a group of responses dis¬
t-T schedules as a comprehensive schedule classifica¬
plays a specified property. For example, responses
tion system, it does not incorporate fixed- and variable-
might have to be emitted with a particular force,
ratio schedules directly. Response count does not enter
duration, or form (topography) or to occur in a cer¬
into the specification of a schedule; probability of
tain locus. Differentiation schedules are involved in
reinforcement is applied only to individual responses.
shaping new responses, but they also encompass cer¬
Although performance typical of ratio schedules can
tain unchanging requirements. Interresponse-time
be obtained by appropriate manipulation of the
(IRT) schedules establish the time between successive
temporal parameters, this does not mean that response
responses as the requirement. If the time must equal
count is an irrelevant independent variable. Schedules
or exceed the specified value, this is an IRT > t sched¬
still can be specified based on fixed and variable num¬
ule. If the response must occur before a specified time
bers of responses without reference to temporal param¬
period elapses, this is an IRT < t schedule. If the
eters. Similar performances generated by ratio and t-j
reinforcer is presented whenever a specified response
schedules pose the challenge of finding characteristics
has not occurred for a certain time period, this is an
common to both types. The t-r schedules do not in
R > t schedule. The IRT > £, IRT < t, and R > t
themselves explain ratio performance.
schedules all are differentiation schedules involving
intervals between responses as a prerequisite for rein¬
forcement. Since in an R > t schedule, not emitting a Extinction
certain response is treated as if it was a response, rein¬
In an extinction schedule no reinforcer is pre¬
forcement is manifested by a decreased frequency of
sented, Extinction is not a schedule of reinforcement,
the criterion response.
but it is included here to provide a comprehensive
The IRT > t, IRT < t, and R > t designations re¬
list of common scheduling operations. The various
place DRL, DRH, and DRO. The problem with the
schedules described so far (time, ratio, interval, differ¬
old usage is that it confused a theoretical account of
entiation, l~r, extinction) can be combined in various
the effects of the schedules (differential-rein for cement-
ways to produce compound schedules. Together they
of-low-rate, differential reinforcement-of-high-rate, dif¬
comprise all reinforcement schedules known to date.
ferent! al-reinforcement-of-other [or not-] responding)
with the simple description of the prescription for
reinforcer delivery.
TYPES OP CONTROLLING RELATIONS:
VARIABLES AND EFFECTS

f—t Schedules
Direct and Indirect Variables
Schoenfeld and his colleagues (Schoenfeld et ah,
1972) have devised schedules based on temporal pa¬ A schedule states the conditions that must obtain
rameters combined with varying probability of rein¬ for a reinforcer to be delivered. These prerequisites
forcement for single responses. The probability of are formal properties. All schedules arrange that cer¬
reinforcer presentation occurring in any part of a re¬ tain conjunctions of events must obtain at the mo¬
peating time cycle can be varied between 0.00 and ment of reinforcer presentation, although individual
1.00, either dependent on or independent of a re¬ schedules differ in what these events must be. These
sponse. If the cycle duration is fixed and the probabil¬ formally imposed prerequisites are the direct variables
ity of a reinforcer following the first response of a imposed by a schedule. In ratio schedules, for exam¬
cycle is 1.00, it is equivalent to an FI schedule; if the ple, presentation of the reinforcer depends on the
probability of a reinforcer for the first response of a execution of a certain number of responses, so that it
cycle is less than 1.00, it is an RI schedule. If the is a formal requirement that this number of responses
probability of reinforcement occurring for all re¬ precede every reinforcer.
sponses is greater than 0.00 but less than 1.00, this is Other variables are not imposed directly. Although
an RR schedule. These are limiting cases. Combina¬ the time between successive reinforcer presentations is
tions of cycle lengths, periods of reinforcer availabil¬ not specified by a fixed-ratio schedule, the characteris¬
ity, and probabilities of reinforcement for each re¬ tics of performance establish a certain time period.
204 SCHEDULES OF REINFORCEMENT

And, although a time schedule does not require that

any particular response occur, some behavior must
precede the reinforcer. Indirect variables are those
that are imposed without being explicitly prescribed
by the schedule. One problem in a theoretical analy¬
sis of reinforcement schedules is to specify these in¬
direct variables and when and how they influence
performance.
It appears that any variable that occurs directly
under one type of schedule can occur indirectly under
others. For example, the time separating the rein¬
forced response from the one preceding—the inter¬
response time (IRT)—is imposed at a specific value
under IRT > t and IRT < t schedules. Under any
schedule, however, some IRT precedes the reinforcer.
Or interreinforcer time, which is specified directly 20 Minutes
under time schedules, arises indirectly under ratio
schedules. This is not to say that the variable is neces¬ Fig. 1. Performance of a pigeon under FI £>-min and FT 5-min
sarily exerting an effect under any schedule, but sim¬ schedules. The response pen offset at food presentations. (From
Zeiler, 1908.)
ply that it is imposed either directly or indirectly.
The fact that some schedules require what others
permit provides a methodology for an experimental
analysis of schedule effects. The hypothesis that an behavior, he found that each bird performed some
indirect variable (e g., interreinforcer time) has effects consistent ritual. It seemed that this occurred because
under some schedule can be evaluated by studying a particular behavior happened to occur in close
how performance is affected when it is imposed as an temporal contiguity with food presentation (in chap¬
explicit requirement (e.g.? in time or interval sched¬ ter 5 of this volume Staddon offers a different interpre¬
ules)* When imposed directly it must produce the tation). This temporal relation increased the proba¬
effects it is assumed to exert indirectly. This experi¬ bility of the response, even though the relation was
mental strategy is in the tradition of Skinner (1938), adventitious. Additional research indicating many
Ferster and Skinner (1957), and Morse (1966). similarities between response-dependent and response-
independent schedules suggests that the essential na¬
Stereotypic and Dynamic Effects ture of the response-reinforcer relation is temporal
(Herrnstein, 1966; Zeiler, 1972a). For example, as
Performance under a particular schedule is gen¬
shown in Figure 1, both types of schedule can main¬
erally uniform among different subjects and in the
tain the responses that precede them, and both have
same subject over prolonged periods of time. Each
similar effects on how the responses are distributed in
schedule accomplishes this by arranging certain inter¬
time (the pattern of responding). In addition, both
actions among characteristics of performance and the
bring responding under the control of the exterocep¬
controlling direct and indirect variables.
tive stimuli present when the reinforcer appears
These interactions can have two effects. The first is
(Morse & Skinner, 1957). Such data imply that response-
that certain characteristics of behavior may be re¬
dependent reinforeer presentation increases the proba¬
peated in the same form in the future. The produc¬
bility of the response because the dependency guaran¬
tion of repetitive stereotyped behavior is the defining
tees that the effective temporal relation will occur.
attribute of reinforcement: the response preceding the
In all effective respondent-dependent schedules (ex¬
reinforcer increases in frequency. The second effect is
cept those involving delayed reinforcement) the speci¬
dynamic: performance changes from one instance to
fied response occurs close in time to the reinforcing
the next.
event. The result of this contiguity is that the response
is maintained at a substantial level. The precise rate
A Pervasive Stereotypic Effect:
and temporal patterning of the response are deter¬
Response Dependency
mined by the particular schedule. The delivery of a
When Skinner (1948) observed pigeons after giving reinforcer following a single response is always an im¬
them food every 15 seconds without regard to their portant determinant of the tendency to respond, but
Michael Zeiler 20 5

the schedule modulates the rate of responding and

determines how successive occurrences of the response
are distributed in time.

The Asymmetry of Reinforcement

and Extinction

Herrnstein (1966) and Morse (1966) noted that be¬

havior typically is acquired rapidly and lost slowly
(although the loss is accelerated if there are numerous
exposures to extinction). This asymmetry means that
in all schedules a single reinforcer presentation gen¬
erates numerous subsequent repetitions of the refer¬
30 Minutes
ence response. It is this property of reinforcement that
is described by Skinner’s (1938) concept of the reflex
Fig. 3. Performance of a pigeon under an FR 150 schedule. The
reserve. response pen reset at each food presentation.
Two experiments illustrate the large effects of a
few reinforcer presentations. Skinner (1938, pp. 86- indirect variable. If, for some reason, the interrein-
90) allowed rats to adapt to the experimental chamber forcer time should lengthen, response rate might de¬
and to the sound of the food magazine. He then pre¬ crease. The consequence would be to increase inter-
sented a food pellet following one press and changed reinforcer time still further, thereby again reducing
the schedule to extinction. As shown in Figure 2, rate and producing an even longer interreinforcer
more than 60 presses occurred before the response rate time. Figure 3 illustrates such an effect, indicating
returned to the preconditioning level. Also, Neuringer that it is perhaps not felt immediately but may
(1970) demonstrated that pigeons given food for three cumulate over several interreinforcer periods. Or an
successive key pecks emitted approximately 150 pecks unusually short interreinforcer time might increase
in a subsequent extinction phase. A general effect of a rate, thereby producing still shorter times and conse¬
reinforcing stimulus is to generate substantial quan¬ quently increasing rate still further. An intermediate
tities of the response that precedes it (a stereotypic interreinforcer time, however, might not change the
effect). prevailing rate and therefore would recur in successive
ratios. Variables operating in this way are said to have

Dynamic Effects
dynamic effects. Dynamic effects do not all change be¬
havior in one direction. When variables are at a high
Some interactions between performance and con¬ level, they may operate to change behavior in such a
trolling variables lead to change rather than to stereo¬ way that a low level follows. This is the way the
typy. The important factor is the level of the variable number of responses emitted per reinforcer presenta¬
in question. Consider, for example, the role of inter- tion is hypothesized to operate under fixed-interval
reinforcer time in fixed-ratio schedules where it is an schedules; it will be discussed in detail in the next
section.
Dynamic effects can only occur when the level of a
variable is free to change, so they are typically effects
of indirect variables. However, if the schedule re¬
quirements were to be changed depending upon the
characteristics of performance, it would be possible to
observe whether a direct variable has dynamic effects.
Adjusting schedules, which will be discussed in a
later section, have this provision.
Dynamic effects play an important part in deter¬
60 Minutes mining the frequency of responding under schedules
of intermittent reinforcement. They are particularly
Fig. 2. Responding in a rat produced by a single food presen¬ significant in schedules that maintain a high average
tation. The first response was followed by a food pellet; later
number of responses per reinforcer presentations, but
responses had no scheduled consequences. (Traced from Skinper,
1938, p. 87, Figure 15.) they also occur elsewhere. The fixed-interval schedule.
206 SCHEDULES OF REINFORCEMENT

which readily shows how these dynamic effects in¬ the last session of FR 600 and the immediately suc¬
fluence response frequency, provides the focus of the ceeding session involving an FI 40-min schedule. The
next section. The consideration of direct and indirect bird was studied under FR 1000 as well. The FR 1000
variables operating in fixed-interval schedules leads schedule (not shown) did not sustain responding, i.e.,
into the analysis of the determinants of response fre¬ the bird responded infrequently during sessions as
quency under the other major schedules. long as 16 hours and never completed a ratio. Re¬
sponding was maintained with FR 600, but there were
very long pauses and many hours between successive
VARIABLES DETERMINING food presentations. With the FI 40-min schedule, how¬
RESPONSE FREQUENCY ever, there was an average of well over 1000 responses
in each 40-min period. Ferster and Skinner (1957)
Although interval schedules require only a single show numerous records with several thousand re¬
response per reinforcer presentation, they maintain sponses occurring in an interval with no sign of
many more. At moderate and large parameter values, strained behavior.
a fixed-interval schedule will maintain a larger aver¬ Why can a fixed interval schedule maintain so
age number of responses than can be maintained by many responses? An answer to this question helps to
ratio schedules. For example, Ferster and Skinner reveal the variables responsible for response rate
(1957, pp* 518-520) correlated one stimulus with an under both interval and ratio schedules.
FI 5-min schedule and another with an FR 275 sched¬
ule (multiple FI 5-min FR 375). Responding was
Response Number in Fixed-interval Schedules: The
severely strained under the fixed-ratio schedule with
Herrnstein and Mor5e (1958) Experiment
periods of 80 minutes and more occurring without a
response. However, if the fixed-interval stimulus was An important factor in the ability of fixed-interval
introduced during the pauses, more than 275 re¬ schedules to maintain a high average number of re¬
sponses often were emitted within the 5-min period. sponses per reinforcer is simply that they require only
In general, it is difficult to maintain responding with one. Herrnstein and Morse (1958) drew attention to
fixed-ratio schedules higher than FR 300 even after this apparent paradox in their investigation of a con¬
prolonged exposure to lower values. Yet an average of junctive fixed-interval, fixed-ratio schedule. Since, in
300 responses per reinforcer presentation is main¬ a conjunctive schedule, the reinforcer is delivered
tained routinely with fixed-interval schedules. Figure when both individual schedule requirements have
4 shows cumulative records for the same pigeon under been met, the direct effects of the schedule involved
both the minimum number of responses specified by
the fixed-ratio component and the minimum interrein-
PIGEON 136 forcer time followed by a single response specified by
the fixed-interval component. In a conjunctive FI 15-
min FR 40 schedule, for example, a reinforcer is
presented following the first response after 15 minutes
if at least 39 responses have occurred earlier. Other¬
wise, the reinforcer is presented as soon after 15 min¬
utes as the 40th response is emitted. The conjunctive
FI FR schedule imposes minimum response require¬
ments on the fixed-interval schedule, the minimum
value depending on the parameter of the fixed-ratio
component. (It also imposes a minimum interrein-
forcer interval in a fixed-ratio schedule.) Herrnstein
and Morse maintained the interval value at 15 min¬
utes and varied the ratio value from zero (a simple
fixed-interval schedule) up to 240.
The left panel of Figure 5 shows the average num¬
Fig. 4. Cumulative records for the last session of training of a
pigeon with FR 600 and the first session with the FI 40-min ber of responses per 15-min interval under each ratio
schedule. The response pen reset at 1100 responses and with requirement. As the ratio was increased, the number
food presentation. Offsets of the event pen on the FI record
indicate when food became available for the next response.
of responses per interval decreased. Both birds emitted
Breaks in the FR record indicate periods with no responses. close to 300 responses per interval with the simple
Michael Zeiler 207

0 40 80 120160200 240 0 40 80 120160200 240

RESPONSES REQUIRED RESPONSES REQUIRED

PER INTERVAL PER INTERVAL

Fig. 5. The left panel shows the number of responses per 15

min; the right panel shows the mean interreinforcer time. (Data
from Herrnstein 8c Morse, 1958. (c) 1958 by the Society for the
Experimental Analysis of Behavior Inc.)
10 Minutes

Fig. 6. Performance of a pigeon under a conjunctive FR 1 FT

fixed-interval schedule, but averaged 100 or less when 2-min schedule compared with an El 2-rnin schedule. The re¬
food presentation required 240 responses. When 240 sponse pen reset at each food presentation,

responses were required, one bird took more than 4

hours to obtain food (right panel), yet without the
ratio requirement an average of more than 240 re¬ subsequently. This is a dynamic effect„ because the
sponses was emitted in 15 minutes (left panel). value obtaining at one time can produce a different
These data indicate that an important factor in value later which will then itself determine the next
fixed-interyal performance is that the schedule does value and $o forth. It is an effect of an indirect vari¬

not require more than one response. It is important, able, because the schedule does not specify how many
though, that this response be in close temporal con¬ responses (beyond one) must occur.
tiguity to the reinforcer. A conjunctive FT FR 1 According to this hypothesis, a high frequency of
schedule also requires a single response, hut it does responding in an interval caused by preceding low-
not guarantee that it immediately precede the rein- response intervals. The simple fixed-interval schedule
forcer. The data shown in Figure 6 corroborate allows as few as one response, but the addition of a
Morgan's (1970) and Shull's (1970) reports that such a fixed-ratio requirement means that there must be at
schedule maintains responding, but at a substantially least the number of responses specified by the ratio.
lower level than a comparable fixed-interval schedule. Therefore, imposing a fixed-ratio requirement reduces
If the response requirement is entirely eliminated by responding by preventing a dynamic effect responsible
changing a fixed-interval to a fixed-time schedule, re¬ for high numbers of responses.
sponding will eventually either fall to a low level or The role of variation in the number of responses
cease (Herrnstein, 1966; Zeiler, 1968). The responding per reinforcer is evident from comparisons of variable-
maintained by a fixed-interval schedule evidently in¬ ratio and fixed-ratio schedule. Ferster and Skinner
volves something other than the simple requirement (1957, pp. 407-410) established responding under a VR
of a single response per reinforcer presentation and/or 360 schedule, and they then changed the schedule to
the temporal regularity of the reinforcing stimulus. FR 360. There were more responses under the vari¬
Herrnstein and Morse attributed the high average able ratio; in fact, the fixed ratio did not always main¬
frequency of responding on the fixed-interval schedule tain responding. At the same average number of re¬
to the dynamic effect of the indirect variable number sponses per reinforcer, therefore, variable numbers can
of responses per reinforcer. Consider the following maintain more responses than fixed numbers.
hypothesis: The number of responses in an interval The maximum number of responses per reinforcer
is determined by the number of responses in pre¬ can be restricted without affecting the ability of a
ceding intervals. High-response intervals (many re¬ schedule to sustain responding. Neuringer and
sponses per reinforcer) generate few responses in Schneider (1968) used an FI 30-sec schedule in which
subsequent intervals; low-response intervals (few re¬ each response prior to the last produced a blackout
sponses per reinforcer) generate many responses (an intertrial interval). By varying the duration of the
 208 SCHEDULES OF REINFORCEMENT

sion (second-order deviations); response rate changed

within individual intervals (third-order deviations);
individual responses tended to occur in groups (fourth-
order deviations). First-order deviations have not re¬
ceived attention subsequently, while fourth-order
deviations may occur with all schedules (Blough, 1963;
Skinner, 1938). The distinctive characteristics of fixed-
interval performance are the second- and third-order
deviations. The third-order deviation—the pattern of
responding or the distribution of responses in the
time between successive reinforcer presentations—is a
most important characteristic of different schedules
and will be treated separately. The second-order devia¬
tions—the varying number of responses per interval-
are of main concern now. Other investigators have
also found that this deviation remains after extended
exposure to fixed-interval schedules (Cumming 8c
Schoenfeld, 1958; Dews, 1970). A satisfactory explana¬
tion of fixed-interval performance must explain the
variability in response number per interval. This
Fig. 7. Latency m seconds of the pigeons’ first response after variability is illustrated in Figure 8.
fOOd pr6SGnt3diOfl (filled points) and the time Retween successive Dews (1970) has shown that under an FI 3-min
subsequent responses (open points). (Left) Maximum number schedule the number of responses in an interval can
of responses per rcinforcer on an LI 30-sec schedule. (Right)
Minimum interreinforcer time on an FR 15 schedule. (Data
vary over nearly a 50-fold range. In analyzing the rela¬
from Neuringer & Sehn^ide*, 1908.) tions among the number of responses emitted in 200
consecutive 3-min fixed intervals. Dews found two
interesting phenomena. The first was shown by classi¬
blackout and measuring the interval in real time (by fying intervals in terms of which of six class intervals
adding the blackout durations to the time spent re¬ described the number of responses. There was a gen¬
sponding), they restricted the total number of re¬ eral tendency for intervals with many responses to
sponses that could occur. For example, if the blackout follow intervals with many responses. After an un-
duration was 4.96 sec, no more than six responses
could occur. As in the ordinary fixed-interval schedule,
the reinforcer could follow a single response. The left
FI 15-Min
panel of Figure 7 shows that blackout durations rang¬
ing from ,34 sec (maximum of 88 responses per rein¬
forcer) to 4.96 sec (maximum of six responses) did not
change the latency of each response. To the extent
that this discrete-trial procedure involving intertrial
intervals of different durations is related to the typical
fixed-interval schedule, it shows that behavior is un¬
affected by restricting the maximum number of re¬
sponses. The important factor, as shown by Herrn-
stein and Morse (1958), is that the possibility of few
responses be preserved.

Cyclicities in Responding

Skinner (1938, pp. 123-126) found that responding

Fig. 8. Performance of a pigeon under an FI 15-min schedule.
under fixed-interval schedules varied in four ways.
The response pen reset at each food presentation. Offsets of the
There were oscillations in the number of responses event pen indicate when the 15-min interval timed out. The
per session (first-order deviations); response frequency numerals adjacent to each interval indicate the number of
responses in the interval to the nearest five responses as mea¬
changed from one interval to the next in each ses¬ sured from the record.
Michael Zeiler 209

predictable number of high-response intervals, one or disclose its effects on behavior. What happens when
more low-response intervals occur and the cycle re¬ number of responses per reinforcer is manipulated
peats. There may also be a series of intervals having directly?
about the mean number of responses per interval. As
Dews says, there is “irregular periodicity that was seen
Number of Responses and Fixed-Ratio Performance
as a waxing and waning of the prevailing numbers of
responses in sequence of intervals” (p. 59). It seems Felton and Lyon (1966) and Powell (1968) extended
evident, therefore, that the relations controlling the Ferster and Skinner’s (1957) investigations of the
number of responses in successive intervals may not effects on pigeons of varying the fixed-ratio value.
operate immediately from one interval to the next but Both found that the duration of the initial pause in¬
instead are cumulative effects over at least several creased as the ratio increased. In some subjects the
intervals. postpause rate decreased with increases in the ratio,
A second kind of cyclicity was revealed by ignoring and in others the changes were less clear. Thus over¬
the absolute number of responses and considering all response rate (total responses divided by pause
only the direction of change from one interval to the time plus the time spent responding) depends on, but
next. Dews found an alternation pattern in which is not linearly related to, the number of responses per
intervals tended to be preceded and followed by inter¬ reinforcer.
vals having more responses. “A second-order effect, This relation corresponds to that hypothesized to
alternation, was occurring during the session to a account for response number fluctuations with fixed-
slight degree but . . . quantitatively this effect was interval schedules and to the results reported by
small (and as a matter of fact, inconsistent from sub¬ Herrnstein and Morse (1958) with conjunctive FI FR
ject to subject)” (p. 58). Shull’s (1971) data on sequen¬ schedules. Here, too, a small number of responses per
tial relations among initial pause durations can be reinforcer generated subsequent high average rates
interpreted as also showing a small alternation ten and many responses generated lower average rates.
dency on the assumption that pause duration and re¬ Since the hypothesized indirect effects were consistent
sponse number covary. Randolph and Sewell (1968) with those found when the variable was imposed
found that low-response intervals do tend to be fol¬ directly, the conclusion that the number of responses
lowed by intervals with many responses, but that there per reinforeer operates as an indirect determinant
is not an equally strong tendency for many responses gams plausibility,
to be followed by few. In general, then, there is a long¬ The fixed-ratio schedule docs not allow the number
term effect of number of responses per reinforcer felt of responses to have dynamic effects, because the nuim
over several intervals and a more immediate but her is held constant. Instead, it reveals how behavior
smaller effect seen from one interval to the next. is related to response number, With sufficiently high
Herrnstein and Morse's (1958) explanation of fixed- constant numbers (high ratios), a point is reached at
interval responding in terms of the dynamic effects of which overall response rate drops and responding is
the number of responses per reinforcer suggests that poorly maintained in the absence of special histones.
there should be oscillation in response number in suc¬ The precise nature of these histories is not well under¬
cessive intervals. In fact, the hypothesis that response stood.
number has dynamic effects seems to originate in Skin¬
ner’s (1938) and Ferster and Skinner’s (1957) attempts
Interreinforcer Time
to explain the cyclicities. According to these views,
intervals containing few responses generate intervals In general, at values under which fixed-ratio sched¬
containing many responses, and these in turn generate ules maintain responding readily, the responses are
few responses which then produce many responses and emitted at higher rates than occur with fixed-inter¬
the cycle repeats. Apparently, the effects of a high re¬ val schedules generating about the same average num¬
sponse interval are not immediately to generate few ber of responses per reinforcer. This conclusion is de¬
responses in the next interval. Instead, effects seem to rived from Ferster and Skinner’s (1957) separate
reveal an accumulation over several successive experiments involving fixed-ratio and fixed-interval
intervals. schedules. If this is in fact the case, the number of
In fixed-interval schedules, number of responses per responses per reinforcer cannot be the sole determi¬
reinforcer operates indirectly and its role must be in¬ nant of response rate, because response number can
ferred. However, fixed-ratio schedules establish num¬ be the same under both types of schedule while rate
ber as an explicit independent variable and thereby varies.
210 SCHEDULES OF REINFORCEMENT

The reason that equal responses per reinforcer can achieved for the first bird under VR. For one pair of
generate higher rates on ratio than on interval sched¬ birds the variable-ratio schedule generated two- to
ules lies in the relation of responding to the time be¬ threefold higher rates than did the yoked variable-
tween reinforcer presentations (interreinforcer time). interval schedule. Interreinforcer time could not be
The interreinforcer time is a function of response rate responsible for the difference since it was the same;
with ratio schedules, but its minimum is specified by instead, the important factor had to be the depen¬
the parameter value of interval schedules. When in¬ dence of interreinforcer time on response rate for the
terreinforcer time is controlled on ratio schedules, re¬ ratio bird.
sponse rate is affected markedly. Consider again A second effect occurred for another pair of birds.
Herrnstein and Morse’s (1958) conjunctive FI FR Responding could not be maintained under the vari¬
schedules, but this time from the point of view of the able-ratio schedule after the change from the initial
ratio component. Interreinforcer time is not free to variable interval. That one pair of birds revealed a
decrease to less than 15 min because of the FI 15-min higher rate under VR than VI while the other stopped
requirement. Consequently, the minimum yalue is responding under VR is not contradictory. When re¬
controlled. Also consider that Felton and Lyon (1966) sponding is well maintained by a ratio schedule, the
found that under simple FR 150 schedule birds aver¬ rate is higher than with an interval schedule gen¬
aged somewhat more than 200 seconds to emit 150 re¬ erating comparable numbers of responses per rein¬
sponses. Herrnstein and Morse7 in contrast found forcer; however, interval schedules will maintain a
that with a conjunctive FI 15-min FR 120 schedule number of responses per reinforcer that cannot be sus¬
the shortest average interreinforcer time was more tained by ratio schedules.
than SO min (Figure 5? right panel). The importance Responding in interval and ratio schedules, in sum¬
of leaving the minimum interreinforcer time uncon¬ mary, reflects the operation of the number of re¬
trolled to obtain high rates under ratio schedules sponses per reinforcer and the interreinforcer time.
seems evident. This was also shown by Neuringer and The response number characteristics of interval sched¬
Schneider (1968): They u§ed an FR 15 §chednle in ules arise from the dynamic role of varying numbers
which each response before the last produced a black¬ of responses per reinforcer combined with constant
out period, while the fixed-ratio schedule held the interreinforcer times: the rate characteristics of ratio
number of responges constant. The right panel of schedules arise from constant numbers of responses
Figure 7 §h<g>w§ that th§ lunger the mt^rreinfpreer per reinforcer combined with behavior-dependent in¬
time (the longer the blackouts)* the longer was the terreinforcer times. The parameters of each of these
latency of each response. These data are further evi¬ schedules are important in determining the perfor¬
dence that the relation between response and rein- mance engendered, because they establish the levels of
forcer rates is one reason for the high rates under ratio the variables either directly or indirectly.
schedules.

Variable-ratio schedules of moderate value main¬

Tondem Schedules with FI and FR Components
tain a high and fairly constant rate of responding
(Ferster & Skinner, 1957, Chapter 7). As in the case of The present analysis cannot explain why tandem
fixed-ratio schedules, the high rates depend on the FI FR schedules should have the effects that they
eireular relation between response rate and lnterrein- sometimes do. In such a schedule, completion of a
lorccment time. This was shown by Ferster and Skin¬ fixed interval requirement initiates the ratio require¬
ner (pp. 399-407) in the following way, ment, and completion of the ratio then produces the
Pigeons were matched in their stable response rates reinforcer. There are no stimulus changes correlated
under a variable-interval (VI 5-min) schedule. Then with the different schedule components. Ferster and
one bird was changed to a variable-ratio schedule in Skinner (1957, pp. 416-422) found that a tandem FI
which the number of responses per reinforcer was 45-min FR 10 schedule increased response rate above
chosen to match the number emitted on the variable- that occurring with an FI 45-min schedule. Note that
interval schedule. The second bird was yoked to the the tandem schedule places restrictions on both the
first. When the first received food, the second obtained minimum interreinforcer time and the minimum
food for its next response. No other responses were number of responses per reinforcer. The rate increase,
required for the second bird. This generated a vari¬ therefore, is unexpected. Ferster and Skinner (p. 429)
able-interval schedule for the second bird, with inter¬ also showed that responding was maintained when the
reinforcer time under the VI schedule matching that ratio component was increased to FR 400. Parametric
Michael Zeiler 211

analyses are necessary to show whether rate increases the first component was a ratio of FR 25, FR 50, or
are general effects or occur only when the interval FR 100 and the second involved an FR 2 schedule
component is so large that it maintains a high average with the 2 responses separated by a blackout. A com¬
number of responses per reinforcer by itself. puter recorded the time spent in the first component
(interreinforcer interval). It then also recorded the
The Relation Between Responses Per Reinforcer
latency of the first response of the second component
and Interreinforcer Time
and adjusted the duration of the blackout so that it
ended when the latency plus the blackout equaled the
Experiments studying the effect of fixed-ratio size interreinforcer interval of the first component. The
on response rate consistently have confounded the next response produced food. The result was that the
direct effects of responses per reinforcer with the in¬ interreinforcer time was the same in both components,
direct effects of interreinforcer time. As ratio size was but one required 25, 50, or 100 responses whereas the
increased, the birds took more time to obtain each other required 2.
reinforcer. Therefore, increased interreinforcer time Interreinforcer time in the first component in¬
rather than, or in addition to, increased response num¬ creased as the ratio was raised. Figure 9 shows that the
ber may have produced the lower overall rate. Changes latency to the first response of both components in¬
in both variables occurred in Herrnstein and Morse’s creased as well. Since the response requirement was
(1958) experiment as well. Under the conjunctive FI constant at two in the second component, interrein¬
FR schedule any time the ratio requirement had not forcer time clearly was relevant. In addition, however,
been met by the time the 15-min interval had elapsed, the latency was always shorter under the 2-response
interreinforcer time increased. The right panel of requirement than with the 25-, 50-, or 100-response
Figure 5 shows that interreinforcer time did increase requirement even though interreinforcer times were
as a function of the size of the ratio, and the left panel matched in both. In other words, response number
shows that response rate decreased accordingly. The was important independent of interreinforcer time.
effects observed by Herrnstein and Morse may have Although the response-produced blackouts in this and
been due to responses per reinforcer, to interreinforcer Neurmger and Schneider’s experiment probably
time, or to some combination of the two. exerted distinctive effects that themselves still need to
Neuringer and Schneider’s (1968) attempt to sepa¬ be isolated, the experiments do suggest that both in-
rate the two variables was only partially successful.
When they maintained an FR 15 schedule while vary¬
ing interreinforcer time, they found that, as the time
increased, response latency (time to the first response)
increased as well (Figure 7, right panel). This demon¬
strated the role of interreinforcer time independent of
the numbers of responses. Further, when they con¬
trolled interreinforcer time by establishing an FI 30-
sec schedule while limiting the number of responses
that could be emitted, latency was the same inde¬
pendent of response number (Figure 7, left panel).
Thus they demonstrated that restricting the maximum
number of responses had no significant effect on re¬
sponding. They did not, however, control how few
responses could occur, i.e., the minimum number of
responses per reinforcer. Since the minimum number
was unrestricted under all conditions, there was no
interference with the property now hypothesized to
induce high rates, and consequently no effect was
observed.
An experiment by Crossman, Heaps, Nunes, and FIXED RATIO (FRn)
Alferink (1974) showed that minimum number of re¬
Fig. 9. Latency of the first response on FR 25, FR 50, and FR
sponses exerted effects with interreinforcer time con¬
100 schedules and on FR 2 schedules equated for interreinforcer
trolled. They arranged a multiple schedule in which time. (Data from Crossman et al, 1974.)
212 SCHEDULES OF REINFORCEMENT

terreinforcer time and minimum response require¬ maintain many responses (e.g., Herrnstein & Morse,
ment are important independent sources of control. 1957).

The Regenerating Power of Interval Schedules Adjusting Fixed-Ratio Schedules

The interval schedule tends to have the number of Research involving adjusting fixed-ratio schedules
responses per reinforcer regress toward the mean. further supports the hypothesis that schedules will
When responding is strong and many responses occur, maintain a large average number of responses if they
the number of responses per reinforcer becomes high, provide a reinforcer whenever responding weakens.
and responses subsequently occur less frequently. In an adjusting fixed-ratio schedule, the ratio require¬
When responding is weak; this means few responses ment is changed based on some behavioral criterion.
per reinforcer and a subsequent high response rate. For example, Ferster and Skinner (1957, pp. 718-720)
The occurrence of a reinforcer when the tendency to imposed an FR 160 schedule. Whenever the bird
respond is low may have another related effect: should paused for 2 min or more before making the first re¬
the organism stop responding for a time exceeding the sponse, the schedule became FR 1. This meant that
interval parameter Value, the HCXt rCSpOUSC Will be whenever the FR 160 schedule failed to support
followed by the reinforcer and responses will be re¬ responding, a single response produced the reinforcer.
generated. The adjusting schedule maintained a high response
Ratio schedules do not share this regenerating char¬ rate without long pauses; only occasionally did the
acteristic: no matter how much time elapses without a FR 1 condition operate. Comparisons with other data
response; a remfore^r will not occur following the suggest that the average rate was higher than might
next instance, unless it just happens that the next be expected with an FR 160 schedule alone.
one ends the ratio. Because ratio schedules do not In a second experiment (pp. 720-721) Ferster and
provide a reinforcer just when It Is needed to revive Skinner employed a continuously adjusting schedule.
weak behavior, they are usually involved when a re¬ (Actually, it wag an interlocking schedule, since, in a
sponse is poorly maintained (Morse, 1Q66, p, 86). given component one parameter decreased as a func¬
If this regenerating potential is important, a vari¬ tion of the other.) Whenever the initial pause was less
able-ratio schedule that matched a high-valued fixed- than 25 sec, the ratio increased by 5 responses. During
interval schedule in the sequence of response numbers the first 25 sec the ratio decreased slowly unless there
per reinforcer would not maintain responding as well was a response (the rate of decrease is not specified).
as the fixed interval. The variable-ratio schedule does Responding was maintained in three birds with ratios
not automatically adjust the response requirement of 445, 600, and 650 responses respectively. Such ratio
downward when performance weakens. Since a cor¬ values do not sustain behavior easily; for them to do
respondence between weak behavior and a low ratio so, it appears necessary to adjust requirements to the
would be entirely fortuitous, the variable-ratio sched¬ ongoing behavior.
ule lacks a perfectly tuned regenerating characteristic. The most systematic study of adjusting schedules
The only relevant information appears in Ferster and was that of Kelleher, Fry, and Cook (1964, Experi¬
Skinner’s (1957) comment about variable-ratio sched¬ ment 1) with squirrel monkeys. It differed from the
ules: “As m all schedules requiring a number of re¬ Ferster and Skinner experiments in that pausing
sponses, the bird will stop responding altogether if affected the ratio requirement in a subsequent rather
the average number goes beyond a certain value” (p. than in the current component (by Ferster and Skin¬
391). In contrast, there is no apparent limit to how ner’s definition, it was an adjusting as opposed to an
high an average number of responses can be achieved interlocking schedule). A session began with an FR 10
with interval schedules. If higher rates should occur schedule. If two successive postreinforcement pauses
on the variable-ratio schedule, however, it would sug¬ were shorter than time t, the ratio changed to the next
gest that the regenerating potential is not important. higher value. Sixteen ratio values increased in steps
The IRT>f schedule has the same built-in regen¬ from FR 10 to FR 1000. If two successive pauses ex¬
erating property as interval schedules. Under IRT>f ceeded t, the ratio decreased by one step (except that
schedules, whenever responding becomes weak enough one pause exceeding t would reduce the ratio from
that the organism pauses beyond the parameter value, FR 1000 to FR 870). The ratio remained constant if
the next response is followed by the reinforcer. As two successive pause durations were alternately longer
would be expected, therefore, these schedules also and shorter than t. The values of t were 1, 2, 4, 8, and
Michael Zeiler 213

15 min. Kelleher et al. pointed out that under simple cessive reinforcer presentations. These patterns of re¬
fixed-ratio schedules, performance usually shows long sponding are stable and characteristic of the schedule.
pauses with schedules higher than FR 100. With the The fixed-interval pattern is Skinner’s (1938) third-
adjusting schedule, the average maintained fixed-ratio order deviation from responding at a steady rate. Skin¬
value increased with increases in t, but all levels of t ner describes it as follows: “Deviations of a third
produced averages higher than 100 responses per rein¬ order appear as depressions in the rate of elicitation
forcer. Average ratio values over 200 were typical with after the periodic reconditioning of the reflex. They
t values from 2 to 8 min, and the average number of are followed typically by compensatory increases, so
responses was over 400 when t was 15 min. So a fixed- that the total rate is unchanged” (p. 125). Much sub¬
ratio schedule that lessens the response requirement sequent research has confirmed that there typically is a
when responding weakens maintains a substantial pause followed by responding. Postpause responding
level of responding. may display either continuous positive acceleration,
positive acceleration changing to a steady high rate,
Interlocking Fixed-Ratio, or an abrupt transition from no responding to a high
Fixed-Interval Schedules
rate (e.g., Cumming & Schoenfeld, 1958; Ferster 8c
A schedule that would seem to adjust ratio require¬ Skinner, 1957; Schneider, 1969). Responding under
ments to the momentary strength of responding is the variable-interval and random-interval schedules is
interlocking fixed-ratio, fixed-interval schedule. In fairly constant between successive reinforcers. Catania
such a schedule, the fixed-ratio value decreases as time and Reynolds (1968) found that the precise nature of
elapses since the preceding reinforcer and reaches the variable-interval pattern depended on how the
FR 1 when the interval has elapsed. The rate of de¬ constituent intervals were selected and how they were
crease in the ratio value is a function of the interval distributed. Response rate can be either positively
parameter: longer intervals mean slower decreases* If accelerated, negatively accelerated, or approximately
responses are emitted at a high rate, the reinforcer linear depending on the distribution of interr^m-
occurs after a number of responses close to the initial forcer intervals. Ratio schedules also produce charac¬
ratio value; if responding is slow, the reinforcer occurs teristic patterns. With fixed-ratio schedules, there is a
after fewer responses. pause followed by a transition to responding at a high
Parametric data on interlocking FR FI schedules rate? the pause duration is a function of the fixed-j-atio

were reported by Berryman and Nevin (1902). Unfor¬ size (e.g., & Lyon, 1966; Powell, 1968). Fre¬
Felton

tunately, from the point of view of the present argu¬ quently, the transition is abrupt, but sometimes there
ment, the ratio value used (FR 32) itself maintained is positive acceleration, and at times responding may
responding quite well. Combining the FR 32 schedule slow somewhat at the end of the ratio (Ferster 8c Skin¬
with a fixed-interval schedule in an interlocking FR ner, 1957). Under moderately valued schedules (e.g.,
FI reduced the overall rate of responding, with the FR 30) the rate is often very stable after the initial
amount of decrease positively related to the interval transition (Cott & Weiss, 1972; Weiss 8c Gott, 1972).
value* Apparently, the possibility of a reinforcer Variable-ratio and random-ratio schedules tend to
occurring when behavior weakens is unnecessary with produce responding at a high steady rate, at least as
small ratio schedules; under these conditions the judged from cumulative records (Ferster 8c Skinner,
occasional delivery of a reinforcer via an interval 1957; Schoenfeld et al., 1972). What variables are re¬
schedule seems to suppress responding. An interlock¬ sponsible for the patterns engendered by each
ing FI FR schedule should markedly increase respond¬ schedule?
ing if the ratio schedule is so large that it does not
maintain behavior very well by itself. This prediction
Temporal Placement of the Reinforcing Stimulus
has not yet been tested.
A differentiating characteristic of interval schedules
is the regularity of reinforcer presentation in relation
RESPONSE PATTERNING: THE to time. In fixed-interval schedules the reinforcer
TEMPORAL ORGANIZATION occurs at regular times (given that response rate is
OF BEHAVIOR sufficiently high—as it invariably is—to produce the
reinforcer close to when it becomes available), and in
Different schedules have distinctive effects on the variable-interval and random-interval schedules it
way responses are distributed in the time between suc¬ occurs irregularly. The placement of the stimulus in
214 SCHEDULES OF REINFORCEMENT

time plays a major role in determining the pattern of and time schedules) or number of responses per rein¬
responding. forcer (ratio schedules). With the fixed schedules, the
parameter value indicates where each reinforcer is
Interval and Time Schedules
located; however, with variable schedules, the param¬
eter indicates only the average location. (In random
Shifting from an interval to a time schedule main¬ schedules there is no indication of the location but
tains the temporal placement of the reinforcing stim¬ simply information about the probability of reinforcer
ulus while changing the tendency to respond. Interval presentation at any moment or for any response.) In
and time schedules can be matched for minimum an attempt to describe responding under variable-
interreinforcer interval, but they differ in their ability interval schedules, Catania and Reynolds (1968) have
to maintain a specific response. Time schedules main¬ related response rate at a given point in postreinforcer
tain a lower response rate than do interval schedules time to the availability of the reinforcer at that point.
(Appel & Hiss, 1962; Herrnstein, 1966; Zeiler7 1968). They proposed two measures of temporal placement
In fact, it is not unusual for the response to stop alto¬ of the reinforcer: (1) probability of reinforcement;
gether under time schedules. and (g) local rate of reinforcement.
Interval and time schedules do, however, maintain
the same pattern of responding (see Figure 1). One
Probability of Reinforcement
(Zeder, 1968) studied the effects of chang¬
ing from variable to fixed schedules and vice versa; One measure of reinforcement density within a
the transitions involved both interval and time sched¬ time period is how often remlorcers occur m a given
ule If the preceding schedule was of the same type in interval relative to how often the interval occurs.
terms of rcinforcer regularity, the same pattern was Catania and Reynolds (1968) call this probabilty of
maintained; if the schedules were of different types, reinforcement, or reinforcements per opportunity.
the pattern changed accordingly. In this respect it Consider a variable-interval 1-min schedule with the
made no difference whether the schedules were inter¬ following 20 interreinforcer intervals occurring in
val or time, although there were large effects on re¬ some irregular order.* 3, 7, 10, 22, 28, 30, 42, 45, 50, 54,
sponse rate. 1 he temporal pattern of reinforcer de¬ 6?, 65, 66, 67, 85, 90, 95, 105, 134, and 140 sec, These
liveries controlled the pattern of responding* individual intervals can be categorized in successive
Interval and time schedules generate similar pat¬ 20-sec class intervals as shown in column (a) of Table
terns despite the differences they produce in the be¬ 1. The result is seven class intervals. Column (b) shows
havior oCCttrriiig at the moment of reinforcer presen¬ the number of reinforcers in each of these class inter¬
tation {the interval schedule guarantees that it be a vals. Not each class interval has an equal opportunity
specific response, whereas the time schedule allows the for occurrence, since each can only occur when a
response to vary). These observations are not in ac¬ shorter interval is not in force. For example, the 21-
cord with analyses of patterning that stress the impor¬ 40-See class interval exists only when one of the three
tance of the events occurring m close contiguity with members of the 0-20-see class is not in effect, whereas
th<g reinforcer, Dew§ (1969) provides an additional the 121-140-sec class interval can only occur if no
indication that the precise quantitative relations ot> individual interval less than 121 sec is in force.
taining between responses and reinforcers do not con¬ Column (c) shows the number of opportunities for
trol patterning. Dews compared performance on a each class interval in each cycle of the 20 individual
fixed-interval schedule with that occurring when a interreinforcer intervals. The probability of rein¬
small fixed-ratio requirement or a short delay of rein¬ forcement for each class interval is computed by
forcement was added to the end of the interval. Some dividing column (b) by column (c). These probabil¬
of these manipulations changed the rate of respond¬ ities are shown in column (d); they represent the prob¬
ing, but none affected the pattern. As long as the ability of a reinforcer presentation in that region of
temporal placement of the reinforcing stimulus was time since the last reinforcer, given that the region
not markedly disturbed, the pattern was unchanged. has been reached.
Catania and Reynolds (1968) studied a number of
Quantitative Measurement of Temporal
different types of variable-interval schedules differ¬
Placement of the Reinforcing Stimulus
entiated by the way the individual interreinforcer
intervals were selected. All of these had the same
Fixed and variable schedules can be differentiated mean value; i.e., each was a particular-valued variable-
in terms of variability of interreinforcer time (interval interval schedule, but they all differed in the probabil-
Michael Zeller 215

Table 1 Quantitative Measurement of Reinforcer Placement in VI Schedules

(a) (b) (d)

(c) (/)
Class
(e)
Probability of Time Spent Local Rate of
Intervals Number of Reinforcer in Interval Reinforcement
(sec) Reinforcers Opportunities Presentation (sec) (Reinforcers 1 hr)

0-20 3 20 .150 360 30.0

21-40 3 17 .176 298 36.2
41-60 4 14 .286 228 63.2
61-80 4 10 .400 136 105.9
81-100 3 6 .500 87 124.1
101-120 1 3 .333 44 81.8
121-140 2 2 1.000 32 225.0

ity of reinforcement occurring with respect to succes¬ and 30-sec intervals) in which the class begins but is
sive class intervals. In three types of VI (arithmetic, terminated prior to 40 sec. The computations for the
linear, constant probability), reinforcement probabil¬ other class intervals are equivalent. The local rate of
ity in an interval and the response rate in that inter¬ reinforcement (column f) is computed by dividing the
val (local response rate) both increased as time number of reinforcers presented in a class interval
elapsed. However, in two other types (geometric, (column b) by tbe total time spent in that interval
Fibonacci), local response rate decreased as probability (column e).
of reinforcement increased. Therefore, probability of Figure 10 depicts the local rate of reinforcement
reinforcement in an interval cannot generally explain and the rates of responding at successive time periods
local response rate under variable-interval schedules. after reinforcer presentation in several types of vari¬
able-interval schedule (Catania 8c Reynold?! 1968).
Local Rate of Reinforcement
The correspondence between local response rate and
local rate of reinforcement was closer than the corre*
Gatama and Reynolds devised another measure of
spondence between local response rate and probabil¬
reinforcer density. This measure takes into account
ity Of reinforcement. Th§ fit i§ not exact, primarily be¬
not simply how often a given class interval occurs,
cause of departures at low and intermediate values;
but also how much time is spent in that interval rela¬
tive to the number of reinforce! presentations. Con¬ but the curves do not diverge as they sometimes do
for probability of reinforcement.
sider the 0-20-sec class interval in Table 1. Except
when the 3-, 7-, or 10-sec interreinforcer intervals are
in effect, the interval lasts for 20 sec. (This computa¬
Variable Interval
tional procedure is modified slightly from Catania
and Reynolds by not splitting the time between suc¬
cessive classes.) This will occur in 17 of the 20 inter¬
reinforcer intervals, yielding a total time of 340 sec
spent in the 0-20-sec class interval. In addition, the
organism will spend 3 sec in that class interval when
the 3-sec interval is scheduled, 7 sec when the 7-sec
interval is in effect, and 10 sec when the 10-sec interval
prevails. Over the 20 interreinforcer intervals, there¬
fore, the 0-20-sec class interval will be in effect for a
total of 360 sec. The time spent in each class interval 0 1.0 2.0 0 1.0 2.0 0 1.0 2.0 0 1.0 2.0 3.0 4.0 5.0 6.0

is shown in column (e) of Table 1. The 21-40-sec class Relative Time since Reinforcement
interval will be in effect a total of 298 sec, because
Fig. 10. Local rate of reinforcement (upper panels) in reinforce¬
there will be 3 intervals in which the class is never ments per hr and local rate of responding (lower panels) on
reached, 14 in which it will be in effect for the entire four different types of variable-interval schedule. Time units
20 sec (the 14 interreinforcer intervals longer than 40 on the abscissa are interreinforcer times relative to the average
interreinforcer interval (the VI value). (Data are for pigeons
sec), and a total of 18 sec (divided among the 22-, 28-, from Catania & Reynolds, 1968.)
216 SCHEDULES OF REINFORCEMENT

Local Rate of Reinforcement and the Table 2 Local Rate of Reinforcement in FI Schedules
Fixed-interval Pattern
(a) (b) (c) (d)
Catania and Reynolds (1968, Experiment 4) found Class Time Spent Local Rate of
Intervals in Interval Number of Reinforcement
that the local rate of reinforcement analysis did not
(sec) (sec) Reinforcers (Reinforcers / hr)
describe fixed-interval performance. Note that with
fixed-interval schedules the local rate of reinforce¬ 0-20 185 1 19.5
ment is 0.00 for all class intervals save the last, when 21-40 168 2 42.9
it is 1.00. Yet substantial responding occurs prior to 41-60 140 0 0.0
the end of the interval. 61-80 124 1 29.0
81-100 109 1 33.0
Several theorists have proposed that fixed-interval
101-120 . 83 2 86.7
performance constitutes two phases, one correspond¬ 121—140 49 1 73.5
ing to the period of not responding (the pause period) 141-160 23 2 313.0
and the other corresponding to the period of respond¬
ing (the work period). This general concept originated
The local response rate in a given class interval is
with Skinner (1938) and has been elaborated by
the total number of responses occurring in that inter¬
Schneider (1969) and Shull, Guilkey, and Witty
val divided by the total time spent in that interval.
(197£). Schneider noted, as Shull (1970) later con-
This procedure is analogous to that used with variable-
firmed, that the work period is of variable duration
interval schedules by Catania and Reynolds, except
(the period of pausing varies from one interval to the
that the segments are defined with respect to the on¬
next). He suggested, therefore, that the work period
set of responding in each interval.
i§ correlated with a variable-interval schedule, where¬
Figure 11 shows the local rate of responding and
as the pause period is a period q£ extinction, Ac¬
the local rate of reinforcement computed as described
cording to Schneider, the pattern observed under a
above for 150 successive reinforcer presentations un¬
fixed-interval schedule is the outcome of extinction
der an FI 5-min schedule, The detailed correspon¬
followed by a variable-interval gchedule.
dence between the two curves is not close, although
Schneider’s account suggests the following applica¬
generally both curves increase for three of the four
tion of Catania and Reynolds’ (1968) local rate of
birds. The curves resemble those found by Catania
r£itt£orc£M£rit analysis. Define the lnterreinforcer in¬
and Reynolds with arithmetic variable-interval sched¬
tervals by measuring the time spent responding in
ules. Hence fixed-interval performance follows no
each individual fixed interval. The only difference
more (but also no less) precisely from a local rate of
from VI is that the interreinforcer intervals are cal¬
reinforcement analysis than does variable-interval
culated from beha.vi6r rather than imposed directly.
Compute the local rates of reinforcement from these
performance. The conceptualisation of fixed-interval

values. For example, if, under an FI 5-min schedule

the organism began responding 3 min prior to the end Fixed Interval
of a particular interval, the value of 180 sec would be
assigned to that interval. Consider 10 such intervals
having the values of 5, 30, 40, 65, 100, 110, 115, 130,

145, and 160 sec. Column (a) of Table 2 shows these

values arranged into 20-sec class intervals, and column

(b) shows the total amount of time spent in each. For
example, if responding began 65 sec before reinforcer
presentation, 20 sec was spent in each of class inter¬
vals 0-20, 21-40, and 41-60, and 5 sec was spent in
interval 61-80. No time was spent in any of the longer
classes. The number of reinforcers obtained in each
class interval appears in column (c). For example, if
responding began 30 sec before the end of the fixed
interval, the reinforcer occurred in the 21-40-sec class.
The local rate of reinforcement for each class interval Fig. 11. Local rate of reinforcement in reinforcements per hr
and local rate of responding on a FI 5-min schedule in pigeons.
(column d) was obtained by dividing column (c) by
The reference point is the break point. See text for a detailed
column (b). explanation. (Davis & Zeiler, unpublished data.)
Michael Zeiler 217

performance as a combination of an extinction and 60 sec. Under one condition any food presentation
a variable-interval schedule does not conflict with available in the first 60 sec occurred only if a key peck
what is known about variable-interval performance. had not occurred for 10 sec (R > 10 sec). If the food
Whether this will help in understanding either fixed- was available after 60 sec, it was produced by pecking
interval or variable-interval behavior is still unclear. according to a variable-interval schedule. Under an¬
One is now confronted with explaining variable- other procedure the schedule orders were reversed: a
interval performance, and it perhaps may be best peck was required in the first 60 sec, and the R > t
understood as a combination of fixed-interval sched¬ schedule was in effect thereafter.
ules. The first procedure produced pauses followed by a
substantial rate of key pecking, and the second pro¬
duced pecking followed by pausing. Thus the tem¬
Patterning Under Ratio Schedules
poral placement of the reinforcing event with respect
Ratio schedules establish certain inevitable rela¬ to two different forms of behavior determined the
tions between time and the reinforcing stimulus, al¬ probability with which each occurred at a given point
though these are indirect rather than direct effects of in time.
the schedule. In fixed-ratio schedules the time that
elapses from the first opportunity to respond until the
The Components of Temporal Placement
reinforcer appears depends on ratio size and response
rate. A reason for the initial pause and for its direct The temporal pattern of reinforcers explains the
relation to ratio size (Felton 8c Lyon, 1966; Ferster & pattern of responding on different schedules. Temporal
Skinner, 1957; Powell, 1968) may be the relation be¬ placement refers to the occurrence of reinforcers in
tween one reinforcer and the time until the next. The time in relation to some reference point, e.g., the last
pause might be attributed to the zero probability of reinforcer presentation or the beginning of a trial.
reinforcement for the first response, but then it is Attempts to further analyze temporal placement have
focused on two processes: temporal discrimination and
difficult to understand why the first response should
ever occur. Since the differential presentation of the delay of reinforcement.
reinforcer with respect to time always produces re¬
The Theoretical Background
sponding prior to the time the reinforcer is available,
a pause followed by responding occurs. The impor¬ The use of delayed reinforcement to explain pat¬
tance of the interreinforcer interval is further implied terning m sequences originated with Hull's (19S?) Re-
by Killeen's (1969) finding that pausing was about the Count of why rats chose the shorter of two paths to the
same in a fixed-ratio schedule and in an interval same goal and why they ran faster in the first section
schedule yoked to it in terms of interreinforcer time. of a short alley than in the first section of a longer al¬
The shorter pause under variable-ratio schedules ley. Because the distance to reward differs, responding
would follow from the occasional occurrence of the is reinforced with different delays. On the assumption
reinforcer close in time to the preceding one. Given that habit strength is inversely related to delay, choice
the possibility of temporal control, it would be of of the shorter path or faster running in the shorter
interest if Catania and Reynolds’ (1968) local rate of alley indicates that sequences of responses are influ¬
reinforcement analysis of variable-interval perfor¬ enced by their temporal remoteness from the reinforc¬
mance would apply equally well to behavior under ing stimulus.
variable-ratio schedules. This account of pausing fol¬ In 1952 Hull distinguished between experiments
lowed by responding under ratio schedules is similar involving short and long alleys and those involving a
to that offered by Skinner (1938), Ferster and Skinner delay period imposed after a single response. The first
(1957), and Morse (1966). type presents reinforcers immediately upon the com¬
pletion of the final response of the entire sequence
Temporal Placement and Not Responding (i.e., upon entering the goal box), so that the rein¬
forcer is delayed with respect to earlier parts of the
Staddon’s (1970, 1972) experiments provide further sequence. Hull described such procedures as involving
evidence that the pattern of responding depends on a gradient of reinforcement within a chain (chaining
the temporal placement of the reinforcing event in delay). The second procedure imposes delay after the
relation to responses. During some intervals food was terminal response; it does not involve intrinsic de¬
available in the first 60 sec following a food presenta¬ lays due to the relations between early and later
tion; during other intervals it was only available after parts of sequences (nonchaining delay).
218 SCHEDULES OF REINFORCEMENT

Spence (1956) pointed out that Hull distinguished be discriminated, and it has never gone beyond a re¬
chaining and nonchaining delay procedurally but not iteration of the independent variables; e.g., it happens
theoretically. Spence, in contrast, believed them to be when reinforcement occurs at a certain point in time.
the outcome of different processes. He proposed that Thus the process interpretation simply tacks on tem¬
behavior in the chaining situation was due to the poral discrimination to the relation between the inde¬
similarity of goal box stimuli to stimuli occurring pendent variable and observable performance. A more
earlier in the chain. Response decrements in the non¬ meaningful usage of temporal discrimination proposes
chaining situation were attributed to other responses that elapsed time may function as a stimulus. A test¬
emitted during the delay period, and because they able implication is that the time elapsing from the start
occur closer in time to the reinforcer these other re¬ of an interval has discriminative properties. Since, in a
sponses compete with the target response. Whereas fixed-interval schedule, responding is never reinforced
Hull believed that response strength was the outcome just after the interval begins, this period may serve as
of delayed reinforcement in both chaining and non- a discriminative stimulus controlling a low rate of
chaimng delay situations, Spence maintained that the responding. The same is true of fixed-ratio schedules,
two situations had little in common and that neither because responding just after reinforcer presentation
represented a direet functional relation between re¬ is never followed by another reinforcer. The analysis
sponse strength and delay. can also be extended to patterning under variable
Dews’s (1962) interpretation of fixed-interval sched¬ schedules, where a reinforcer does sometimes occur
ules is very similar to Hull’s analysis of chaining de¬ just after the preceding one. The period immediately
lay, and Dews, like Hull, used data from nonchaining- following reinforcer presentation does not come to
delay experiments to support the hypothesis. Skinner function as a negative discriminative stimulus, and
(1938), on the other hand, separated the two situa¬ responses are emitted at a more stable rate.
tions. His explanation of fixed-interval performance
resembled Spence’s account of chaining delay* It drew
Experiments on Temporal
heavily on elapsed time as a stimulus; i.e., responding
Discrimination
depends on the similarity of the temporal stimuli at
any point to the temporal stimulus correlated with Catania (1970) described several experiments deal¬
rcinforccr presentation, This account emphasized stim¬ ing with the discrimination of time. The temporal dis¬
ulus properties rather than a theoretical gradient of crimination experiment is identical to other experi¬
reinforcement. Since Skinner also dealt with non¬ ments on stimulus control, differing only in that time is
chaining delay in terms of competing adventitiously the stimulus dimension manipulated. In a study of
reinforced responses, his accounts of both chaining wavelength discrimination, the experimenter correlates
and nonchaining delay are similar to Spence’s. certain wavelengths with a reinforcing stimulus and
In the case of chaining delay, then, the issue is Others With different consequences; in a study of tem¬
whether responding depends on the proximity to the poral discrimination, different time durations are
end of the cham (Hull, Dews) or on the similarity of selectively correlated with the reinforcer.
the conditions at any point to those present at the In a temporal discrimination experiment, Stubbs
moment of reinforcer presentation (Skinner, Spence). (1968) had the center member of a three-key display
Is fixed-interval performance due to the effect of a transilluminated with white hght for 1 of 10 dura¬
gradient of reinforcement on response strength, or tions. Which of the two side keys was then correlated
does it depend on the time since the beginning of the with reinforcer availability depended on the preced¬
interval in relation to the total interval duration? An ing duration: if the center key had been lit for 1 of
answer to this question would indicate whether delay the 5 shorter durations, responses to one of the keys
of reinforcement or temporal discrimination is the produced food. If it had been lit for one of the longer
major controlling aspect of the temporal placement of durations, responses to the other key produced food.
reinforcement. By plotting responding to each key as a function of
the preceding white key duration, Stubbs was able to
Temporal Discrimination
compute the Weber fraction for discrimination of
The term temporal discrimination has a confusing time. There was dear differential responding with
history. Sometimes it has been used as a process to ex¬ respect to duration as the antecedent stimulus. An¬
plain behavior—e.g., the fixed-interval pattern is the other example of a temporal discrimination experi¬
outcome of the discrimination of time. But this shifted ment was provided by Reynolds and Catania (1962).
the problem to an explanation of how time comes to Pigeons were exposed to a dark key for durations
Michael Zeller 219

ranging from 3 to 30 seconds. When the key was 1964; Dews, 1960; Skinner, 1938, pp. 139 ff.). This pro¬
then illuminated, responding was reinforced only if cedure also has little relevance to fixed-interval sched¬
the preceding dark key duration had been a specific ules when the presumed delay period for each response
value (e.g., 3 sec). The birds responded at the highest is unaffected by subsequent responses.
rate following the duration correlated with reinforcer The situation most nearly equivalent to fixed-inter¬
availability and at progressively lower rates with in¬ val schedules is one without delay stimuli and resets.
creasingly different durations. Clearly, duration can Dews (1960) arranged delays of 10, 30, or 100 sec be¬
have discriminative properties. tween a key peck and food delivery. Responding was
maintained with the 100-sec delay, but at a lower rate
Delay of Reinforcement than with the 10-sec delay. Of the two birds given the
30-sec delay, one had a higher rate at 30 sec than at
Delay of reinforcement describes the time from a
10 sec, and the other had a lower rate at 30 sec. To
response until the reinforcer appears. Dews (1962)
some extent, then, the function relating response rate
hypothesized that delay of reinforcement is a deter¬
to delay of reinforcement corresponds to the hypothe¬
minant of fixed-interval patterning, since the rein¬
sized role played by delay in fixed-interval perfor¬
forcer follows not only the last response in the inter¬
mance.
val, but also earlier ones, albeit with longer delays.
Explicit delays of reinforcement added to fixed-
Responses occurring closest to the reinforcing stimu¬
interval schedules have large effects whether responses
lus are subjected to the shortest delay; responses more
do (Skinner, 1938, pp. 139-150) or do not (Dews, 1969)
remote from reinforcement are subjected to longer
reset the delay timer. Skinner used either an FI 4-min
delays. "The progressive increase in rate of respond¬
or an FI 5-min schedule with delays of reinforcement
ing through the fixed interval would be based on a
ranging from 0 to 8 §£<5 in 2-§££ step. Rate declined
declining retroactive rate-enhancing effect of the
33% with a 2-see delay and more than 50% with 8 sec;
reinforcing stimuli as the delay between response and
in fact, Skinner concluded that the 6-sec delay prob¬
reinforcement is increased” (Dews, 1962, p. 373).
ably did not maintain responding at all. Dews added
a 1-see delay of reinforcement to an FI 3-min sched¬
Studies of Delayed Reinforcement
ule. Because responses did not reset the delay tiffiev,
In order to evaluate the possibility that delay of the actual delay could range from 0 to 1 sec (Pews
reinforcement influences fixed-interval performance, mentions that the average delay was about Z60 msec):
it is necessary to observe its direct effects. Such re* Response rate was about half that occurring with im¬
search has manipulated the delay interval by having mediate reinfereer presentation. These data suggest
the reinforcer Occur at varied times after the response. that a reinforcing stimulus has a small effective range
The experiments have differed in whether ot not re¬ in time. If delays as short as 1 sec sharply reduce
sponses during the delay period reset the delay timer response rate? the high average level of respond¬
and in whether or not there was a distinctive stimulus ing maintained minutes or even hours prior to the
correlated with the delay period. appearance of the reinforcer under fixed-interval sched¬
A distinctive stimulus correlated with the delay ules cannot be attributed to the delay of reinforce¬
usually eliminates responding during that period; thus ment gradient. Morgan (1970) drew the same conclu¬
it makes little difference whether or not responses sion from his comparison of fixed-interval schedules
reset the delay timer. Responding may either decrease with a conjunctive fixed-time, fixed-ratio schedule
as a function of delay (e.g., Hamilton, 1929; Pierce, that maintained the same single-response requirement
Hanford, 8c Zimmerman, 1972) or be maintained with and interreinforcer interval, but did not guarantee
delays as long as 24 hr (Azzi, Fix, Keller, 8c Rocha e that a response be contiguous with the reinforcer.
Silva, 1964; Ferster, 1953; Ferster 8c Hammer, 1965). Skinner’s, Dews’s, and Morgan’s experiments demon¬
These correlated stimulus conditions would not, in strate that substantial responding is maintained only
any event, seem to be relevant to fixed-interval per¬ if there is very close temporal contiguity between the
formance, since the fixed-interval schedule provides final response and the reinforcer.
no delay stimulus.
Delay of Reinforcement in
In the absence of distinctive stimuli, there is a
difference depending on whether or not responses Schedules Involving

during the delay period reset the delay timer. If each Response Number

response prolongs the delay (reset condition), respond¬ Delay of reinforcement may operate when sev¬
ing typically declines with increased delays (Azzi et al.. eral responses must precede each reinforcer presen-
220 SCHEDULES OF REINFORCEMENT

tation. Catania (1971) required a sequence of re¬ procedure involves discrete trials. Responding is not
sponses to either one or two keys. If all of the free to occur at any time, but is limited to periods
responses except the first had to be to one key (key marked by distinctive stimuli and separated by inter¬
A), while the first response had to be to the other trial intervals during which responses have no sched¬
key (key B), the rate of responding to key B decreased uled consequences. In some trials (72-trials) a rein¬
as the number of subsequent key A responses was in¬ forcer is available; in others (A-trials) it is not. There
creased from 1 to 11. In another experiment, the can be a series of A-trials between successive 72-trials
sequence was always four responses. In one set of con¬ (e.g., Dews, 1966a), or, as shown in Figure 12, there
ditions the last response and two of the other three can be alternating A- and /2-trials (e.g., Dews, 1962;
had to be to key A, while the remaining response had Jenkins, 1970). The location of an A-trial is specified
to be to key B. The closer the key B response was to either by the time from the end of the preceding
the reinforcer, the higher was the rate of responding 72-trial to the end of the A-trial (the 720-A interval)
to key B. Catania attributed the results to the differ¬ or the time from the end of the A-trial to the begin¬
ential delay of reinforcement inherent in his pro¬ ning of the next 72-trial (the N-Rc interval). The in¬
cedures. The closer key B responses were to the rein¬ terreinforcer intervals are T0-Tc in the free operant
forcer, the shorter the delay and hence the higher the procedure and Rq-Rc in the trial procedure.
response rate. ■The T0-A interval as the controlling factor ap¬
Catania (1971) observed that when the required se¬ pears in accounts that emphasize elapsed time as a
quence of responses was changed, the interreinforcer discriminative stimulus (time from the beginning of
time increased substantially at first but then decreased. the interval); the A— Tc interval is crucial for delay-o£-
The decrease implies that the probability of the spe¬ reinforcement explanations (the time between a re¬
cifically required sequence increased, Yet Catania re¬ sponse and the next reinforcer presentation). It is clear
ported that stereotyped sequences did not occur. from Figure 12, however, that the T0-A and N-Tc
Perhaps the changes in interreinforcer time are at¬ intervals are not independent, given that the T0-Tc
tributable to changed interactions between two inde¬ interval (the total fixed interval) is held constant.
pendent responses occurring at different rates. They Since the two are perfectly negatively correlated, their
could also be due to modification of the emitted re¬ effects cannot be separated. In addition, two lines of
sponse unit. evidence suggest that they must be considered to¬
gether.
Interval Relativity When Dews (1970) plotted relative rate in each

Further consideration indicates that temporal dis¬

crimination and delay of reinforcement actually are To-N-1-N-Tc
not distinguishable in interval schedules. The tem¬ N
poral relations in two types of experimental arrange¬ \
ment are shown in Figure 12 (derived from Jenkins, N - Period Free Operant
1970), The free operant procedure describes the
typical fixed-interval or fixed-time schedule in which
the prevailing exteroceptive stimuli do not change
■R0-N T—N-Rc-1
until the reinforcer appears. Some event (time zero,
or T0) marks the beginning of the interval; often R0 N Rc

this event is the end of a reinforcer presentation \ \1 \

R N R Trial
accompanied by a stimulus change such as the il¬ Trial Trial Trial Procedure
lumination of the key and/or houselight. The orig¬
inating event also could be the end of a blackout
TIME ..»
period, or any other stimulus that marks the onset
of the interval. There is then a continuous period Figure 12. Temporal intervals. The upper segment (free oper¬
(A-period) in which a reinforcer is not available. ant) illustrates the typical fixed-interval schedule. TQ indicates
The completion of the interval (time completed, or the stimulus change indicating the beginning of the interval.
The stimulus continues until T , i.e., the presentation of the
Tc) occurs when the reinforcer appears. Any par¬ reinforcer that ends the interval. N is any point during the
ticular point within the A-period is referred to as A period between TQ and T . The lower segment (trial procedure)
and can be localized either with respect to T0 (the illustrates the condition involving restricted opportunities to
respond. R refers to trials in which a reinforcer is available, N
T0-N interval) or Tc (the N-Tc interval). Another to trials in which it is not.
Michael Zeiler 221

one-fifth interval, curves for fixed intervals of 30 sec, tive data do not show very orderly results, the rank-
300 sec, and 3000 sec fell on top of one another. The order correlation between these relative proximity
relative patterns were independent of fixed-interval measures and responding on the TV-trials was .88, with
value. Even though equivalent fifths represented very the sign being either positive or negative depending
different absolute time periods from the beginning or on whether the i70-TV or the N-Rc interval is used. In
to the end of the fixed interval, relative position and these eight conditions of Jenkins’s discrete-trial pro¬
patterning was constant. cedure, as in the fixed-interval schedule, responding
Jenkins (1970) used a trial procedure to separate, at any point during the interreinforcer interval was
absolute time from the beginning, and time to the end related to the proportion of the total interval that had
of the interval. The variables manipulated were the elapsed (or was remaining) at that point.
time from the end of one TTtrial to the end of the
TV-trial (.R0-N interval) and the time from the end of Conclusion
the TV-trial to the beginning of the next 17-trial (the
N—Rc interval). (Jenkins described the independent The temporal location of a point in an inter¬
val can be specified from either the beginning or
variables somewhat differently; he measured time
the end of an interval. Changes in the relative loca-
from the beginning of each trial. The present descrip¬
tion of the point always mean relative changes in both
tion is used to facilitate the reanalysis of his data.)
the time from the beginning of the interval and the
Table 3 shows eight conditions (columns a and b) and
the results (column f).
time to the end of the interval, and thus these two
must be totally confounded. It cannot be asserted,
therefore, that either of the periods is responsible for
Table 3 Intervals Stud! ed by Jenldns (1970)
an effect. The important factor is the location of a
INTERVALS RELATIVE PROXIMITY given psmt relative te> the overall duration of the
(max. duration in sec) interval (cf, Jenkms’sj 1970, relative proximity prin¬

(a) {d) ciple).

(*> (c) (e) (f)
Average
R0-Rc R0-NI N-Rc/ Responses
R0-N N-Re Interval Rq~Rg Rq~Rg
on N-trials SEQUENCES ANE> UNITS

H 0 62 .07 .13 6.7

m 8 m ,95' ,05 8,0 Tbe Cliajnlng Hypothesis
u 108 162 .33 .67 3.3
154 108 262 .59 .41 3.5 Schedules of intermittent reinforcement impose or*
154 8 162 .95 .05 12.9 der on the numerous responses that occur between
154 207 361 .43 .57 67 §ueee§§ive veinforeers: the behavior generated ha§ a
54 20 74 .73 .27 6.1
sequential organisation, Historically, it has been com¬
154 20 174 .89 .11 8.6
mon to explain sequences in terms of response chain*
All intervals are given in terms of their maximum duration mg. The essence of the chaining hypothesis is that
in see. ^ach r^5pQn§c provides the stimulus (or part of the
stimulus complex) for the next response. These
Neither the absolute R9-N nor N-Rc intervals in¬ response-produced stimuli havo an eliciting or dis¬
dependently determined responding in the ^/-trials, criminative stimulus function with respect to the
since responding was not closely related to the value next response, and they may have a reinforcing func¬
of either. But it is important to note that whenever tion with respect to preceding responses. Individual
there was a change in either the R0-N or the N-Rc learning theories differ about whether the chain is
intervals, there was either a change in the other as composed of reflexes or of instrumental responses, but
well and/or a change in the time from the end of one the concept of each response acting as a stimulus for
jR-trial to the beginning of the next (the R0-Rc inter¬ the next is common to all.
val). Column (c) of Table 3 shows the R0-Rc interval Platt and Johnson (1971) reviewed data showing
for each condition. Columns (d) and (e) show the that behavior can have stimulus properties. Behavior
R0-N and N-Rc intervals as proportions of the R0~RC under mixed FR FR schedules (two fixed-ratio sched¬
intervals, i.e., they describe the relative proximity of ules occur in some order, but with no exteroceptive
an TV-trial to the overall interreinforcer period. stimulus indicating which one is in effect) shows dif¬
Although analyses based on the absolute quantita¬ ferential sensitivity to the two ratio requirements. In
 222
SCHEDULES OF REINFORCEMENT

one type of experiment, Rilling and McDiarmid her (1966) objected to it as inherently untestable be¬
(1965), Pliskoff and Goldiamond (1966), and others cause it is based on purely inferred stimuli, while
have shown that pigeons can make appropriate choices Lashley (1951) believed that the speed and precision
on the basis of the size of the fixed ratio just com¬ with which many sequences are executed made it un¬
pleted. In another type of experiment, Platt and John¬ tenable to assume that each response is controlled by
son (1971) delivered food to a rat only if the rat’s feedback from the preceding response. Data reported
approach to the food tray was preceded by N lever by Taub and Berman (1968) and Lashley (1917) show
presses. Approaches prior to N lever presses resulted that precise sequences occur in organisms deprived of
in a time-out and reset the response number require¬ sensory feedback from their responses. If chaining oc¬
ment. The mean number of responses emitted per tray curs, perhaps it does so only under specialized condi¬
entry slightly exceeded the number required at all tions of differential reinforcement.
values of N. These data support the hypothesis that
organisms can respond differentially to their own be¬
havior. Response Units

In all these experiments different consequences

The Specification qf a Unit
were arranged following the emission of different
numbers of response*. There i? n9 evidence that re¬ Units other than the individual response might be
sponses function as discriminative stimuli in the affected by reinforcement and play a role in schedule-
absence of explicit differential reinforcement. Animals controlled performance, For example, Morse (1966)
in extinction after fixed-ratio training will emit yery suggested the interresponse time as a reinforceable as¬
kng runs of responses before pausing (Ferstcr 8c Skin¬ pect of behavior; and Staddon (1967) proposed that
ner, 1957, pp. 57=63); this provides no evidence for the entire temporal pattern of responding may be a
sensitivity to the number of responds previously re¬ unit. The experimental evaluation of whether a cer¬
quired for reinforcer presentation. Data reported by tain form of behavior is a unit requires clarifying the
Overmann and Denny (1974) suggest that the animals different meanings of response units. (Although re¬
are controlled by tbs Exteroceptive stimulus changes sponses are emphasized here, a parallel case can be
correlated with completion of the ratio rather than made for stimulus units.)Three different kinds of re¬
by response number. Control by response number ap¬ sponse unit can he distinguished. These are described
pears to require that there be different schedules cor¬
here as formal, conditionable, and theoretical units.
related with different numbers q£ responses; it does One kind of response unit refers to the class of be¬
«9t arise automatically through the differential rein¬ havior that the experimenter prescribes as prerequisite
forcement of response number inherent in simple for a reinforcer presentation; this is simply the opera¬
fixed-ratio schedules.
tional definition of the measured response. This is the
Chaining has been proposed as being involved m
formal response unit. The formal unit is always un¬
performance under fixed-interval schedules (Ferstcr & ambiguous, but it need not be experimentally inter¬
Skinner^ 1957; Skmnsr, 1938), but this hypothesis no esting or useful. To be so, a formal unit must obey a
longer seems tenable. According to the chaining hy¬ plasticity principle: its probability of occurrence
pothesis, sequential changes in responding are due to should be affected by its consequences. Some formal
the stimulus properties of preceding responses ‘ con¬ units display this plasticity, whereas others do not.
sequently, the pattern should not survive interrup¬
The term operant has been used to describe modifi¬
tions in the flow of responses. Dews (1962, 1965a, able units; here they will be referred to as condition-
1965b; 1966a, 1966b) interpolated stimuli that dis¬ able response units. Conditionable units, like formal
rupted responding at various points during fixed in¬ ones, are unambiguous: if some behavior is required
tervals. Although this should break the hypothetical for reinforcer presentation and it increases in proba¬
chain, the basic fixed-interval pattern endured: when bility, it is a conditionable response unit.
responding occurred, it was appropriate to the tem¬ The term response unit may also be used to refer to
poral location within the interval regardless of the something inferred rather than observed directly. A
response rate immediately preceding. Similar results response, a stimulus-response relation, or some cogni¬
have been obtained by others (Farmer 8c Schoenfeld, tive activity, can be postulated to underlie observed
1966; Ferster 8c Skinner, 1957, pp. 213 ff.; McKearney, performance. Inferred units are being used when it is
1970). Chaining, therefore, is not necessary for fixed- asserted that organisms learn turning responses, or to
interval performance. approach certain locations in a maze, or interresponse
The chaining hypothesis has limited value. Kelle- times, or entire sequences of behavior. These inferred
 223
Michael Zeiler

units will be referred to as theoretical response units. controlling observable performance. Given the one-to-
The interresponse time illustrates the distinction one correspondence between response and inferred
between formal, conditionable, and theoretical units. stimulus properties, however, the two treatments ap¬
If the interresponse time is specified as the require¬ pear to be equivalent.
ment for the delivery of a reinforcing stimulus, it is
the formal unit. If it should be altered by the imposi¬
IRTs as Theoretical Units
tion of these consequences, it is a conditionable unit.
It is a theoretical unit when it is used to explain per¬ Ferster and Skinner (1957) demonstrated that IRT
formance under a schedule in which some other be¬ reinforcement could explain the response rate differ¬
havior is specified as the formal unit. For example, in ences in interval and ratio schedules. If responses tend
a fixed-ratio schedule the formal unit is the sequence to occur in bursts (Skinner’s, 1938, fourth-order devia¬
of required responses. However, if the behavior that tion from a steady rate), under ratio schedules it is
emerges is then explained as the outcome of differen¬ likely that the response requirement will be met during
tial reinforcement of short interresponse times, then a burst. Therefore, the IRT correlated with reinforcer
the interresponse time is a theoretical unit. In other presentation is likely to be in the short range of those
words, a theoretical unit is one that is hypothesized to emitted. Ratio schedules do in fact generate homoge¬
underlie observed behavior. neous sequences of short IRTs and those are the ones
Evaluating the plausibility of a particular theo¬ emitted at the moment of reinforcement (Gott & Weiss,
retical unit requires a strategy analogous to that used 1972; Weiss & Gott, 1972). In interval schedules, how¬
to evaluate indirect variables: the hypothesized unit ever, it is more likely that the reinforcer will follow a
must be studied directly. This means that the theo¬ period of pausing (more time elapses during a pause,
retical unit must be specified as a formal unit. If it hence making it more likely that the reinforcer will be¬
then proves to be conditionable, and if it is affected come available). Therefore, longer IRTs will be prefer¬
as it is hypothesized to be when it is used to explain a entially correlated with the reinforcing stimulus. Morse
given performance, the theoretical construct gains (1966) showed formally that more short IRTs will be
plausibility. reinforced on ratio than on interval schedules, and
Dews (1969) confirmed that the terminal IRT in fixed-
interval schedules is likely to be longer than the ones
Interresponse Time
immediately preceding. Morse also developed a model
An interresponse time (IRT) was initially concep¬ for explaining patterning as the consequences of the
tualized as a stimulus, Skinner (1938, pp. 247-284) relation between IRTs and reinforcement.
described time since the preceding response as a dis¬ The selective reinforcement of IRTs accounts for
criminative stimulus controlling the emission of the what are otherwise puzzling data. Consider response
next response, and Anger (1956) treated the interre¬ rate under tandem FI FR schedules. Rate is higher
sponse time in the same way. Reynolds (1966) demon¬ than on a simple fixed-interval schedule (Ferster 8c
strated that an IRT could indeed function as a Skinner, 1957). The addition of a fixed ratio at the
discriminative stimulus by showing that if responding end of the fixed interval makes it more likely that the
in the second component of a chained schedule was reinforcer will follow a short IRT, and thereby prefer¬
followed by food only if the IRT in the first compo¬ entially reinforces short IRTs. If the emitted IRT dis¬
nent exceeded 18 seconds, response rate in the second tribution changes in the direction of the reinforced
component varied as a function of the preceding IRT. IRT distribution, it is to be expected that rate should
Thus the IRT as an antecedent event produced differ¬ increase with the tandem schedule.
ential responding, i.e., it exerted stimulus control over Some data from t- and r-schedules also fit an IRT
responding. analysis. In many of these schedules a reinforcer is
Morse (1966) treated the IRT as a shaped and rein¬ available for a single response occurring at some point
forced property of behavior rather than as an ante¬ in time. Restricting the duration of the time period
cedent stimulus for a response. In the typical schedule has large effects on response rate: rate increases as
of reinforcement in which IRTs are postulated as availability decreases. Morse (1966) has suggested that
controlling behavior, the IRT is a property of emitted the period of reinforcer availability determines which
behavior and is not manipulated as a stimulus. In¬ IRTs are preferentially reinforced. The shorter the
stead, stimulus properties are inferred from respond¬ availability period, the more the schedule favors the
ing. If the IRT is treated as a differentiated response reinforcement of short IRTs and the higher the con¬
unit, unobservable stimuli need not be postulated as sequent response rate.
224 SCHEDULES OF REINFORCEMENT

The IRT as a Conditionable under IRT > t, but without the IRT requirement (a
Response Unit variable-interval schedule yoked to the IRT > t sched¬
ule). The response rates and distributions of IRTs
The IRT as a theoretical unit has the potential to obtained under the IRT > t and yoked VI schedules
explain major aspects of schedule-controlled perfor¬ differed greatly, showing that the specific differential
mance. What is necessary, then, is to examine IRTs as reinforcement of IRTs was an important factor in
formal response units and see if they are conditionable. IRT > t performance.
Tests of IRTs as conditionable responses derive from Alleman and Platt (1973) used what they termed a
IRT > t and IRT < t schedules, which require that percentile reinforcement schedule involving IRT rein¬
an IRT either exceed or be less than a certain value forcement. Platt (1973) has described such arrange¬
for a reinforcer to occur. Several experiments (e.g., ments as shaping schedules; the prescribed behavior
Malott 8c Cumming, 1964; Richardson & Loughead, at any moment shifts with the nature of the behavior
1974; Staddon, 1965) have shown that if an IRT must being emitted. In Alleman and Platt’s experiments
exceed some value, the distribution of emitted IRTs the critical aspects of performance were the emitted
changes in the appropriate direction. Similar results IRTs. Only IRTs more extreme than a certain pro¬
occur when a terminal IRT requirement is added to a portion of the recently preceding IRTs were rein¬
variable-interval schedule; i.e.7 when the schedule is a forced. Under some conditions, the requirement in¬
paced VI or a tandem VI IRT > t (Anger, 1956; volved IRTs shorter than those preceding; under
Slump, 1967). Since the probability of the specific others, it involved longer IRTs. For example, the
IRT that is required increases (although at long IRT IRT might have to be shorter than 95% of the previ¬
requirements very few of the emitted IRTs may be ous IRTs. The emitted IRT distribution then became
long enough to produce the reinforcer), the IRT is a progressively peaked at the lower end. A parametric
conditionable response unit. study showed that when the IRTs had to be among
Quantitative analyses of the effects of temporal the least frequent 5 or 10% the distribution shifted
differentiation schedules have shown similar results toward longer or shorter IRTs depending on which
whether the unit involved was the IRT or some aspect was required. These effects occurred even if the same
of performance other than the interresponse time number of responses per reinforcer were maintained
(Catania, 1970; DeCasper & Zeiler, 1974). Performance over the range of percentile requirements. By selec¬
can be described by the power function T — ktnj tively reinforcing extreme IRTs, the IRT distributions
where T is the emitted duration, t is the required changed in the appropriate direction, Plough (1966)
duration, and k and n are constants fit to the data. A showed the complementary effect; variable IRTs were
small range of values of k and n are necessary to de¬ produced by always reinforcing the least frequent ones
scribe performance m most of these experiments. In without regard to direction.
fact, DeCaspcr and Zeiler have suggested that the The data show, therefore, that an IRT is a condi¬
effects of temporal differentiation schedules may be tionable response, It is reinforceable directly when it
independent of particular response units and dura¬ is specified as the formal response, and the changes in
tions. behavior are not attributable to changed reinforcer
A problem in interpreting rate changes under density.
IRT > t and other temporal differentiation schedules
is that as the time parameter increases, the number of
IRTs and Reinforced IRTs
responses per reinforcer and average interreinforcer
time both increase as well. If response rate is a func¬ What do these data imply about behavior under
tion of reinforcer density, IRTs would change as well schedules not specifying the value of terminal IRTs
since they are the components of rate. Actually, how¬ directly? Anger (1956) suggested that IRT reinforce¬
ever, this is not the major reason for the effects. Ferster ment operates to control performance with such sched¬
and Skinner (1957, p. 460) found that adding an ules. He found that under variable-interval schedules
IRT > t requirement to a variable-interval schedule the frequency distribution of IRTs correlated closely
reduced rate without markedly affecting the average with the frequency distribution of reinforced IRTs.
interreinforcer time. Richardson (1973) has also shown Those IRTs that were most often correlated with the
that interreinforcer time is not responsible for the reinforcing stimulus occurred more frequently; the
IRT distributions in simple IRT > t schedules. An¬ shapes of the IRT distributions corresponded with the
imals were given exactly the same temporal distribu¬ shapes of the distribution of reinforcers per hour per
tions of reinforcer presentations that they obtained IRT.
 225
Michael Zeiler

Later analyses have shown that this relation is proximately linearly related to either relative fre¬
probably not significant. Blough and Blough (1968) quency or magnitude of reinforcement, but it did not
were the first to point out that the dependency of the match either. Shimp (1969) found that with equal
reinforcer distribution on the IRT distribution is reinforcement frequency for both IRT classes P
mathematically forced, because “on a VI schedule, the matched the relative reciprocal (term used by Hawkes
probability of reinforcement at a given IRT bin must & Shimp, 1974, here called RR) of the classes. The
increase with the number of responses in that bin” RR was computed by considering the lower bound of
(pp. 26-27). Reynolds and McLeod (1970) showed that the short class (t11) and the upper bound of that class
a correlation between emitted and reinforced IRTs (t12) together with lower (L i) and upper (t2,2) bounds
holds regardless of the distribution of emitted IRTs. of the longer class. The relative reciprocal of the
Blough and Blough examined whether the distribu¬ short class is:
tion of reinforcers among various IRTs determined
the IRTs emitted subsequently. If this were the case, jVfW + _
it might be expected that the IRT distribution would 1/fl, 1/ 1 /1*2,11 + f/^2,2/
+ l/L.2/ ~t~
shift in the direction of those IRTs that earlier had
received the most reinforcers per hour. There was no The values of RR and P matched when both IRT
indication that IRT distributions changed in this classes were reinforced equally. However, matching of
manner. RR and P seems only part of a general function which
Anger (personal communication) suggests that IRT varies according to the absolute values of the short

reinforcement occurs in the context of the increase in class of IRTs. Matching occurs when tl x is in the
the frequency of a response due to reinforcement, the 1-4-sec range, and departures occur at other values
decrease due to nonreinforced responses, recovery of (Hawkes & Shimp, 1974).
responses after a period of nonreinforcement, and per¬ Hawkes and Shimp review the series of experiments
haps a rate-enhancing effect of extinction as well, in some detail. What emerges is the possibility of a
These factors interact to produce oscillations in emit¬ general function relating IRT reinforcement to be¬
ted IRTs. Thus IRTs are the outcome of at least four havior, and that the function interacts systematically
influences rather than being simply due to frequency with other variables to determine response rate. As
of reinforcement. the work ha§ expanded to deal with more than two
Since 1968 Shimp and his colleagues have been de¬ IRT classes, it has come increasingly into contact with
veloping a systematic quantitative account of IRT what perhaps occurs under schedules of reinforcement
reinforcement. The various experiments have in not involving explicit IRT requirements (Shimp,
volved a paced VI schedule (Anger, 1956; Ferster & 1973). This way of considering IRT reinforcement

Skinner, 1957), in which the reinforcer occurs follow¬ might show correspondences between what happens
ing the first appropriate IRT emitted when a VI when IRTs are specifically required and when they
schedule has made the reinforcer available. Shimp’s are adventitiously correlated with the reinforcing
experiments have involved at least two IRT bands, stimulus on interval and ratio schedules.
each defined by a lower and upper bound. The pro¬
cedure involves distinctive stimuli presented only Sequences as Response Units
during IRT bands that are eligible for reinforcement.
Thus reinforcement occurs only during the stimulus,
Sequences as Theoretical
and the stimulus is correlated with the IRT require¬
Response Units
ments. The purpose of introducing stimuli in this
way is to gain precise control over responding, but it Skinner (1938) speculated that, in fixed-ratio per¬
seems possible that discriminative stimulus control formance, the effective response unit might involve all
may influence the performance in other ways as well. of the behavior extending between successive rein¬
In any event, the dependent measure is the percentage forcer presentation. Later, Mowrer and Jones (1945)
of IRTs, P, falling in one of the bands. Considering reasoned that if ratios did in fact generate units com¬
the shorter class, P equals the number of IRTs in the posed of all of the responses, resistance to extinction
short class divided by the number of IRTs in both should be a function of the number of units and a
classes. constant number of ratios would occur in extinction.
In 1968 Shimp applied different relative frequen¬ Their failure, as well as Boren’s (1961) and Weissman
cies or magnitudes of reinforcer presentation to 1.5- and Crossman’s (1966), to obtain a perfect correspon¬
2.5-sec and 3.5-4.5-sec IRTs. The value of P was ap¬ dence between the hypothesized units and number of
226 SCHEDULES OF REINFORCEMENT

responses in extinction seemed to oppose a response fixed ratio was treated as a single response and was
unit hypothesis. This failure could be due to the reinforced according to a DRL schedule. Marr (1971)
properties of extinction itself. Extinction is not a has confirmed this finding with sequence schedules in
passive condition, but instead exerts effects of its which distinctive stimuli were correlated with each
own; these often include an initial intensification of successive fixed-ratio component and the ratio per¬
responding prior to the well-known decrement (cf. formance produced the reinforcer according to either
Amsel, 1967; Morse, 1966). Also, extinction involves a a fixed-interval or a fixed-ratio schedule. The com¬
change in stimulus presentations, and the way stimuli ponent sequences resembled single responses in other
are presented in extinction can affect the number of ways. For example, Kelleher (1966) showed that with
responses emitted (Overmann 8c Denny, 1974). De¬ the FI 10-min (FR 20) second-order schedule the time
spite these complications, predictions of the response taken to emit successive ratios shortened as the inter¬
unit hypothesis of resistance to extinction, although val progressed. This can be compared with the short¬
not completely accurate, were not totally out of line. ening of successive interresponse times, when single
In the Mowrer and Jones study, for example, the num¬ responses are reinforced according to fixed-interval
ber of responses in extinction was linearly related to schedules. Shull, Guilkey, and Witty (1972) also found
the number of responses previously required. substantial similarity in the emission of successive
Data using a different schedule support the re¬ fixed ratios and successive individual responses under
sponse unit analysis. Day and Platt 1972) used a fixed- fixed-interval schedules.
constant number (FCN) schedule: rats had to emit a
certain number of responses, but food was presented
Sequences as Conditionable
only if they approached the food tray after they had
Response Units
made those responses. Early approaches reset the re¬
sponse requirement. The schedules used were FCN 1, Are sequences reinforceable units—that is, is their
FCN and FCN In terms of the number of re¬ probability of occurrence a function of their conse¬
sponses emitted in extinction, these data replicated quences? To determine that and thereby to determine
those of Mowrer and Jones. However, it had been the plausibility of sequences as theoretical response
pointed out by Demiy, Wells, and Maatsch (1957) units, it is necessary to impose the sequence as a
that the number of responses in the ratio may not be formal unit and to observe whether it is conditionable.
the appropriate measure of the response unit. Instead, It is well to be cautious in concluding that a sequence
it would be the sequence of responses preceding each actually required by a schedule. For example, a
tray approach, and the important dependent variable ratio schedule imposes a sequence o£ n responses as a
in terms of resistant to extinction would be the formal requirement, and it demonstrates that such a
number of approaches. Day and Flatt found that in sequence is conditionable (reinforceable). It does not
extinction the groups did not differ in the number of show that the particular pattern according to which
such approaches, and they concluded that the response ratios are emitted is conditionable, because the sched¬
unit hypothesis was tenable. ule does not require any particular pattern. Interval
The stable sequences observed in successive com¬ schedules specify nothing at all about sequences, since
ponents of second-order schedules involving fixed-ratio they require only a single response. Many responses
components fit a unitary response interpretation. In may occur in highly organized form, but interval
this research, a fixed-ratio sequence was treated as a schedules do not demonstrate that either total interval
single response unit, and it was reinforced according performance or any portion of it other than the single
to some schedule. Findley (1962) used fixed-ratio com¬ response is a conditionable response unit.
ponents in a variety of complex schedules involving Findley s (1962) and Kelleher, Fry, and Cook’s
sequences of the same or of different responses and (1964) procedures demonstrate the necessary condi¬
found that typical ratio patterns appeared in each tions for observing that an aspect of schedule per¬
component. Kelleher (1966) demonstrated that the formance is conditionable. In their experiments, a
ratio pattern was maintained when the first FR 20 reinforcer was presented at the completion of a fixed
completed after 10 min—an FI 10-min (FR 20) second- ratio only if the postreinforcement pause exceeded a
order schedule—resulted in food presentation. Lee and minimum duration. They found that the pause dura¬
Gollub (1971) found the same when ratio perfor¬ tions changed in accordance with the requirements,
mance was followed by a reinforcer on a fixed-ratio so that pause behaved as a conditionable unit. These
schedule, and Kelleher, Fry, and Cook (1964) also findings justify interpretation of the pause as a re¬
observed characteristic ratio performance when the sponse unit on ordinary fixed-ratio schedules.
Michael Zeiler 227

Transition States and Steady States

Since initial pause duration and overall duration

of fixed-ratio sequences are conditionable, they could
be reinforced under ordinary fixed-ratio schedules. If
so, they could play a role in producing the stereotypy
in pausing and interreinforcer time often observed in
fixed-ratio performance. They cannot, however, be
important in the development of steady-state behav¬
ior. If they were, the pause and ratio durations ini¬
tially correlated with the reinforcer should tend to
predominate. This does not usually occur. Consider a
pigeon changed from one fixed-ratio schedule to a
larger one. Immediately following the transition the
pause is short and the ratio duration is also short.
Eventually both durations lengthen despite these early
duration-reinforcer relations. Apparently, other vari¬
ables operate to establish the pause and ratio dura¬
tions, and then they may come to operate as indirect
Required Duration determinants of performance once they have become
more or less stable.
Fig, 13. Mean ratio time in seconds (initial pause time plus Figure 14 shows the frequency distributions of the
time spent responding) emitted by a pigeon as a function of
duration of the pauses prior to the first response in
the ratio time required for food presentation under an FR 30
schedule. (From DeGasper 8c Zeiler, 1974; Zeiler, 1970, 1972.) the first 25 sessions of FR SO after FR 1 training, Over

This differential reinforcement procedure has been

extended to other aspects of fixed-ratio performance.
Figure IS shows the effects of requiring that the SeaSion
duration of an entire ratio—i.e., the time from the first 1- 4
opportunity to respond until the completion of the
ratio—must exceed some particular value. Other ex¬
periments investigated the effects of requiring that
5-8
the duration be shorter than a specified value (Zeiler,
1970, 1972b). In all of the experiments, emitted
durations were related to required durations by the
9-12
same power function applicable to other temporal
differentiation procedures (see page 224) (DeCasper
Sc Zeiler, 1974). The duration of an entire ratio,
therefore, is a reinforceable aspect of behavior. Since 13-16
the total time taken to execute a ratio—the initial
pause time plus the time spent responding—belongs
not to any single response, but instead describes
17-20
a property of the entire sequence, the experiments
demonstrated that the ratio sequence can be shown
to have unitary properties. More precisely, they
showed that duration is a reinforceable aspect of the 21-25
sequence. In one experiment (Zeiler, 1972b) the most
orderly aspect of behavior was shown to be the over¬ 2 8 14 20 24 >24
all duration of the ratios: the initial pause times and
the rate of responding after the pause did not show Pause Time
equally predictable effects. The total sequence was
Fig. 14. Probability distribution of initial pause lengths (in
more orderly than were either of the two major com¬
seconds) in the first 25 sessions of an FR 30 schedule after an FR
ponents. I history with pigeons. Each session consisted of 30 ratios.
228 SCHEDULES OF REINFORCEMENT

formance is reinforced directly but develop their

characteristics for other reasons. It may be, as Dews
(1970) suggested, that “when a subject is exposed re¬
peatedly to a consistent schedule, patterns of respond¬
ing may become sufficiently consistent to enable par¬
ticular aspects of the patterns themselves to be related
reliably to the schedule. The very reliability of the
relation may lead to the further strengthening of
those particular aspects of the pattern” (p. 59). Per¬
haps unitary aspects of fixed-ratio performance do not
play a role in establishing the final behavior, but do
operate to maintain steady-state performance once it
develops.

SUMMARY AND CONCLUDING REMARKS

Tke Controlling Variables

The preceding account of reinforcement schedules

differs from those of Ferster and Skinner (1957) and
Morse (1966) only in the variables that are given ma¬
jor emphasis. The general approach is basically the
same: the assumption is that schedule controlled be¬
2 $ M 20 -24 havior is multiply determined and that each hypothe¬
sized controlling variable must he studied directly to
Run Time determine if it operates as expected.
Response rate and patterning appear to be con¬
15- Ptf6fe5.feilifcy distribution of run-time lengths ^seconds
from ttic first to the last response) in the same sessions as in trolled by different variables. This view differs some¬
F|gurf 14, what from Ferster and Skinner’s and Morse’s in that
they tended to deal with both characteristics simul¬
taneously. A major departure is the present emphasis
fh£ £3Ur§£ Of th£§£ g@§§ioilsf it appeared that those on factors other than the quantitative properties of
pause durations that occurred more fr^qu^ntly (!,§,, the response occurring the moment of reinforcer pres¬
those that were most often followed SO responses later entation.
by the reinforcing stimulus) became still more pre¬ There are two types of variables responsible for
dominant. This could be interpreted as indicating the performance, the direct variables specified by the
differential reinforcement of particular pause dura¬ schedule and the indirect variables that derive from
tions and the consequent increase in probability. Simi¬ performance. The variables have two types of effect,
lar data were obtained for overall ratio duration. A stereotypic and dynamic, differentiated by whether
yery different effect is revealed in Figure 15, which they tend to maintain the same behavior or to change
shows the distribution of run times (the time from the performance. With respect to response rate, one sig¬
1st to the 30th response) for the same 25 sessions. The nificant factor is the particular response dependency
shorter run times came to predominate, even though that guarantees close temporal contiguity between a
they were not the times correlated with reinforcer particular response class and the reinforcing stimulus.
presentation in the earlier sessions. It is not likely, This variable operates directly under response-depen¬
therefore, that a specific run time is being reinforced dent schedules and indirectly under response-indepen¬
in the development of simple fixed-ratio performance. dent schedules, and it has the effect of maintaining or
In conclusion, it is not clear whether the reinforce¬ increasing the probability of the response that pre¬
ment of unitary properties operates to determine be¬ cedes the reinforcer. This is the most molecular as¬
havior under fixed-ratio schedules. Perhaps pause time pect of the present approach. Other hypothesized
and ratio duration are directly reinforced and run variables encompass larger time periods and larger
time is not. Or perhaps none of these aspects of per¬ groups of responses. The number of responses per
Michael Zeiler 229

reinforcer and interreinforcer time were considered to and if the behavior is not disrupted, it is concluded
be major factors. Also, the availability of the rein¬ that the variable is not important. This approach also
forcer when responding weakens may determine requires caution. If a performance is actually over¬
whether a given schedule will maintain many re¬ determined—i.e., if the behavior under a given
sponses per reinforcer presentation. Thus responding schedule arises because several variables operate inde¬
under a given schedule reflects the conjunction of the pendently to produce the same performance—the elimi¬
response dependency, number of responses per rein¬ nation of one variable may not have a noticeable
forcer, interreinforcer time, and the regenerating effect. Yet the variable may actually play a role. There
power of the schedule. This account was able to deal are enough precedents (e.g., the sensory basis of maze
with interval and ratio schedules and some complex learning, recovery of function in the brain) to warrant
schedules, but other complex schedules may involve consideration of this possibility.
additional variables.
Patterning was explained by a single variable, the
Schedules As Fundamental
placement of the rein forcer in time. In interval sched¬
ules the tendency to respond at any instant is deter¬ Morse and Kelleher (1970) have maintained that
mined by the location time relative to the overall in¬ schedules are fundamental determinants of behav¬
terval duration. ior, because the particular events scheduled may be
Schedules that specify certain events as prerequi¬ less important than is the schedule itself. In this sense,
site for reinforcer presentation (e.g., a certain inter¬ it is the schedule in relation to ongoing behavior that
response time or a certain behavioral sequence) reveal is fundamental in determining subsequent perfor¬
that such events can exert control over responding. mance. There is another sense in which schedules are
Their role in the absence of such specification is un¬ fundamental. If each schedule represents a particular
clear. conjunction of variables, the only way of arranging
that conjunction is by establishing that schedule.
Some Reflections on Methodology
Fixed-interval schedules establish (1) a response-
reinforcer dependency involving a single response} (?)
d he rationale for the methodology advocated here a nearly fixed interreinforcer time; (3) an unlimited
is that the effects of hypothesized variables must be range of possible number of responses per reinforcer
assessed by studying those variables directly. This in successive intervals; (4) a reinforcer following a
approach has been followed consistently. Thus when single response when responding i§ not well main¬
number of responses per remforcer was proposed as a tained; (5) presentation of the remforcer at a fixed
source of control in fixed-interval performance, its time since the beginning of the interval. Only the
mode of operation was assessed by reference to fixed- fixed-interval schedule specifies that all of these condi¬
ratio schedules, which control it directly. The same tions will occur, Each variable may be analyzed, and
procedure was followed with every hypothesized con¬ fixed-interval performance may be understood as the
trolling variable. The approach has its limitations outcome of these component events, hut the schedule
and potential pitfalls. Inherent in it is the idea that is fundamental in that it alone can arrange these pre¬
variables have relatively simple effects and that com¬ cise interactions. To the extent that the precise inter¬
plicated interactions do not occur. Consider, for ex¬ action of multiple variables is responsible for a dis¬
ample, the assessment of number of responses per tinctive performance, each schedule is a fundamental
reinforcer by the use of ratio schedules. As the num¬ arrangement.
ber (ratio size) is manipulated, interreinforcer time The problems faced by the experimenter interested
changes as well, Interreinforcer time is analyzed di¬ in understanding schedule performance are difficult
rectly in interval schedules, but here responses per because of the complexities of the relationships. How¬
reinforcer will vary. If, in fact, the two variables in¬ ever, the fundamental role of schedules in psychology
teract in some complicated way, the function of either demands the attempt. It is impossible to study be¬
cannot be easily ascertained. Other as yet unknown havior either in or outside the laboratory without en¬
experimental designs will be necessary to evaluate countering a schedule of reinforcement: whenever be¬
such interactions. havior is maintained by a reinforcing stimulus, some
Another approach to the analysis of controlling schedule is in effect and is exerting its characteristic
variables is subtractive. A variable is prevented from influences. Only when there is a clear understanding
operating (cf. Dews’s work on interrupting the hy¬ of how schedules operate will it be possible to under¬
pothesized response chain in fixed-interval schedules), stand the effects of reinforcing stimuli on behavior.
230 SCHEDULES OF REINFORCEMENT

REFERENCES DeCasper, A. J., 8c Zeiler, M. D. Time limits for complet¬

ing fixed ratios, III: Stimulus variables. Journal of the
Experimental Analysis of Behavior, 1974, 22, 285-300.
Alleman, H. D., 8c Platt, J. R. Differential reinforcement Denny, M. R., Wells, R. H., 8c Maatsch, J. L. Resistance
of interresponse times with controlled probability of to extinction as a function of the discrimination habit
reinforcement per response. Learning and Motivation, established during fixed-ratio reinforcement. Journal of
1973, 4, 40-73. Experimental Psychology, 1957, 6, 451-456.
Amsel, A. Partial reinforcement effects on vigor and per¬ Dews, P. B. Free-operant behavior under conditions of de¬
sistence. In K. W. Spence 8c J. T. Spence (Eds.), The layed reinforcement, 1: CRF-type schedules. Journal of
psychology of learning and motivation (Vol. 1). New the Experimental Analysis of Behavior, 1960, 3, 221-234.
York: Academic Press, 1967. Dews, P. B. The effect of multiple SA periods on respond¬
Anger, D. The dependence of interresponse times upon ing on a fixed-interval schedule. Journal of the Experi¬
the relative reinforcement of different interresponse mental Analysis of Behavior, 1962, 6, 369-374.
times. Journal of Experimental Psychology, 1956, 52, Dews, P. B. Behavioral effects of drugs. In S. M. Farber 8c
145-161. R. H. L, Wilson (Eds.), Conflict and creativity. New
Appel; J. B.; 8c Hiss, R. H, The discrimination of contin¬ York; McGraw-Hill, 1963.
gent from noncontingent reinforcement. Journal of Com¬ Dews, P. B. The effect of multiple SA periods on respond¬
parative and Physiological Psychology, 1962, 55, 37-39. ing on a fixed-interval schedule, II: In a primate. Jour¬
Ami* R., Fix, D. 3. R., Keller, F. 5., 8c Rocha e Silva, M. I. nal of the Experimental Analysis of Behavior, 1965, 8,
Exteroceptive control of response under delayed rein¬ 53-54. (a)
forcement, Journal of the Experimental Analysis of Be■ Dews, P. B. The effect of multiple SA periods on responding
havlors 1964, 7, 159-162. on a fixed-interval schedule. III: Effects of changes in
Berryman, Rm & Neyin, J, A, Interlocking schedules of pattern of interruptions, parameters, and stimuli. Jour¬
reinforcement* Journal of the Experimental Analysis of nal of the Experimental Analysis of Behavior, 1965, 8,
Behavior, 1969, h 213-223, t 427-435. (b)
Blouoh, D. S. Interresponse time as a function of continu Dews, P, B. The effect of multiple SA periods on respond¬
ous variables: A new method and some data. Journal ing on a fixed-interval schedule, IV; Effect of continuous
Of the Experimental Analysis of Behaviorj 1963, 6, 937- SA with only short SD probes. Journal of the Experi¬
246. mental Analysis of Behavior, 1966, 9, 147-151. (a)
SLOUCH, D. S. The reinforcement of least-frecjuent mter- Dews, P. B. The effect of multiple SA periods on responding
response times. Journal of the Experimental Analysis of on a fixed-interval schedule, V: Effects of periods of
Behavior, 1966, 9, 581-591. complete darkness and of occasional omissions of food
Blouch, P. M., & Slouch, D. S. The distribution of inter¬ presentations. Journal of the Experimental Analysis of
response times in the pigeon during variable-interval Behavior, 1966, 9, 573-578. (b)
reinforcement. Journal of the Experimental Analysis of Dews, P. B. Studies on responding under fixed-interval
Behavior, 1968, 11j 23-27. schedules of reinforcement: The effects on the pattern
Boren, J. J, Resistance to extinction as a function of the of responding of changes in requirements at reinforce¬
fixed ratio. Journal of Experimental Psychology, 1961, ment. Journal of the Experimental Analysis of Behavior,
hi, 304-308. 1969, 12y 191-199.
Catania, A. C. Reinforcement schedules and psychophysical Dews, P. B. The theory of fixed-interval responding. In
judgments: A study of some temporal properties of be¬ W. N. Schoenfeld (Ed.), The theory of reinforcement
havior. In W. N. Schoenfeld (Ed.), The theory of rein¬ schedules. Englewood Cliffs, N.J.; Prentice-Hall, Inc.,
forcement schedules. Englewood Cliffs, N. J.: Prentice- 1970.
Hall, Inc,, 1970. Farmer, J., 8c Schoenfeld, W. N. Varying temporal place¬
Catania, A. C- Reinforcement schedules; The role of re¬ ment of an added stimulus in a fixed-interval schedule.
sponses preceding the one that produces the remforcer. Journal of the Experimental Analysis of Behavior, 1966,
Journal of the Experimental Analysis of Behavior, 1971, 9, 369-375.
15, 271-287. Felton, M., Lyon, D. O. The post-reinforcement pause.
Catania, A. G., & Reynolds, G. S. A quantitative analysis Journal of the Experimental Analysis of Behavior, 1966,
of the responding maintained by interval schedules of 2, 131-134.
reinforcement. Journal of the Experimental Analysis of Ferster, C. B. Sustained behavior under delayed reinforce¬
Behavior, 1968, 11, 327-383. ment. Journal of Experimental Psychology, 1953, 45,
Crossman, E. K., Heaps, R. S., Nunes, D. L., 8c Alferink, 218-224.
L. A, The effects of number of responses on pause Ferster, C. B., 8c Hammer, G. Variables determining the
length with temporal variables controlled. Journal of effects of delay in reinforcement. Journal of the Experi¬
the Experimental Analysis of Behavior, 1974, 22, in mental Analysis of Behavior, 1965, 8, 243-254.
press. Ferster, C. B., 8c Perrott, M. C. Behavior principles. Engle¬
Cumming, W. W., 8c Schoenfeld, W. N. Behavior under wood Cliffs, N.J.; Prentice-Hall, Inc., 1968.
extended exposure to a high-value fixed interval rein¬ Ferster, C. B., 8c Skinner, B. F. Schedules of reinforcement.
forcement schedule. Journal of the Experimental Anal¬ Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1957.
ysis of Behavior, 1958, 1, 245-263. Findley, J. D. An experimental outline for building and
Day, R. B., 8c Platt, J. R. Several tests of the response-unit exploring multioperant behavior repertoires. Journal of
hypothesis of extinction. Paper read at the meeting of the Experimental Analysis of Behavior, 1962, 5, 113-116.
the Psychonomic Society, 1972. Gott, C. T., 8c Weiss, B. The development of fixed-ratio
Michael Zeiler 231

performance under the influence of ribonucleic acid. forcement. Quarterly Journal of Experimental Psychol¬
Journal of the Experimental Analysis of Behavior, 1972, ogy, 1970, 22, 663-673.
18, 481-497. Morse, W. H. Intermittent reinforcement. In W. K. Honig
Hamilton, E. L. The effect of delayed incentive on the (Ed.), Operant behavior: Areas of research and applica¬
hunger drive in the white rat. Genetic Psychology Mono¬ tion. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966.
graphs, 1929, 5, 131-207. Morse, W. H., 8c Kelleher, R. T. Schedules as funda¬
Harvey, J. A. Behavior analysis of drug action. Glenview, mental determinants of behavior. In W. N. Schoenfeld
Ill.: Scott, Foresman, 1971. (Ed.), The theory of reinforcement schedules. Englewood
Hawkes, L., 8c Shimp, C. P. Choice between response rates. Cliffs, N.J.: Prentice-Hall, Inc., 1970.
Journal of the Experimental Analysis of Behavior, 1974, Morse, W. H., 8c Skinner, B. F. A second type of “super¬
21, 109-115. stition” in the pigeon. American Journal of Psychology,
Herrnstein, R. J. Superstition: A corollary of the principles 1957, 70, 308-311.
of operant conditioning. In W. K. Honig (Ed.), Operant Mowrer, O. H., 8c Jones, H. Habit strength as a function
behavior: Areas of research and application. Englewood of the pattern of reinforcement. Journal of Experi¬
Cliffs, N.J.: Prentice-Hall, Inc., 1966. mental Psychology, 1945, 35, 293—311.
Herrnstein, R. J., & Morse, W. H. Some effects of response- NEURINGER, A. J. Superstitious key pecking after three peck-
independent positive reinforcement on maintained oper¬ produced reinforcements. Journal of the Experimental
ant behavior. Journal of Comparative and Physiological Analysis of Behavior, 1970, 13 127-134.
Psychology, 1957, SO, 461-467. Neuringer, A. J., 8c Schneider, B. A. Separating the effects
Herrnstein, R. J., 8c Morse, W. H. A conjunctive schedule of mterreinforcement time and number of interrein¬
of reinforcement. Journal of the Experimental Analysis forcement responses. Journal of the Experimental Analy¬
of Behavior, 1958, 1, 15-24. sis of Behavior, 1968, 11, 661-667.
Hull, G. L. The goal gradient hypothesis and maze learn¬ Overmann, S. R., & Denny, M. R. The free-operant partial
ing. Psychological Review> 1932, 39, 25-43. reinforcement effect: A discrimination analysis. Learn*
Hull, C. L, A behavior system. New Haven; Yale Univer¬ ing and Motivation, 1974, 3, 248-257.
sity Press, 1952. Pierce, G. H., Hanford, P. W., & Zimmerman, J. Effeets of
Jenkins, H. M. Sequential organization in schedules of different delay of reinforcement procedures on variable-
reinforcement. In W. N. Schoenfeld (Ed.), The theory of interval responding. Journal of the Experimental Anal¬
reinforcement schedules. Englewood Cliffs, N.J.: Pren¬ ysis of Behavior, 1972, 18, 141-146.
tice-Hall, Inc., 1970. Platt, J. R. Percentile reinforcement: Paradigms for ex*
Kelleher, R. T. Chaining and conditioned reinforcement, perimental analysis of response shaping* In G. H. Bower
In W. K. Honig (Ed.), Op erant behavior: areas of re¬ (Ed.), Psychology of learning and motivationAdv9tT\GGS
search and application. Englewood Cliffs, N.J.1. Prentice- in research and theory (Vol. 7). New Vork: Academic
Hall, Inc., 1966. Press, 1973.
Kelleher, R. T., Fry, W., & Cook, L. Adjusting fixed-ratio Platt, J. R., & Johnson, D. M. Localization of position
schedules in the squirrel monkey, Journal of the Experi¬ within a homogeneous behavior chain: Effects of error
mental Analyses of Behavior, 1964, 7, 69-77. contingencies. Learning and Motivation, 1971, Z, 380-
Kelleher, R, T,, & Mqrse, W, H. Determinants of the 414.
specificity of behavioral effects of drugs. Ergebnisse der Pliskoff, S. & Goldiamond, I. Some discriminative prop¬
Physiologic, 1968, 60, 1 56. erties of fh^d ratio performance In the pigeon. Journal
Killeen, P. Reinforcement frequency and contingency as of the Experimental Analysis of Behavior. 1966, 9, 1~9,
factors in fixed-ratio behavior. Journal of the Experi¬ Powell, R. W. The effect of small sequential changes in
mental Analysis of Behavior, 1969, 12, 391-395.
fixed-ratio size upon the post-reinforcement pause. Jour¬
Lashley, K. S. The accuracy of movement in the absence of
nal of the Experimental Analysis of Behavior, 1968, 1L
excitation from the moving organ. American Journal of
589-593.
Physiology, 1917, 43, 169-194.
Randolph, J. J., & Sewell, W. R. A chained adjusting
Lashley, K. S. The problem of serial order in behavior. In
ratio schedule. Psychological Reports, 1968, 22, 989-995.
L. A. Jeffress (Ed.), Cerebral mechanisms in behavior.
Reynolds, G. S. Discrimination and emission of temporal
New York: Wiley, 1951.
intervals by pigeons. Journal of the Experimental Analy¬
Lee, J. K., 8c Gollub, L. R. Second-order schedules with
sis of Behavior, 1966, 9, 65-68.
fixed-ratio components: Variation of component size.
Reynolds, G. S., &: Catania, A. C. Temporal discrimination
Journal of the Experimental Analysis of Behavior, 1971,
15, 303-310. in pigeons. Science, 1962, 135, 314-315.
Malott, R. W., 8c Cumming, W. W. Schedules of inter¬ Reynolds, G. S., 8c McLeod, A. On the theory of interre¬
response time reinforcement. Psychological Record, 1964, sponse-time reinforcement. In G. H. Bower (Ed.), The
14, 211-252. psychology of learning and motivation (Vol. 4). New
Marr, M. J. Sequence schedules of reinforcement. Journal York: Academic Press, 1970.
of the Experimental Analysis of Behavior, 1971, 15, Richardson, K. A test of the effectiveness of the differential-
41-48. reinforcement-of-low-rate schedule. Journal of the Exper¬
McKearney, J. W. Rate-dependent effects of drugs: Modifi¬ imental Analysis of Behavior, 1973, 20, 385-391.
cation by discriminative stimuli of the effects of amobar- Richardson, W. K., 8c Loughead, T. E. Behavior under
bitol on schedule-controlled behavior. Journal of Experi¬ large values of the differential-reinforcement-of-low-rate
mental Analysis of Behavior, 1970, 14, 167-175. schedule. Journal of the Experimental Analysis of Be¬
Morgan, M. J. Fixed interval schedules and delay of rein¬ havior, 1974, 22, 121-129.
232 SCHEDULES OF REINFORCEMENT

Rilling, M. E., 8c McDiarmid, C. G. Signal detection in Spence, K. W. Behavior theory and conditioning. New
fixed-ratio schedules. Science, 1965, 148, 526-527. Haven: Yale University Press, 1956.
Schneider, B. A two-state analysis of fixed-interval respond¬ Staddon, J. E. R. Some properties of spaced responding in
ing in the pigeon. Journal of the Experimental Analysis pigeons. Journal of the Experimental Analysis of Be¬
of Behavior, 1969, 12, 667-687. havior, 1965, 8, 19-27.
SCHOENFELD, W. N., COLE, B. K., BLAUSTEIN, J., LACHTER, Staddon, J. E. R. Asymptotic behavior: The concept of the
G. D., Martin, J. M., 8c Vickery, C. Stimulus schedules: operant. Psychological Review, 1967, 74, 377-391.
The t-r systems. New York: Harper 8c Row, 1972. Staddon, J. E. R. Temporal effects of reinforcement: A
Shimp, C. P. The reinforcement of short interresponse negative “frustration” effect. Learning and Motivation,
times. Journal of the Experimental Analysis of Behavior. 1970, 1, 227-247.
1967, 10, 425-434. Staddon, J. E. R. Reinforcement omission on temporal
Shimp, C. P. Magnitude and frequency of reinforcement go-no-go schedules. Journal of the Experimental Analy¬
and frequencies of interresponse times. Journal of the sis of Behavior, 1972, 18, 223-229.
Experimental Analysis of Behavior, 1968, 11, 525-535. Stubbs, A. The discrimination of stimulus duration by
Shimp, C. P. The concurrent reinforcement of two inter¬ pigeons. Journal of the Experimental Analysis of Be¬
response times? The relative frequency of an interre¬ havior, 1968/22, 223-238,
sponse time equals its relative harmonic length. Journal Taub, E., 8c Berman, A. J. Movement and learning in the
of the Experimental Analysis of Behavior, 1969, 12, 403- absence of sensory feedback. In S. J. Freedman (Ed.),
411. The neuropsychology of spatially oriented behavior.
SHIMP, C. P. Synthetic variable-interval schedules of rein¬ Homewood, Ill.: Dorsey Press, 1968.
forcement. Journal of the Experimental Analysts of Be¬ Weiss, B., 8c Gott, C. T. A microanalysis of drug effects on
havior, l 973, 19, 311-330.
fixed-ratio performance in pigeons. Journal of Pharma¬
Shull, R. L. The response-reinforcement dependency in
cology and Experimental Therapeutics, 1972, 180, 189-
fixed-interval schedules of reinforcement. Journal of the
202.
Experimental Analysis of Behavior, 1970, 14, 55-60.
Weissman, N. W., k Crossman, E. K. A comparison of two
Shull, R. L. Sequential patterns In post-reinforcement
types of extinction following fixed-ratio training. Journal
pauses on fixed-interval schedules of food. Journal of the
of the Experimental Analysis of Behavior, 1966, 9, 41-46.
Experimental Analysis of Behavior, 1971, 15, 221-231.
Zeiler. M. D. Fixed and variable schedules of response-
Shull, R, L,, Guilkey, M., 8c Witty, W. Changing the
response unit from a single peck to a fixed number of
independent reinforcement. Journal of the Experimental
pecks m fixed-interval schedules. Journal of the Experi¬ Analysis of Behavior, 1968,11, 405-414.
mental Analysis of Behavior, 1972, 15, 1 93—200. Zeiler, M. D. Time limits for completing fixed ratios.
SlDMAN, M. Tactics Of seientipc research. New York: Basic Journal of the Experimental Analysis of Behavior, 1970,
Rooks, 1960. 14, 275-286.
Skinner, B. F. The behavior of organisms. New York: Ap- Zeiler, M. D, Superstitious behavior in children: An exper¬
pleton-Oentury-Grofts, 1938. imental analysis, In H. W, Reese (Ed,), Advances in
Skinner, B. F. “Superstition” in the pigeon. Journal of child development and behavior (Vol. 7), New York;
Experimental Psychology, 1948, 168— l72« Academic Press, 1972- (a)
Skinner B. F. A case history m scientific method. In S, Zeiler, M. D. Time limits for completing- fixed ratios, II:
Koch (Ed.), Psychology! A study of a science (Vol. 2). Stimulus specificity. Journal of the Experimental Analy¬
New York, McGraw-Hill, 1959. sis of Behavior, 1972, 18, 243-251. (b)
 9

Choice in
Concurrent Schedules and
a Quantitative Formulation
of the Law of Effect*

Peter de Villiers

INTRODUCTION choice can be used to quantify the relative reward

value of different conditions of reinforcement.
Since the early 1960s there has been much operant Second, rates of responding to each of two alterna¬
conditioning research using continuous choice pro¬ tives are far more sensitive to the frequency and mag¬
cedures. Several factors suggest the utility of these nitude of reinforcement for each alternative than are
procedures for quantifying the effects of reward and response rates in a single response situation (Catania,
punishment on behavior, the law of effect. 1963a; Herrnstein, 1961).
First, in continuous choice procedures (called con¬ The most persuasive argument for any measure of
current schedules) two or more alternative schedules response strength is an orderly relation between that
of reinforcement are simultaneously available and the measure and the frequency, duration, or immediacy
animal continually chooses between responding to of reinforcement. Relative performance and reinforce¬
one alternative or the other. Thus the number of re¬ ment measures obtained from concurrent schedules
sponses or amount of time the animal allocates to each show just such an orderly relation. Many studies have
alternative during an experimental session may be shown that a simple linear “matching” relation holds
considered a measure of its preference. In this way between relative response rates (or time distribution)
and relative frequency (Herrnstein, 1961), magnitude
# The preparation of this chapter was supported by grants (Brownstein, 1971; Catania, 1963a) and immediacy of
from the National Institute of Mental Health and the National
reinforcement (Chung 8c Herrnstein, 1967). This rela¬
Science Foundation to Harvard University. Thanks are due to
many people for helpful suggestions, encouragement, and great tion is described by the following equations:
patience, in particular to Bill Baum, Arturo Bouzas, Peggy
Burlet, Jill de Villiers, Richard Herrnstein, Werner Honig,
James Mazur, Arlene Pippin, John Staddon, Dan Weber, and Tj,
-or-—— =-or -— or-(1)
r, h <»i ...
James Wilkinson. I acknowledge my indebtedness to work by
R1 -f- R2 T1 + T2 + r2 i1 + i2 cli + #2
Peter Killeen for many of the points raised in the discussion,
and to several graduate students in the Harvard University
operant conditioning laboratory for permission to quote un¬
published data.
where Rx and R2 are the number of responses per ses-

233
234 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

sion to each of the two alternatives, 7\ and T2 are

SCHEDULE SCHEDULE
the times spent responding on each schedule, and CONTROL 1 OR 2
SCHEDULE 1 SCHEDULE 2

and r2 are the frequencies of reinforcement for the

alternative responses; i1 and i2 represent different im¬
mediacies (the reciprocal of the delay of reinforce¬
OhO
ment), and % and a2 different amounts of reinforce¬
ment for the two alternatives. From the matching
relation, Herrnstein (1970) has formulated a powerful
set of equations that describe the relation between
response strength and reinforcement parameters in
single-response situations as well as concurrent sched¬
ules.
This chapter is largely concerned with an evalua¬ Fig. 1. Representation of two methods of programming con¬
current schedules. The pigeon on the left is responding on a
tion of Herrnstein’s equations as a quantification of
changeover-key concurrent schedule (Findley, 1958), the pigeon
the law of effect. After briefly describing concurrent on the right on a two-key concurrent schedule (Herrnstein,
schedule procedures, the chapter assesses the empirical 1961.)
basis and generality of the matching relation and its
status as a general principle. Certain prerequisite
conditions for matching are discussed, together with When equal concurrent interval schedules are pro¬
the role of procedural factors or more molecular proc¬ grammed in this way, rapid alternation between the
esses. Herrnstein’s (1970) quantitative formulation is two schedules is the dominant response pattern
then considered in detail—in particular, its ability to (Herrnstein, 1961; Skinner, 1950). This is conducive to
incorporate results of earlier runway studies in the the development of concurrent superstitions (Catania,
tradition of Hull and Spence, and its relation to be¬ 1966), the adventitious correlation of one response with
havioral contrast phenomena. Finally, alternative or reinforcement programmed for another. Responses to
more general quantitative models of choice in con¬ one alternative thus come partially under the control
current schedules are discussed. of the reinforcement schedule associated with the
other. The best illustration of this comes from a study
of one pigeon by Catania and Cutts (1963). Pecking at
CONCURRENT SCHEDULES two keys was maintained by a concurrent VI 1-min VI
2-min schedule. Then the reinforcement for the VI
Two different methods of programming concurrent 2-min key was discontinued (extinction). The pigeon
schedules have generally been used. In one of these continued pecking at a steady though somewhat re¬
(Herrnstein, 1961), the animal switches back and forth duced rate (about 15 responses per min) on the ex¬
between two spatially separated response keys or lev¬ tinction key throughout the 12 1-hr sessions. Catania
ers, each associated with a different reinforcement and Cutts reported similar results from 13 human sub¬
schedule. In the second (Findley, 1958), the animal jects pressing two buttons on a concurrent VI 30-sec
switches between two schedules programmed on the EXT schedule. Most of the subjects responded at a
same key by responding on a second changeover (CO) substantial rate on the button associated with extinc¬
key; each schedule is correlated with a different stim¬ tion. Although never explicitly reinforced, responses
ulus. The first method will be referred to as a two- on that button often occurred in close temporal prox¬
key or two-lever concurrent, the second as a CO-key imity to reinforcement (increments of a counter) for
concurrent schedule. The difference between the two the other button. Several early studies of concurrent
procedures is illustrated in Figure 1. schedules also found control of one response by the
In both procedures the two component schedules schedule for another (e.g., Sidman, 1958). Ferster and
are programmed independently and continuously. If Skinner (1957, Chapter 13), for example, programmed
a reinforcement opportunity is programmed by a a concurrent VI FI (fixed-interval) schedule, each
variable-interval (VI) schedule for response A while schedule associated with a different key. On a single
the animal is making response B, that reinforcement FI schedule almost no responding occurs early in the
is held until the animal again makes response A. interval, but in the concurrent schedule considerable
Consequently, the probability of reinforcement from responding was maintained on the FI key early in the
one schedule increases with the time spent responding fixed interval, probably by accidental correlation with
on the other schedule. VI reinforcements for pecks on the other key.
Peter de Villiers 235

To separate the two schedules a changeover delay sponding on each key because the concurrent sched¬
(COD) is usually added to the concurrent schedule ules run continuously and the response alternatives
procedure (Herrnstein, 1961). The COD specifies the are simultaneously available to the animal through¬
minimum time interval that must elapse between a out the experimental session. Calculation of response
CO and a subsequent reinforced response. In a two- rates with respect to the time that a given stimulus is
key (or two-lever) concurrent procedure the COD is in effect is more customary for multiple schedules,
usually timed from the first response on a given key where each schedule runs and is available to the an¬
(or lever) after a CO (Herrnstein, 1961); in CO-key imal only in the presence of a particular stimulus.
procedures it is usually timed from responses on the However, particularly in the CO-key concurrent sched¬
CO key (Catania, 1966). Alternatively, the COD can ule, the amount of time that the animal spends in the
be programmed from the last response on a key before presence of the stimuli associated with each schedule
a CO (Findley, 1958), but since this selectively rein¬ can also readily be calculated. This enables the calcu¬
forces slow COs, the two other methods are preferred. lation of local response rates—i.e., the number of re¬
Subsequent mention of the COD will refer to the first sponses made on each schedule divided by the time
two methods of scheduling, unless otherwise specified. spent responding on that schedule.
The COD therefore insures a separation in time The relation between response rates and reinforce¬
between response A and the reinforcement of response ment frequencies in concurrent schedules is usually
B, preventing the adventitious reinforcement of AB considered in terms of relative measures (Herrnstein,
sequences. Its effectiveness in this respect is illustrated 1970): relative overall response rate [the number of
by a condition of the Catania and Cutts (1963) pigeon responses made to one alternative divided by the
experiment in which a 1-sec COD was introduced. total number of responses during the session—i.e.,
When the schedule for one key changed to extinction, Ri/(R1 + R2)] and relative reinforcement frequency
responding on the extinction key rapidly declined to [ri/(ri + r2)]. Similarly, relative time distribution is
zero. Without the COD, responding on the extinction measured in terms of the time spent in the presence
key continued at a steady rate. In the Catania and of the stimulus associated with one schedule divided
Cutts human experiment the introduction of a COD by the total session time [T1/(T1 + T2)]. In two key
of between 2 and 15 sec in duration substantially re¬ concurrent schedules, the time distribution is calcu¬
duced and in many cases eliminated responding on lated in terms of cumulative in ter changeover time for
the extinction button. each alternative. Exclusive preference for one alterna¬
The dominant patterns of responding on equal tive is shown by a relative response rate or relative
concurrent VI schedules with a COD shows runs of time distribution of 1.00 or .00, indifference between
responses of roughly COD duration on one schedule the alternatives by relative values of .50. The relation
alternating with runs of responses of similar duration between ratios of responses (R1/R2) or of time (T1/T2)
on the other schedule. When unequal VI schedules and ratios of obtained number of reinforcements
are programmed concurrently, the duration of re¬ (ri/r2) for each of the alternative has also frequently
sponse runs maintained by the schedule programming been examined (e.g., Baum, 1974a; Baum 8c Rachlin,
more frequent reinforcement increases, and COs are 1969; Staddon, 1968).
less frequent (Catania, 1966). In the absence of the
COD the switching response itself is a primary com¬
ponent of the behavior, and the main effect of changes THE MATCHING RELATION IN
in reinforcement is on switching; with a COD the CONCURRENT VI SCHEDULES-
main effect is on the rate of responding to each al¬ REINFORCEMENT FREQUENCY
ternative.
In 1961 Herrnstein first demonstrated that when
Measurement of Choice or Preference
two independent VI schedules arranged reinforce¬
ments for concurrent responses and a COD was in
Response rates in concurrent schedules are usually effect, there was a matching relation between relative
calculated in terms of the number of responses made overall response rates and relative reinforcement fre¬
on each schedule divided by the total session time quency. Throughout his experiment the two VI sched¬
(minus time consumed by reinforcement)—i.e., over¬ ules set an overall maximum rate of reinforcement at
all response rates. Rate of responding to each alterna¬ 40 per hr, but the number of reinforcements allocated
tive is calculated with respect to overall session time to each key was systematically varied. At all distribu¬
rather than the time the animal actually spends re¬ tions of the reinforcements, Herrnstein found that the
236 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

varied from 15 to 120 per hr. For each of four pigeons,

relative rate of responding in each component sched¬
ule matched the relative frequency of food that the
schedule provided.
In an experiment by Baum (1972), a pigeon lived
in the experimental situation. All of its food was
obtained by pecking at two keys, each associated with
a separate VI schedule. A 1.8-sec COD was in effect
throughout the experiment. The bird was free to eat
to satiation and one alternative alone was often suffi¬
cient to fulfill the bird’s normal food requirements.
Distribution of responses between the alternatives was
therefore often unnecessary, but the pigeon neverthe¬
less made thousands of responses on each key each
day. For a wide range of relative reinforcement fre¬
quencies the proportion of pecks allocated to either
key equaled the proportion of food obtained by pecks
at that key.
In a subsequent experiment, Baum (1974b) ex¬
tended the matching relation to wild pigeons in a
RELATIVE REINFORCEMENT FREQUENCY more natural habitat. A version of the standard op¬
erant conditioning apparatus was placed in the attic
Fig. 2. The relative frequency of responding to one alterna¬ of a wooden frame house in Cambridge, Massachu¬
tive in a two-key concurrent VI VI schedule as a function of setts. A flock of about 20 free-ranging wild pigeons
the relative frequency of reinforcement for that alternative. The
diagonal line shows matching between the relative frequencies. that inhabit the attic were trained to peck at two
(Data from three pigeons, Herrnstein, 1961.) keys for access to grain. A narrow perch in front of
the keys allowed only one pigeon at a time access to
the keys and food, but the pecks of the group were
pigeons’ relative response rates approximately equalled
treated as an aggregate. Over a wide range of concur¬
the relative reinforcement frequencies for the two
rent VI VI schedules without a COD the pigeons’ pro¬
alternatives;
portion of pecks at a key approximately equaled the
proportion of grain presentations obtained from it.
Ri _ r± Nevin (1969) used a two-key concurrent procedure,
R 1 -f- R% ri + ^2 ^ ^ but presented the two keys simultaneously in discrete-
choice trials. Independent concurrent VI schedules
where R is response rate, r is reinforcement frequency, which ran during the intertrial interval as well as
and the subscripts denote the two alternatives. The during the choice trials arranged reinforcements.
data from the three pigeons in the experiment are When a reinforcer was scheduled for a key it was held
shown in Figure 2. and timing of the intervals in that schedule stopped
until the next choice trial in which that key was
chosen. The first peck in a trial terminated the trial:
Response Matching if a reinforcer had been scheduled for that key the
response produced 4-sec access to grain. Choice trials
Since Herrnstein’s 1961 study, matching between without a peck lasted for 2 sec. The intertrial interval
relative response rate and relative reinforcement fre¬ (ITI) was 6 sec long, during which time the keys
quency has been demonstrated in both kinds of con¬ were dark. Pecks during the ITI extended it for 6
current schedule procedure and for several different sec from the last peck. The proportion of responses
species. (choices) made by the pigeons to each key closely
McSweeney (1975) reinforced pigeons’ treadle matched the proportion of reinforcements produced
presses on several two-treadle concurrent VI VI sched¬ by each key in this discrete-trials procedure, as it does
ules with a 2-sec COD. The rate of reinforcement for in concurrent VI schedules with continuous access to
one alternative remained constant at 30 food presen¬ the keys.
tations per hr; the rate of reinforcement for the other Schroeder and Holland (1969) reported a further,
Peter de Villiers 237

perhaps more exotic, confirmation of the response¬ rate of lever pressing and time distribution matched
matching relation with human subjects. Their sub¬ the obtained distribution of reinforcements, but only
jects were required to monitor deflections of pointers when the COD was greater than 7.5 sec.
on four dials arranged in a square with two on the A different method for programming a CO-key con¬
left and two on the right. An eye movement camera current schedule was used by Stubbs and Pliskoff
recorded macrosaccadic eye movements scanning each (1969). One VI programmer arranged the reinforce¬
of the dials. A fixation on a dial after looking away ments for both the schedules, each schedule being
toward the other dials counted as one response. Look¬ associated with a different color stimulus on the main
ing horizontally or diagonally between the two pairs response key. When a reinforcement was programmed,
of dials was defined as a changeover. A change in fixa¬ it was allocated to one schedule or the other according
tion between two vertically arranged dials was there¬ to different probabilities. Thus whenever a reinforce¬
fore a response but not a changeover, while a change ment opportunity was arranged for one schedule, no
in fixation between left- and right-hand dials was further reinforcements could be arranged for either
both. Pointer deflections were delivered to the two schedule until the available reinforcement had been
left-hand dials on one variable-time (VT) schedule obtained. This procedure forces the subject to respond
and to the right-hand dials on a second, independent occasionally on both alternatives in order to obtain
VT schedule. Scheduled deflections were assigned reinforcement for either alternative, even if one of the
with an equal probability to the upper or lower dial schedules is extremely unfavorable when compared
on each side. Signal presentation was contingent on with the other. The advantage of the Stubbs and
looking toward the side for which it was scheduled, Pliskoff procedure lies in insuring that the obtained
but was independent of which of the two dials on relative reinforcement frequency must equal the
that side was being fixated.1 When a short COD was scheduled relative frequency. On the other hand, the
programmed between crossover eye movements and two alternative schedules are no longer independent
signals, the pattern of scanning changed from fixating of one another.
the four dials in succession or in a Z-shaped pattern to Stubbs and Pliskoff found matching for both rela¬
vertical scanning of the dials on either side with tive response rates and relative time for three pigeons
fewer crossovers. All reinforcements scheduled to at five different relative reinforcement frequencies. In¬
occur before the COD timed out were held until the creasing the COD from 2 sec, the value at which
end of the COD and then delivered. With a 2.5-sec matching was first obtained, through 32 sec produced
COD for one subject and 1.0-sec CODs for the others, little systematic change in relative response rates when
all six subjects in the experiment matched relative relative reinforcement frequency was kept constant at
scanning eye movement rates (number of fixations .75. In this procedure, responding on the two sched¬
per min) on each side to the relative signal frequen¬ ules is somewhat constrained since the subject has to
cies on each schedule. respond on the less favorable schedule in order to
maintain the overall frequency of reinforcement.
However, this requirement by no means forces the
Matching of Both Responses and Time subject toward matching, and might be expected to
favor indifference. Nevertheless, excellent matching
Catania (1963b) used a CO-key procedure so that was obtained.
time spent in each component could be accurately In contrast to these results, Schmitt (1974) failed to
measured. With a 2-sec COD in effect, he found that find matching with humans in a conventional CO-key
pigeons approximately matched both the relative re¬ concurrent schedule procedure. Five subjects pressed
sponse rates and the relative amount of time spent in a button for increments of a counter, each worth a
each component to the relative frequency of food. number of cents, on one of two independent VI sched¬
Similar results for pigeons were reported by Silberberg ules each associated with a different stimulus light.
and Fantino (1970) using a two-key procedure, but They could change schedules by operating a toggle
with CODs varying between .88 and 3.5 sec. With switch. A 1.5-sec COD stipulated the time that had to
rats as subjects and brain stimulation as the rein¬ elapse after a changeover before a button press could
forcer Shull and Pliskoff (1967) also found that relative be reinforced. In almost all of the experimental condi¬
tions relative response rates and relative time distribu¬
1 Reinforcement presentation was thus independent of the tion did not match the relative frequency of reinforce¬
vertical scanning eye movements, but the schedule was not
ment, the departure usually being toward indifference.
strictly a VT schedule since the reinforcements were contingent
on looking toward the appropriate side. Apart from the difference in subjects, reasons for the
238 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

discrepancy between these results and the studies on Baum and Rachlin plotted the results of this experi¬
nonhuman species are not readily apparent. ment in terms of Equation 3—i.e., the relative time
spent in the presence of either the red or green stim¬
ulus—the data points from most of the pigeons system¬
Time Matching
atically fell below the matching diagonal in a bowed
In most of these studies, local response rates were curve. However, they found that the results could be
found to be the same for each schedule, and response expressed in terms of the following equation:
matching resulted from the matching of relative time
distribution to relative reinforcement frequency.
Brownstein and Pliskoff (1968) and Baum and Rach- T r
1 2 r2
lin (1969) therefore argued that time and not response
allocation underlies the matching relation.
The ratio of the times spent on the two sides of the
Brownstein and Pliskoff (1968) demonstrated that
chamber was directly proportional to the ratio of the
the relative time spent in either component of a CO-
rates of reinforcement provided on the two sides.
key concurrent schedule matched the relative fre¬
When the logarithms of the time ratios are plotted
quency of food provided in that component even in
against the logarithms of the reinforcement ratios,
the absence of any key pecking for the food. The
Equation 4 specifies a linear function of the form
pigeons in their experiment changed the color of a
log (T1/T2) = 1.00 log (ri/r2) + log k, where 1.00 is
stimulus light by pecking a single CO key, but the
the slope and log k the intercept of the function on
reinforcements in each component were delivered
the F-axis when log (r1/r2) — 0. The data from one of
independently of the birds’ behavior according to
Baum and Rachlin’s pigeons is shown in Figure 3,
different VT schedules. The pigeons therefore chose
plotted in terms of both Equation 3 and the logarith¬
between two different frequencies of food delivery
mic form of Equation 4. The linear function of least
each correlated with a different stimulus color. In this
squares fit to the log data is shown in the lower panel
situation they matched the proportion of the session
together with the percentage of the variance in the
time spent in the presence of a color to the proportion
dependent variable accounted for by the function.
of reinforcements associated with that stimulus:
The value of k is given by the antilog of the intercept;
in this case k = .54. The slope of the function approx¬
Tl _ ri m imates 1.00 as specified by Equation 4.
T1+T2~r1 + r2 W
When k — 1.0, Equation 4 is identical to the
matching relation described by Equation 3. A A-value
Baum and Rachlin (1969) pointed out that pigeons
different from 1.0 signifies a constant proportional
tend to peck at a constant rate when they are respond¬
bias toward one side of the chamber or toward one
ing, with the majority of interresponse times (IRTs)
schedule. Such a bias shows up as a constant displace¬
falling between .3 and .5 sec (Blough, 1963). Long¬
ment from the matching diagonal when the logs of
term response rate varies with the duration of pauses
the ratios are plotted against one another, as in the
between bursts of responses at the constant rate. With
lower panel of Figure 3. Hence Equation 4 (propor¬
such a constant rate of responding, time spent re¬
tional ratio matching) and its response equivalent.
sponding determines the number of responses. Hence,
Baum and Rachlin argued, time spent responding is
the most general measure of response frequency for
(5)
repetitive responses like key pecking or lever pressing.
Baum and Rachlin studied a response that can only
be measured in terms of time spent responding: stand¬ are more general than Equations 2 and 3 (matching
ing in a particular location. Standing on one or the of relative proportions). They account for such factors
other side of an experimental chamber was reinforced as position or color preferences, or even preferences
on two concurrent VI schedules. A 4.25-sec COD was arising from qualitatively different reinforcers (see p.
signaled by the illumination of a white light in the 252). The bias parameter k takes account of our im¬
chamber, during which time no reinforcement was perfect knowledge of the reinforcers at work in the
presented. Post-COD standing on either side of the experimental situation. For example, in the Baum
chamber was correlated with the illumination of and Rachlin (1969) study there were two feeders. If
either a red or green stimulus light, each associated they did not yield equal quantities of food, k ^ 1.0 to
with a different frequency of food presentation. When the extent that the units of r1 and r2 were not equiv-
Peter de Villiers 239

Baum (1975) recently reported time matching for

humans in a vigilance task. Subjects were required to
monitor red or green signals projected on a trans¬
lucent plastic screen. The signals were arranged ac¬
cording to a single VI schedule and assigned as red
or green with varying probabilities (cf. Stubbs and
Pliskoff, 1969). By pressing down one of two telegraph
keys the subjects illuminated the screen with either a
green (left key) or red (right key) floodlight for the
duration of the press, enabling them to detect the red
or green signals. (Opposite colors made the signals
visible.) As long as the appropriate floodlight was on,
the signal remained on until it was turned off by
pushing a button next to the depressed key. Turning
off a signal incremented a “score” counter by 1. To
r, reduce subjects’ changeover rates a 2-sec COD was in
effect and a response cost was programmed on a vari¬
rl+r2
able-ratio (VR) 3 for release of a key after it had been
depressed. The response cost consisted of the incre¬
ment by 1 of a second counter, the final tally of which
was subtracted from the “score” counter to determine
the overall session score. Pilot studies had indicated
that the response cost was necessary to prevent sub¬
jects from simply alternating as rapidly as possible be¬
tween the two keys.
To maintain the subjects’ interest throughout the
experiment it was given a gamelike appearance. As
captain of a spaceship under enemy siege, the subject
defended himself by detecting and destroying two
types of enemy missiles: red ones and green ones. The
colored floodlights represented the appropriate sen¬
sors; the response cost represented a “hit” by the
enemy when a sensor was “deactivated.” Subjects
competed for a monetary bonus given for the highest
session score (missiles detected minus “hits”) for each
Fig. 3. Data from one pigeon in Baum and Rachlin’s (1969) block of five sessions.
shuttlebox choice procedure plotted in terms of relative time
With a 2-sec COD and a VR 3 response cost, two of
and reinforcement distributions (top panel) and in terms of
the logarithms of the time and reinforcement ratios (bottom the three subjects showed excellent matching be¬
panel). The heavy diagonal line represents perfect matching; the tween the ratio of the times for which each key was
fine diagonal line plotted through the data points is the least-
depressed and ratio of detections of each type of
squares regression line for those points. The regression function
and the percentage of data variance that it accounts for is signal. The third subject was nearer indifference than
given in the lower panel. predicted by matching, but when the response cost in¬
creased to a “hit” for every release, changeover rate
declined and excellent matching was obtained over a
alent. Other biases can also be considered as rein¬ wide range of signal ratios.
forcers not identified in the independent variable.
Baum (1974a) has therefore suggested that choice in
Assessment of the Empirical Evidence for Matching
concurrent schedules is best described by the ratio of
responses or times spent responding as some function How strong a generalization is the matching rela¬
of the ratio of reinforcements. Where relative pro¬ tion from the data just reviewed? Proportional ratio
portion matching is obtained, proportional ratio matching (Equations 4 and 5) specifies that the regres¬
matching must also hold, but the converse is not sion line relating log response or time ratios to log
necessarily true. reinforcement ratio have a slope of 1.0. A slope greater
 240 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

Table 1 Percentage of data variance accounted for by least-squares regression lines and best fit line of 1.0 slope relating
log response ratio to log reinforcement ratio for both individual subjects and groups

BEST FIT LINE NO. OF

INDIVIDUAL DATA LEAST-SQUARES LINE OF 1.0 SLOPE POINTS COD

Herrnstein {1961)
P231 1.36X- .04 (99.0%) 1.00X — .12 (92.2%) 5 1.5 sec
P055 .80X-.01 (98.3%) 1.00X +.01 (92.3%) 5 1.5 sec
Catania {1963b)
PI 17 .83X-.01 (90.5%) 1.00X — .02 (86.2%) 8 2 sec
P243 •77X + .01 (92.9%) 1.00X — .03 (87.2%) 8 2 sec
P294 .80X + .03 (90.3%) 1.00X (84.8%) 8 2 sec
Stubbs if Pliskoff {1969)
P103 .86X + .08 (96.0%) l.OOX-f .04 (93.7%,) 4 2 sec
P104 1.24X-.01 (92.6%) 1.00X + .06 (89.1%) 4 2 sec
P108 •94X + .02 (98.4%) 1.00X + .01 (98.0%,) 4 2 sec
Silberberg if Fantino (1970)
A •74X - .05 (99.9%) 1.00X — .06 (87.8%,) 3 .88 sec
E .77X - .06 (99.7%) 1.00X — .06 (91.0%) 3 .88 sec
G •94X -f .07 (48.0%) l.OOX-f .04 (42.0%) 3 1.75 sec
B .55X-.18 (61.0%) 1.00X + .10 (20.5%) 3 1.75 sec
G .63X - .05 (99.0%) 1.00X + .10 (66.1%) 3 3.5 sec
H .93X + .17 (99.8%) l.OOX-f. 16 (99.2%) 3 3.5 sec

P101
,
Trevitt, Davison & Williams (1972)
.52X + .02 (94.9%) 1.00X — .04 (10.9%,) 4 3 sec
P102 1.01X — .15 (98.8%) 1.00X — .15 (98.5%,) 4 3 sec
P105 .61X- .17 (91.5%) 1.00X — .21 (65.8%) 4 3 sec
P106 .73X - .04 (97.9%) 1.00X — .07 (85.5%) 4 3 sec
Baum (1972)
A .95X-.01 (99.4%,) 1.00X-.01 (99.2%,) 5 1.8 sec
McSweeney (1975)
8422 .74X + .03 (93.8%,) 1.00X — .04 (89.6%) 4 2 sec
8772 .89X-.01 (86.8%,) 1.00X — .04 (85.6%) 4 2 sec
8845 1.01X — .03 (95.3%) 1.00X-.03 (95.1%) 4 2 sec
8927 •79X - .02 (84.2%) 1.00X-.08 (83.7%) 4 2 sec
GROUP DATA

Herrnstein (1961) 1.1 IX-.02 (92.3%) 1.00X — .03 (91.4%) 12 1.5 sec
Catania (1963b) .80X + .01 (90.7%) 1.00X-.02 (86.3%) 24 2 sec
Stubbs if Pliskoff (1969) 1.01X + .03 (92.5%) l.OOX-f .04 (90.4%,) 12 2 sec
Silberberg if Fantino (1970) .85X + .04 (92.9%) 1.00X -f .05 (90.0%) 18 .88-3.5 sec
Trevitt, Davison, if Williams (1972) .76X - .10 (88.7%) 1.00X- .12 (80.7%,) 16 3 sec
Baum (1974b) 1.03X+.05 (99.3%) 1.00X + .04 (99.1%) 5 no COD
McSweeney (1975) .85X-.01 (87.2%) 1.00X - .02 (84.2%,) 16 2 sec
Peter de Villiers 241

Table 2 Percentage of data variance accounted for by least-squares regression lines and best fit line of 1.0 slope relating
log time ratio to log reinforcement ratio for both individual subjects and groups

BEST FIT LINE NO. OF

INDIVIDUAL DATA LEAST-SQUARES LINE OF 1.0 SLOPE POINTS COD

Catania (1963b)
\ /
PI 17 MX - .04 (94.8%) 1.00X — .05 (94.5%) 8 2 sec
P243 .76X - .06 (86.0%) 1.00X + .02 (77.2%) 8 2 sec
P294 .94X - .04 (93.4%) 1.00X — .05 (93.1%) 8 2 sec
Brownstein & Pliskoff (1968)
P93 1.04X + .01 (98.6%) 1.00X + .01 (98.5%) 5 2 sec
Baum & Rachlin (1969)
P334 .84X - .25 (97.3%) 1.00X - .24 (92.5%) 11 4.25 sec
P360 .63X-.12 (98.1%) 1.00X-.11 (64.4%) 11 4.25 sec
P488 1.09X-.29 (96.5%) 1.00X-.29 (95.4%) 11 4.25 sec
P489 1.15X-.49 (90.3%) 1.00X — .49 (88.8%) 11 4.25 sec
P490 1.29X — .06 (96.3%) 1.00X — .08 (91.5%) 11 4.25 sec
P496 .98X - .27 (98.5%) 1.00X - .26 (97.9%) 11 4.25 sec
Stubbs & Pliskoff (1969)
P103 1.03X + .11 (97.5%) 1.00X + .12 (97.4%) 4 2 sec
PI 04 1.24X+.00 (92.8%) 1.00X + .07 (89.4%) 4 2 sec
P108 1.07X + .01 (99.0%) 1.00X + .03 (98.5%) 4 2 sec
Silberberg & Fantino (1970)
A •87X-.21 (99.7%) 1.00X — .21 (97.5%) 3 .88 sec
E 1.06X — .18 (96.2%) 1.00X — .18 (95.9%) 3 .88 sec
C .58X + .45 (99.8%) 1.00X + .21 (48.6%) 3 1.75 sec
B .84X — .12 (98.7%) 1.00X — .02 (90.2%) 3 1.75 sec
G .97X-.08 (96.8%) 1.00X — .06 (96.6%) 3 3.5 sec
H 1.22X — .06 (96.3%) 1.00X-.01 (95.6%) 3 3.5 sec
Trevitt, Davison, & Williams (1972)
P101 .60X - .02 (94.5%) 1.00X - .06 (35.0%) 4 3 sec
PI 02 1.1 IX-.08 (95,9%) 1.00X — .08 (95.0%) 4 3 sec
P105 .70X- .10 (94.0%) 1.00X — .12 (80.9%) 4 3 sec
PI 06 .98X - .08 (98.2%) 1.00X- .08 (98.1%) 4 3 sec
Baum (1975)
Doug 1.16X-.08 (90.6%) 1.00X — .08 (82.8%) 10 2 sec
Noa .98X + .03 (93.3%) 1.00X +.04 (92.6%) 10 2 sec
John I .67X (96.1%) 1.00X + .01 (70.2%) 10 2 sec
John II .94X + .15 (93.7%) 1.00X + .15 (93.3%) 11 2 sec
GROUP DATA

Catania (1963b) .89X-.01 (90.2%) 1.00X-.02 (88.7%) 24 2 sec

Brownstein & Pliskoff (1968) .94X + .02 (97.5%) 1.00X + .02 (96.6%) 12 2-7.5 sec
Baum & Rachlin (1969) 1.0 IX-.24 (89.4%) 1.00X- .24 (89.0%) 66 4.25 sec
Stubbs & Pliskoff (1969) 1.11X + .04 (95.0%) 1.00X + .07 (94.0%) 12 2 sec
Silberberg & Fantino (1970) 1.07X — .05 (93.9%) 1.00X - .05 (93.4%) 18 .88-3,5 sec
Trevitt, Davison, & Williams (1972) .88X-.08 (91.0%) 1.00X - .09 (89.4%) 16 3 sec
Baum (1975) .93X (87.6%) 1.00X (85.6%) 30 2 sec
242 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

than 1.0 represents overmatching, a stronger prefer¬ However, several methodological considerations must
ence for the schedule providing the more frequent be taken into account in evaluating the data from
reinforcement than that predicted by matching. A some of the studies.
slope less than 1.0 represents undermatching, a weaken The strongest evidence for systematic undermatch¬
preference for the richer schedule than that predicted. ing comes from Trevitt et al. (1972). In this experi¬
In the following analysis a least-squares line of best ment four different concurrent VI VI values were
fit was calculated for the data from all of the pub¬ studied in the course of an experiment on choice in
lished experiments on concurrent VI VI schedules in two-key concurrent VI FI schedules. All of the data
which at least three different ratios of reinforcement points for the concurrent VI VI schedule were ob¬
frequency were studied. Data points showing exclusive tained after long exposure to the different VI FI
preference for one or other schedule where the rein¬ values, five VI FI conditions being studied before the
forcement ratio was either 0 or oo were excluded from first VI VI. In both the VI FI and VI VI conditions,
the analysis, since neither of these can be expressed both keys were illuminated with white light, and the
as a logarithm, although they are in keeping with the FI schedule was always associated with the same key.
matching relation. To evaluate Equations 4 and 5 the During the VI VI procedure three of the four pigeons
percentage of data variance in the dependent variable showed a bias toward the key that had been correlated
accounted for by proportional ratio matching—i.e., by with the VI schedule and preferred throughout the
a line of 1.0 slope—was also calculated in each case. concurrent VI FI. All four pigeons showed regression
This analysis is shown in Table 1 for response ratios lines for response ratios with very similar slopes to
for both individual and group data. The group regres¬ those obtained for them in the VI FI conditions. Both
sion lines were calculated from all the data points Trevitt et al. and Nevin (1971) demonstrated that re¬
provided by the subjects in that experiment. Table 2 gression lines relating response ratios to obtained
shows the same analysis for time ratios.2 reinforcement ratios on concurrent VI FI schedules
The matching relation accounts for over 80% of have slopes of considerably less than 1.0. The prior
the data variance for 18 of the 23 individual subjects exposure to concurrent VI FI schedules could there¬
in Table 1 (response ratios), and for 22 of the 27 sub¬ fore have affected the VI VI data in this study.
jects in Table 2 (time ratios). It fails to account for a The Silberberg and Fantino (1970) study raises
substantial proportion of the data for only 4 subjects, another consideration: control for order effects. The
2 in the Trevitt, Davison, and Williams (1972) study two pigeons that showed particularly marked under¬
and 2 in the Silberberg and Fantino (1970) study. In matching, subjects B and C, were exposed to only
fact, for the response ratios of the latter 2 pigeons, three different relative reinforcement frequencies,
even the least-squares line accounts for little of the with the same key providing more frequent reinforce¬
variance. A possible explanation for the marked ment in each case. Subject B was successively exposed
deviation of these subjects from matching is con¬ to relative reinforcement frequencies of .33, .20, and
sidered later in this section. .11, subject C to relative frequencies of .67, .90, and
Group data are important, especially for the studies .88. Any order effects in which the previous reinforce¬
in which only a few points were obtained for each ment conditions affected responding on the new rein¬
subject. Here the matching relation accounts for over forcement schedules (so-called hysteresis effects—Baum,
80% of the variance in response ratios and for over 1974a; Stevens, 1957) could have produced the flat
85% of the variance in time ratios for all the experi¬ regression lines for the pigeons.3
ments. In all cases, the matching line is less than 8% Another major factor to be taken into account is
worse than the regression line of least-squares fit. the role of the COD in matching. Many studies
Nevertheless, an important point in assessing the (Brownstein 8c Pliskoff, 1968; Herrnstein, 1961; Shull
matching formulation is whether the deviations from 8c Pliskoff, 1967) have shown that a minimum COD
matching are systematic. For time ratios (Table 2) duration is necessary for matching to be obtained, but
there is no systematic deviation. The individual re¬
3 Undermatching in terms of a slope <1.0 does not neces¬
gression lines vary equally on both sides of matching, sarily mean that the subject is nearer indifference between the
with a median slope of .98. For response ratios (Table schedules than is predicted by the matching relation. A subject
1) the slopes of the individual regression lines do tend could actually respond more on the preferred key than matching
predicts, yet still produce a regression line of slope <1.0. For
toward undermatching, with a median slope of .80. example, if relative reinforcement frequencies of .50, .60, .70, and
.80 were studied (i.e., if the same key always provided the more
2 In calculating the group regression line for Baum’s (1975) frequent reinforcement), relative response rates of .58, .66, .74,
data, the results from John I rather than John II were used, and .82 would produce a regression line with a slope flatter than
since the experimental parameters were the same for John I and 1.0. This was the case for the two pigeons that produced the
the other two subjects. flattest functions in the study by Silberberg and Fantino (1970).
 Peter de Villiers 243

there is no particular reason to believe that the same matching on a minimum COD duration severely
minimum COD value will suffice for all subjects. Just limits the generality of the principle. This would fol¬
as there are individual and species differences in many low if matching occurred only at certain arbitrary
discrimination tasks, it is likely that different COD COD values. In fact, although different minimum
values will be needed for individual subjects before COD values may be required for different species and
they properly discriminate the two reinforcement even for different individual subjects, matching is
schedules in a concurrent procedure. For example, a found for all values of the COD greater than this
considerably longer minimum COD of 5 to 10 sec is minimum (Allison 8c Lloyd, 1971; Shull 8c Pliskoff,
apparently needed for rats (de Villiers 8c Millenson, 1967; Stubbs 8c Pliskoff, 1969). The matching relation
1972; Shull 8c Pliskoff, 1967) than for pigeons (about 1 is therefore not an artifact of any particular COD
to 3 sec—Catania, 1966; Herrnstein, 1961). Brownstein duration, although it requires a minimum separation
and Pliskoff (1968) reported that different COD values of the two schedules in time.
(between 2 and 7.5 sec) were needed to obtain match¬ On the other hand, Pliskoff (1971) has argued that
ing for each of their three pigeons, and Schroeder and response or time distributions may themselves be by¬
Holland (1969) needed a longer COD for one of their products of changeover (CO) responding, which de¬
human subjects before matching was observed. pends on both the COD duration and the relative
At all COD durations less than the minimum re¬ reinforcement frequency. With a fixed COD duration,
quired for matching the subjects are nearer indiffer¬ CO rate decreases as the relative reinforcement fre¬
ence between the two schedules and show a flatter quency diverges from .50; and with a fixed relative
function relating response and reinforcement ratios reinforcement frequency, CO rate decreases as the
than that predicted by the matching relation—i.e., COD increases (Shull 8c Pliskoff, 1967; Stubbs 8c
undermatching. Yet most studies have programmed Pliskoff, 1969). But the results of Stubbs and Pliskoff s
the same short COD for all subjects, usually between (1969) experiment do not support Pliskoff’s conclusion
1 and 3 sec in duration. Therefore, it is not surprising that matching is determined by CO responding. They
that in most studies some subjects show regression fixed the relative reinforcement frequency at .75, and
lines with slopes less than 1.0. It is worth noting that although overall CO rate declined systematically with
only one of the studies shown in Tables 1 and 2 increasing COD duration, relative response rate and
(Brownstein 8c Pliskoff, 1968) varied the COD duration time allocation did not change. Matching was ob¬
for individual subjects until matching was obtained tained at all COD values from 2 through 32 sec. This
at the first relative reinforcement frequency. The only suggests that relative reinforcement frequency is the
subject in that experiment for which three different crucial variable in determining response and time
relative reinforcement rates (besides 1.0) were studied matching.
produced a regression line relating time and rein¬ Nevertheless, the role of the minimum COD values
forcement ratios with a slope of 1.04. The regression required for matching in concurrent schedules has
line for the group of three subjects, including all the still to be clarified. Catania (1966) and Herrnstein
data points, was Y = MX + .02 (with 97.5% of the (1961, 1970) suggest that the COD separates the two
data variance accounted for)—close to perfect schedules in time and so reduces the adventitious
matching. reinforcement of left-right or right-left response se¬
The matching relation, therefore, holds only under quences. As mentioned earlier, Catania and Cutts
certain conditions. A COD of sufficient duration must (1963) showed that without a COD, responses to one
be used for each subject, and the reinforcement sched¬ alternative came partially under the control of the
ules should be run in balanced order across the two other reinforcement schedule. One function of the
keys to obviate order effects. Many of the studies cited COD is therefore to prevent such concurrent super¬
in Tables 1 and 2 did not satisfy these conditions. stitions by introducing a delay between a response to
Since most of the known factors that lead to systematic one alternative and the reinforcement from the other.
deviations from matching lead to undermatching Pliskoff (1971) has suggested that the COD func¬
(Baum, 1974a), this outcome will be obtained in ex¬ tions to punish the CO response since it specifies a
periments that fail to control for them. period of time during which no response will be rein¬
forced if the subject changes over. By thus decreasing
The Role of the COD in Matching CO rate, the COD separates the two schedules. Delays
of increasingly longer duration produce larger decre¬
The importance of the COD in the matching rela¬ ments in CO rate, suggesting that increasing the COD
tion raises questions about the generality of matching. may be comparable to increasing shock intensity in a
Indeed, it could be argued that the dependency of punishment paradigm.
244 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

There are certain similarities between perfor¬ post-COD response rates. Relative response rates
mances on concurrent schedules with CODs and those within the COD approximated .50 (indifference)
in which the CO response is explicitly punished by a while post-COD response rates overmatched the over¬
shock or by a time-out from reinforcement. Todorov all frequency of reinforcement. Only when these two
(1971) demonstrated that CO rate decreased and rela¬ response rates were added did overall relative re¬
tive response rate increased as the intensity of shock sponse rate closely match relative reinforcement fre¬
or duration of time-out increased, as they do with in¬ quency. From these results, Silberberg and Fantino
creasing COD duration (Allison & Lloyd, 1971; Shull concluded that the matching relation depends on the
& Pliskoff, 1967). interaction of COD and post-COD response patterns
However, the parallel breaks down under further and on the perseverance of the COD response burst
scrutiny. Shull and Pliskoff (1967) observed that at into the post-COD period on the preferred key. Since
all COD durations, relative response rates and time the probability of reinforcement on one VI schedule
distributions continued to match the relative obtained increases the longer the subject spends responding on
reinforcement frequencies, and the local response rates the other schedule, they suggested that the high re¬
tended to be the same on both schedules. On the other sponse rates during the COD on both keys reflect the
hand, Todorov (1971) found that with increasing increased local probability of reinforcement imme¬
punishment of the CO response, the relative rate of diately after a CO.
responding increased more rapidly than the relative Pliskoff (1971) similarly observed that with equal
time distribution, and the local rates of responding concurrent VI schedules, the response rate during the
deviated more and more from equality. Relative rein¬ COD was higher than the post-COD response rate at
forcement frequency did not change significantly as all COD durations between .33 and 27 sec. But
punishment increased, so that neither response nor Pliskoff found that response rate during the COD was
time matching were obtained at time-out durations highest with a 1-sec COD and then declined as the
longer than 3 sec or at shock intensities higher than COD increased, whereas the post-COD response rate
4 mA. was fairly constant at all COD Values. Unfortunately,
In another procedure, Stubbs and Pliskoff (1969) the same analysis has not been performed when rela¬
programmed a fixed-ratio (FR) requirement of 20 tive reinforcement frequencies varied, but it suggests
responses on the CO key while the main key was that the matching relation is not an artifact of the
darkened and the VI programmers stopped—i.e., the COD. The difference between COD and post-COD
FR functioned like Todorov’s time-out punishment. response rate decreases considerably as the COD
With the FR requirement in effect, relative response lengthens, so response bursts during the COD must
rate again overmatched the distribution of reinforce¬ contribute a different relative amount to the overall
ments and local response rates deviated from equality. rate of responding as the COD varies. Yet the match¬
These two experiments therefore fail to demonstrate ing of overall relative response rate to the relative fre¬
any functional equivalence between the COD and quency of reinforcement is maintained at all these
direct punishment of the CO response. COD durations (Shull & Pliskoff, 1967; Stubbs &
It is nevertheless questionable whether these pro¬ Pliskoff, 1969).
cedures constitute an adequate test of the hypothesis Other results suggest that the high response rates
that the COD punishes changeovers. Since the rein¬ generated during the COD are unnecessary for match¬
forcement schedules did not run during the timeout ing if the CO rate is low enough to separate the sched¬
or the FR contingent on the CO response, the ules in time. Kulli (unpublished data) showed that
punishers interacted with the VI schedules in a way pigeons’ CO rates systematically decreased with in¬
that the COD does not. Similarly, the shock used as creasing body weight. Even without a COD, relative
a punisher by Todorov could reduce the relative response rates matched relative reinforcement fre¬
value of the food reinforcement (de Villiers 8c Millen- quency when the birds were at 100 to 110% of their
son, 1972; Millenson 8c de Villiers, 1972). PliskofFs preexperiment ad libitum weights.
(1971) punishment hypothesis therefore remains as a In brief, there is substantial evidence that a certain
possible explanation of the COD’s effects on respond¬ amount of separation and differentiation between the
ing. two reinforcement conditions is a necessary condition
Silberberg and Fantino (1970) proposed that the for matching in concurrent schedules and that this
COD has a more complex role in the matching rela¬ separation can be produced by several methods that
tion than simply separating the two schedules in time. reduce CO rate. However, any role played by the
They reported that response rates during the COD COD beyond merely producing this separation is not
period following a CO were considerably higher than clear (cf. Kulli’s data).
 Pefer de Villiers 245

MAXIMIZING OR MATCHING

Shimp (1966, 1969a) has argued that matching is

not fundamental, but is produced by more molecular
interaction between choices and the probability of
reinforcement. He scheduled reinforcements probabil¬
istically for choices in a discrete-trial procedure
(Shimp, 1966). A contingency similar to that in a VI
schedule was employed in that a reinforcement pro¬
grammed for a particular choice remained available
until produced by the subject. The probability of
reinforcement on one key therefore increased while
the bird responded on the other key. Shimp found
that both the initial postreinforcement choices and
the sequential changes in choice probabilities between
reinforcements corresponded to the differences in
probability of reinforcement for each choice arranged
by the schedule. He concluded that the overall match¬
ing found in concurrent VI schedules is a by-product
of the subjects’ tendency to maximize—i.e., to choose
the alternative with the higher momentary probability
of reinforcement on each choice trial. TRIALS SINCE LAST CHOICE OF KEY
However, discrete-trial experiments by Nevin
Fig. 4. The two step functions plot changes in the relative
(1969) and Herrnstein (unpublished data), using con¬ momentary probability of reinforcement with the number of
current VI schedules, produced contrary results. Both trials since the last choice of each key in a discrete-trial con¬
Nevin and Herrnstein obtained matching, but sequen¬ current VI VI schedule. The open and closed circles plot the
relative number of responses on each key, averaged across two
tial changes in choice probabilities and postreinforce¬ pigeons, as a function of the number of trials since the last
ment choices did not correspond to the momentary choice of that key. The horizontal lines show the overall rela¬
probability of reinforcement on each key. Nevin did tive reinforcement rates on the two VI schedules. (Herrnstein,
unpublished data.)
find that choices were determined by changes in the
relative frequency of reinforcement within sequences
of trials between reinforcements, but as shown in forced response when given a choice between two in¬
Figure 4, Herrnstein found no relation between the compatible responses that differ in reinforcement.
pigeons’ choices and the changing relative probability Herrnstein and Loveland suggest that in a choice be¬
of reinforcement. Momentary maximizing is there¬ tween two VI schedules the animal adjusts its be¬
fore not necessary for matching in concurrent VI havior to equalize the ratio of responses to reinforce¬
schedules. ments on each alternative, which produces matching:
It is an empirical question whether matching can
be explained by some combination of more molecular
R2
processes, but as Herrnstein (1970) has pointed out,
there is no logical reason to assume that the matching Ti ~~ r2
relation must be explained at a molecular level.
Ri _ ri
Choice behavior in concurrent schedules may be more
R2 ^2
orderly at the level of the matching equation than
at the level of local response rate variations, sequences R1 ri
and
of choices, or relative changeover frequencies. Ri + R2 t1 -f- r2 (6)
Nevertheless, the matching relation can perhaps be
incorporated into a more general model of maximiz¬
ing payoff per response. Indeed, Herrnstein and Love¬ where Rt and R2 stand for responses to each of the
land (1975) have argued that an implicit assump¬ two alternatives and r1 and r2 stand for the corre¬
tion of maximizing is present in our thinking about sponding number of reinforcements. At any given
the interaction between reinforcement and behavior. moment, the animal performs that response which
It is an integral part of our conception of reinforce¬ seems to have the most favorable response-to-rein-
ment that an animal engages in the more highly rein¬ forcement ratio. By responding to that alternative it
246 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

drives its response-to-reinforcement ratio up toward cause these responses tend to be emitted in bursts of
that for the other alternative. When the other sched¬ responses at a constant rate (Blough, 1963). The time
ule offers the more favorable ratio, the animal spent pecking then determines the number of pecks
switches to it. In choice between two unequal ratio emitted, and overall rate of responding over a session
schedules, where the experimenter fixes the response- varies with the duration of pauses between the bursts.
to-reinforcement ratios, this maximization implies The time spent in each component of a CO-key
exclusive preference for the alternative with the concurrent schedule, or cumulated interchangeover
smallest ratio of responses to reinforcements. In fact, time in a two-key procedure, cannot directly measure
Herrnstein (1958) and Herrnstein and Loveland (1975) the time spent responding as Baum and Rachlin de¬
have shown that on concurrent FR FR or VR VR fine it, because it will always include time spent at
schedules, pigeons will respond exclusively to the other activities—e.g., grooming. These measures of
alternative with the shortest radio requirement, pro¬ time distribution between the two schedules will only
vided that the difference between the two ratios is be directly proportional to time spent responding if
larger than some minimum value. the proportions of time spent in other activities in
This analysis of the role of maximizing in the each component were invariant across experimental
matching relation leads to an interesting prediction conditions. Baum and Rachlin (1969) therefore sug¬
for performance in concurrent VI VR schedules. Ac¬ gest that the relative number of responses may pro¬
cording to the matching relation, an animal’s re¬ vide the best measure of the relative time spent re¬
sponse-to-reinforcement ratio on the VI schedule sponding. As long as the time required for a response
should come to equal the response-to-reinforcement remains fairly constant, the number of responses will
ratio on the ratio schedule (see Equation 6). But be directly proportional to time spent responding.
should the VI schedule be short enough so that the But it then becomes difficult, if not impossible, to
obtained reinforcements per response for the VI sched¬ distinguish empirically between response matching
ule rise above the value fixed for the ratio schedule, and matching in terms of time spent responding.
responding to the ratio alternative should cease, since While there are choice situations in which an analysis
it no longer can offer the most favorable ratio of re¬ in terms of number of responses would be arbitrary
sponses to reinforcements. Herrnstein (1970, 1971) (Baum 8c Rachlin, 1969; Brownstein 8c Pliskoff, 1968),
showed that pigeons do match response ratios to rein¬ there are also situations in which an analysis in terms
forcement ratios on VI VR schedules over a wide of time would be arbitrary. Nevin (1969) and Herrn¬
range of schedule values. But once the VI was rich stein (unpublished data) both found response match¬
enough or the VR high enough so that the same rate ing in discrete-trial procedures in which the pigeon
of reinforcement could be obtained for fewer re¬ had only occasional pecks at the key.
sponses on the VI, responding on the VR tended to On the other hand, Rachlin (1973) argues that by
cease; that is, relative response rate drifted toward allocating time to each schedule in a concurrent sched¬
exclusive preference for the VI schedule. ule procedure, the animal equalizes the local fre¬
quency of reinforcement for each alternative. Since
local response rates are also equal on concurrent VI
TIME MATCHING AS THE FUNDAMENTAL VI schedules (Catania, 1966; Killeen, 1972b; Shull 8c
MATCHING PROCESS Pliskoff, 1967), overall response matching results from
the matching of time allocation. But time allocation
Brownstein and Pliskoff (1968), Baum and Rachlin in the sense of equalizing local reinforcement fre¬
(1969), and more recently Rachlin (1973) have all quencies refers to the total time spent in each com¬
argued that matching of relative time allocation to the ponent (or to cumulated interchangeover time), not
relative frequency of reinforcement is more basic than to time spent responding as defined by Baum and
response matching in concurrent schedules. They Rachlin (1969). Local response and reinforcement
demonstrated time matching in situations in which rates are calculated in terms of the time spent in each
there were no response requirements apart from the component, and the evidence cited by Rachlin (1973)
allocation of time to the particular stimuli associated in support of his theory shows time matching in this
with each schedule. Baum and Rachlin (1969) pro¬ sense (Catania, 1966; Killeen, 1972b; Shull 8c Pliskoff,
posed that even a series of repetitions of discrete re¬ 1967; Silberberg 8c Fantino, 1970).
sponses such as key pecks or lever presses can be But Herrnstein (1970, 1971) reported that pigeons
thought of as periods of time spent engaging in a con¬ matched relative response rates, but not the relative
tinuous activity (key pecking or lever pressing), be¬ cumulated interchangeover time, to the relative rein-
 Peter de Villiers 247

forcement frequency in a two-key concurrent VI VR

schedule. In this situation, rate of reinforcement is
essentially independent of rate of responding on the
VI but is directly proportional to response rate on
the VR.4 Since local rates of pecking on VR schedules
are faster than those on VI schedules for the same fre¬
quency of reinforcement, matching cannot hold for
both relative rate of responding and relative cumu¬
lated interchangeover time. For each of the 10 pigeons
studied by Herrnstein, matching described relative
rate of responding over a wide range of different VI
and VR values, although local response rates were
faster on the VR.
Herrnstein and Loveland (unpublished data) re¬
cently obtained similiar results in a CO-key concur¬
rent VI VR procedure. The data are analyzed here in
terms of Equations 4 and 5 (proportional ratio match¬
ing). All four pigeons in the study matched response
ratios to obtained reinforcement ratios, but with a
proportional bias toward the VR schedule; i.e., the
pigeons responded a constant proportional amount
more on the VR at all reinforcement ratios (except
(0 or oo). Figure 5 plots the logarithm of the response
and time ratios against the logarithm of reinforce¬
ment ratios for each pigeon. The regression lines of
best fit to these data and percentage of the variance
accounted for are also shown. One pigeon matched
both time and responses, but for three other birds the
regression lines for time ratios are much flatter than
those for response ratios. However, since the range of
reinforcement ratios studied was narrow, the group
Fig. 5. The relation between log response ratio and log rein¬
data should also be examined, i.e., all the data points
forcement ratio, and between log time ratio and log reinforce¬
provided by the four pigeons. The least-squares regres¬ ment ratio, for four pigeons responding on CO-key concurrent
sion lines for these data are 1.03X — 0.15 for response VI VR schedules. The heavy diagonals represent unbiased
matching; the fine diagonals represent the regression lines fitted
ratios and 1.0 IX + 0.09 for time ratios. Thus while
to the data by the method of least-squares. The percentage of
the individual pigeons showed better response match¬ the data variance accounted for by each regression line is also
ing, the slopes for the group data functions were close given. (Herrnstein & Loveland, unpublished data.1)
to 1.0 for both responses and time.
La Bounty and Reynolds (1973) studied pigeons in time to relative reinforcement. They conclude that
a two-key concurrent FI FR schedule. This schedule time matching is not more basic than response match¬
has similiar properties to the VI VR schedule; local ing, since the latter can hold without the former.
response rate is higher on the FR than the FI, and However, a different picture emerges when their data
reinforcement rate on the FR is directly proportional are reanalyzed in terms of proportional ratio match¬
to response rate. The experimenters report approx¬ ing, which accounts for bias toward one schedule.
imate matching between relative response rate and Time ratios in fact fit proportional ratio matching
relative reinforcement frequency for four of six better than response ratios for four of the six pigeons.
pigeons, but state that no pigeons matched relative The slopes of the regression lines relating time
ratios and reinforcement ratios for five of the pigeons
4 It should be noted that relative reinforcement frequency in
a concurrent VI VR schedule is not strictly an independent were .88, .88, .99, .81, and .66; for response ratios they
variable since reinforcement rate on the VR schedule is de¬ were .84, .79, .89, .75, and .86 for the same pigeons. All
pendent on response rate on the VR. But this is often the case the pigeons showed a bias in time ratios toward the
with short VI schedules as well, and the matching relation holds
between relative reinforcement rate and obtained, not scheduled, FI schedule, but the same bias was not found for re¬
relative reinforcement frequency. sponse ratios. The sixth pigeon was either indifferent
248 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

between the two schedules or else showed exclusive entation by pecking at a CO key. Different COD
preference for the FR key. Thus in this study response durations (2, 5, and 7 sec) were scheduled for each
matching was no better than time matching; both bird. The slopes of the regression lines relating time
response and time distributions tended to somewhat ratios to reinforcement duration ratios were .70, 1.20,
undermatch the reinforcement distribution. A possible and .95 for the three pigeons. The pigeon for which
explanation of the undermatching found in concur¬ the shortest COD was programmed (2 sec) was under¬
rent FI FR schedules will be considered later in the matching, and the pigeon with the 5-sec COD was
chapter (under the heading “Applications of Herrn- overmatching. The regression line for the group data
stein’s Equations to Other Schedules”). was .95X -f- .02. De Villiers and Millenson (1972) also
To summarize, both response and time matching reported response matching to relative duration of
are found in concurrent schedules, and the best gen¬ condensed milk reinforcement for three rats, but they
eral conclusion at present is that the distribution of only examined one relative duration (.75).
reinforcement in a concurrent schedule governs the In a two-key concurrent schedule, Fantino, Squires,
distribution of behavior. Sometimes behavior is best Delbruck, and Peterson (1972) varied the overall fre¬
measured in terms of the time allocated to each sched¬ quency of grain reinforcements while keeping the rela¬
ule, sometimes by the rate of responding. In some tive duration constant at .80 (1.5 and 6 sec). Relative
situations only one measure is appropriate (e.g., Baum response rates and the relative total reinforcer time on
8c Rachlin, 1969; Brownstein 8c Pliskoff, 1968; Nevin, the 6-sec key increased as reinforcement became more
1969), but why one measure sometimes works better frequent, because the pigeons spent more time re¬
than the other when both are available (Hollard 8c sponding on that key. Fantino et al. analyzed their
Davison, 1971) remains to be determined. data in terms of relative response rates and reported
that the pigeons failed to match either relative hopper
duration or relative total reinforcer time (hopper
THE GENERALITY OF THE MATCHING duration times the number of reinforcements on that
RELATION key). But if the data are reanalyzed in terms of pro¬
portional ratio matching (Equation 5; repeated here
As Catania (1966) indicated in the first substantial for the reader’s convenience).
review of choice in concurrent schedules, the signif¬
icance of the matching relation depends on the range
of conditions over which it occurs. We have already
considered a substantial amount of evidence for
matching to relative frequency of reinforcement in which allows for bias toward one key, then matching
concurrent VI schedules, but does the matching rela¬ is found (Baum, 1974a). A regression line of slope 1.08
tion hold for other reinforcement parameters (e.g., and a A: of 2.4 accurately describe the ratio data. The
magnitude, immediacy, or quality of reinforcement); pigeons of Fantino et al. were therefore matching
for different schedules besides the concurrent VI VI; total reinforcer time, but with a constant proportional
or for negative reinforcement? bias toward the 1.5-sec key. Such a systematic bias at
all frequencies of reinforcement could arise if the time
spent eating on the 6-sec reinforcement was in fact less
Magnitude of Reinforcement
than 4 times the 1.5-sec reinforcement.
The data on reinforcement magnitude are equiv¬ The above results support response or time match¬
ocal, both matching and undermatching being re¬ ing to reinforcement duration, but several other
ported in studies of concurrent schedules. Catania studies have failed to find matching to either relative
(1963a) found that two of his three pigeons matched duration or total reinforcer time. Walker, Schnelle,
their distribution of responses to the relative dura¬ and Hurwitz (1970), using rats as subjects, varied the
tions of grain reinforcement provided by two equal duration of access to sucrose solution and reported
VI schedules. But he examined only one relative rein¬ only poor matching between relative response rates
forcement duration besides equality. Brownstein and relative reinforcer durations. In a subsequent
(1971) investigated choice between two pairs of un¬ experiment, Walker and Hurwitz (1971) also varied
equal reinforcement durations in addition to equality the access to sucrose, but scheduled reinforcements
in the time allocation procedure used by Brownstein by a single VI schedule (Stubbs 8c Pliskoff, 1969), there¬
and Pliskoff (1968). Three pigeons chose between two by insuring that reinforcement frequency did not co¬
equal VI schedules of response-independent grain pres¬ vary with reinforcement duration. They, too, found
Peter de Villiers 249

that preference for an alternative was less extreme from pigeons in a wavelength discrimination with dif¬
than would be predicted by matching to the relative ferent hopper durations unless the two durations were
duration of reinforcement. However, in both these signaled by hopper lights of different intensity. The
studies the COD was only 2 or 3 sec, too short for peak shift is readily obtained when reinforcement fre¬
matching in concurrent schedules with rats. Moreover, quency varies in a discrimination procedure (Dysart,
the COD was programmed from the last response on a Marx, McLean, 8c Nelson, 1974; Guttman, 1959). Thus
lever before a CO (Findley, 1958), making it func¬ pigeons might match relative reinforcement durations
tionally even shorter than the usual COD (Catania, more closely if differential signals were provided for
1966). Shull and Pliskoff (1967) and de Villiers and the hopper durations.
Millenson (1972) found that CODs longer than 5 sec Schneider (1973) avoided a difficulty common to all
were necessary before rats matched either frequency these studies—that of determining whether the actual
or duration of reinforcement. relative quantity of reinforcer obtained by the subjects
Todorov (1973) studied concurrent schedules in equals that scheduled by the timers for hopper dura¬
which frequency and duration of reinforcement co¬ tion. He varied the number of food pellets provided
varied. The pigeons in his experiment considerably during each reinforcement in a two-key concurrent
undermatched the relative total reinforcer time, and schedule. A single VI schedule arranged the rein¬
frequency of reinforcement affected choice more than forcements, which were assigned with an equal prob¬
did duration. But this experiment used an extremely ability to each key. A 1.5-sec COD operated through¬
complex procedure. Three different pairs of VI sched¬ out the experiment. Schneider noted that a fixed
ules were programmed on the main key in randomized quantity of reinforcement on a key maintained more
combinations, each pair in operation for 20 of the 60 rapid responding when it was delivered frequently in
reinforcements per session. The durations of rein¬ small amounts than when it was delivered infre¬
forcement for the three different VI schedules were quently in large amounts. When reinforcement
then varied in blocks of sessions. The procedure was frequency remained constant the pigeons under¬
therefore a multiple concurrent schedule, and it is matched the relative quantity of reinforcement at
difficult to assess the reinforcement interactions that three different pellet ratios (1:1, 1:3, and 1:7). Slopes
such a schedule might produce. (Interactions in of the regression lines relating log response ratios to
multiple schedules are discussed later in the chapter). log pellet ratios were .19, .41, .43, and .58 for the four
Furthermore, Todorov used a 1-sec time-out con¬ pigeons. However, when the number of pellets per
tingent on a CO response in place of a COD. Whether reinforcement remained constant, undermatching was
the time-out is functionally equivalent to a COD also obtained for the same ratios of reinforcement
(see p. 244) or whether it separated the schedules frequency (slopes of .35, .46, .64, and .63). In view of
sufficiently is uncertain. The flat function relating the excellent matching reported by Stubbs and
relative response rate to relative total reinforcer time Pliskoff (1969) for frequency using the same sched¬
observed by Todorov is typical of the function found uling procedure and a 2-sec COD, Schneider’s result
in the absence of a COD, so it is possible that the is surprising. The 1.5-sec COD was possibly too short
pigeons never fully distinguished the three different for his pigeons, and it is unfortunate that he did not
component schedules. However, Todorov did find that also study choice at longer COD durations.
frequency of reinforcement had a greater effect on Unpublished data from an experiment by de Vil¬
relative response rate than did reinforcement dura¬ liers and Balboni suggest another reason why Schnei¬
tion. This could represent either a problem of dis¬ der did not find matching to relative quantity of food.
crimination or a fundamental difference in the rela¬ De Villiers and Balboni studied two rats responding
tionship between response strength and various on a two-lever concurrent VI schedule for different
reinforcement parameters. Mariner and Thomas (1969) numbers of food pellets. A single VI 30-sec schedule
argued that pigeons have difficulty discriminating be¬ arranged reinforcements which were assigned with
tween different hopper durations because they cannot equal probability (.50) to each lever. With one pellet
tell that they are in the longer duration until the time of food for responding on the left lever and five pellets
of the shorter duration has passed. Mariner and on the right, the COD was systematically increased
Thomas failed to obtain peak shift5 (Hanson, 1959) from 5.5 through 15 sec. Relative response rate on the
5 The peak shift is a displacement of the peak of a post¬ five-pellet lever increased between 5.5-sec and 7.5-sec
discrimination generalization gradient away from the stimulus COD duration but declined with further lengthening
associated with the higher frequency of reinforcement (S+) in
the direction away from the stimulus associated with extinction of the COD. Changeover rate decreased with increas¬
or the lower frequency of reinforcement (S-). ing COD length. As the lower panel of Figure 6 indi-
250 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

three pellets for responding on each lever, the prob¬

ability of reinforcement assignment to each lever was
varied through the sequence: .50:.50, .15:.85, and
.75:.25. Regression lines fitted to the mean data from
the two rats were .89X — .01 (responses) and .94X —
CD
o .09 (time): The slopes of the individual functions were
.91 and .86 for responses, and .77 and 1.11 for time
ratios. One rat therefore matched responses and the
other rat approximately matched time allocation to
the ratios of the obtained frequencies of reinforce¬
LOG
Nl
ment.
Nr
The results suggest that matching to relative mag¬
COD DURATION (SEC)
55 75 10 125 15 75
COO DURATION (SEC)
nitude of reinforcement does not hold in the Stubbs
55 75 10 125 15 75

juu[ -
-
-
and Pliskoff (1969) single-tape scheduling procedure
regardless of COD length and changeover rate. The
single-tape procedure forces the animal to respond for
R4 R5
some time on the alternative with the smaller rein¬
forcer since half the reinforcement opportunities are
Fig. 6. (Top:) The mean log response ratios (left panel) and
mean log time ratios (right panel) of two rats responding on a scheduled for that side. Reinforcements assigned to
single-tape concurrent VI schedule plotted as a function of log that alternative must be obtained before the VI tape
pellet ratio ATL/NR. The open symbols represent data from the
starts timing intervals again for either alternative. In¬
condition in which relative frequency of reinforcement was kept
constant at .50 and the relative number of food pellets for each creasing the COD reduces changeovers and separates
alternative varied; the filled symbols represent the condition the two alternatives in time, but it forces the animal
in which number of pellets per reinforcement was the same
to spend even longer responding on the alternative
for both levers and relative frequency of reinforcement varied.
The least-squares regression lines and the percentage of the with the small reinforcer in order to restart the tape.
data variance that they account for are given in the upper Since the Stubbs and Pliskoff procedure, which varies
lefthand corner of each panel for frequency and the lower
relative frequency of reinforcement, does not force the
right-hand corner for number of pellets. (Bottom:) The bar
graphs show the deviation from matching to relative pellet animal toward the less frequently reinforced alterna¬
number (.83) (relative response rate minus relative number of tive to the same degree, matching is found at suitably
pellets) for each rat as the COD duration varied. (From de
Villiers 8c Balboni, unpublished data.)
long COD durations.
A recent experiment on magnitude of reinforce¬
ment by Iglauer and Woods (1974) used indepen¬
cates, the smallest deviation from matching to the dently programmed VI 1 min schedules, thus avoiding
relative pellet number of .83 occurred at the 7.5-SCC the constraints on responding inherent in the single¬
COD, and matching to relative pellet number was not tape procedure. They varied the dosage of intravenous
observed at any COD value. cocaine injections available on each schedule, a
In the second condition of the experiment the parameter of reinforcement magnitude more imme¬
COD was set at 7.5 sec and the number of food pellets diately discriminable than duration. Drug dosage was
for responding on each lever varied through the fol¬ manipulated by varying the volume of a constant-
lowing sequence: 1:5, 3:3, 4:2, and 5:1. The VI 30-sec concentration cocaine solution injected over a con¬
schedule continued to assign reinforcements with an stant time period. Reinforcers therefore differed in
equal probability to the two levers. The open symbols volume but not in concentration or duration. The full
in the upper panels of Figure 6 show the mean log procedure was a modification of the standard two-
response ratios and mean log time ratios (cumulated lever concurrent schedule in that a single response on
interchangeover time) of the two rats plotted against a center lever initiated the concurrent schedule on
the mean log pellet ratios. The least-squares regression two adjacent levers. Responding on one lever pro¬
lines fitted to the data were .45X + .03 (responses) duced a constant dose of .1-mg/kg/injection, while the
and .53X — .26 (time). Slopes of the individual func¬ dosage of cocaine associated with the other lever
tions were .52 and .38 for responses, .62 and .46 for varied from .025 to .4 mg/kg/injection. During rein¬
time ratios. forcement, one of two pumps injected the cocaine
The upper panels of Figure 6 also give the mean solution for 35 sec, followed by a 5-min blackout of
response and time ratio data for the third condition of the chamber for the drug to take effect. At the end of
the experiment. With the same COD duration and the blackout a single response on the center lever
Peter c/e Villiers 251

again initiated the choice procedure. A 1.5-sec COD taken as the actual delay interval for what was nom¬
was in effect during the concurrent schedule com¬ inally immediate reinforcement, Chung’s pigeons were
ponent. actually matching the relative reciprocal of delay
Since relative reinforcement frequency covaries to on each key. The constant represents the time taken
some extent with relative response rate in the two- for the pigeon to lower its head to the feeder and
tape procedure, Iglauer and Woods calculated the begin eating.
relative drug intake on the two levers. Drug intake On the other hand, Shimp (1969b, Experiment II)
represents the number of reinforcements received on a found that if pigeons in a similar choice procedure
lever multiplied by the drug dose available on it. For had to peck after the delay blackout to obtain rein¬
both monkeys on the concurrent schedule, relative forcement, two of the three birds still matched relative
response rates matched relative drug intake over a response rates to the relative reciprocal of blackout
wide range of values. Regression lines relating log duration. Since there was now no time interval im¬
response ratio to log drug intake ratio (excluding posed between the last peck and food presentation,
points of exclusive preference) were 1.08X — .06 and Shimp argued that the blackout and not the delay of
1.11X + .01 for the two monkeys. reinforcement was crucial in Chung and Herrnstein’s
Two other monkeys responded in a concurrent experiment. Shimp’s procedure is much like requiring
chain schedule procedure (Autor, 1969; Herrnstein, a lengthy prereinforcement interresponse time (IRT)
1964), in which responding during an initial two- stipulated by the blackout duration. Shimp (1969b,
lever concurrent VI VI schedule (choice link) led to Experiment I) and Moffitt and Shimp (1971) demon¬
one of two equal-valued single-lever FR schedules for strated that in concurrent VI schedules in which
cocaine (terminal links). These monkeys matched rela¬ different IRTs are required for reinforcement on the
tive response rates during the choice link to their two schedules, pigeons match response rates to the
relative drug intake in the terminal links. relative reciprocal of IRT duration.
On a whole, these studies suggest that frequency of But experiments by Herbert (1970) do not support
reinforcement may have a greater effect than magni¬ Shimp’s interpretation of Chung and Herrnstein’s re¬
tude on choice in concurrent schedules with short sults. Herbert used a single VI schedule and assigned
CODs, but the matching relation applies to total rein¬ the reinforcements with an equal probability to the
forcement received (amount times frequency) when two keys. In the first experiment, reinforcement for
appropriate concurrent schedule procedures are em¬ each key was delayed by a blackout, as in Chung and
ployed. Herrnstein’s study. Two relative delay values besides
equality were examined. The slopes of regression lines
Immediacy of Reinforcement relating ratios of responses to ratios of immediacy of
reinforcement were .86, .73, and .86 for the three
The application of the matching relation was ex¬ pigeons. The short 1-sec COD or the single-tape pro¬
tended to immediacy of reinforcement (1/delay) by cedure (see p. 249) could account for the under¬
Chung and Herrnstein (1967). For four pigeons, VI matching observed, especially for one bird. In a
reinforcement on one key (the standard key) was de¬ second experiment with the same pigeons, Herbert
layed for 8 sec, while on the other key (the experi¬ repeated Shimp’s procedure requiring a response be¬
mental key) the delay of reinforcement on an equal VI fore food presentation after the blackouts. Only one
schedule varied from 1 to 30 sec. For two other of the three pigeons continued to approximate match¬
pigeons the standard key delay was 16 sec. During the ing to the relative reciprocal of blackout duration; the
delay of reinforcement the experimental chamber was other two were much closer to indifference at three
blacked out. The slopes of the least-squares regression different relative blackout durations. Matching was
lines relating ratios of responses to ratios of imme¬ therefore considerably impaired by the added response
diacy of reinforcement on the two keys were .92 for requirement in this study.
the 8-sec standard delay group and 1.05 for the 16-sec In Herbert’s third experiment reinforcement on
group. The 16-sec pigeons showed a consistent bias one key was immediate, whereas it followed a black¬
toward the experimental key, but the slopes of both out delay on the other key (as in Chung’s 1965 pro¬
functions were close to perfect matching. In an earlier cedure). But an equal number and duration of re¬
experiment, Chung (1965) had studied choice between sponse-contingent blackouts were also programmed at
immediate and delayed reinforcement in equal con¬ variable intervals on the immediate key, indepen¬
current VI schedules. Chung and Herrnstein (1967) dently of the reinforcement. Under these conditions,
demonstrated that if a small constant (1.6 sec) was no pigeon matched the relative immediacy of rein-
252 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

forcement when a small constant was taken as the stein’s results remains uncertain. (See Fantino, chapter
actual delay on the immediate key. When the relative 11 in this volume.)
frequency of reinforcement was fixed at .50, relative
response rate on the key with immediate reinforce¬
ment increased as a linear function of increasing de¬ Qualitatively Different Reinforcers
lay on the other key as opposed to the exponential
function obtained by Chung (1965). On the basis of In all the above experiments the physical dimen¬
these results, Herbert questions Chung and Herrn¬ sions of a given reinforcer were varied. But what if
stein’s interpretation of their findings in terms of the reinforcers differ in quality? In this case the sub¬
matching to relative immediacy of reinforcement. ject may prefer one of the reinforcers over the other,
However, the pigeons in Herbert’s third experiment even at equal frequencies of reinforcement. Hollard
did not choose between immediate and delayed rein¬ and Davison (1971) studied pigeons under two-key
forcement as was suggested; they actually chose be¬ concurrent VI schedules with food as the reinforcer
tween immediate reinforcement plus punishment by on one key and ectostriatal brain stimulation as the
response-contingent time-out from food (blackout) reinforcer on the other. A single VI schedule assigned
and delayed reinforcement. The relation between this reinforcements according to different proportions to
procedure and choice between two different delays the two keys (Stubbs 8c Pliskoff, 1969). The brain
of reinforcement (Chung 8c Herrnstein, 1967; Herbert, stimulation parameters were kept constant while the
1970, Experiment I), for which approximate match¬ frequency of food reinforcement was varied. Although
ing is found, is unclear. The effects of punishment on all the pigeons showed a constant proportional prefer¬
choice in concurrent schedules is considered later in ence for the food at all frequencies of reinforcement,
the chapter. proportional ratio matching accurately described the
Nevertheless, the possibility remains that the relation between ratios of time spent responding for
pigeon’s choice in Chung and Herrnstein’s experiment food or brain stimulation and ratios of the number of
was influenced more by rate of reinforcement than by reinforcements of each kind. Individual regression
the delay per se. Their procedure is formally similar lines relating the log of the time ratios to the log of
to a concurrent chain schedule in which responding the reinforcement ratios were: Y = 1.05X -f .74;
in an initial concurrent VI VI schedule leads to un¬ Y = 1.01X + .27; and Y = .98X + .78. On the other
equal-length terminal links (blackouts) ending with hand, all three pigeons produced regression lines re¬
noncontingent food—i.e., different fixed-time (FT) lating response ratios to reinforcement ratios with a
schedules. The shorter terminal link (i.e., the shorter flatter slope than that predicted by matching, the
delay period) has the higher rate of reinforcement, individual slopes being .79, .65, and .85. Biased time
and the relative frequency of reinforcement equals the matching but not response matching was therefore
relative reciprocal of blackout duration. Neuringer found for the different qualities of reinforcement.
(1969) demonstrated that pigeons are indifferent be¬ Brown and Herrnstein (1975) have pointed out that
tween FT and FI terminal links in a concurrent chain the matching relation might not hold where the
schedule. This suggests that relative response rates in different reinforcers interact, as food and water do
the choice link of a concurrent chain schedule should (Bolles, 1967). Since animals tend to drink following
show the same functional relation to relative terminal food consumption (Bolles, 1967), increasing the fre¬
link reinforcement frequency whether the terminal quency of food reinforcement might enhance the
links are FT schedules (as in Chung and Herrnstein’s value of a constant frequency of water reinforcement.
study) or FI schedules. But Duncan and Fantino On the other hand. Wood, Martinez, and Willis
(1970) found greater preference for the shorter of two (1975) found that increasing the FR requirement in a
FI terminal links than was predicted by matching to concurrent FI (food) FR (food or water) schedule had
the obtained relative frequency of reinforcement, al¬ different effects depending on whether the FR rein¬
though they investigated a range of FI durations forcement was food or water. When both reinforcers
similar to the delay duration employed by Chung and were food, increasing the FR also increased the
Herrnstein. Neuringer (1969) himself obtained under¬ animal’s responding on the FI; but when the FR
matching between relative response rates in the choice reinforcer was water, responding on the FI for food
link and the relative reciprocal of differing FI and FT was unaffected by changes in the FR. Systematic varia¬
terminal link durations. Thus the role of rate of rein¬ tion of the frequency of food and water reinforcers in
forcement as opposed to delay in Chung and Herrn¬ concurrent schedules is necessary to determine the
Peter de Villiers 253

form of the choice function and the nature of any in¬ both directions must be equal. Thus the less preferred
teraction between the reinforcers. choice receives disproportionately more punishment
than the more preferred and is more suppressed.
De Villiers and Millenson (1972) also studied the
Punishment and Choice in Concurrent Schedules
interaction between positive reinforcement and aver¬
Few studies have quantified the effects of punish¬ sive stimulation in a choice procedure. They superim¬
ment on choice in concurrent schedules. Holz (1968) posed a conditioned aversive stimulus (a conditioned
punished concurrent responses maintained by differ¬ suppression procedure—Estes and Skinner, 1941) on
ent frequencies of reinforcement in a two-key concur¬ a two-lever concurrent VI schedule with different
rent VI 1-min VI 4-min schedule. Each response on durations of reinforcement programmed for each re¬
the two keys was punished by a brief electric shock. sponse. They suggested that the aversiveness of the
As the shock intensity increased, response rate on both conditioned suppression procedure summated with
keys progressively decreased. Nevertheless, at all shock the reinforcement for the two responses, subtracting a
intensities, as long as the pigeons continued to re¬ constant value from each. If this were the case, the
spond, the proportion of responses to each key relative reinforcing value of the preferred schedule
matched the proportion of reinforcements obtained would increase,
from that key. In Holz’s procedure every response
was punished, so the proportion of punishments (D - c) rt
->- (7)
equaled both the proportion of responses and the (r-i — c) + (r2 - c) r1 + r2 w
proportion of reinforcements on the two keys as long
as the pigeon matched. If the shocks had been pro¬ where rx > r2 and r2 ^ 0, and so should the relative
grammed so that this proportionality no longer held— performance measures if matching to relative value
e.g., by equal VI or FI punishment schedules—some were retained. In fact, de Villiers and Millenson
interaction between the positive reinforcement and found increased preference for the lever associated
the punishment might be expected, and the simple with the bigger reinforcement during the preshock
matching relation might not hold. Azrin and Holz stimulus. A similar interaction between the food rein¬
(1966) reported that when only one of the concurrent forcement and the aversive value of the punishment
responses was punishment, the alternative response could account for the increased preference for the
increased in frequency while the punished response more favored key in the Todorov (1971) study.
was rapidly suppressed. However, no systematic study This interpretation of Todorov’s results receives
has yet quantified the interactions between punishing support from a recent experiment by de Villiers (un¬
and rewarding consequences of responding in a con¬ published data). Three pigeons responding on a two-
current schedule paradigm (cf. Rachlin 8c Herrnstein, key concurrent VI 1-min VI 3-min schedule for grain
1969). received intermittent punishment with brief electric
As discussed earlier, Todorov (1971) punished the shock for pecks at each key. Punishments were ar¬
CO response in a CO-key concurrent VI schedule with ranged by a single VI 15 sec schedule and assigned
electric shock or time-out from reinforcement. Rela¬ with an equal probability (.50) to each key, thus main¬
tive response rate and relative time allocation for the taining the relative frequency of punishment at .50. A
preferred schedule increased sharply as the shock in¬ 3-sec COD was programmed for food but not for
tensity was increased or the time-out duration length¬ punishment throughout the experiment. In the ab¬
ened, but the relative reinforcement frequency did sence of punishment the pigeons closely matched rela¬
not also increase. The relative response and time tive response rates and relative time allocation
measures therefore deviated more and more from (cumulated interchangeover time) to the relative fre¬
matching in the direction of the preferred schedule quency of reinforcement of .75. But with increasing
as the punishment increased. This deviation from intensity of punishment, the relative response and
matching can be explained by the differences between time distributions deviated more and more from
the Holz (1968) and Todorov procedures. In the matching toward the key with more frequent rein¬
simple continuous punishment procedures used by forcement. Overall response rate on the VI 3-min key
Holz, each choice is punished in proportion to the was much more suppressed by punishment than that
number of responses made to it. However, in on the VI 1-min key. Figure 7 depicts the results from
Todorov’s procedure, each choice receives the same each pigeon.
number of punishments per session, since the COs in Further systematic investigation of the effects of
254 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

on two different VI schedules by a 2-min time-out

64 65
67
from a train of shocks. There were sizable hysteresis
effects, but proportional ratio matching, Equation 4,
pi '

nn
—
provided a good fit to the data for two of the four
n _D_ _Q
L_Jt=r-
pigeons; i.e., those pigeons matched the ratio of time
>
UJ
2
CO
U
spent on each side of the chamber to the ratio of the
Q uq

frequencies of time-out provided on that side. The

data from the other two pigeons deviated from the
matching relation in opposite directions. For one
|—I
X +.1
U pigeon, a regression line of greater slope than 1.0
H

hlH
<
2 fitted the time and reinforcement ratios, but this
m n n pigeon died after only the ascending series of schedule
u values. The data could therefore be affected by the
-
strong order effects found in this experiment. The
0 3 4.5 6 0 0 3 4.5 6 7.5 0 0 3 4.5 6 0 second pigeon undermatched the ratios of time-out
PUNISHMENT INTENSITY (ma) frequencies, and Baum suggests that this may be due
to the short COD used in the study (1 sec). This bird
showed a constant high rate of COs at all schedule
values and also undermatched ratios of positive rein¬
forcement in Baum and Rachlin’s earlier study when
the COD was as long as 4.25 sec. But the equivalence
of COD values for positive and negative reinforce¬
ment in concurrent schedules remains to be demon¬
strated, so this explanation is tentative.
Despite the difficulties involved in working with
Fig. 7. Deviation of relative response rates (top panels) and aversive contingencies, Baum’s results suggest that sub¬
relative time distribution (center panels) of three pigeons from sequent research on concurrent schedules of negative
relative frequency of reinforcement (relative frequency of re¬ reinforcement could be extremely fruitful. The match¬
sponding or relative time minus relative frequency of rein¬
forcement) on a concurrent VI 1-min VI 3-min schedule as a ing relationship may provide a means of integrating
function of punishment intensity. Relative frequency of pun- both positive and negative reinforcement into the
ishment was the same on each key. Shown in the lower panels same conceptual framework (see also pp. 260 and
are the suppression ratios for each key, calculated in terms of
FU), where B is response rate on a key in a particular
270).
punishment condition and A is response rate on the same key
in the base line condition (no punishment). (From de Villiers,
Unpublished data.) Different Schedules of Reinforcement

Earlier in the chapter it was shown that the match¬

different punishment or conditioned suppression ing relation is readily extended to concurrent VI VR
parameters on a wider range of relative reinforcement schedules (Herrnstein, 1970, 1971) and to concurrent
frequencies is needed to determine the function that FI FR schedules (La Bounty & Reynolds, 1973),
relates the positive and negative consequences of though in the latter case the subjects tended to under¬
choice. The sensitivity of concurrent performances to match. In concurrent ratio schedules, on the other
both positive reinforcement and punishment param¬ hand, matching cannot occur except trivially, since a
eters (Catania, 1966; Holz, 1968) indicates that this ratio schedule fixes a proportionality between num¬
paradigm could be very useful for study of the inter¬ bers of responses and numbers of reinforcements. In
actions between positive and aversive control of fact, for most pairs of ratio values on concurrent VR
behavior. VR or FR FR schedules the subjects maximize rein¬
forcements per response—i.e., respond exclusively to
Negative Reinforcement the alternative with the shortest ratio requirement
(Herrnstein, 1964; Herrnstein & Loveland, 1975).
Baum (1973) used the same shuttlebox situation as The relation between maximizing and matching was
Baum and Rachlin (1969) and four of the pigeons discussed in an earlier section.
from that study in a concurrent VI escape procedure. What about choice between concurrent interval
Standing on either side of the chamber was reinforced schedules other than the concurrent VI VI? Both
Peter de Villiers 255

Nevin (1971) and Trevitt, Davison, and Williams power function in choice between VI and FI
(1972) investigated choice in two-key concurrent VI FI schedules.
schedules. Nevin reported that relative frequency of A study of choice in CO-key concurrent FI FI
responding on the FI key depended on the relative schedules (White 8c Davison, 1973) indicates that the
frequency of reinforcement on that schedule, but did pattern of responding is crucial in determining
not match it. Instead, the ratio of responses on the FI whether or not the matching relation holds in con¬
to responses on the VI was a power function of the current interval schedules. When typical VI perfor¬
ratio of reinforcements for each schedule, with an mance was found on both FI schedules—i.e., a fairly
exponent of approximately .50: constant response rate between reinforcements on
each schedule—matching between response ratios and
reinforcement ratios occurred. Similarly, when typical
FI performance was found on both schedules—i.e., a
postreinforcement pause followed by a rapidly accel¬
erating response rate on each schedule—matching was
Two of his pigeons tended to favor the VI key while also observed. However, when differing response pat¬
one favored the FI key. terns were generated by the two schedules, VI re¬
The power function is more general than the sponding on the shorter FI schedule and FI respond¬
matching relation described by proportional ratio ing on the longer schedule, a power function with an
matching (Equation 5). While proportional ratio exponent close to .50 (individual exponents ranging
matching specifies that the exponent shown in Equa¬ from .38 to .57) related the response and reinforce¬
tion 8 should be 1.0, the power function formulation ment ratios on the two schedules. This exponent is
allows the exponent to vary on both sides of 1.0. Thus the same as that found by Nevin (1971) for concurrent
it can account for both undermatching (slope <1.0) VI FI schedules.
and overmatching (slope >1.0) to the reinforcement
ratios. It should be noted that a suitable rescaling of
Choice Between Interresponse Times
the reinforcement variable—for example, by taking the
square root of reinforcement frequency in the case of The pattern of responding on concurrent schedules
Equation 8—would give proportional ratio matching. can also be altered by reinforcing only particular
The question of rescaling the reinforcement variable bands of interresponse times (IRTs). In experiments
to give proportional ratio matching is discussed in by Staddon (1968) and Shimp (1968, 1969b) two classes
more detail on pp. 275 ff. of IRTs on a single key were differentially reinforced.
Trevitt et al. (1972) also found a power function The pigeons therefore chose between emitting two
relation between the ratio of responses or time spent different IRTs on the same key, each associated with a
responding on each of the schedules and the ratio of different frequency of reinforcement. In Shimp’s
reinforcements. All of their birds showed a consistent experiments, but not in that of Staddon, the two
preference for the VI schedule. The exponents fitted IRT bands were signaled by discriminative stimuli on
to the Trevitt et al. data vary between .38 and .75 for the key.
response ratios, with an exponent of .62 for the aver¬ It is difficult to characterize the reinforced operant
aged data. Similar slopes were found for each pigeon in these paced schedules, since a large amount of col¬
in control VI VI conditions, but the preference for lateral behavior is generated by such schedules
the VI key was less marked. Comparing the VI FI and (Kramer 8c Rilling, 1970). Nevertheless, orderly rela¬
VI VI functions, Trevitt et al. therefore concluded tions were obtained between response rates and rein¬
that there was a constant proportional preference for forcement frequencies. Shimp (1968, 1969b) demon¬
the VI schedule in the VI FI choice. As discussed strated that the relative frequency of each IRT
earlier, however, the VI VI data from this study are approximately matched the relative reciprocal of its
somewhat equivocal because of the long preexposure length, and the relative rate at which each of the two
to the VI FI conditions during which the FI schedule IRTs was emitted was a monotonically increasing
was always associated with the same key; so the con¬ function of its relative frequency of reinforcement.
stant proportional preference for the VI may not be a Staddon (1968) divided all of the responses made by
general finding. his subjects into two component distributions under
Nevertheless, both studies indicate that matching the control of the short and long IRT contingencies,
of response or time ratios does not occur in concur¬ respectively. The pigeons’ allocation of responses to
rent VI FI schedules. These ratios are related by a each of these distributions (response ratios derived
 256 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

from the component distributions) was related to the therefore applies even in situations in which the pat¬
ratios of reinforcement of the two IRTs by a power tern of responding on each key is constrained by
function with an exponent of approximately .67 (a paced schedules, provided that the response constraints
range of .59 to .77 for individual birds). are the same on the two keys. The more the constraints
Moffitt and Shimp (1971) used a two-key concurrent deviate from equality, the more relative response rates
procedure in which one class of IRTs was reinforced or response ratios deviate from matching.
on one key and a second class of IRTs was reinforced Other experiments by Shimp (1970) and Hawkes
on the other key. A single VI programmer arranged and Shimp (1974) have established some of the bound¬
reinforcements for both keys, and these were assigned ary conditions for matching to the relative reciprocal
to each key according to different probabilities. In one of the reinforced IRTs. When two different signaled
experiment, the relative reinforcement frequencies IRT bands are reinforced equally frequently on a
were equal, but the lengths of the two reinforced single key, preference for the shorter of the IRTs in¬
IRTs varied. The relative rate of responding on a key creases from near indifference at very low overall rates
approximately equaled the relative reciprocal of the of reinforcement until it reaches an asymptote ap¬
length of the IRT reinforced on that key. In a second proximating the matching-to-relative-reciprocal value
experiment, the relative reinforcement frequency for between 20 and 30 reinforcements per hr (Shimp,
two particular IRTs, one on each key, was varied. 1970). Hawkes and Shimp (1974) concurrently rein¬
The reinforced IRTs for one group of pigeons were forced two signaled IRT bands on a single key with
the same as those used by Shimp (1968), 1.5 to 2.5 sec equal frequencies. They maintained the relative re¬
and 3.5 to 4.5 sec. For a second group of pigeons the ciprocal of the shorter IRT band at .70 but varied the
reinforced IRTs were those used by Staddon (1968), absolute values of the two reinforced IRTs. Relative
2 to 3 sec and 10 to 11 sec. rate of emission of the two IRTs only matched the
Monotonically increasing, negatively accelerated relative reciprocal of their lengths when the shorter
functions described the relation between relative re¬ IRT band was between 1.5 and 2.5 sec. This was
sponse rate and relative reinforcement frequency for roughly the lower bound of the shorter class of IRTs
each key, similar to the functions obtained for the used in previous experiments that reported matching
same IRTs by Shimp (1968) and Staddon (1968). If to the relative reciprocal (Moffitt & Shimp, 1971;
the data are plotted in terms of ratios of responses Shimp, 1969b, 1971). When the lower bound of the
and reinforcements, the group of three pigeons ex¬ shorter IRT band was less than 1.0 sec the pigeons
posed to the IRTs used by Staddon (1968) produce were nearly indifferent between the two IRTs. And
power functions with exponents of .73, .58, and .69, when both of the reinforced IRTs were longer than
respectively. The group of pigeons exposed to the 2.5 sec the pigeons showed greater preference for the
IRTs taken from Shimp’s earlier experiment produce shorter IRT than was predicted by relative reciprocal
power functions with larger exponents, .91, .86, and matching.
.63, respectively, nearer the 1.0 exponent predicted To summarize, the matching relation applies in a
by the biased matching relation. All of the pigeons wide range of choice situations, to several different
showed a marked bias toward the shorter of the two parameters of reinforcement besides frequency, and to
IRTs, in keeping with the preference for shorter choice between several different schedules of rein¬
IRTs suggested by Shimp (1968). Choice between two forcement besides Vis. In concurrent VI FI and paced
concurrent IRTs therefore produces similar functions schedules, however, the slope of the function relating
relating response ratios (or proportions) to IRT choice and reinforcement for each alternative differs
lengths and to reinforcement ratios (or proportions) from that predicted by matching, and even the gen¬
whether the IRTs are programmed on two keys or eralized matching equations including a bias or pref¬
together on one key. erence parameter (Equations 4 and 5) do not handle
The results of these experiments demonstrate that the data. They account for deviations in the intercept
the closer together the two reinforced IRTs are in of the function from that predicted by matching—i.e.,
length, the higher the power function exponent and for a consistent preference for one alternative—but
the closer the approximation to matching. Indeed, not for deviations in slope from 1.0. In the next sec¬
Shimp (1971) showed that when the same two bands tion of the chapter I shall consider how a quantitative
of IRTs are reinforced on each key in a two-key con¬ formulation of the relation between response strength
current schedule, close matching occurs between rela¬ and reinforcement, derived from the matching rela¬
tive response rate on one key and relative rein¬ tion, could account for the deviations from 1.0 slope
forcement rate on that key. The matching relation found in these studies.
Peter de Villiers 257

ABSOLUTE RATES OF RESPONDING AND all sources in the situation. The parameter k can be
A QUANTITATIVE LAW OF EFFECT thought of as the total amount of behavior sustained
by all the reinforcement available to the animal in the
We have seen that in a wide range of choice situa¬ experimental situation. It is measured in the same
tions the relative rate of responding is directly deter¬ units as the stipulated response—e.g., in responses per
mined by the obtained relative frequency of reinforce¬ min or running speed units7 (see Herrnstein, 1974).
ment. However, the matching relationship between The relation of Equation 9 to the matching equation
response proportions or ratios and the corresponding then becomes clear:
reinforcement proportions or ratios may not be the
most fundamental way to quantify the relation be¬ ri
tween response strength and reinforcement. Response n n
and reinforcement proportions or ratios remain in¬ 2 Ri 2 n
i=0 i=0
variant over large changes in overall response rate or
reinforcement frequency. What about the absolute n
k 2 Ri (10)
response rates in choice situations? 7=0
Herrnstein (1970) reasoned that the relative fre¬
kr1
quency of reinforcement should determine not only
Ri
the relative response rates but also the absolute rates n
2 n
of responding. As Herrnstein pointed out, at every 7=0
moment of possible action a set of alternative re¬
sponses confronts the animal, so that each action is
Single Schedules—Frequency of Food Reinforcement
the outcome of a choice. No matter how the experi¬
menter tries to control the extraneous sources of In a single-response procedure, Equation 9 becomes
reinforcement for responses other than those stipu¬
lated in the experiment, the subject will always have
distractions available, even if they are merely con¬ Rx (11)
n + re
cerned with its own body or physiological processes.
Thus even on single-response procedures the subject where Rlf rlf and k are as specified for Equation 9.
is in a concurrent situation, although the experi¬ The parameter re represents the total reinforcement
menter may monitor only one of the alternative re¬ besides ry in the experimental situation6—i.e., all the
sponses and reinforcers. other reinforcers that a subject brings with itself or
Absolute response rate on single and concurrent finds in the experimental setting. The equation as¬
schedules can therefore be considered a function of sumes that they can be given a value in terms of the
the frequency of reinforcement for that response rela¬ units of reinforcement contingent on R±; thus re is
tive to all the other sources of reinforcement for com¬ measured in terms of the same units as rlt a frequency,
peting responses. Herrnstein (1970, 1971) suggested a amount, or concentration of reinforcement (Herrn¬
direct proportionality between the overall relative stein, 1974).
frequency of reinforcement and the absolute rate of Herrnstein (1970) tested this equation with data
responding. For a situation containing n alternative from two experiments investigating the effects of fre¬
sources of reinforcement he proposed a general equa¬ quency of food on key-peck rate in pigeons. The best
tion of the form
data come from an exhaustive study of single VI
schedules by Catania and Reynolds (1968). Six pigeons
kr1 were exposed to VI schedules with frequencies of rein¬
(9) forcement ranging from 8 to 300 reinforcements per
n
2 n
i=0 6 In his 1970 paper “On the Law of Effect,” Herrnstein used
the expression r0 to mean all reinforcements besides n—i.e.,
where Rx is the rate of emission of the stipulated re¬ 5 n — n — ra. But in his later paper (Herrnstein, 1974), he used
r0 in a more restrictive sense of reinforcements that come
sponse and r1 is the frequency or magnitude of rein¬
spontaneously and are not conditional upon any responses.
forcement for that response. The parameter k repre¬ Therefore, re is used to denote more generally all reinforcements
sents the asymptotic response rate in the absence of spontaneous or contingent on responses other than n—i.e., all
extraneous sources of reinforcement (Herrnstein 8c Loveland,
any reinforcement for competing response—i.e., when
1974). I shall follow the more recent usage of re throughout the
r1 equals 2 the total amount of reinforcement from chapter.
 258 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

eter values than those for Catania and Reynolds’

subjects. With a k of 130 pecks per min and re of 210
reinforcements per hr, Equation 11 accounts for 94.7%
of the variance in mean response rate for Chung’s
pigeons.
Can the equation be extended to other parameters
of reinforcement besides frequency, and to other mea¬
sures of response strength besides rate of key pecking?
De Villiers and Herrnstein (in press) have carried out
a comprehensive review of the literature on the rela¬
tion between several measures of response strength
(e.g., running speed in an alley or latency to respond
in a discrete-trial situation, as well as the rate of emis¬
sion of repetitive responses like key pecking or lever
pressing) and several parameters of both positive and
negative reinforcement. For each study the data were
either taken from tables or estimated from figures. A
least-squares fit of Equation 11 to the data from
groups or individual subjects was performed, and the
percentage of the variance in the dependent variable
accounted for by the equation was calculated.7
REINFORCEMENTS/HOUR

Fig. 8. Rate of responding as a function of frequency of food Magnitude of Food Reinforcement

reinforcement for six pigeons responding on single VI schedules.
The least-squares fit of Equation 11 to the data is plotted for The most extensive study of magnitude of food
each pigeon. The k- and revalues and the percentage of the
variance in response rate accounted for by the functions are reinforcement is that by Crespi (1942). Five groups of
also shown. (Data from Catania and Reynolds, (1968) as plotted rats, 12 to 20 to a group, ran down a straight alley
by Herrnstein, R. J. On the law of effect. Journal of the Experi¬ for different weights of dog chow, ranging from .02 to
mental Analysis of Behavior, 1970. © 1970 by the Society for the
Experimental Analysis of Behavior, Inc.) 5.12 grams. Equation 11 describes remarkably well the
relation between the quantity of food and the mean
running speed (1/time in sec) for each group of rats,
hr. Figure 8 shows the least-squares fit of Equation 11 accounting for 99.6% of the variance in running
to the data from each of the pigeons. The values of speed. Appendix A summarizes this and several other
k and re and the percentage of data variance ac¬ studies investigating the effects of magnitude of food
counted for by the equation in each case are given in reinforcement on response strength. In the study of
the bottom right-hand corner of each panel. With a k Davenport, Goodrich, and Hagguist (1966), individual
of 66.3 responses per min and an re of 7-3 reinforce¬ data were also available for four macaque monkeys
ments per hr; the equation also accounts for 91.3% of lever-pressing for a varying number of pellets on a
the data variance when response rate is averaged VI 1-min schedule. Once again the equation provides
across the pigeons for each VI value. an excellent fit to the data, accounting for 99.9, 90.4,
Chung (1966) reinforced pigeons’ responses on a 90.1, and 96.6% of the variance in response rate for
tandem FR 1 Fix schedule, where x represents a given each monkey. For six of the remaining studies in
duration after the first postreinforcement response. Appendix A (Beier, 1958; Di Lollo, 1964; Hutt, 1954;
The first response after the postreinforcement pause Keesey 8c Kling, 1961, Experiment II; Logan, 1960, 12-
started an FI timer, and the first response after the hour group; Zeaman, 1949), which show group data
timer timed out produced food reinforcement. The from rats and pigeons. Equation 11 accounts for over
length of the FI was varied, determining both rate of
responding and rate of reinforcement. By this method, 7 A digital computer iteratively calculated the mean-squared
reinforcement rates of up to 2,000 per hr were ob¬ deviation of obtained response rates from those predicted by
Equation 11 for a wide range of k- and revalues varying on
tained, many times the maximum value investigated
either side of those derived from a best fit by eye. The smallest
by Catania and Reynolds (1968). Herrnstein (1970) mean-squared deviation thus obtained was subtracted from the
demonstrated that Equation 11 also accounts for total variance in response rate and the result divided by the
total variance to determine the percentage of data variance
Chung s results, though with somewhat higher param¬ accounted for by the equation.
Peter de Villiers 259

90% of the variance in the measure of response the magnitude of reinforcement, since the frequency
strength in five cases, and for over 80% in the sixth. of brain stimulation covaries with response rate on
Since k in runway studies is measured in response these schedules. At higher intensities or durations of
speed units, 100/time in seconds to run the runway, it stimulation, motor effects of the brain stimulation also
varies with runway length. Hence the ^-values are not tend to interfere with responding. Keesey (1962, 1964)
usually comparable across studies. therefore used a VI 16-sec schedule to investigate the
There are three exceptions to the remarkably good effects of several parameters of stimulation of the pos¬
fit of Equation 11 to the data on magnitude of rein¬ terior hypothalamus on rate of lever pressing in rats.
forcement. Keesey and Kling (1961, Experiment I) Over a wide range of values, Equation 11 accurately
studied four pigeons key-pecking for different-sized depicts the relation between response rate and either
chick-peas (varying from four quarter-peas to four the duration, intensity, or pulse frequency of the
whole peas) on a VI 4-min schedule. Catania (1963a) brain stimulation, accounting for 94.1, 92.9, and
studied two pigeons key-pecking for three different 96.9% of the data variance, respectively. The least-
durations of grain reinforcement (3, 4.5, and 6 sec) on squares fit to the data from both of Keesey’s studies is
a VI 2-min. Logan (1960, Experiment 55B) studied six given in Appendix B.
rats per group receiving varying numbers of food pel¬ Gallistel (1969) argued that lever-press rate on a VI
lets (1, 3, 6, and 12) for running down a straight alley. schedule of brain stimulation might not be a valid
At 12 hr of food deprivation, running speeds of the measure of the magnitude of reinforcement, since re¬
groups conformed to Equation 11; but at 48 hr of sponse rate is also sensitive to the time since the last
deprivation running speed ceased to be a monotonic stimulation. To control for any variation in the after¬
function of number of pellets. In the other two devi¬ effects of the last brain stimulation, Gallistel gave all
ant studies (Catania, 1963a; Keesey & Kling, 1961, his subjects a series of 10 priming stimulations before
Experiment I) the experimenters observed minimal they ran down a straight alley. Any differences in run¬
changes in response rate with increases in reinforce¬ ning speed should then be a function of the magni¬
ment magnitude. Two factors could possibly account tude of the rewarding stimulation in the goal box.
for the insensitivity of key-peck rate to variations in Nine rats, implanted in three different rewarding
reinforcement in these two studies. First, only a very areas of the brain, ran the straight alley for a varying
narrow range of magnitudes was investigated, no¬ number of .01-msec pulses of electrical stimulation.
where near the range studied by Crespi (1942) or the Appendix B shows that the least-squares fit of Equa¬
range of reinforcement in studies of frequency of food tion 11 accounts for a substantial proportion of the
(Catania 8c Reynolds, 1968). Second, the pigeons were variance in running speed for each rat, even in some
studied at high-drive levels in both experiments (as cases where there was little variation in running speed.
were the rats in the disconfirming condition of Lo¬ The equation accounts for the behavior very well (i.e.,
gan’s study). Herrnstein (1970) has argued that the better than 90%) for five out of the nine subjects.
reinforcement from sources other than ^—namely, re
—will be extremely small relative to r1 itself under con¬
Quality of Reinforcement
ditions of high drive. Response rates would therefore
be very close to k over a fairly wide range of ^-values, Numerous experiments have investigated the effects
since r1 would approximate 2 rt. The data variance in of different sugar concentrations on response strength.
both pigeon studies was minimal. Guttman (1954) studied seven concentrations of su¬
In brief, Herrnstein’s equation can be readily ex¬ crose (between 2 and 32%) and the same seven con¬
tended to the relation between response strength and centrations of glucose with rats lever-pressing on a
magnitude of food reinforcement for several responses VI 1-min schedule. The rats responded faster for
and species, though with a few exceptions. sucrose than for glucose at each concentration, but
Equation 11 accurately describes the relation between
Brain Stimulation response rate and concentration for both reinforcers,
accounting for 93.7 and 98.7% of the data variance.
The magnitude of reinforcement from intracranial Appendix C summarizes the data from this and
brain stimulation varies with several parameters of several other studies on concentration of sucrose. In
stimulation—e.g., the intensity, duration, or pulse fre¬ Guttman’s (1953) experiment two different conditions
quency of the stimulation. But variations in response were studied. In one a different group of about 20
rate for brain stimulation on continuous reinforce¬ rats was exposed to each concentration; in the other,
ment schedules do not provide an accurate measure of 20 rats were exposed to all four concentrations. In
260 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

both cases the equation accounts for over 90% of the form substantially to Equation 11. Both experiments
variance in mean response rate. In analyzing the data used rats as subjects and food as reinforcement, but
from rhesus monkeys obtained by Conrad and Sidman Logan measured speed in a runway and Silver and
(1956), response rates for the 60% sucrose solution Pierce measured rate of lever pressing. In each case,
were excluded because the experimenters report con¬ including both high- and low-drive conditions in
siderable satiation at this concentration. Two drive Logan’s experiment, the equation accounts for over
levels were used: 48 and 72 hr of food deprivation. As 90% of the variance in group data.
in the case of Logan’s (1960) experiment on magni¬
tude of reinforcement, the equation accounts for
rather more of the variance in response rate at the
Frequency of Negative Reinforcement
lower drive level (96.2 versus 71.8%).
A pair of more extensive studies by Schrier (1963, De Villiers (1974) demonstrated that Herrnstein’s
1965), also with rhesus monkeys and a lever-press equation for absolute response strength in single
response, used considerably shorter sessions to avoid schedules of positive reinforcement can be extended
satiation effects. Over a range of concentrations from to VI avoidance schedules, with shock-frequency re¬
10 to 50%, the equation fits both the average and duction (shocks avoided per min) as the reinforcer for
individual data remarkably well. For 10 of the 14 avoidance (Herrnstein, 1969; Herrnstein 8c Hineline,
available individual functions, the variance accounted 1966). In de Villiers’s experiment, lever-press re¬
for was greater than 90%, and for none did it fall be¬ sponses canceled the delivery of shocks scheduled at
low 70%. For the three separate group averages, Equa¬ variable intervals. If no lever press was made, all of
tion 11 accounts for over 95% of the variance in each the scheduled shocks were presented. The first re¬
case. sponse made after a scheduled shock, whether or not
that shock had been presented, prevented the delivery
of only the next scheduled shock. Extra responses
Immediacy of Positive Reinforcement (1/Delay)
between two scheduled shocks did not avoid further
The most comprehensive data on delay of reinforce¬ shocks. All the scheduled shocks could therefore be
ment in a single-response situation come from Pierce, avoided if the rat responded at least once within every
Hanford, and Zimmerman (1972). Four rats respond¬ intershock interval, but the durations of the intervals
ing on a VI 1-min schedule for food experienced varied unpredictably. On this VI avoidance schedule
delays of reinforcement varying from .5 to 100 sec. Dur¬ both received shock rate and shock-frequency reduc¬
ing the delay a cue light was illuminated and respond¬ tion (scheduled shock rate minus received shock rate)
ing had no programmed consequences. Equation 11 can be measured, and response rate is not constrained
again provides an accurate description of the results by any fixed temporal relations between responses and
of Pierce et al., with immediacy of reinforcement shocks, as it is on the free operant avoidance sched¬
(1/delay) as r±. It accounts for 96.1% of the variance ules usually studied (Sidman, 1953, 1966).
in mean response rate and for 80.8, 98.3, 78.9, and Four rats were exposed to an ascending and de¬
97.4% of the variance for each of the four rats. The scending series of VI avoidance schedules ranging
same rats were studied in a second condition in which from a VI 15-sec (four programmed shocks per min)
the lever was retracted during the delay of reinforce¬ to a VI 75-sec (.8 programmed shocks per min). A wide
ment. The equation fits the data from this condition range of response rates was produced by each rat on
as well, accounting for 95.0% of the variance in mean the different schedules. Response rates are plotted
response rate and for 97.8, 98.6, 94.9, and 92.9% of the against shock-frequency reduction (shocks avoided per
individual data variances. min) for each rat in Figure 9. The least-squares fit of
Appendix D gives the least-squares fit of the equa¬ Equation 11 is shown for each animal. Where there
tion to several other studies on delay of reinforcement were two determinations of response rate for a given
as well as that of Pierce et al. The Perin (1943) experi¬ VI value, the mean of the two was used in calculating
ment is noteworthy in that it measured latency to the best fit of the equation. Equation 11 accounts for
lever press in a discrete-trial procedure as a function over 95% of the variance in response rate for each rat
of delay of reinforcement. The equation here accounts (see Figure 9). Reinforcement from extraneous sources,
for the relation between speed of responding (100/ ^e> in VI avoidance schedules represents the reinforce¬
latency in sec) and immediacy of food presentation ments for freezing, crouching, defecating, etc. scaled
(1/delay). The remaining studies (Logan, 1960; Silver in terms of their equivalent value in shocks avoided
8c Pierce, 1969) summarized in Appendix D also con¬ per min, the reinforcer for the avoidance response.
Peter de Villiers 261

why parameter values in the two studies differ by as

much as they do.
Dinsmoor and Hughes (1956) and Harrison and
Abelson (1959) varied the duration of time-out from
aversive stimulation contingent on a lever-press escape
response. Dinsmoor and Hughes measured latency to
lever press in a discrete-trial escape procedure and
observed a monotonically increasing function relating
speed of response (100/latency in sec) and duration of
time-out. The equation accounts for 99.5% of the
variance in speed of response for the rats that were
escaping from a .2-mA shock. For the rats escaping
from the stronger shock—.4 mA—variance in response
speed was small and only about 75% of it was ac¬
counted for by the equation. Harrison and Abelson
SHOCKS AVOIDED / MINUTE measured response rate on VI escape from a loud
noise. They found substantial order effects and ex¬
Fig. 9. Rate of responding as a function of shock-frequency posed only one rat to complete ascending and descend¬
reduction for four rats responding on single VI avoidance
schedules. The least-squares fit of Equation 11 to the data is ing orders, but if response rates are averaged across
plotted for each rat. The k and rg values and the percentage the several determinations for each time-out duration,
of the variance in response rate accounted for by the equation over 90% of the variance is accounted for by Equa¬
are also shown. Filled circles represent the mean response rate
of two determinations for a given VI value; open circles repre¬
tion 11.
sent single determinations, (de Villiers, 1974.)

Immediacy of Negative Reinforcement

Magnitude of Negative Reinforcement
Fowler and Trapold (1962) conducted a thorough
Several experiments have investigated the effects of study of delay of escape in a straight runway. Delay
different amounts of voltage reduction between the of voltage reduction in the goal box varied over five
alley and goal box on a rat’s running speed in a values between 1 and 16 sec for groups of five rats
straight runway. Three of these studies (Bower, Fowler, each. The observed relation between running speed
8c Trapold, 1959; Campbell 8c Kraeling, 1953; Seward, in the runway (100/time in sec) and immediacy of
Shea, Uyeda, 8c Raskin, 1960) are summarized in Ap¬ escape (1/delay in sec) in the goal box conforms to
pendix E. All of them used different groups of rats for Equation 11, 92.6% of the variance in running speed
each voltage reduction value. In fitting Equation 11, being accounted for by the equation with a k of 94.0
de Villiers and Herrnstein (in press) took running (running speed units) and an re of .06 (1 /delay units).
speed as the response measure and the reduction in These results, together with other data from experi¬
voltage as the reinforcer. Extraneous reinforcement re ments varying delay of negative reinforcement, appear
is expressed as a voltage reduction, its value scaled in in Appendix F.
the units of the independent variable. Thus in the Tarpy (1969) had rats escape from electric shock
study of Bower et al. (1959) the value of the reinforce¬ by pressing either of two levers in a discrete-trial pro¬
ment for competing responses such as freezing or cedure. The experiment’s main purpose was to investi¬
defecating is equivalent to a voltage reduction of gate preference as a function of differential delays be¬
338 V. For all three experiments, Equation 11 ac¬ tween a press on either lever and shutting off the
counts for over 85% of the data variance. shock. But de Villiers and Herrnstein (in press) tested
Woods and his co-workers (1965, 1966) used an in¬ Equation 11 with data from conditions in which the
teresting variant of the runway escape procedure. delay of escape was the same for both levers; 89.5% of
Their rats swam down a straight alley filled with very the variance in response speed (100/latency in sec) for
cold water in order to be placed in a goal box con¬ five delays between 1 and 16 sec (excluding the 0-delay
taining much warmer water. The independent vari¬ condition) is accounted for. Tarpy and Koster (1970)
able was the increase in water temperature between varied delay of discrete-trial escape from electric shock
the alley and goal box; the dependent variable was by rats responding on a single lever. Using the three
swimming speed (100/time in sec). Equation 11 ac¬ nonzero-delay values, 94.6% of the variance in re¬
counts for these data very well, though it is not clear sponse speed is accounted for by Equation 11. Leem-
262 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

ing and Robinson (1973) also varied delay of escape physical scales for the independent variables, such as
from electric shock by rats, but the escape response electrical pulses per second or percentage concentra¬
was running from one compartment to the other in a tion by weight. In contrast, studies in psychophysics
shuttlebox. The equation accounts for 86.0% of the indicate that many dimensions of experience are not
variance in speed to respond (100/latency in sec). accounted for by the physical scales, unless they are
Finally, Appendix F also contains Moffatt and transformed (Stevens, 1957). At least two explanations
Koch’s (1973) data from human subjects listening to of the simplicity of the independent variables are
a Bill Cosby comedy record. Occasionally the record¬ possible.
ing would stop and the subject could restart it by First, some of the simplicity may reflect the rela¬
pressing a panel. The primary independent variable tively small ranges of the independent variable used
was the delay between the panel response and the on¬ in the studies. The range from 1 to 9 food pellets or
set of the recording. For the three nonzero delays .28 to 2.7 msec of brain stimulation is not quite an
studied. Equation 11 accounts for over 95% of the order of magnitude, while the usual psychophysical
variance in speed of panel depression. experiment uses several orders of magnitude. If such
broad ranges of reinforcement could be used in stud¬
Summary
ies of animal behavior, rescaling of the reinforcement
variable might become necessary or a different equa¬
The remarkable generality of Herrnstein’s equation tion might need to be formulated.
is apparent from this survey. The behavior of rats, Second, several of the independent variables sur¬
pigeons, monkeys, and (in the one ease we found) veyed here have been shown to produce matching of
people, is equally well accounted for, whether the relative response rate to relative reinforcement (see
behavior is lever pressing, key pecking, running speed, Equation 1) in concurrent schedules. Variables that
or response latency in a variety of experimental set¬ obey Equation 1 should also obey Equation 11 (see pp.
tings. The reinforcers can be as different as food, 257 and 266). Therefore the fit of Equation 11 to
sugar water, escape from shock or loud noise or cold the data on frequency, magnitude, and immediacy of
water, electrical stimulation of a variety of brain loci, reinforcement only further confirms the matching re¬
or turning a comedy record back on. Out of 53 tests lation for choice.
of Equation 11 on group data, the least-squares fit of
the equation accounts for over 90% of the variance in
The Constancy bf the /^-parameter
the dependent variable in 42 cases, and for over 80%
in another six cases. Out of 45 tests on individual In a recent paper Herrnstein (1974) suggests that
data, the equation accounts for over 90% of the the equation can be made even stronger. He argues
variance in 32 cases, and for over 80% in another that a formal implication of the matching relation is
seven cases. The literature appears to contain no evi¬ that for any given response the parameter k must re¬
dence for a substantially different equation than main constant across different qualities or quantities
Equation 11. Where the equation fails to account for of reinforcement or changes in the animal’s drive
most of the data variance, that variance is negligible level, as long as the response topography does not
in all cases but one. In the one exception (Logan, change. The ^-parameter is just a measure of behav¬
1960, Experiment 55B) supposedly asymptotic run¬ ior, such as responses per min, and represents the
ning speed was a sharply nonmonotonic function of amount of behavior the observed response would
number of food pellets. Other experiments on pellet show if there were no reinforcement for competing
number, including one by Logan (I960), found mono- responses. When there are other sources of reinforce¬
tonic functions, all conforming to Equation 11. This ment, k measures the total frequency of all responses
equation therefore provides a powerful but simple in units commensurate with the measure of the re¬
framework for the quantification of the relation be¬ sponse being studied. The ^-parameter is therefore
tween response strength and both positive and nega¬ simply “the modulus for measuring behavior” (Herrn¬
tive reinforcement. stein, 1974), and the sole influence on its size is the
Perhaps the most surprising feature of the close fit chosen response form itself.
of the equation to the data is that it did not require Several of the studies discussed above test the con¬
any ad hoc transformations of the reinforcement vari¬ stancy of k across different qualities and quantities of
ables. For example, studies of brain stimulation (Ap¬ reinforcement or across different drive levels. Only
pendix B) or sucrose reinforcement (Appendix C) are cases in which over 90% of the variance was ac¬
well accounted for by Equation 11 by using ordinary counted for by Equation 11 will be considered here,
Peter de Villiers 263

since the value of k is then well specified. When the bell and Kraeling (1953), in which rats ran clown an
equation was fitted to Guttman’s (1954) data on the electrified runway for reduction in voltage between
same eight rats lever-pressing for either sucrose or the alley and goal box. The &-value derived for the
glucose, practically the same &-value was derived for more intense alley shock condition was over twice that
the two reinforcers (15.6 versus 16.1 responses per derived for the less intense shock condition: 228 to
min). The reinforcement from other sources (re) was 106 running speed units. The data on the constancy
much smaller for the sucrose function (7.1% versus of k therefore remains equivocal, and further experi¬
11.0%), in keeping with rats’ preference for sucrose, ments must examine the range of conditions under
the sweeter solution. Kraeling (1961) ran different which k does or does not remain constant.
groups of rats in a runway for sucrose solutions of
varying concentration. Three separate functions relat¬
ing running speed to sucrose concentration were ALTERNATIVE THEORIES OF
found: one for each of three different durations of RESPONSE STRENGTH
access to the sucrose in the goal box. The three
^-values given by the least-squares fit of Equation 11 Shimp (1974) has argued against Herrnstein’s equa¬
were 89.4, 87.9, and 88.7 running speed units, impres¬ tion as an adequate theory of response strength in VI
sive confirmation of the constancy of k considering schedules on the grounds that it neglects the role of
that different groups of rats ran in the different condi¬ the distribution of reinforced interresponse times
tions. Logan (1960) ran groups of rats under both (IRTs) in determining the overall response rate.
high- and low-drive levels (hours of deprivation) for Shimp points out that the mean rate of responding is
different immediacies of food reinforcement. The the reciprocal of the mean of a distribution of IRTs,
^-values fitted to his data are 64.9 and 64.5. In the which is in turn determined by the distribution of
experiment of Seward et al. (1960), rats ran down an reinforcements for those IRTs. He suggests that the
electrified runway for a reduction in voltage between pigeon in fact distributes its time among different
the alley and goal box. For two different alley volt¬ IRTs according to how frequently each is reinforced
ages, A-values of 161 and 146 running speed units relative to the others; i.e., the basic response is one of
were derived, about a 10% difference in k across drive pausing between key pecks rather than key pecking
levels. In the comparable experiment on escape from per se.
cold water (Woods 8c Holland, 1966), the rats swam Shimp (1974) studied three pigeons responding on
from cold water to warmer water, and temperature in¬ a “synthetic” VI schedule (Anger, 1973; Shimp, 1973).
crease was the reinforcement variable. For two differ¬ Food reinforcement for key pecking was arranged by
ent alley temperatures the ^-values were 108 and 114 a single VI schedule and a programming device that
swimming speed units, an even better agreement be¬ assigned the reinforcements equally to each of ten
tween ^-values than that found by Seward et al. (1960) classes of IRTs ranging from 1.0 to 6.0 sec in .5-sec
for electric shock. classes. Overall reinforcement frequency was varied
While the above studies support Herrnstein’s hy¬ between 1 and 70 per hr by changing the VI schedule.
pothesis that k remains constant as reinforcement and Overall response rate on this schedule was a mono-
drive values change, some of the data analyzed do not. tonically increasing, negatively accelerated function of
In Keesey’s (1964) experiment the same 10 rats lever- overall reinforcement frequency, much like that ob¬
pressed on a VI 16-sec schedule for different durations served with normal VI schedules by Catania and
of brain stimulation under two intensity conditions, Reynolds (1968). Shimp argued that this overall re¬
3.0 and 1.5 mA. In this case, the least-squares fit of sponse rate function could be decomposed into two
Equation 11 produced a much larger k for the 3.0-mA time-allocation functions: (1) the time allocated by
condition (21.0 versus 14.9 responses per min), while the pigeon to all of the reinforced IRTs as a function
revalues were very similar. In Schrier’s (1965) experi¬ of overall reinforcement frequency—i.e., the percent¬
ment on monkeys lever-pressing for five different con¬ age of session time taken up by all of the IRTs emit¬
centrations of sucrose, two different quantities, .33 ted in the range from 1.0 to 6.0 sec; and (2) the time
and .83 cc, were investigated. The ^-values for the allocated to any particular class of IRTs as a function
mean response rates of the monkeys were 88 and 66.5 of the reinforcement rate obtained by that class of
responses per min in the two conditions. Individual IRTs—i.e., the number of emissions of that IRT class
A-values also showed considerable variation across the times its lower bound duration. At asymptotic overall
two quantities (see Appendix C). Finally, the largest response rate (over 20 reinforcements per hr) the rela¬
discrepancy was found in the experiment by Camp¬ tive amount of time allocated to a particular IRT
 264 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

class roughly matched its relative frequency of rein¬ tion for response strength based on assumptions about
forcement. The pigeons spent about the same amount the excitatory and inhibitory effects of reinforcement
of time responding in each IRT class since each was on behavior. Whereas Herrnstein assumes that asymp¬
assigned the same frequency of food; therefore, they totic response rate (k) for a given response is constant
made fewer of the long IRTs and more of the shorter across different drive and reinforcement conditions,
ones. The asymptotic overall response rate in re¬ Catania assumes that responding increases linearly
sponses per min could therefore be predicted by a with increasing reinforcement,
combination of the percentage of session time taken
up by IRTs between 1.0 and 6.0 sec and the frequency = Krx (12)
of emission of each of the IRT classes in that range.
This asymptote varies with the distribution of rein¬ This effect combines with an inhibitory effect of total
forcements across IRT classes; thus Shimp argues that reinforcement on the response, whether the reinforcers
there may be no such thing as a constant asymptotic originate from other sources or from the response it¬
response rate across reinforcement and drive condi¬ self (Catania, 1963b, 1969; Rachlin & Baum, 1969,
tions, as required by Herrnstein’s theory. Given the 1972):
distribution of reinforced IRTs, the experimenter
could in fact predict the asymptotic response rate, C
since pigeons match the time allocated to each IRT Ri = Krx X
C + %r (13)
class to its relative reinforcement frequency (pro¬
vided that the overall reinforcement rate is above 20 KCr1
per hr). C + 2r
In assessing Shimp’s argument against Herrnstein’s
molar formulation, however, several considerations where C is a constant that depends on the magnitude
should be taken into account. First, while Shimp of the inhibitory effect of the reinforcers, and 2 r is
(1973, 1974) and Anger (1956, 1973) have shown that the total reinforcement obtained from all sources.
an animal’s behavior is sensitive to the differential Setting KC to a new constant k gives the following
reinforcement of particular IRTs, the extension of equation:
their analysis to responding on normal VI schedules is
unclear. On VI schedules with a constant probability kr±
of reinforcement over time the differential reinforce¬ *i = (14)
C + Xr
ment of IRTs is minimal unless the animal has a
propensity to emit IRT$ of a particular duration. Here 2 r includes only the specified sources of rein¬
Since pigeons tend to emit IRTs of about .3 to .5 sec forcement, not extraneous unspecified sources as in
duration (Blough, 1963), those IRTs may be differen¬ Herrnstein’s Thus when rL is the only scheduled
tially reinforced, but such an effect has not been con¬ reinforcement, 2 r = r± and Equation 14 is mathe¬
clusively demonstrated. matically equivalent to Herrnstein’s equation for
Second, Shimp’s synthetic VI schedule had impor¬ single-response situations. The constant C is derived
tant differences from the normal VI. In his procedure from the data, as is re. Catania’s equation therefore
the same 10 IRT classes were reinforced equally fre¬ accounts for as much of the data summarized in the
quently at all VI values; but if differential reinforce¬ earlier sections of the chapter as does Herrnstein’s
ment of IRTs takes place in a normal VI, different equation.
VI schedules should differentially reinforce different While there is not yet empirical data to distinguish
IRTs. Each mean response rate on the function relat¬ between these two formulations, Catania (1973) argues
ing responding to reinforcement frequency would that Herrnstein’s equation assumes that the variety of
therefore be the outcome of a different time-allocation reinforcers subsumed under re do not interact in any
function. Herrnstein’s quantitative formulation ac¬ complex way with rlf the scheduled reinforcer. But
counts for the entire function, not only its asymptote. there is evidence for interaction between such rein¬
Finally, there seems to be no way to extend Shimp’s forcers as food and water (Bolles, 1967). However,
account of responding on VI schedules to the great Catania’s equation also fails to account for the data
wealth of data on other measures of response strength on choice between food and water reinforcers. In the
(e.g., running speed or latency) that Herrnstein’s experiment of Wood et al. (1975), rate of water rein¬
equation accounts for so well. forcement on an FR schedule did not affect rate of
Catania (1973) has proposed an alternative equa¬ responding on a concurrently scheduled FI schedule
Peter de Villiers 2 65

for food, as it should if food and water simply sum in proximity of responses to reinforcement. The latter
2 r in Equation 14. Therefore, neither equation read¬ principle determines the temporal locatiorr of be¬
ily handles interactions between reinforcers. havior under schedule control, accounting for the
temporal pattern of responding typically found on FI
and FR schedules, in which the relative proximity to
APPLICATION OF HERRNSTEIN'S reinforcement increases with time since the last rein¬
EQUATIONS TO OTHER SCHEDULES forcement. On VI schedules with a random distribu¬
tion of intervals, differential reinforcement of par¬
Herrnstein’s equation for single schedules. Equa¬ ticular IRTs is minimized and the relative proximity
tion 11, applies best to VI schedules, in which the of responses to reinforcement does not vary with post¬
probability of a reinforcement being scheduled after reinforcement time. Consequently, response rate is
one has been presented is approximately constant fairly constant over time since reinforcement and is
over time. Its extension to other schedules, such as directly related to relative reinforcement frequency
FIs, is more difficult. Overall response rate on differ¬ (see p. 258) (Catania 8c Reynolds, 1968). Staddon
ent FI schedules typically does not fit the equation showed that these two factors can qualitatively ac¬
very well. However, Schneider (1969) has argued for count for many of the properties of behavior on FI,
a two-state analysis of well-learned FI performance. In VI, FR, and VR schedules.
the first state, beginning immediately after reinforce¬ The data considered in the earlier sections of this
ment, response rate is very low and approximately chapter suggest that Herrnstein’s equation provides
constant over different FI values. At some variable an accurate quantitative formulation of one aspect of
time, on the average about two-thirds of the way schedule-controlled behavior: the relation between
through the FI, there is an abrupt transition (or response strength and relative reinforcement fre¬
break point) to a high and approximately constant re¬ quency or magnitude. Further research must specify
sponse rate. Rate of responding in the second state is the way this factor interacts quantitatively with other
an increasing, negatively accelerated function of rein¬ factors such as the relative proximity to reinforcement
forcement frequency in that state, much like the func¬ or differential reinforcement of IRTs to determine
tion obtained for VI schedules. Schneider suggests that behavior on any given schedule.
in a sense the pigeon is on a VI schedule in the second
state, the interreinforcement intervals being deter¬
Concurrent Schedules
mined by the bird’s break point distribution. Herrn¬
stein’s equation actually provides an accurate account Equation 11 can also be applied to absolute re¬
of second-state response rate in Schneider’s experi¬ sponse rates in two-response concurrent schedules.
ment, though with somewhat higher parameter values The term re then breaks down into r2 + re, where r2
than those usually found for pigeons on VI schedules. is the reinforcement rate associated with the second
With a k of 147 responses per min and an re of 24 response. Thus:
reinforcements per hr, the equation accounts for
92.2% of the variance in mean second-state response
rate for Schneider’s pigeons.
Several theorists have argued that the operation of
at least two factors determines the pattern and rate of where k and re are as already defined for Equation 11.
responding on schedules of intermittent reinforce¬ Herrnstein (1970) demonstrated that this equation ac¬
ment. Morse (1966) proposed that most schedule- counts for the absolute response rates obtained by
controlled behavior results from the joint effects of Catania (1963b) for pigeons in a CO-key concurrent
reinforcement on response strength and the differen¬ procedure. The fit of the equation to the data is
tial reinforcement of certain interresponse times shown in Figure 10. The left-hand panel shows re¬
(IRTs). Thus ratio schedules tend to maintain higher sponse rate on each schedule as a function of rein¬
response rates than interval schedules for the same forcement frequency for each when the total rate of
frequency of reinforcement possibly because there is reinforcement was held constant at 40 per hr. The
more selection for long IRTs on interval than on right-hand panel shows response rate on each sched¬
ratio schedules (Killeen, 1969). Staddon (1972) simi¬ ule when the reinforcement frequency on Schedule 2
larly argued that responding on intermittent rein¬ was held constant at 20 per hr and that on Schedule 1
forcement schedules is determined by two factors: the varied from 0 to 40 per hr. The straight line and two
relative frequency of reinforcement and the relative curves represent the plot of Equation 15 with k- and
266 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

kr-t
Ri =_*i + r2 + re_Tl
Ri + R2 kri kr2 r1 + r2
- _j-
*i + r2 + re r2 + rt + re

If the asymptotic response rates (k) or the revalues on

the two schedules differ, however, the ^-parameters or
the denominators in Equation 16 will not cancel out,
and matching will not be obtained. Herrnstein’s
formulation of the relation between absolute response
rates and reinforcement therefore predicts deviations
from matching in many cases. For example, in the
paced schedules of Staddon (1968) and Moffitt and
Shimp (1971), the reinforced IRTs were of different
durations and hence the responses would differ in
asymptotic response rate for the same frequency of
reinforcement. They may also differ in their revalues,
Fig. 10. The absolute rate of responding on each schedule as a since more collateral behavior will be generated in
function of the rate of reinforcement for each alternative in a the one case than in the other.
CO-key concurrent VI VI schedule (Catania, 1963b). For the
left panel, the overall frequency of reinforcement was held Similarly, Equation 15 might account for the devi¬
constant at 40 reinforcements per hr, while varying complemen- ation from matching with concurrent VI FI (Nevin,
tarily for the two alternatives. Each point is here plotted above
1971; Trevitt et ah, 1972) and FI FR (La Bounty &
the reinforcement rate at which it was obtained. For the right
panel, the frequency of reinforcement for Schedule 2 was kept Reynolds, 1973) schedules. On FI or FR schedules, the
constant at 20 reinforcements per hr, while varying between value of re probably changes with the differing prob¬
0 and 40 per hr for Schedule 1. The points here are plotted
ability of reinforcement over time. While re should be
above the reinforcement rate on Schedule 1 at the time they
were obtained, The values of K and re were used for the smooth fairly constant over time on a VI schedule, its value
curves in both panels. (Catania 1963b,)
on an FI or FR schedule is likely to be high immedi¬
ately following reinforcement, in the postreinforce¬
ment pause, and low immediately prior to reinforce¬
h>-ValU@£ US Shown in the ftglive. The same parameter ment. Equal ^-values for the two schedules in these
values wer£ Used for the Wcti<?*i§ in both panels, procedures therefore cannot be assumed, as would be
Only one data point deviates substantially from the required for matching. If the differing mean revalues
plotted function, and the equation accounts for 91.0% for each schedule could be ascertained, then the devi¬
of the variance in response rates in the left-hand panel ation from matching should be accounted for by
and 90.3% of the variance in the right-hand panel. Equation 15. However, until the application of Equa¬
Equation 15 was also fitted to the mean absolute tion 11 (Herrnstein's equation for single schedules) is
response rates on each key from Chung and Herrn- extended to FI and FR schedules, this explanation of
stem's (1967) study of choice between different im¬ the deviations from matching must remain specula¬
mediacies of reinforcement. For the pigeons with an tive.
8-sec delay on the standard key, the equation accounts
for 90.5% of the variance in absolute response rate on
Multiple Schedules and Behavioral Contrast
the two keys. The lvalue was 60 responses per min;
fe was .002 (1/delay units). The equation was not fit¬
In Equation 15 two reinforcement schedules are
ted to the data for the two pigeons with a 16-sec assumed to exert their full effect on each response,
standard delay since they showed a consistent bias since the schedules are simultaneously available to the
toward the variable-delay key. These data therefore subject. Numerous operant conditioning experiments,
also provide substantial confirmation for Herrnstein’s however, reveal interactions of a lesser magnitude be¬
system of equations.
tween successive reinforcement conditions and the re¬
Equation 15 predicts matching in concurrent sched¬ sponse rates they maintain.
ules, since the denominators and ^-parameters cancel Reynolds (1961a) first demonstrated that response
out when relative response rate and reinforcement fre¬ rate in the first component of a multiple schedule de¬
quency are calculated: pends not only on the frequency of reinforcement in
Peter de Villiers 267

that component, but also on the reinforcement in the The parameter m varies between 0 and 1.0 and repre¬
second component. He trained one group of pigeons sents the degree of interaction between the two rein¬
on a multiple VI VI schedule and another group on a forcement conditions. In concurrent schedules interac¬
multiple VR VR, component duration for both groups tion is maximal, so m equals 1.0, and Equations 15
being 3 min. In both cases, the rate of responding in and 17 are identical. Matching then follows as shown
the first component of the schedule increased con¬ in Equation 16. The modified equation is therefore
siderably when reinforcement was discontinued in the proposed to account for both matching in concurrent
second component, although the reinforcement fre¬ schedules and behavioral contrast in multiple sched¬
quency in the first component was unchanged. Reyn¬ ules.
olds called the effect “behavioral contrast.” Subsequent Equation 16 accurately describes the stable behav¬
research has confirmed the phenomenon of behav¬ ioral contrast reported in several experiments that in¬
ioral contrast and extended it to different species vestigated a sufficient range of reinforcement frequen¬
even if the responses or the reinforcers in the two cies or durations. For example, Lander and Irwin
components are different in kind (Beninger, 1972; (1968) varied reinforcement frequency in a multiple
Premack, 1969). Reynolds (1961b, 1963) himself dem¬ VI 3-min VI v-min schedule and Nevin (1968) varied
onstrated that a change in the relative reinforcement the duration of nonresponding required for reinforce¬
rate in each component of the schedule is the major ment in the DRO component of a multiple VI 3-min
determinant of stable contrast. As long as reinforce¬ DRO x. Rachlin and Baum (1969) manipulated the
ment in the interacting component was sustained at duration of reinforcement associated with one key
the same level, contrast failed to occur, whether or not from 1 to 16 sec while the VI schedule associated with
the pigeons responded in that component (Reynolds, a second key remained constant. Illumination of the
1961b). Bloomfield (1967a) found that both positive key light signaled the availability of reinforcement on
(increased response rate) and negative (decreased re¬ the key for which reinforcer duration varied; rein¬
sponse rate) contrast were determined by changes in forcement on the constant VI schedule was unsig¬
the relative reinforcement frequencies. Response rate naled. Since the pigeons in Rachlin and Baum’s study
in the VI component of a multiple schedule varied in¬ only responded on the signaled key when the key light
versely with the frequency of reinforcement in the was illuminated—i.e., when reinforcement was avail¬
second component, whether that component was a able—the conditions of stimulation, reinforcement,
DRL or an FR schedule. Only the relative reinforce¬ and responding on the two schedules were successive.
ment frequency and not differences in response rates Their procedure could therefore be considered a mul¬
or patterns of responding on the two component tiple VI 3-min CRF schedule, even though two re¬
schedules had any effect on the direction or degree of sponse keys were used. In all three of these studies,
behavioral contrast observed (Bloomfield, 1967a). Re¬ response rate on the unchanged schedule varied in¬
sponse rate in the presence of a given stimulus is versely with the frequency or duration of reinforce¬
therefore determined by the frequency of reinforce¬ ment for the other schedule. With appropriate param¬
ment during all of the stimuli that successively control eter values selected, the averaged group data from
the subject’s behavior (Reynolds, 1961b). each of the studies never deviates by more than 6 re¬
These results agree qualitatively with Herrnstein’s sponses per min from a perfect fit to Equation 17. For
(1970) general formulation of the law of effect; re¬ 15 of 18 independent data points the deviation of data
sponse rate is directly related to its relative reinforce¬ from theory is less than 3 responses per min (approx¬
ment frequency. In addition, Herrnstein (1970) showed imately 6%) (Herrnstein, 1970).
how a simple modification of Equation 15 could pro¬ The subjects in these experiments were pigeons
vide an accurate quantitative account of many stable and the reinforcer was access to food, but de Villiers
interactions between response rate and reinforcement (1972) extended Herrnstein’s equation for behavioral
in multiple schedules. Since only one schedule oper¬ contrast to rats responding on multiple random-inter¬
ates in each component, interaction between multiple val (RI) avoidance schedules. Lever-press responses
schedules is presumably less than that in concurrent canceled the delivery of shocks scheduled at random
schedules. Therefore, Herrnstein inserted an interac¬ intervals, and the scheduled rate of shock was varied
tive parameter into Equation 15, the equation describ¬ in the two components of the multiple schedule.
ing absolute response rate in concurrent schedules: When shock-frequency reduction (scheduled shock rate
minus received shock rate) was substituted for and
r2 in Equation 17 (Herrnstein, 1969; Herrnstein 8c
rx + mr2 + re Hineline, 1966), the equation provided an accurate
268 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

schedules when inserted into Equation 11, Herrn-

stein’s equation for single schedules. The deviation of
observed from predicted response rate on the single
schedules averaged only 5.5% of the observed response
rate, and was never greater than 3 responses per min.
Equation 17 also makes predictions about relative
response rates on multiple schedules. If m < 1.0, the
predicted relation between relative response rate and
relative reinforcement frequency is plotted by a fam¬
ily of curvilinear functions, the curvature of which
depends on the magnitudes of rlf r2, re> and m (Herrn-
stein, 1970). An empirical test of this relation was
provided by an experiment by Reynolds (1963) which
systematically varied reinforcement rates in both
components of a multiple VI VI schedule. Figure 11
shows the data from Reynolds’s experiment and the
relative response rate curve predicted by Equation 17
for the given parameter values. The value of re was
set to zero for these curves, but the revalues typically
obtained for pigeons on VI schedules would change
the functions only minimally. Once again the theory
provides a good fit to the data.
At this point, however, two limitations on the ap¬
plication of Equation 17 must be noted:

1. The equation not only assumes that a simple multi¬

plicative mechanism governs the degree of inter¬
action between the two reinforcement conditions,
but it also assumes the most symmetrical of multi¬
ple schedules. It applies best to a schedule in which
the response forms are the same in both compo¬
nents—i.e., in which they have equal asymptotic
rates (k), in which the interaction is the same going
from one component to the other in either direc¬
tion (m), and in which extraneous sources of rein¬
forcement are kept constant in the two components
(r&)> Yet behavioral contrast occurs in multiple
schedules in which many or all of these symmetries
are violated (Premack, 1969). Such asymmetries
0 .2 A -6 .8 LO would considerably complicate a quantitative anal¬
ysis of the contrast effects, but it need not change
the framework of the analysis—i.e., a framework
RELATIVE which stresses relative reinforcement frequency (or
REINFORCEMENT RATE magnitude) as a determinant of response strength.
2. Second, the equation is not intended to account for
Fig. 11. The relative rate of responding as a function of the
relative frequency of reinforcement in one component of a short-term contrast effects in multiple schedules
multiple VI VI schedule, for each of three pigeons. The such as those discussed by Terrace (1966a, 1966b,
smooth curves plot Equation 15, with re — 0 and m set to the 1972) or even more transient changes in response
values indicated. (From Reynolds, 1963.)
rates reported by Nevin and Shettleworth (1966)
and Bernheim and Williams (1967). Nevertheless,
Herrnstein (1970) has indicated how Equation 17
quantitative description of the long-term positive and
could be easily modified to handle the contrast
negative contrast effects obtained in this study. The
effects described by Terrace (1966a) by assuming
values of the k- and r6-parameters calculated for each that an extinction component is aversive (Terrace,
rat from the behavioral contrast data even predicted 1966b) and adds a negative value to the denom¬
subsequent response rates on single RI avoidance inator of the equation.
Peter de Villiers 269

Matching in Multiple Schedules must. Equation 17 therefore predicts that a subject’s

performance in a multiple schedule should approach
Nevertheless, within the situations to which it di¬ matching as the motivation for the scheduled rein¬
rectly applies, Equation 17 makes a number of inter¬ forcer declines.
esting predictions. It not only predicts matching in The basic procedure for the entire experiment was
concurrent schedules, but also predicts matching in a multiple VI 1-min VI 4-min schedule with 2-min
multiple schedules under particular conditions. components. The body weight of the pigeons was
First, if the reinforcement frequency in one com¬ systematically varied from 80 to 110% of their ad
ponent of the multiple schedule became extremely libitum weight at the beginning of the experiment.
large relative to re and the scheduled reinforcement Finally, the pigeons responded on the multiple sched¬
in other components, the denominators of the equa¬ ule with a cupful of food in the experimental cham¬
tions governing response rate in each of the compo¬ ber. For all five pigeons, absolute response rates de¬
nents would approach equality. When the denomina¬ clined steadily with increasing body weight. At the
tors are equal, matching is predicted (Equation 16). same time, relative response rate in each component
Data reported by Nevin in 1974 (cited by Herrnstein, asymptotically approached the matching value (.80
1970) test this prediction. Nevin studied pigeons in a and .20, respectively). Two pigeons matched at 100%
three-component multiple schedule. Two of the com¬ body weight and three pigeons matched at 110% body
ponents were conventional VI 1-min and VI 3-min weight. With free food in the chamber, response rates
schedules, each lasting for 1 min, and the third com¬ dropped to low levels, between 5 and 15 responses per
ponent consisted of a 30-sec blackout of the chamber min; but the relative response rates for all five pigeons
separating the two VI components. The independent closely approximated matching.
variable was the frequency of noncontingent rein¬ Finally, Equation 17 predicts matching in multiple
forcement provided during the blackout. As predicted schedules when m approaches 1.0—i.e., when the in¬
by Equation 17, response rates in the two VI compo¬ teraction between the two components is maximal, as
nents decreased steadily as reinforcement frequency in concurrent schedules. The more the two reinforce¬
in the blackout increased from zero to 360 per hr ment conditions are separated in time by long com¬
(Herrnstein, 1970). Furthermore, relative response ponent durations, the less interaction there is likely
rates in the VI components approached matching: .75 to be between them, the smaller the contrast effect,
for the VI 1-min and .25 for the VI 3-min. Averaged and the greater the deviation from matching. On the
over the four pigeons, relative response rate in the VI other hand, the more the multiple schedule approx¬
1-min component went from .54 when there was no imates a CO-key concurrent schedule with rapid al¬
reinforcement in the blackout to .72 when the black¬ ternations between the two reinforcement conditions,
out reinforcement frequency was 360 per hr—i.e., 6 the larger the contrast effects and the closer the ap¬
times the combined rate of reinforcement in the VI proximation to matching predicted by Equation 17.
components. The denominators in Equation 17 for Experiments by Shimp and Wheatley (1971) and
the two VI schedules therefore became more and more Todorov (1972) provide a direct test of this predic¬
determined by the frequency of reinforcement in the tion. They varied component duration in a multiple
blackout. VI VI schedule with asymmetrical reinforcement fre¬
Second, similarly, if the values of r1 and r2 became quencies. As component duration shortened, relative
extremely small relative to re, reinforcement from ex¬ response rate increased for the richer VI and decreased
traneous sources in the two components, the denom¬ for the other schedule. At the longer component dura¬
inators in Equation 17 for each component would be tions (>10 sec), relative response rate undermatched
determined by re. As the denominators approach relative reinforcement frequency on the richer VI, but
equality, relative response rates should also approach with very brief components of 5 to 10 sec the relative
matching. response rate reached its maximal value and matching
Herrnstein and Loveland (1974) manipulated the was observed. With 5-sec components, Shimp and
relative importance of rx and r2 by varying their Wheatley obtained matching to a wide range of rela¬
pigeons’ body weights. They argued that it is unneces¬ tive reinforcement frequencies in the two components.
sary to know the specific reinforcers that constitute re In a similar procedure, but with rats as subjects
in order to conclude that re should become a larger and multiple VI avoidance schedules, de Villiers
fraction of the denominator as the pigeons get less (1974) demonstrated the same relation between rela¬
and less hungry. Whatever reinforcers re contains, tive response rate and relative frequency of negative
they should not covary with hunger the way rx and r2 reinforcement (shock-frequency reduction) as compo-
 270 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

where R/ is response rate after behavioral contrast.

The magnitude of behavioral contrast (percentage
increase in responding in the unchanged component)
is then calculated as follows:

kry kry
Ri - Ri r, + re rt + mrx + r6
ri X 1UU — -- X 100
Ri krx
r, + mr1 + re

mr1
Fig. 12. The relation between relative response rate and relative
X 100 (20)
ri + re
reinforcement frequency for three pigeons responding on multi¬
ple VI VI food schedules with 5-sec components (Shimp &
According to this equation, the magnitude of behav¬
Wheatley, 1971) (left-hand panel); and for three rats responding
on multiple VI VI avoidance schedules with 40-sec components ioral contrast increases as increases, assuming that
(nght-hand panel), The diagonal lines represent matching be¬ m and re are greater than zero and remain constant
tween the relative frequencies, (de Villiers, 1974.)
across the experimental conditions.
Spealman and Gollub (1974) tested this prediction
nent duration varied. At the longer component dura¬ with eight pigeons. Four of the pigeons responded on
tions, relative response rate undermatched the relative a multiple VI 30-sec VI 30-sec schedule, the other four
shock-frequency reduction; but as component dura¬ on a multiple VI 180-sec VI 180-sec schedule, The
tion shortened, the relative response rate increased components of the multiple schedules alternated every
until it was maximal at 40-sec components. At this 180 seconds. The second-component schedule for each
component duration matching was obtained for three group of pigeons was then changed to extinction.
different relative shock-frequency reduction values. Contrary to the prediction of Herrnstein's equation,
Figure 12 summarizes the matching data from Shimp the magnitude of behavioral contrast for the group
and Wheatley (1971) for positive reinforcement and with less frequent reinforcement (VI 180-sec) was
from de Villiers (1974) for negative reinforcement. greater than the contrast shown by all but one of the
pigeons on the VI 30-sec. However, a critical assump¬
Contrary evidence tion underlying the experiment is that the mean val¬
ues of 77i and Te do not differ across the two groups of
Certain limitations of Herrnstein's equation for pigeons, and this assumption may not be justified. It
multiple schedules have already been discussed. In is plausible to suggest that extraneous sources of rein¬
addition, recent experiments indicate that several of forcement (re) may be larger with less frequent con¬
its predictions are not supported by the data. tingent reinforcement and that the interaction be¬
As Spealman and Gollub (1974) pointed out. Equa¬ tween components (wi) may be greater with more
tion 17 predicts that the higher the reinforcement rate frequent reinforcement. But both of these effects
in an equal-valued multiple VI VI schedule, the would work in favor of the prediction by Equation 17
larger the behavioral contrast found in the unchanged —greater behavioral contrast with more frequent rein¬
component when the second component is changed to forcement—and against the observed results.
extinction. When reinforcement frequencies in the Another prediction based on Equation 17 is that
two components are equal, r, = r2 and Equation 17 response rate on a multiple VI VI schedule will be
can be rewritten as lower than that on one of the VI schedules in isola¬
tion. Parameter R/ in Equation 19 represents response
kr1
rate on a single VI schedule (cf. Equation 11), and
(18)
r, + + re Ri > Ri in Equation 18 if m > 0. Herrnstein (1970)
cited unpublished data from Terrace showing that
If reinforcement is no longer scheduled in the second pigeons’ response rates on a single VI 60-sec schedule
component, response rate in the first component is were higher than their response rates on a multiple VI
governed by the following equation: 60-sec VI 60-sec schedule. De Villiers (1972) also found
higher response rates on a single VI 15-sec avoidance
kr1
schedule than in either component of a multiple VI
Ri' (19)
*1 + re 15-sec VI 15-sec avoidance schedule with rats. How-
Peter de Villiers 271

Table 3 Mean absolute response rate in each component of the multiple schedule as a
function of component duration, along with mean relative response rates and
relative reinforcement frequencies for the component with more frequent rein¬
forcement, for the studies of Shimp and Wheatley (1971) and Todorov (1972)

SHIMP AND WHEATLEY (1971)

Component Relative Relative
Duration Responses) min Response Reinforcement
(in sec) VI1-min VI 4-min Rate in VI1-min Rate in VI 1-min

180 53.1 25.4 .68 .81

60 48.8 19.8 .71 .80
30 52.7 19.8 .73 .79
10 67.5 23.9 .74 .80
5 83.2 19.7 .81 .80
2 90.1 27.1 .77 .81

TODOROV (1972)
Component Relative Relative
Duration Responses) min Response Reinforcement
(in sec) VI 30-sec VI 90-sec Rate in VI 30-sec Rate in VI 30-sec

300 47.3 31.9 .60 .74

150 52.7 32.1 .62 .75
40 52.9 26.8 .66 .75
10 57.2 23.6 .71 .75
5 58.4 27.0 .68 .75

ever, in a second experiment Spealman and Gollub sults, since m should be smaller with longer compo¬
(1974) found that neither the pigeons on the VI 30-sec nents; yet Spealman and Gollub obtained sizable
nor the pigeons on the VI 180-sec schedules responded contrast effects.
faster on those schedules in isolation than on multiple Finally, there is strong evidence that the process by
schedules with the same Vis as components. which matching develops in multiple schedules with
One possible explanation of Spealman and Gol- short components is different from that predicted by
lub’s results is that m had gone to zero by this stage of Equation 17. Provided that k and re do not change,
their study. The single VI schedule was the fourth the equation predicts that as component duration
experimental condition, preceded and followed by a shortens and m approaches 1.0, response rates in both
multiple VI VI schedule condition. Since the pigeons components should decrease. Lowest response rates
were run for long periods of time in each condition, as should occur at the component duration where inter¬
long as 70 30-min sessions, and the components of the action is greatest and matching is obtained. Table 3
multiple schedule were signaled by red versus green summarizes the average mean response rates in each
key lights throughout, m may well have approached component of the multiple schedule for the pigeons
zero by the end of the experiment. Equal rates of re¬ studied by Shimp and Wheatley (1971) and Todorov
sponding would then be expected for the single and (1972). As component duration decreased, so did re¬
multiple VI schedules. But if m = 0 in the last multi¬ sponse rate in the component with less frequent rein¬
ple-schedule condition of the experiment, response forcement, until it reached its lowest value at the
rate in that condition should be higher than that in component duration where approximate matching
the first multiple-schedule condition, where m was was found. In contrast, response rate in the compo¬
clearly greater than 0 (since substantial behavioral nent with more frequent reinforcement increased with
contrast was obtained). For only three of the seven decreasing component duration, and was highest at
pigeons run in the last multiple-schedule condition is the shortest duration. De Villiers (1972) found no
that the case; for another three pigeons response rates clear relation between absolute response rates and
are lower. Differences in component durations—180 component duration in his experiment on multiple
sec in Spealman and Gollub’s study versus 90 sec in VI avoidance schedules, but his results were affected
Terrace’s—might also explain the discrepancy in re¬ by prolonged adaptation of the animals to the electric
 272 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

shock. This analysis and Spealman and Gollub’s re¬ In a second experiment, however, Killeen (1972a)
sults therefore severely question the adequacy of showed that the response matching found in multiple
Equation 17 as an account of the interactions between VI schedules with brief components (Shimp & Wheat-
response rate and reinforcement frequency in multi¬ ley, 1971; Todorov, 1972) is different in kind from
ple schedules. that found in concurrent VI schedules. If component
durations are fixed at equal values while reinforce¬
ments are programmed by unequal VI schedules in
AN ALTERNATIVE THEORY OF MATCHING the two components, the subject can no longer equal¬
AND BEHAVIORAL CONTRAST ize local reinforcement frequencies. The distribution
of time to the two schedules is fixed by the experi¬
In a recent paper, Rachlin (1973) raised two ques¬ menter so the local reinforcement rates approximate
tions about the adequacy of Equation 17 in account¬ those set by the VI schedules in operation in each
ing for both behavioral contrast and matching. He component. Local response rates therefore deviate
argued first that the equation implies that the same from equality in the two components. With brief
process underlies matching in both multiple and con¬ components, response rate-reinforcement interaction
current schedules. Yet Killeen (1972a) found clear is maximal and the relative local response rate matches
differences between the patterns of behavior that lead the relative local frequency of reinforcement provided
to matching in the two procedures. by the two VI schedules.
In one experiment, Killeen divided his subjects From Killeen’s results, Rachlin (1973) concluded
into yoked pairs. Each pair contained a leader pigeon that response matching in concurrent schedules re¬
that was exposed to a CO-key concurrent schedule. sults from the subject's distribution of time. Extra
Follower pigeons were yoked to the leaders so that responses on the preferred schedule that lead to
the schedule in operation on the main key for these matching come from the extra time spent responding
pigeons was determined by the changcovers of the there, not from a higher local response rate. On the
leaders. Whenever a reinforcement was programmed other hand, matching and behavioral contrast in mul¬
for a response by the leader, it was also available for tiple schedules are indicative of an increased response
the follower. Killeen observed that leader and fol¬ output when the relative reinforcement frequency is
lower pigeons produced very similar patterns of re¬ increased. Equation 17 fails to distinguish between
sponding, although the followers were on a true these two processes.
multiple schedule and had no control over alternation Second, Rachlin argued that the equation does not
between the schedules. The absolute response rates account for both inhibitory and excitatory effects of
were a little higher for the leader birds, but both reinforcement on responding. Rachlin and Baum
groups of pigeons matched relative overall response (1972) found that free reinforcement scheduled at
rates to relative reinforcement frequency. The leader variable intervals inhibited key pecking maintained
pigeons al§o matched the time they allocated to each by a constant frequency of contingent reinforcement,
schedule to the distribution of the reinforcements. in accordance with Herrnstein’s equations. In sharp
The local rates of reinforcement in the concurrent contrast with these results, several recent experiments
and yoked conditions were therefore constant across have demonstrated excitatory effects of free reinforce¬
the two component schedules. Killeen noted that the ment on a pigeon’s key pecking.
local response rates were also equal in both condi¬ For example, Staddon and Simmelhag (1971) pre¬
tions so that relative local response rate matched sented food reinforcement to a pigeon at fixed time
relative local reinforcement frequency at .50. He con¬ intervals, irrespective of the bird’s behavior. They
cluded that in concurrent schedules the behavior ob¬ observed that after exposure to this procedure for a
served is not so much response matching as it is short period of time, pecking (at either the key or the
equalizing. The pigeon adjusts the time spent in each walls of the chamber) began to predominate just prior
component schedule so that the local reinforcement to food presentation. Staddon (1972) argues that stim¬
rates are equal. Overall relative response rate match¬ uli signaling the imminent presentation of food elicit
ing therefore results from the more basic process of pecking (as in autoshaping procedures—Brown Sc
time allocation. If this equalizing process is performed Jenkins, 1968; Gamzu Sc Williams, 1971), even if peck¬
for, rather than by, the subject, as it was for the yoked ing has no effect on food presentation. Thus a base
pigeons, equal local response rates and overall re¬ line response of key pecking is facilitated by a brief
sponse matching will be found even in multiple sched¬ light stimulus signaling free food (LoLordo, 1971).
ules. Herrnstein and Loveland (1972) and Williams and
Peter de Vi fliers 273

Williams (1969) report that a pigeon will peck at a inforcement alternates with a signal for low reinforce¬
brief stimulus signaling food even if the peck avoids ment rates, it elicits pecking in pigeons in both multi¬
or postpones the food presentation. There appears to ple schedules and autoshaping procedures.
be a special relation between pecking and food for a Subsequent experiments have determined some of
pigeon such that pecking becomes the dominant re¬ the conditions in which autoshaped pecking is pro¬
sponse in the bird’s repertoire and tends to be emit¬ duced in the Gamzu and Schwartz procedure. Redford
ted when food is imminent. Other responses that can and Perkins (1974) found that pigeons produced sub¬
be maintained by food reinforcement, such as treadle stantial rates of key pecking in the component as¬
pushing, do not show the same special linkage to food sociated with the higher relative frequency of free
for pigeons. LoLordo (cited by Staddon, 1972) reports reinforcement only if the color stimuli signaling the
that a base line of treadle pushing is suppressed by a two components were localized on the key. Substantial
light stimulus for free food because the pigeons peck positive contrast was also found with pigeons pecking
at the signal key. Other research suggests that the on a conventional multiple VI VI schedule when the
special relation between pecking and food is species- schedule changed to multiple VI EXT (extinction)
specific for pigeons and that other responses may be only if the component stimuli were localized on the
linked to food reinforcement for other species (Bre¬ key. When the components of the multiple schedule
land 8c Breland, 1961). in both procedures were signaled by a change in the
Staddon and Simmelhag (1971) therefore define a color of the houselight, none of Redford and Perkins’s
class of “terminal responses” that bear a special rela¬ pigeons showed positive contrast or autoshaped peck¬
tion to a particular reinforcer. These responses are ing. This result is in keeping with an autoshaping
excited by the mere presence of the reinforcer or by account of positive contrast, since autoshaping of key
stimuli that signal its imminent presentation, and do pecking in standard procedures can only be obtained
not need to be selected for by the reinforcement con¬ when the prefood stimulus is located on the response
tingencies. Different terminal responses may be linked key (Wasserman, 1973). However, it should be noted
to different reinforcers for a particular species. On the that Redford and Perkins failed to get differential re¬
other hand, “interim responses” are not directly re¬ sponding in the VI and EXT components of the
lated to the reinforcer and tend to be emitted during multiple schedule when these were signaled by a
periods of low reinforcement probability. change in the color of the houselight. They ran only
An experiment by Gamzu and Schwartz (1973) first five multiple VI EXT sessions, and those alternated
raised the question of the relation between response with VI VI sessions. If their pigeons failed to dis¬
elicitation in the presence of reinforcements and be¬ criminate the change in reinforcement conditions in
havioral contrast. They presented free reinforcement the EXT component, behavioral contrast would not
to pigeons in both components of a multiple schedule. be expected to occur in the other component.
A color change of the key light signaled each compo¬ Schwartz (1973) demonstrated that key pecking is
nent, but pecking the key had no programmed conse¬ not elicited in the Gamzu and Schwartz procedure if
quences. When the rate of free reinforcement was the schedule components are signaled by the presence
equal in the two components, no key pecking oc¬ and absence of an auditory stimulus. Once key peck¬
curred. But when the rate of reinforcement in one ing was established with a change in key color sig¬
component was reduced, the pigeons began to peck naling the components, however, it could be trans¬
the key in the other component. Frequency of rein¬ ferred from the control of the visual stimulus to a
forcement in that component was unchanged, but it tone, although it was maintained at a somewhat re¬
was now associated with a higher relative reinforce¬ duced level. Subsequently, the pecking could be reini¬
ment rate. Gamzu and Schwartz observed that the tiated by the tone signal after an extinction procedure.
number of responses made in the unchanged compo¬ Schwartz suggests that for pigeons preexperimental
nent approximated the increase in responding found relations exist among food, visual stimuli, and peck¬
in normal contrast experiments using similar rates of ing, but not for auditory stimuli. Thus a pigeon’s key
contingent reinforcement. They therefore suggested peck cannot be autoshaped with an auditory prefood
that normal positive behavioral contrast consists of stimulus.
the instrumental responding appropriate to the sched¬ The results of two experiments by Keller (1974—
ule in the unchanged component, plus the extra pecks cited by Rachlin, 1973) also support an autoshaping
elicited by the stimulus signaling a higher relative analysis of positive contrast. Pecking on one key (in¬
rate of reinforcement in that component. If a stimulus strumental key) in the experimental chamber pro¬
associated with a high frequency or probability of re¬ duced reinforcement according to a multiple VI 30-sec
274 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

EXT schedule. The stimuli associated with each com¬ tive value (in most cases the relative reinforcement
ponent of the schedule, however, were presented on a frequency) of the component schedules.
second key (signal key). Pecking at either key in the To the extent that Rachlin emphasizes relative
VI component produced a feedback click; pecking in reinforcement frequency as the major determinant of
the EXT component produced no feedback. After matching and contrast, his analysis is similar to Herrn-
training on the multiple VI 30-sec EXT schedule, the stein’s; but Rachlin stresses the differences between
pigeons were run on a multiple VI 30-sec VI 30-sec the underlying response rate and reinforcement inter¬
schedule until responding was stable in both compo¬ actions in multiple and concurrent schedules. His
nents. When reinforcement in the second component account of matching in multiple schedules is sup¬
was again discontinued, only one of the three pigeons ported by the analysis of the response rate data from
showed temporary behavioral contrast on the instru¬ Shimp and Wheatley (1971) and Todorov (1972) given
mental key in the VI 30-sec component; the other two on p. 271, while Herrnstein’s account is not. As the
pigeons showed no contrast on the instrumental key interaction between the components increases, Rach¬
but began to peck the signal key in the unchanged lin suggests that the pigeon’s terminal response for
component. food—pecking—should be facilitated in the period of
In Keller’s second experiment there were three high relative reinforcement frequency and more and
components in the multiple schedule, each associated more inhibited in the period of low relative reinforce¬
with a different color key light on the signal key. The ment frequency. In fact, peck rate increased in the
pigeons were trained on a multiple VI 1-min VI 1-min component with higher reinforcement frequency and
EXT schedule, and then the component schedules decreased in the other component, with decreasing
were varied, They were either a VI 1-min or EXT. component duration in both studies.
None of the pigeons showed behavioral contrast on However, Rachlin’s theory of the processes produc¬
the instrumental key during a VI component when ing the two kinds of response matching is itself inade¬
one or two of the other components was EXT; all quate in several respects. Response matching not only
three pigeons pecked at a substantial rate on the sig¬ occurs in concurrent VI VI schedules where local re¬
nal key during the VI components but not during the sponse rates are equal in each schedule. It is found in
EXT components, Signal key response rate was high¬ concurrent VI VR schedules in which the local rates
est when only one component was a VI. Rachlin differ considerably on the two schedules and the
(1973) concludes that i£ the signal and instrumental subject cannot equalize both local response rates and
pecks were superimposed on a single key, as in normal reinforcement frequency (Herrnstein, 1970, 1971;
multiple schedules, positive behavioral contrast would Herrnstein & Loveland, unpublished data). It is also
have been observed. obtained in discrete-trial concurrent schedules where
In view of these results, Rachlin (1973) suggests time allocation applies only arbitrarily (Nevin, 1969;
that whereas response matching m concurrent sched¬ Herrnstein, unpublished data). Hence more direct re¬
ules is a by-product of a basic process of time alloca¬ sponse matching occurs in concurrent schedules than
tion, matching and contrast in multiple schedules Rachlin’s theory implies.
result from more direct excitatory and inhibitory Furthermore, the generality of Rachlin’s explana¬
effects of relative reinforcement frequency on respond¬ tion of behavioral contrast in multiple schedules is
ing. A stimulus associated with a higher relative rein¬ questionable. The results of an unpublished experi¬
forcement frequency elicits extra pecks, while a stim¬ ment by Bouzas question the generality of Keller’s
ulus associated with a lower relative frequency inhibits findings. Bouzas trained six pigeons on a multiple VI
pecking. In more general terms, Rachlin argues that 1-min VI 1-min schedule with 30-sec components dur¬
“terminal” (Staddon 8c Simmelhag, 1971) or auto¬ ing which the instrumental key in the chamber was
shaped responses are excited during periods of high always white. In one experimental condition the com¬
relative reinforcement frequency, while “interim” re¬ ponents were signaled by a color change on a second
sponses are inhibited. During periods of low relative (signal) key (i.e., Keller’s procedure); in a second con¬
reinforcement frequency the opposite process occurs: dition the components were signaled by the presence
terminal responses are inhibited and interim responses or absence of a tone. The six pigeons were studied in
excited. Hence both positive (excitatory) and negative both conditions, three receiving Keller’s procedure
(inhibitory) contrast effects occur. Matching is ob¬ first and three receiving the tone condition first.
served in both multiple and concurrent schedules be¬ When Bouzas discontinued the reinforcement in the
cause the two different processes that produce it are second component, five of the six pigeons showed posi¬
reactions to the same independent variable—the rela¬ tive contrast in the first component in the tone condi-
Peter de Villiers 275

tion. In the Keller procedure, four of the six pigeons reinforcement. But it has become clear from the
showed positive contrast on the instrumental key in studies discussed here that no one analysis of behav¬
the absence of any pecking at the signal key. A fifth ioral contrast can handle all of the phenomena usu¬
pigeon did peck at the signal key, but did not show ally subsumed under that term. We must distinguish
contrast even when those extra pecks were added on between short-term, transient effects that may be emo¬
the instrumental key. Bouzas then studied three of the tional in origin (Terrace, 1966b, 1972) and long-term,
pigeons in the Keller procedure with shorter compo¬ stable interactions between successive reinforcement
nent durations. The birds were trained on a multiple conditions (Bloomfield, 1967b; Herrnstein, 1970).
VI 1-min VI 1-min schedule with 10-sec components, Other contrast effects seem to be related to excitatory
and then switched to a VI 1-min VI 3-min. None of effects of some reinforcers on particular responses
the pigeons showed contrast on the instrumental key (Gamzu & Schwartz, 1973; Staddon, 1972), and may
in this procedure; but two of the three birds pecked even be species-specific. Indeed, there is no reason to
the signal key a substantial amount during the VI believe that all of these contrast effects can be ex¬
1-min component. Adding these extra pecks to re¬ plained by the same process; many different phenom¬
sponse rate on the instrumental key produced posi¬ ena may be involved.
tive contrast for these two birds. But the difference in
results between the studies of Keller and Bouzas can¬
not be explained by component durations; Keller DISCUSSION
used 1-min and 2-min components, longer than either
of the values used by Bouzas. Frequency of reinforce¬ Preceding sections of this chapter have considered
ment could be a factor, as only one of Keller’s three a wide range of choice procedures in which a simple
pigeons in his first experiment pecked at the signal matching relation holds between relative performance
key when he changed from a multiple VI 1-min VI measures and relative measures of reinforcement.
1-min to a multiple VI 1-min EXT schedule. Bouzas They have shown how a quantitative theory of re¬
used VI 1-min schedules. On the other hand, Keller sponse strength derived largely from the matching
obtained signal key pecking from all three pigeons in relation can account for a remarkable range of data
the three-component multiple VI VI EXT schedule on response rate-reinforcement interactions in single,
in which VI l-min components were used. multiple, and concurrent schedules. But they have
The contrast effect found by Bouzas for the pigeons also indicated that some choice procedures do not pro¬
with the tone signal is also surprising in terms of an duce the matching relation (Moffitt Sc Shimp, 1971;
autoshaping analysis of contrast, since a pigeon’s key Nevin, 1971; Staddon, 1968; Trevitt et al., 1972; White
peck can only be autoshaped with visual and not audi¬ 8c Davison, 1973). To what extent do these results
tory stimuli. Similarly, LoLordo et al. (1974) found weaken the matching relation as a quantitative law of
that a brief stimulus signaling free food only facili¬ choice in concurrent schedules?
tated the base line key pecking if a visual signal was Rachlin (1971) has argued that despite these em¬
presented on the response key. An auditory prefood pirical data, the matching relation “is not an em¬
stimulus did not facilitate responding. But West¬ pirical law but a statement of assumptions made prior
brook (1973) also obtained positive contrast for six to empirical test” (p. 249). It derives directly from
pigeons when the components of the multiple sched¬ our intuitions about unconstrained choice and as such
ule were signaled by white noise versus a tone. The it is not subject to disproof by empirical data. The
contrast effects found with auditory stimuli signaling theoretical matching law as formulated by Rachlin
components seem to be smaller than those found with states that
visual stimuli on the key, but they are not readily
explained by the autoshaping theory of contrast. (For
a detailed discussion of the strengths and weaknesses (21)
of an autoshaping account, including a more detailed
review of Keller’s results, see chapter 3 by Schwartz
and Gamzu in this volume.) (22)
In brief, a large body of data on stable behavioral
contrast and matching in multiple schedules can be where T represents the time allocated, R the number
accounted for by. a particular view of response strength of responses made, and V the value of reinforcement
—namely, that the strength of a response is directly consequent on each alternative; r is the reinforcement
related to its relative frequency (or magnitude) of frequency, a the amount of reinforcement, i the im-
276 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

mediacy of reinforcement, and % any other parameters ing relation, not any intuitions about choice, that
of reinforcement for the two alternatives. brought to the attention of researchers in operant
Rachlin believes that the matching law embodied conditioning the utility of the concurrent schedule
in Equations 21 and 22 is a tautology or analytic procedure and the advantages of a relative measure of
statement about how reinforcing value should be behavior in quantifying the law of effect. The value
measured. Wherever there are deviations from the of an empirically based matching law lies in the ex¬
simple matching relation (Equation 1), transforma¬ tent to which it makes valid predictions in new situa¬
tions on the independent variable dictated by ob¬ tions, as well as in its utility for ordering data and
tained time (or response) ratios can make the law fit pointing out anomalies. As Harre (1960) noted: “It is
the data. A difference between programmed and only when we begin to apply the law to cases further
obtained reinforcement (Premack, 1965) or the ac¬ and further removed from the cases upon which it
tions of unknown reinforcers (i.e., x1 and in Equa¬ was based that we begin to run serious risks of a con¬
tion 22) can always be invoked to minimize the trary case appearing and disconfirming the law” (p.
deviation from matching. In these ways the theoreti¬ 154). Three possibilities are open to us when the in¬
cal matching law cannot be empirically disproved or evitable counterinstances to the matching law are
supported. For Rachlin the value of the matching law found (Harre, 1960):
lies in specifying the assumptions made in any choice One, we can uphold the law and note the contrary
experiment and circumscribing our search for hidden case. “This will be the appropriate action when there
reinforcers so that matching will hold in choice situa¬ are various doubts in the investigator's mind about
tions,
the control of extraneous factors in the experimental
A recent theoretical paper by Killeen (1972b), how¬ set up” (Harre, p. 154). I have argued that this is the
ever, raises several arguments against Rachlin's ap¬ appropriate action in the case of studies by Schmitt
proach to the matching relation, Killeen argues first (1974) for humans and Trevitt et al. (1972) for pi¬
that the empirical matching relation (Equation 1) geons, and in the case o£ several experiments where
must be distinguished from Rachlin/s theoretical undermatching probably resulted from a GOD dura¬
matching law (Equation 21). The empirical matching tion that was too short.
equations clearly are not examples of a law "not sub¬ I wo, we may merely state the limitations on the
ject to empirical test,” since concurrent schedule law (e.g., Boyle’s law does not hold at very high pres¬
experiments could well have found some other rela¬ sures). In the present chapter I have stated some
tion between relative performance measures and rein¬ limitations on the application of the matching rela¬
forcement. On the other hand, Rachlin’s matching tion. Matching does not hold m the absence of a COD
law is indeed a tautology, since it assumes that time or when the COD is too short. Kulli’s unpublished
ratios define value ratios. But Killeen suggests that
experiment showing matching in the absence of a
thi§ only gsnsratss a redundant intervening variable GOD at low drive levels (p. 244) suggests that rate of
—i.e., value. Since time ratios must equal value ratios, alternation between the schedules, and not the COD
the former could be substituted for the latter when¬ procedure per se, is the crucial factor. If the rate of
ever they occur, and the notion of value adds nothing alternation is too high, responses to one alternative
to Rachlin’s analysis. "The generality of Rachlin's come under partial control of the other reinforcement
law is uninteresting since it can be obtained in any schedule.
situation where one cares to postulate an intervening White and Davison (1973) have suggested that the
variable equal to the data in question” (Killeen, 1972b, matching relation fails to hold when different patterns
p. 491). of responding are generated by the two schedules—
Rachlin asserts that Equation 21 is “derivable from e.g., on concurrent VI FI schedules (Nevin, 1971;
the assumption that an organism choosing between Trevitt et al. 1972) or on concurrent paced schedules
alternatives is under no constraints except those the (Staddon, 1968; Moffitt Sc Shimp, 1971). In these cases
contingencies of reinforcement impose” (p. 249); it the schedule contingencies tend to constrain response
codifies our intuitions about choice. However, like rates on the two schedules. When the patterns of re¬
Killeen (1972b), I do not see how this must follow, as sponding are more similar—e.g., on concurrent FI FR
many other relationships are possible. A far more schedules—the deviation from matching is much
complicated relation could hold between responding smaller (LaBounty 8c Reynolds, 1973); and when the
or time allocation and the reinforcement variables; response patterns are the same on the two keys, even
why not T1 — T2 — V1 — V2 (see Lea 8c Morgan, 1972)? though responding is paced by the reinforcement con¬
Moreover, it was the success of the empirical match¬ tingencies (Shimp, 1971), matching is obtained.
Peter de Villiers 277

An important goal of future research will be the quencies seems to be different for choice between two
determination of further limitations on the matching Vis and choice between a VI and an FI schedule.
relation. It will then be necessary to explain why Staddon (1968, 1972) has argued that a power func¬
matching does not hold in particular situations (see p. tion relation between behavior and reinforcement
266) and to assess the power of the matching law in ratios—i.e., a specific case of Killeen’s equation—ac¬
terms of the range of situations for which it gives an curately describes all of the present data on choice in
accurate quantitative description of choice. concurrent schedules:
Third, we may attempt to “formulate a new gen¬
eralization which will include all the results obtained
Ti Ri rf
under all sets of conditions that have been investi¬ — or — = k— (24)
gated” (Harre, p. 155). For example, Baum (1974a) T2 R2 r2w

generalized the matching equation to account for bias

or inherent preference for one alternative. In this way where k is a parameter indicating any systematic bias
he could account for data previously thought to con¬ toward one schedule. Equation 24 reduces to propor¬
tradict matching (e.g., Fantino et al., 1972). However, tional ratio matching when the exponent is 1.0; but it
Baum’s proportional ratio matching (Equations 4 and also accounts for choice on paced schedules, concur¬
5) does not handle “results obtained from all sets of rent VI FI and concurrent FI FR schedules in which
conditions [of choice] that have been investigated.” exponents of less than 1.0 were obtained. Staddon
More general mathematical formulations of choice in (1972) suggests that the power function formulation
concurrent schedules have therefore been suggested is required in situations where there is an intrinsic
to replace the simple matching relation. preference for one of the responses or schedules of
Killeen (1972b) proposed that an additive differ¬ reinforcement apart from the frequencies of rein¬
ence model of utility (Tversky, 1969) be considered a forcement associated with each.
“working hypothesis” of the way in which different Although it handles all of the data discussed in
parameters of reinforcement determine choice: this chapter, the implications of the power function
formulation must be further specified before its value
relative to the simpler matching relation can be as¬
II ^1- . MM) # MM ' fijxi)
T2 °r R2~ /j(r2) f2(a2) f3(i2) f4(x2) ' sessed. For example, the factors that lead to differ¬
ences in the power function exponent have yet to be
specified. In some situations in which subjects have a
where T, R, r, a, i, and x are as specified for Equa¬ strong bias toward one or another of the reinforcers
tions 21 and 22. This equation states that choice can the exponent is near 1.0—i.e., the matching exponent
be predicted by a particular concatenation of subjec¬ (Hollard 8c Davison, 1971). Bias or intrinsic preference
tive scales of reinforcement. In the context in which accounts for deviations in the intercept of the choice
each scale is derived, Equation 23 is true by definition, function from (0, 0), but not for differences in the
since the scales are defined by the choice behavior. exponent (Baum, 1974a). Furthermore, while Herrn-
But the equation becomes empirically falsifiable when stein (1970) has extended the matching relation to
extrapolations are made to new choice situations or account for absolute as well as relative response rates,
when several subjective scales are combined (Killeen, the implications of a power function formulation for
1972b). It also assumes that there are no interactions absolute response rates (except where it reduces to the
among the subjective scales, and this too is empiri¬ matching relation) are unclear.
cally testable. Finally, as discussed earlier, Herrnstein’s extension
Both the strength and weakness of Killeen’s formu¬ of the matching relation to absolute response rates
lation lie in its great generality. Current research in¬ may be used to account for some deviations from
dicates that a simple linear scaling of reinforcement matching. For example, if values of k for two differ¬
produces matching for a number of different rein¬ ent response forms were previously determined in
forcement parameters. Then a far more parsimonious single schedules, it should be possible to predict the
matching law than Equation 23 is possible. If the deviation from matching that would be observed in a
more general formulation embodied in Equation 23 concurrent schedule using those two responses (i.e.,
does prove necessary to describe choice in concurrent provided that the revalues were not too dissimilar for
schedules, then the variables that affect each reinforce¬ the two responses). There has been little quantitative
ment function will need to be specified. For example, investigation of choice between different response
the function relating time ratios to reinforcement fre¬ forms in concurrent schedules.
278 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

While there is thus general agreement about the k — 12.4 (responses/min)

empirical relation between response strength and rela¬ re= 10.1 (mg of chow)
99.6%
tive reinforcement, there is yet to be a consensus on
the best mathematical form for expressing it. It is Beier (1958): Running speed (100/time in sec) in a straight
alley for three numbers of pellets between 1 and 13
only as the implications and predictions of each formu¬
pellets. Groups of 6 rats per value.
lation are specified and empirically tested that their
k = 68.0 (running speed units)
relative merits can be evaluated.
re = .67 (pellets)
99.4%
Logan (1960): Running speed (100/time in sec) in a straight
CONCLUSION alley for four different numbers of food pellets between
1 and 12 pellets. Groups of 6 rats per reinforcement
value and drive level.
In 1971, Herrnsteln prefaced a paper entitled
12 hr deprived
"Quantitative Hedonism7' with the statement; “The
k = 45.1 (running speed units)
occasion for this article is the conviction that a pre¬ re — .15 (pellets)
cise statement of reinforcement may he at hand, 90.0%
growing out of the discovery that frequencies of ah 48 hr deprived
tentative forms of behavior occur in the same propor¬ k = 51.0 (running speed units)
tion as the resulting reinforcements” (p, 400), Several rc, = .15 (pellets)
4.9%
precise quantitative statements of response rate-
reinforcement interactions m concurrent, multiple, Keesey & Kling (1961): Experiment I: Rate of key pecking
on a VI 4-min schedule for four different numbers of
and single schedules have been discussed in this chap¬
peas between 1 and 4. Mean response rate of 4 pigeons,
ter. Although the chapter has concluded that the
each exposed to all four values.
available formulations cannot yet be fully evaluated, k = 118.6 (responses/min)
it also indicates that during the past deeade significant re =. .16 (peas)
progress has been made toward the goal of a quanti¬ 12.2%
fied law of effect encompassing both positive and Experiment II: Rate of key pecking on a VI 4-min
negative reinforcement, schedule for three different numbers of hemp seeds be¬
tween 2 and 8 seeds. Mean response rate of 3 pigeons,
each exposed to all three values.
k — 67.0 (responses/min)
re — .45 (seeds)
APPENDIX A
99.3%
Catania (1963a): Rate <?£ key peeking on a VI 2-min sched¬
Lea§f-§q wares fit of Equation 1 1 to the data from ule for three different durations of access to grain be¬
studies of magnitude of reinforcement. The values of tween 3 and 6 seconds. Mean response rate of 3 pigeons,
fc and r& and The percentage of the variance in the each exposed to all three values.
Variable a^auhted for by th£ equation k = 64.6 (responses/mm)
are shown. re ~ .02 (sec of grain access)
1.8%
Gresfi (1942): Running speed (100/time in sec) in a Di Lollo (1964): Running speed (100/time in sec) in a

straight alley for fiye different weights of dog chow Straight alley for three different numbers of food pellets
between .02 and 5.12 g- Groups of 7 to 21 rats per value. between 1 and 16 pellets. Groups of 16 rats per value.
k — 80 (running speed units) k = 38.9 (running speed units)
rg = .2 (g of chow) % ~ 1.0 (pellets)
99.5% 99.7%

Zeaman (1949); Speed (100/latency in sec) to leave the start Davenport, Goodrich, 8c Hagguist (1966): Rate of lever

box of a short runway for six different weights of cheese pressing on a VI 1-min schedule for three different
between .05 and 2.4 g. Groups of 8 to 10 rats per value. numbers of sucrose pellets between 1 and 9 pellets. Four
macaque monkeys, each exposed to all 3 values.
k — 130 (100/latency in sec)
re = .8 (g of cheese) Mean response rate of the four monkeys:
82.1% k — 21.7 (responses/min)
re = 1.7 (pellets)
Hutt (1954): Rate of lever pressing on a periodic-reinforce¬
100.0%
ment 1-min schedule for three different weights of rat
chow between 3 and 50 mg. Groups of 9 rats per value. Individual monkeys:
Peter de Villiers
279

% of the variance
k (responses/min) re (pellets) accounted for

S36 16.5 2.0 99.9

S49 30.0 3.0 90.4
S57 26.0 1.4 90.1
S60 17.0 1.0 96.6

APPENDIX B
Pulse Frequency
k ~ 25.0 (responses/min)
Least-squares fit of Equation 11 to the data from re — 145 (pps)
studies of the reinforcement magnitude of brain stim¬ 96.9%

ulation. The values of k and re and the percentage of Keesey (1964): Rate of lever pressing on a VI 16-sec sched¬

the variance in the dependent variable accounted for ule for five different durations (.25 to 2.0 sec) of poste¬
by the equation are shown. rior hypothalamic brain stimulation at two different
intensities (3.0 and 1.5 mA). Mean response rates of 10
rats, each exposed to all five durations at both intensities.
Keesey (1962): Rate of lever pressing on a VI 6-sec sched¬ 3.0 mA
ule for six different durations (.28 to 2.65 msec), inten¬ k = 21.0 (responses/min)
sities (.5 to 4.0 mA) and pulse frequencies (23 to 138 yg = .15 (sec)
pps) of hypothalamic brain stimulation. IVfean response 98.6%
rates of 10 rats, each exposed to all six values of each 1.5 mA
parameter of brain stimulation. k nm 14.9 (responses/min)
Duration: re = .10 (sec)
k — 15.1 (responses/min) 96.6%
re — .65 (msec)
Gallistel (196:9); Running speed (100/time in sec) in a
94.1%
straight alley for four to six different numbers of brain
Intensity: stimulation puses (between 4 and 384 pulses per rein¬
k = 20.4 (responses/min) forcement. Nine rats, each exposed to several different
r6 = 2.0 (mA) stimulation values.
92.9% Individual rats:
k (running re (pulses % of the variance
speed units) of BS) accounted for

LH 22 107.5 1.2 76.0

LH 30 111.5 5.0 84.2
LH 35 94.2 2.5 96.5
LH 54 115.2 1.2 78.5
DBB 35 106.0 3.8 96.2
DBB 38 110.5 11.9 90.2
DBB 44 110.0 4.0 92.9
PH 35 108.0 18.0 77.8
PH 40 102.0 12.0 95.4

APPENDIX C Guttman (1953): Rate of lever pressing on a periodic-rein¬

forcement 1-min schedule for four different concentra¬
Least-squares fit of Equation 1 1 to the data from tions of sucrose (4 to 32%). Mean response rates of 20
studies of sucrose and glucose concentration. The val¬ rats, each exposed to all four values; and mean response
rate of groups of 20 rats per value.
ues of k and re and the percentage of the variance in
the dependent variable accounted for by the equation Mean of rats:
k — 8.5 (responses/min)
are shown.
re = 4.2 (% sucrose)
92.4%
280 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

Groups of rats: 5 cc:

k — 11.5 (responses/min) k = 89.4 (running speed units)
re — 11.5 (% sucrose) re = 2.4 (% sucrose)
95.6% 100.0%
25 cc:
Guttman (1954): Rate of lever pressing on a VI 1-min k — 87.9 (running speed units)
schedule for seven different concentrations of sucrose re — 2.2 (% sucrose)
and glucose (2 to 32%). Mean response rates of 8 rats, 99.2%
each exposed to all seven values of both reinforcers.
125 cc:
Sucrose:
k = 88.7 (running speed units)
k — 15.6 (responses/min)
re— 1.4 (% sucrose)
re = 7.1 (% sucrose)
95.6%
93.7%
Glucose: Schrier (1963): Rate of lever pressing on a VI 1-min sched¬
k = 16.1 (responses/min) ule for five different concentrations of sucrose (10 to
50%). Mean response rates of 4 monkeys, each exposed
re = 110 (% glucose)
to all five concentrations.
98.7%
k = 37.5 (responses/min)
Conrad & Sidman (1956): Rate of lever pressing on a VI re = 6.1 (% sucrose)
37-sec schedule for five different concentrations of sucrose 95.4%
(2.3 to 30%) at two drive levels (48 and 72 hr of depri¬
vation). Mean of 3 monkeys, each exposed to all five Sgiirier (1965); Rate of leyer pressing on a VI 30-sec sched¬
concentrations at both drive levels. ule for five different concentrations (10 to 50%) and two
48 hr deprivation: different magnitudes (.33 and .83 cc) of sucrose. Re¬
sponse rates of 8 and 6 monkeys, each exposed to all
It — 16.3 (responses/min)
five concentrations, 6 of them exposed to both magni¬
— 3.7 (% sucrose)
tudes.
96.2%
Mean response rate of the 6 monkeys exposed to both
72 hr deprivation: magnitudes:
k — 17.6 (responses/min) .33 cc:
rc — 1.7 (% sucrose) k = 88.0 (responses/mm)
71.8% rg = 16.1 (% sucrose)
95.9%
Kraelinc (1961)! Running speed (100/time in sec) in a
straight alley for three different concentrations (2.4 to .83 cc:
9.1%) and three different magnitudes 5 to 125 cc) of k — 66.5 (responses/min)
sucrose. Groups of 9 rats for each concentration and re — 9,7 (% sucrose)
magnitude. 96.3%

Individual data from the 8 monkeys with .33 cc ol sucrose:

k (responses/ r* (% % of the variance
min) sucrose) accounted for

Ruth 109.1 17.0 94.3

John 61.9 6.9 85.7
Ken 91.0 21.5 98.6
Allan 55.2 .8 70.1
Karen 87.5 9.5 95.2
Joan 131.2 52.4 92.6
Leo 105.9 40.1 98.6
Mae 6.6 27.4 85.4

Individual data from the 6 monkeys with .83 cc of sucrose:

k (responsesj re (%
% of the variance
min) sucrose) accounted for

Ruth 82.4 5.5 84.9

John 77.2 6.0 97.0
Ken 86.5 20.0 97.7
Allan 61.9 7.5 90.8
Karen 68.2 11.2 93.3
Joan 45.8 65.0 98.3
Peter de Villiers 281

APPENDIX D re = .075 (1 / delay in sec)

92.8%
Least-squares fit of Equation 1 1 to the data from stud¬ Silver & Pierce (1969): Rate of lever pressing on a VI 1-
ies of immediacy of positive reinforcement (1/delay min schedule with five different immediacies of rein¬
in sec). The values of k and re and the percentage of forcement (10- to 160-sec delay). Mean of 6 rats, each
exposed to all five delay values.
the variance in the dependent variables accounted
for by the equation are shown. k = 9.8 (responses/min)
re — .02 (1/delay in sec)
95.0%
Perin (1943): Speed of responding (100/latency in sec to
Pierce, Hanford, 8c Zimmerman (1972): Rate of lever press¬
lever press) in a discrete-trial situation with three differ¬
ing on a VI 1-min schedule with five different immedi¬
ent immediacies of reinforcement (2- to 10-sec delay).
acies of reinforcement (.5- to 100-sec delay), the delay
Groups of 25 rats per delay value.
of reinforcement being signaled by a cue light. Also
k = 54.0 (100/latency in sec) three immediacies of reinforcement (10- to 100-sec delay)
re = .29 (1 /delay in sec) with the lever retracted during the delay. Mean response
93.2% rates of 4 rats, each exposed to all of the delay values in
Logan (1960, Experiment 55D): Running speed (100/time both delay conditions:
in sec) in a straight alley for five different immediacies of Cue light:
reinforcement (1- to 30-sec delay). Groups of 10 rats k — 21.4 (responses/min)
per delay value, each run at high and low drive. re — .04 (1 / delay in sec)
High drive: 96.1%
k = 64.9 (running speed units) Lever retracted:
r6 = .016 (1/delay in sec) k = 106.0 (running speed units)
99.1% re — .09 (1/deay in sec)
Low drive: 95,0%
k — 64.5 (running speed units) Individual data from the cue-light condition:
k (responsesI r (delay % of the variance
min) in sec) accounted for
R1 21.5 .03 80.8
R2 23.5 .15 98.3
R3 22.8 .02 78.9
R4 20.9 .07 97.4
Individual data from the retracted-lever condition:
k (responses! re (delay % of the variance
min) in sec) accounted for
R1 219.1 .63 97.8
R2 67.5 .80 98.6
R3 102.3 .23 94.9
R4 26.6 .04 92.9

APPENDIX E 400-V alley:

k = 228.0 (running speed units)
Least-squares fit of Equation 11 to the data from stud¬ re = 701.0 (V)
92.9%
ies of magnitude of negative reinforcement. The val¬
300-V alley:
ues of k and re and the percentage of the variance in
k = 106.0 (running speed units)
the dependent variable accounted for by the equation
re = 125.0 (V)
are shown. 98.9%

Bower, Fowler, 8c Trapold (1959): Running speed (100/

Campbell 8c Kraeling (1953): Running speed (100/time in time in sec) in a straight alley (250-V alley intensity) for
sec) in a straight alley for four different reductions in three different reductions in voltage (50 to 200 V).
voltage (100 to 400 V) with a 400-V alley, and for three Groups of 5 rats for each voltage reduction value.
different voltage reductions (100 to 300 V) with a 300-V k — 185.0 (running speed units)
alley. Groups of 7 rats for each voltage reduction and re - 338.0 (V)
each alley intensity. 99.6%
282 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

Dinsmoor & Hughes (1956): Response speed (100/latency in sec). The values of k and re and the percentage of
in sec to lever press) for four different durations of time¬ the variance in the dependent variable accounted for
out (5 to 40 sec) from .2- and .4-mA shock. Groups of 5
by the equation are shown.
rats per value of time-out and intensity.
.2 mA:
Fowler Sc Trapold (1962): Running speed (100/time in sec)
& = 45.6 (100/latency in sec)
in a straight alley (240-V alley intensity) with shock off¬
r = 89.4 (sec time-out)
set in the goal box delayed for five different delays be¬
99.5%
tween 1 and 16 sec. Groups of 5 rats per delay value.
.4 mA:
k — 94.0 (running speed units)
k — 24.3 (100/latency in sec)
re = .06 (1/delay in sec)
rG — 6.7 (sec time-out)
92.6%
74.1%
Leeming 8c Robinson (1973): Speed (100/latency in sec) of
Harrison 8c Abelson (1959): Rate of lever pressing for four
escape in a shuttlebox for five different immediacies of
different durations of time-out (2 to 20 sec) from loud
offset of a 420-V shock (1- to 16-sec delay). Groups of 10
noise (116 db.) One rat exposed to all four time-out du¬
rats per delay value.
rations.
k = 3.1 (resporase/min)
k = 39,4 (100/latency in sec)
re == ,11 (1/delay in sec)
r = .45 (sec time-out)
Ql.4% 86.0%
Mccpatt 8c Koch (1978): Speed (1/latency m .01 sec) of
Seward, Shea, UYEDA, & Raskin (I960)! Running speed
panel depression to escape time-out from a comedy
(100/time in see) in a straight alley for three different
record for three different immediacies of reinstatement
voltage reductions (65 to 315 V) with two different alley
of the record (3- to 9-sec delay). Groups of 10 human
intensities (315 and 255 V). Groups of 9 rats for each
subjects per delay value.
voltage reduction and alley intensity.
k ~ 304.1 (1 / latency in .01 sec)
315-V alley;
h — 161.0 (running speed units)
rp = 2.0 (1 /delay in sec)
96.6%
rfl«N.0(V)
97.7 % Tarpy (1969): Speed (100/latency in sec) of lever-press

255-V alley: escape response on either of two levers m a discrete-trial

k = 146.0 (running speed units) situation for five different immediacies of offset of a 200-

tp = 35,0 (V)
V shock (1- to 16-sec delay). Groups of 10 rats per delay
86.0% value.
k = 98.1 (100/latency in sec)
Woods, Davidson, & Peters (1964): Swimming speed (100/
■== .33 (1 /delay in sec)
time in sec) in a cold-water tank (15°C water) for three
89.5%
different increases in water temperature (5 to 25°C).
Groups of 10 rats per temperature increase, Tarty 8c Roster (1970): Speed (100/latency in sec) of lever-
h — 103.0 (swimming speed units) press escape response in a discrete-trial situation for
Tp =18,0 (°G) three different immediacies of offset of a 200-V shock
93.6% (1.5- to 6-sec delay). Groups of 10 rats per delay value.
h = 45.5 (100/latency in sec)
Woods & Holland (1966): Swimming speed (100/time in
r€ = .40 (1 / delay in see)
sec) in a cold*water tank for three different increases in
94.6%
temperature (4 to 16°G) with two different water tank
temperatures (15 and 25 6 C). Groups of 16 rats for each
temperature increase and water tank temperature.
25 °G tank water:
k = 108.0 (swimming speed units)
re — 1.3 (°C) REFERENCES
85.9%
15°C tank water: Allison, T. S., 8c Lloyd, K. E. Concurrent schedules of
k — 114.0 (swimming speed units) reinforcement: Effects of gradual and abrupt increases
re — 1.4 (°C) in changeover delay. Journal of the Experimental Analy¬
87.5% sis of Behavior, 1971, 16, 67-73.
Anger, D. The dependence of interresponse times upon the
relative reinforcement of different interresponse times.
Journal of Experimental Psychology, 1956, 52, 145-161.
Anger, D. The effect upon simple animal behavior of differ¬
APPENDIX F
ent frequencies of reinforcement, II: Separate control
of the reinforcement of different IRTs. Journal of the
Least-squares fit of Equation 1 1 to the data from stud¬ Experimental Analysis of Behavior, 1973, 20, 301-312.
ies of immediacy of negative reinforcement (1/delay Autor, S. M. The strength of conditioned reinforcers as a
 Peter de Villiers 283

function of the frequency and probability of reinforce¬ function of drive level and amount of drive reduction.
ment. In D. P. Hendry (Ed.), Conditioned reinforce¬ Journal of Experimental Psychology, 1953, 45, 97-101.
ment. Homewood, Ill.: Dorsey Press, 1969. Catania, A. C. Concurrent performances: A baseline for
Azrin, N. H., 8c Holz, W. C. Punishment. In W. K. Honig the study of reinforcement magnitude. Journal of the
(Ed.), Operant behavior: Areas of research and applica¬ Experimental Analysis of Behavior, 1963, 6, 299-300. (a)
tion. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966. Catania, A. C. Concurrent performances: Reinforcement
Baum, W. M. Choice is a continuous procedure. Psycho¬ interaction and response independence. Journal of the
nomic Science, 1972, 28, 263-265. Experimental Analysis of Behavior, 1963, 6, 253-263. (b)
Baum, W. M. Time allocation and negative reinforcement. Catania, A. C. Concurrent operants. In W. K. Honig (Ed.),
Journal of the Experimental Analysis of Behavior, 1973, Operant behavior: Areas of research and application.
,
20 313-322. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966.
Baum, W. M. On two types of deviation from the matching Catania, A. C. Concurrent performances: Inhibition of
law: Bias and undermatching. Journal of the Experi¬ one response by reinforcement of another. Journal of
mental Analysis of Behavior, 1974, 22, 231-242. (a) the Experimental Analysis of Behavior, 1969, 12, 731-
Baum, W. M. Choice in free-ranging wild pigeons. Science, 744.
1974, 185, 78-79. (b) Catania, A. C. Self-inhibiting effects of reinforcement.
Baum, W. M. Time allocation in human vigilance. Journal Journal of the Experimental Analysis of Behavior, 1973,
of the Experimental Analysis of Behavior, 1975, 23, 45- 19, 517-526.
53. Catania, A. C., 8c Cutts, D. Experimental control of super¬
Baum, W. M., 8c Rachlin, H. C. Choice as time allocation. stitious responding in humans. Journal of the Experi¬
Journal of the Experimental Analysis of Behavior, 1969, , ,
mental Analysis of Behavior 1963, 6 203-208.
12, 861-874. Catania, A. C., 8c Reynolds, G. S. A quantitative analysis

Beier, E. M. Effects of trial-to-trial variation in magnitude of the responding maintained by interval schedules of
of reward upon an instrumental running response. Un¬ reinforcement. Journal of the Experimental Analysis of
published doctoral dissertation, Yale University, 1958. Behavior, 1968, 11, 327-383.
Beninger, R, J. Positive behavioral contrast with qualita¬ Chung, S.-H. Effects of delayed reinforcement in a con¬
tively different reinforcing stimuli. Psychonomic Science, current situation. Journal of the Experimental Analysis
1972, 29, 307-308. of Behavior, 1965, 8, 439-444,
Bernheim, J. W., 8c Williams, D. R. Time-dependent con¬ Chung, S.-H. Some quantitative laws of operant behavior %
trast effects in a multiple schedule of food reinforcement. Unpublished doctoral dissertation, Harvard LTmversity,
Journal of the Experimental Analysis of Behavior, 1967, 1966.
10, 243-249. Chung, S.-H., & Herrnstein, R. j. Choice and delay of
Bloomfield, T. M. Behavioral contrast and relative rein¬ reinforcement. Journal of the Experimental Analysis of
forcement in two multiple schedules. Journal of the Behavior, 1967, 10, 67-74.
Experimental Analysis of Behavior, 1967, 10, 151-158. (a) Conrad, D. G., 8c Sidman, M. Sucrose concentration as rein¬
Bloomfield, T. M. Some temporal properties of behavioral forcement for lever pressing by monkeys. Psychological
contrast. Journal of the Experimental Analysis of Be¬ Reports, 1956, 2, 381-384.
havior, 1967, 10, 159-164. (b) Crespi, L. P. Quantitative variation of incentive and per¬
Blough, D. S. Interresponse time as a function of contin¬ formance in the white rat. American Journal of Psy¬
uous variables: A new method and some data. Journal chology, 1942, 55, 467-517.
of the Experimental Analysis of Behavior, 1963, 6, 237- Davenport, J. W., Goodrich, K. P., 8c Hagguist, W. W.
246. Effects of magnitude of reinforcement in Macaca sped-
Bolles, R. C. Theory of motivation. New York: Harper 8c osa. Psychonomic Science, 1966, 4, 187-188.
Row, 1967. de Villiers, P. A. Reinforcement and response rate inter¬
Bower, G. H., Fowler, H., 8c Trapold, M. A. Escape learn-
action in multiple random-interval avoidance schedules.
ing as a function of amount of shock reduction. Journal Journal of the Experimental Analysis of Behavior, 1972,
of Experimental Psychology, 1959, 58, 482-484. 18, 499-507.
Breland, K., & Breland, M. The misbehavior of organisms. de Villiers, P. A. The law of effect and avoidance: A
American Psychologist, 1961, 16, 681-684. quantitative relationship between response rate and
Brown, P. L., 8c Jenkins, H. M. Auto-shaping of the pi¬ shock-frequency reduction. Journal of the Experimental
geon’s key-peck. Journal of the Experimental Analysis of Analysis of Behavior, 1974, 21, 223-235.
Behavior, 1968, 11, 1-8. de Villiers, P. A., 8c Herrnstein, R. J. Toward a law of
Brown, R. W., 8c Herrnstein, R. J. Psychology. Boston: response strength. Psychological Bulletin, in press.
Little, Brown, 1975. de Villiers, P. A., 8c Millenson, J. R. Concurrent perfor¬
Brownstein, A. J. Concurrent schedules of response-inde¬ mances: A baseline for the study of conditioned anxiety.
pendent reinforcement: Duration of a reinforcing stim¬ Journal of the Experimental Analysis of Behavior, 1972,
ulus. Journal of the Experimental Analysis of Behavior, 18, 287-294.
1971, 15, 211-214. Di Lollo, V. Runway performance in relation to runway-
S. S. Some effects of relative
Brownstein, A. J., 8c Pliskoff, goal-box similarity and changes in incentive amount.
reinforcement rate and changeover delay in response- Journal of Comparative and Physiological Psychology,
independent concurrent schedules of reinforcement. 1964, 58, 327-329.
Journal of the Experimental Analysis of Behavior, 1968 Dinsmoor, J. A., 8c Hughes, L. H. Training rats to press a
11, 683-688. bar to turn off shock. Journal of Comparative and Phys¬
Campbell, B. A., 8c Kraeling, D. Response strength as a iological Psychology, 1956, 49, 235-238.
284 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

Duncan, B., 8c Fantino, E. Choice for periodic schedules Herrnstein, R. J. Secondary reinforcement and rate of
of reinforcement. Journal of the Experimental Analysis primary reinforcement. Journal of the Experimental
of Behavior, 1970, 14, 73-86. Analysis of Behavior, 1964, 7, 27-36.
Dysart, J., Marx, M. H., McLean, J., 8c Nelson, J. A. Peak Herrnstein, R. J. Method and theory in the study of
shift as a function of multiple schedules of reinforce¬ avoidance. Psychological Review, 1969, 16, 49-69.
ment. Journal of the Experimental Analysis of Behavior, Herrnstein, R. J. On the law of effect. Journal of the
1974, 22, 463-470. Experimental Analysis of Behavior, 1970, 13, 243-266.
Estes, W. K., 8c Skinner, B. F. Some quantitative properties Herrnstein, R. J. Quantitative hedonism. Journal of Psy¬
of anxiety. Journal of Experimental Psychology, 1941, chiatric Research, 1971, 8, 399-412.
29, 390-400. Herrnstein, R. J. Formal properties of the matching law.
Fantino, E., Squires, N., Delbruck, N., 8c Peterson, C. Journal of the Experimental Analysis of Behavior, 1974,
Choice behavior and the accessibility of the reinforcer. 21, 159-164.
Journal of the Experimental Analysis of Behavior, 1972, Herrnstein, R. J., 8c Hineline, P. N. Negative reinforce¬
18, 35-43. ment as shock-frequency reduction. Journal of the Ex¬
Ferster, C. B., 8c Skinner, B. F. Schedules of reinforcement. perimental Analysis of Behavior, 1966, 9, 421-430.
Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1957. Herrnstein, R. J., 8c Loveland, D. H. Food avoidance in
Findley, J. D. Preference and switching under concurrent hungry pigeons and other perplexities. Journal of the
scheduling. Journal of the Experimental Analysis of Experimental Analysis of Behavior, 1972, 18, 369-383.
Behavior, 1958, 1, 123-144. Herrnstein, R. J., 8c Loveland, D. H, Hunger and contrast
Fowler, H.7 8c Trapold, M. A. Escape performance as a m a multiple schedule. Journal of the Experimental
function of delay o£ reinforcement, Journal of Experi¬ Analysis of Behavior, 1974, 21, 511-517.
mental Psychology, 1902, 6?j 464-467, Herrnstein, R. J,, 8c Loveland, D. H. Maximizing and
CalosteLj C. R. The Incentive of brain-stimulation reward. matching on concurrent ratio schedules. Journal of the
Journal <?f Comparative and Physiological Psychology, Experimental Analysis of Behavior, 1975, 24, 107-116.
1969,^713-721, Hollard, V,, 8c Davison, M. G. Preference for qualitatively
Camzu, E., 3? Schwarts, B, The maintenance of key-pecking different reinforcers. Journal of the Experimental Analy¬
by stimulus-contingent and response-independent food sis of Behavior, 1971, 16, 375-380.
presentation. Journal of the Experimental Analysis of Hole, W. C. Punishment and rate of positive reinforce¬
Behavior, 1978, IP, $5-72. ment. Journal of the Experimental Analysis of Behavior,
Camzo, E., & Williams, D, R, Classical conditioning of a 1968,11, 285-292.
complex skeletal response. Science, 1971, I7l> 923-925. Butt, p. J, Rate of bar-pressing as a function of quality
GUTTMAN, N. Operant conditioning, extinction, and peri¬ and quantity of food rewards. Journal of Comparative
odic r6iftf6i:C6m£nt m relation to concentration of and Physiological Psychology, 1954, 47} 235-239.
S11CLOS6 used as reinforcing agent. J ournal of Experi¬ ICLAUER, C.? & Woods, J. H. Concurrent performances:
mental Psychology, 1953, 4G, 213-223. Reinforcement by different doses of intravenous cocaine
Guttman, N. Equal-reinforcement values for sucrose and in rhesus monkeys. Journal of the Experimental Analysis
glucose solutions compared with equal-sweetness values. of Behavior, 1971 22, 179-196.
Journal of Comparative and Physiological Psychology, Keesey, R. E. The relation between pulse frequency,
intensity, and duration and the rate of responding for
CUTtmAn, N. Generalization gradients around §tlmul| asso¬ intracranial stimulation, Journal of Comparative and
ciated With different reinforcement schedules. Journal Physiological Psychologyf 1962, 55; 671-678.
of Experimental Psychology, 1959, 58, 335-340. KEESEY, R. E. Duration of stimulation and the reward
Hanson, H; M. Effects of discrimination training on stim= properties of hypothalamic stimulation. Journal of Com¬
ulus generalisation, Journal of Experimental Psychology, parative and Physiological Psychology, 1964, 58, 201-207.
1959, B8t 321-334. Keesey, R. E., 8: Kling, J, W, Amount of reinforcement
HARRE, R. An introduction to the logic of the sciences. and free-Operant responding. Journal of the Experi¬
New York: St. Martin’s Press, 1960. mental Analysis of Behavior, 1961, 4, 125-132.
Harrison, J. M., 8c Abelson, R. M. The maintenance of Keller, K. The role of elicited responding in behavioral
behavior by the termination and onset of intense noise. contrast. Journal of the Experimental Analysis of Be¬
Journal of the Experimental Analysis of Behavior, 1959, havior, 1974, 21, 249-257.
2, 23-42. Killeen, P. Reinforcement frequency and contingency as
Hawkes, L., & Shimf, C. P. Choice between response rates. factors in fixed-ratio behavior. Journal of the Experi¬
Journal of the Experimental Analysis of Behavior, 1974, mental Analysis of Behavior, 1969, 12, 391-395.
21, 109-115. Killeen, P, A yoked-chamber comparison of concurrent
Herbert, E. W. Two-key concurrent responding: Response- and multiple schedules. Journal of the Experimental
reinforcement dependencies and blackouts. Journal of Analysis of Behavior, 1972, 18, 13-22. (a)
the Experimental Analysis of Behavior, 1970, 14, 61-70. Killeen, P. The matching law. Journal of the Experi¬
Herrnstein, R. J. Some factors influencing behavior in a mental Analysis of Behavior, 1972, 17, 489-495. (b)
two-response situation. Transactions of the New York Kraeling, D. Analysis of amount of reward as a variable in
Academy of Sciences, 1958, 21, 35-45. learning. Journal of Comparative and Physiological
Herrnstein, R. J. Relative and absolute strength of re¬ Psychology, 1961, 54, 560-565.
sponse as a function of frequency of reinforcement. Kramer, T. J., 8c Rilling, M. Differential reinforcement
Journal of the Experimental Analysis of Behavior, 1961, of low rates: A selective critique. Psychological Bulletin,
4, 267-272. 1970, 74, 225-254.
Peter de Villiers 285

LaBounty, C. E., 8c Reynolds, G. S. An analysis of re¬ Perin, C. T. A quantitative investigation of the delay-of-
sponse and time matching to reinforcement in concur¬ reinforcement gradient. Journal of Experimental Psy¬
rent ratio-interval schedules. Journal of the Experi¬ chology, 1943, 32, 37-51.
mental Analysis of Behavior, 1973, 19, 155-166. Pierce, C. H., Hanford, P. V., 8c Zimmerman, J. Effects

Lander, D. G., & Irwin, R. J. Multiple schedules: Effects of different delay of reinforcement procedures on vari¬
of the distribution of reinforcements between compo¬ able-interval responding. Journal of the Experimental
nents on the distribution of responses between compo¬ Analysis of Behavior, 1972, 18, 141-146.
nents. Journal of the Experimental Analysis of Behavior, Pliskoff, S. S. Effects of symmetrical and asymmetrical
1968, 11, 517-524. changeover delays on concurrent performances. Journal
Lea, S. E. G., 8c Morgan, M. J. The measurement of rate- of the Experimental Analysis of Behavior, 1971, 16, 249-
dependent changes in responding. In R. M. Gilbert 8c J. 256.
R. Millenson (Eds.), Reinforcement: Behavioral analy¬ Premack, D. Reinforcement theory. In D. Levine (Ed.),
ses. New York: Academic Press, 1972. Nebraska symposium on motivation. Lincoln: University
Leeming, F. C., 8c Robinson, J. E. Escape behavior as a of Nebraska Press, 1965.
function of delay of negative reinforcement. Psycho¬ Premack, D. On boundary conditions of contrast. In J. T.
logical Reports, 1973, 92, 63-70. Tapp (Ed.), Reinforcement and behavior. New York:
Logan, E. A. Incentive. New Haven: Yale University Press, Academic Press, 1969.
1960. Rachlin, H. C. On the tautology of the matching law.

LoLordo, V. M. Facilitation of food-reinforced responding Journal of the Experimental Analysis of Behavior, 1971,
by a signal for response-independent food. Journal of the 15, 249-251.
Experimental Analysis of Behavior, 1971, 19, 49-55. Rachlin, H. C. Contrast and matching. Psychological Re¬
LoLordo, V. M., McMillan, J. C., 8c Riley, A. L. The view, 1973, 80, 217-234.
effects upon food-reinforced pecking and treadle-pressing Rachlin, H. C., 8c Baum, W. M. Response rate as a func¬

of auditory and visual signals for response-independent tion of amount of reinforcement for a signalled concur¬
food. Learning and Motivation, 1974, 5, 24-41. rent response. Journal of the Experimental Analysis of
Mariner, R. W., 8c Thomas, D. R. Reinforcement duration Behavior, 1969, 12, 11-16.
and the peak shift in post-discrimination gradients. Rachlin, H. C., 8c Baum, W. M. Effects of alternative rein¬
Journal of the Experimental Analysis of Behavior, 1969, forcement: Does the source matter? Journal of the Ex¬
12, 759-766. perimental Analysis of Behavior, 1972, 18, 231—241.
McSweeney, F. K. Matching and contrast on several con¬ Rachlin, H. C., 8c Herrnstein, R. J. Hedonism revisited:
current treadle-press schedules. Journal of the Experi¬ On the negative law of effect. In B. Campbell 8c R. M.
mental Analysis of Behavior, 1975, 23, 193-198. Church (Eds.), Punishment and aversive behavior. Engle¬
Millenson, J. R., 8c de Villiers, P. A. Motivational prop¬ wood Cliffs, N.J.: Prentice-Hall, Inc., 1969.
erties of conditioned anxiety. In R. M. Gilbert 8c J. R. Redford, M. E., 8c Perkins, C. C. The role of auto-pecking
Millenson (Eds.), Reinforcement: Behavioral analyses. in behavioral contrast. Journal of the Experimental
New York: Academic Press, 1972. Analysis of Behavior, 1974, 21, 145-150.
Moffatt, G. H., 8c Koch, D. L. Escape performance as a Reynolds, G. S. An analysis of interactions in a multiple
function of delay of reinforcement and inescapable US schedule. Journal of the Experimental Analysis of Be¬
trials. Psychological Reports, 1973, 32, 1255-1261. havior, 1961, 4, 107-117. (a)
Moffitt, M., 8c Shimp, C. P. Two-key concurrent paced Reynolds, G. S. Behavioral contrast. Journal of the Experi¬
variable-interval schedules of reinforcement. Journal of mental Analysis of Behavior, 1961, 4, 57-71. (b)
the Experimental Analysis of Behavior, 1971, 16, 39-49. Reynolds, G. S. Some limitations on behavioral contrast
Morse, W. H. Intermittent reinforcement. In W. K. Honig and induction during successive discrimination. Journal
(Ed.), Operant behavior: Areas of research and applica¬ of the Experimental Analysis of Behavior, 1963, 6, 131-
tion. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966. 139.
Neuringer, A. J. Delayed reinforcement versus reinforce¬ Schmitt, D. R. Effects of reinforcement rate and reinforcer
ment after a fixed interval. Journal of the Experimental magnitude on choice behavior of humans. Journal of
Analysis of Behavior, 1969, 12, 375-383. the Experimental Analysis of Behavior, 1974, 21, 409-
Nevin, J. A. Differential reinforcement and stimulus control 419.
of not responding. Journal of the Experimental Analysis Schneider, B. A. A two-state analysis of fixed-interval
of Behavior, 1968, 11, 715-726. responding in the pigeon. Journal of the Experimental
Nevin, J. A. Interval reinforcement of choice behavior in Analysis of Behavior, 1969, 12, 677-687.
discrete trials. Journal of the Experimental Analysis of Schneider, J. W. Reinforcer effectiveness as a function of
Behavior, 1969, 12, 875-885. reinforcer rate and magnitude: A comparison of concur¬
Nevin, J. A. Rates and patterns of responding with con¬ rent performances. Journal of the Experimental Analysis
current fixed-interval and variable-interval reinforce¬ of Behavior, 1973, 20, 461-471.
ment. Journal of the Experimental Analysis of Behavior, Schrier, A. M. Sucrose concentration and response rates of
1971, 16, 241-247. monkeys. Psychological Reports, 1963, 12, 666.
Nevin, J. A. Response strength in multiple schedules. Schrier, A. M. Response rates of monkeys under varying
Journal of the Experimental Analysis of Behavior, 1974, conditions of sucrose reinforcement. Journal of Compar¬
21, 389-408. ative and Physiological Psychology, 1965, 59, 378-384.
Nevin, J. A., 8c Shettleworth, S. J. An analysis of contrast Schroeder, S. R., 8c Holland, J. G. Reinforcement of eye
effects in multiple schedules. Journal of the Experi¬ movement with concurrent schedules. Journal of the
mental Analysis of Behavior, 1966, 9, 305-315. Experimental Analysis of Behavior, 1969, 12, 897-903.
286 CHOICE IN CONCURRENT SCHEDULES AND A QUANTITATIVE FORMULATION OF THE LAW OF EFFECT

Schwartz,B. Maintenance of key pecking by response-inde¬ lenson (Eds.), Reinforcement: Behavioral analyses. New
pendent food presentation: The role of the modality of York: Academic Press, 1972.
the signal for food. Journal of the Experimental Analy¬ Staddon, J. E. R., 8c Simmelhag, V. L. The “superstition”
sis of Behavior1973, 20, 17-22. experiment: A re-examination of its implications for the
Seward, J. P., Shea, R. A., Uyeda, A. A., 8c Raskin, D. C. principles of adaptive behavior. Psychological Review,
Shock strength, shock reduction, and running speed. 1971, 78, 3-43.
Journal of Experimental Psychology, 1960, 60, 250-254. Stevens, S. S. On the psychophysical law. Psychological
Siiimp, C. P. Probabilistically reinforced choice behavior in Review, 1957, 64, 153-181.
pigeons. Journal of the Experimental Analysis of Be¬ Stubbs, D. A., 8c Pliskoff, S. S. Concurrent responding with
havior, 1966, 9y 433-455. fixed relative rate of reinforcement. Journal of the Ex¬
Shimp, C. P. Magnitude and frequency of reinforcement perimental Analysis of Behavior, 1969, 12, 887-895.
and frequency of interresponse times. Journal of the Tarpy, R. M. Reinforcement difference limen (RDL) for
Experimental Analysis of Behavior, 1968, 11, 525-535. delay in shock escape. Journal of Experimental Psy¬
Shimp, C. P. Optimum behavior in free-operant experi¬ chology, 1969, 79, 116-121.
ments. Psychological Review, 1969, 76, 97-112. (a) Tarpy, R. M., 8c Koster, E. D. Stimulus facilitation of
Shimp, C. P. The concurrent reinforcement of two inter¬ delayed-reward learning in the rat. Journal of Compara¬
response times: The relative frequency of an interre¬ tive and Physiological Psychology, 1970, 71, 147-151.
sponse time equals its relative harmonic length. Journal Terrace, H. S. Behavioral contrast and the peak shift:
of the Experimental Analysis of Behavior, 1969, 12, 403- Effects of extended discrimination training. Journal of
411. (b) the Experimental Analysis of Behavior, 1966, 9, 613—
Shimf, C. P. The concurrent reinforcement of two inter¬ 617. (a)
response times: Absolute rate of reinforcement. Journal Terrace, H. S. Stimulus control. In W. K, Honig (Ed.),
of the Experimental Analysis of Behavior, 1970, 13, 1-8. Operant behavior; Areas of research and application.
Shimp, C. P. The reinforcement of four interresponse times Englewood Cliffs, N,J,; Prentice-Hall, Inc., 1966. (b)
in a two-alternative situation. Joumal of the Experi¬ Terrace, H, S, By-products of discrimination learning. In
mental Analysis of Behavior, 1971, 16, 885-899. G. H, Bower (Ed.), The psychology of learning and
Shimp, C. P. Synthetic variable-interval schedules of rein¬ motivation (Vol. 5). New York; Academic Press, 1972.
forcement. Journal of the Experimental Analysis of Todorov, J. G. Concurrent performances: Effect of punish¬
Behavior, 1973, 19, 311-830. ment contingent on the switching response. Journal of
SlIIMP, C. P. I ime allocation and response rate. Joumal of the Experimental Analysis of Behavior, 197T 16, 51-62.
the Experimental Analysis of Behavior, 1974, 21, 491- Todorov, J. 0- Component duration and relative response
499. rates in multiple schedules, Journal of the Experimental
Shimp, C. P., & Wheatley, W. L. Matching to relative Analysis of Behavior, 1972, 17} 45~50.
reinforcement frequency in multiple schedules with Todorov, J. C. Interaction of frequency and magnitude of
short component duration. Journal of the Experimental reinforcement on concurrent performances. Journal of
Analysis of Behavior, 1971, 13, 205-210. the Experimental Analysis of Behavior, 1973, 19, 451-
Shull, R, E., ft PkisRorr, s, s, Gliangeover delay and con¬ 458.
current schedules: Some effects on relative performance Treyitt, A. J., Davison, M. C., & Williams, R. J. Per¬
measure. Journal of the Experimental Analysis of Be- formance in concurrent interval schedules. Journal of
havior, 1967, 10, 517-527. the Experimental Analysis of Behavior, 1972, 17, 369-
Sidman, M. Avoidance conditioning with brief shock and 374.
no exteroceptive warning signal. Science, 1953, 118, TVERSKY, A. Itttransitivity of prefere nces. Psychological
157-138, Review, 1969, 76, 31-48.
SmMAN; By-products <?£ ^yergiye control. Journal of the Walker, 5. F., 8c Hurwitz, H. M. B. Effects of relative
Experimental Analysis of Behavior, 1958, 1, 265—280.
reinforcer duration on concurrent response rates. Psy-
Sidman, M. Avoidance behavior. In W. K. Honig (Ed.), chonomic Science, 1971, 22} 45-47.
Operant behavior: areas of research and application. WALKER, S. F., Schnelle, J. F.; & Hurwitz, H. M. B. Rates
Englewood Cliffs, N,J,; Prentice-Hall, Inc., 1966. of concurrent responses and reinforcement duration.
Silrrrberg, A., & Fantino, E. Choice, rate of reinforcement, Psychonomic Science, 1970, 21, 173-175.
and the changeover delay. Journal of the Experimental Wasserman, E, A, The effect of redundant contextual
Analysis of Behavior, 1970,13, 187-197. stimuli on autoshaping the pigeon’s keypeck. Animal
Silver, M. P., Sc Pierce, G. H. Contingent and noncontin- Learning and Behavior, 1973, 1, 198-206.
gem response rates as a function of delay of reinforce¬ Westbrook, R. F. Failure to obtain positive contrast when
ment. Psychonomic Science, 1969, 14, 231-232. pigeons press a bar. Journal of the Experimental Analy¬
Skinner, B. F. Are theories of learning necessary? Psy¬ sis of Behavior, 1973, 20, 499-510.
chological Review, 1950, 37, 193-216. White, A. J., 8c Davison, M. C. Performance in concur¬
Spealman, R. D., & Gollub, L. R. Behavioral interactions rent fixed-interval schedules. Journal of the Experi¬
in multiple variable-interval schedules. Journal of the mental Analysis of Behavior, 1973, 19, 147-153.
Experimental Analysis of Behavior, 1974, 22, 471-481. Williams, D. R., 8c Williams, H. Auto-maintenance in the
Staddon, J. E. R. Spaced responding and choice: A pre¬ pigeon: Sustained pecking despite contingent non-rein¬
liminary analysis. Journal of the Experimental Analysis forcement. Journal of the Experimental Analysis of
of Behavior, 1968, 11, 669-682. Behavior, 1969, 12, 511-520.
Staddon, J. E. R. Temporal control and the theory of Wood, K. A., Martinez, E. S., 8c Willis, R. D. Ratio require¬
reinforcement schedules. In R. M. Gilbert 8c J. R. Mil- ment and reinforcer effects in concurrent fixed-interval
Peter c/e Villiers 287

fixed-ratio schedules. Journal of the Experimental Anal¬ ditioning in a water tank: Effects of constant reinforce¬
ysis of Behavior, 1975, 23, 87-94. ment at different levels of drive stimulus intensity.
Woods, P. J., Davidson, E. H., 8c Peters, R. J. Instrumental Journal of Comparative and Physiological Psychology,
escape conditioning in a water tank: Effects of variation 1966, 62, 403-408.
in drive stimulus intensity and reinforcement magnitude. Zeaman, D. Response latency as a function of the amount

Journal of Comparative and Physiological Psychology, of reinforcement. Journal of Experimental Psychology,

1964, 37, 466-470. 1949, 39, 466-483.
Woods, P. J., 8c Holland, C. H. Instrumental escape con¬
 10

Conditioned Reinforcement
schedule effects*

Lewis Gollub

INTRODUCTION If categorized as reinforcers they would be expected to

increase or maintain the frequency of responses they
The topic of conditioned reinforcement lias tradi¬ follow.
tionally been concerned with the question whether a More recent conceptions of the reinforcement pro¬
previously nonreinforcing stimulus could become a cess emphasize the dynamic nature of reinforcement
remforcer through some conditioning operation.1 The (e.g., see Morse 8c Kelleher, chapter 7 of this volume).
answer to such a question depends in large part on In this view, the reinforcement effect of a stimulus
the way the reinforcement process is conceptualized. depends on many factors, such as the organism’s de¬
In the traditional view, reinforcement has been con¬ privation, its past history of exposure to the stimulus,
sidered a static property of a stimulus. Stimuli could specific relations between the behavior elicited or con¬
thus be categorized as reinforcers or nonrein forcers. trolled by a stimulus and the operant behavior under
study (Premack, 1965), and, quite importantly, the
* The author thanks W. H. Morse, who stimulated my early schedule of stimulus presentation. In earlier treat¬
research on chained schedules, R. T. Kelleher, and A. C. Catania ments of conditioned reinforcement, however, the
tor provocative discussions over many years, and D, A. Stubbs for
helpful comments on an earlier draft of this paper. A cordial
reinforcing effectiveness of the conditioned reinforcer
environment for writing the review, and helpful support, were was treated as a fixed property of the stimulus which
provided by Dr. T. Yanagita and K. Ando of the Central could reveal itself under arbitrarily different condi¬
Institute for Experimental Animals, Kawasaki, Japan, while the
author was on sabbatical leave from the University of Maryland.
tions. The paradigm for studying conditioned rein¬
Thanks are due to S. Loftus, R. Crovo, and K. Flowers for help forcement was a two-part experiment in which Part
in preparing the manuscript. This work was supported, in part, One consisted of “training,” where a given type of
by U.S.P.H.S, Research Grant MH-01604.
l This review deals only with positive reinforcement, i.e.,
association with a primary reinforcer was used to im¬
operations involving the presentation of stimuli that increase the bue some arbitrary stimulus with conditioned rein¬
probability of responses they follow. Negative reinforcement is forcing properties, and Part Two was a “test” of
discussed in Chapters 7, 13, and 14. Further references in
this chapter to reinforcement should always be understood to in¬
whether the stimulus was, in the absence of the
clude only positive reinforcement. primary reinforcer, a conditioned reinforcer. The

288
Lewis Gollub 289

major question was, “Under what conditions would a sequence of individual schedule requirements, each of
stimulus that was previously ineffective become a rein- which is accompanied by a characteristic stimulus. For
forcer?’ ’ In attempting to answer this question, the example, consider the sequence in a chained schedule
effect of the putative conditioned reinforcer was in which a variable interval (VI) 2-min component
assessed during extinction with respect to primary precedes a fixed-interval (FI) 2-min component. A
reinforcement. The extinction techniques thus response key is initially lit green. After two minutes,
avoided the possible confounding of response main¬ on the average, the first peck changes the key color to
tenance by primary reinforcement in the test of the red. The first peck that occurs two minutes or more
conditioned reinforcer. after the key becomes red produces a small ration of
In experiments with the extinction methods the food. The key then again becomes green, and the
effects of the putative conditioned reinforcer have chain repeats.
generally been weak, sustaining either relatively few A chained schedule {chain) is denoted by a list of
responses, or lasting for only a short time. This the individual schedules in their order of occurrence;
lability presumably resulted from the fact that the the component stimuli must also be described. Thus,
association with the primary reinforcer which estab¬ the preceding example would be a chain VI 2-min
lished the conditioned reinforcer was, by definition, FI 2-min with green and red lights on the response
broken during the extinction test. (Detailed reviews of key. When the component schedules are identical, a
the experimental literature using extinction tech¬ further shorthand is sometimes used, which specifies
niques can be found in Kelleher 8c Gollub, 1962; the number of components, and the schedule in each
Miller, 1951; Myers, 1958; Wike, 1966.) In addition to (sometimes called the unit schedule). Thus, if both
their empirical frailty, the putative conditioned rein¬ components of a two-component chain were FI 2-min,
forcers were questioned on the grounds that the old or a chain FI 2-min FI 2-min, the schedule could be
response was maintained because of discriminative denoted chain FR 2 (FI 2-min). This indicates that a
effects of the stimulus rather than reinforcing effects primary reinforcer is presented after completion twice
(see the previous reviews and Lott, 1967; Schuster, (fixed ratio 2, or FR 2) of the unit schedule require¬
1969; Wike, 1969). Thus, the question of whether an ment, FI 2-min. A characteristic stimulus is associated
arbitrary stimulus can acquire a long-lasting reinforce¬ with each of the two components (cf. Kelleher, 1966a).
ment effect, and replace a primary reinforcer, fre¬ The parentheses can be read as in mathematical func¬
quently degenerated into inconclusive debate on al¬ tional notation, a chain FR 2 of FI 2-min. This nota¬
ternative stimulus functions. tion is particularly helpful in contrasting results
More recent experiments on conditioned reinforce¬ under chained schedules with those under other
ment have emphasized response maintenance. In these sequential schedules, such as tandem and second-order
procedures, responding is maintained by stimuli that schedules.
have continued association with a primary reinforcer.
Besides avoiding the traditionally intractable ques¬
Overview
tions that arise in the context of extinction pro¬
cedures, these newer techniques are also related more Responding in individual components of chained
closely to the analysis of maintained responding gen¬ schedules is primarily under the control of two vari¬
erally, and the sequential properties of behavior in ables: the component schedule and the temporal loca¬
particular (cf. Kelleher, 1966a). tion of the component with respect to primary rein¬
This review will be concerned primarily with the forcement (cf. Staddon, 1972). The analysis of chained
study of two paradigms, chained and second-order schedules thus emphasizes two aspects of behavior: the
schedules of reinforcement. Both paradigms involve temporal pattern of responding in each component,
sequential response requirements, and the presenta¬ and the overall amount of responding in each com¬
tion of stimuli that bear a scheduled relationship to ponent. In general, the pattern of responding in each
primary reinforcement. In both paradigms behavior is component tends to resemble the pattern that occurs
continually maintained by primary reinforcement. when responding under the same schedule produces
primary reinforcement. Response rate increases from
component to component towards primary reinforce¬
CHAINED SCHEDULES OF REINFORCEMENT ment.
The major emphases of research on chained sched¬
In a chained schedule of reinforcement, a single ules have been the acquisition of responding under
primary reinforcement follows the completion of a chained schedules (transition performances, cf. Sid-
290 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

man, 1960); parametric investigation of factors that schedule had previously accompanied extinction, re¬
determine the level of maintained responding; and, sponse rates increased under the chained schedule;
the analysis of the behavioral functions of the com¬ when pecks in the component had previously pro¬
ponent stimuli of chained schedules. duced food, pecking decreased under the chained
schedule (cf. Ferster, 1953).
Transition Performances Under Chained Schedules Transition performances have not been explored
very extensively since Ferster and Skinner’s observa¬
In a noteworthy series of experiments, Ferster and tions. Gollub (1958) examined responding under
Skinner (1957) examined transition and steady-state 2- and 5-component chained schedules in pigeons that
performances in pigeons under chained schedules fol¬ had previously been trained under comparable
lowing training on related simple and multiple sched¬ tandem schedules. In a tandem schedule, successive
ules. I hey studied two-component chained schedules completion of two or more component schedules is
comprised of eight of the possible nine permutations required before primary reinforcement, but a single
of II, FR, and VI schedule components.2 Pigeons ex¬ stimulus accompanies all components. In Gollub’g
posed to these schedules had previously been trained experiments, the pigeons had never been exposed to
umder related compound schedules with the same the key light colors associated with the earlier com¬
component stimuli. For example, a pigeon was ex¬ ponents of the chained schedule. Gollub found that
posed to a FI 1mm g£h<§dule with a blue key, alter¬ response rates in the initial component of 2-compo¬
nating with a red key during which pecks had no nent chains decreased during the first and second ses¬
consequences (EXT). The blue key light was presented sions, compared to rates under the tandem schedule,
indop^ndontly of key peeks after varying durations and then increased, typically to a value higher than
of the red key light. This arrangement comprised a that attained in the initial component under the
multiple {mult) schedule, mult EXT FI i-mxn. A tandem schedule. One pigeon studied under tandem
chained schedule was later arranged by presenting the and chained schedules with five components had
blue light and its associated FI 1-mm schedule de¬ consistently lower rates in the initial chain compo¬
pendent on a peck after varying durations of the red nent. Kelleher and Fry (1962) also found marked de¬
light that averaged 1 min, a VI 1-min schedule. The creases in the response rate in the initial component
chained schedule was thus chain VI 1-min FI 1-min. within three sessions of chain FR 3 (FI 1 min) follow¬
Under the mult EXT FI 1-min schedule, very low ing 30 sessions under the comparable tandem sched¬
rates of peeking occurred while the key was red, and ule. Thus the changes in performance under chained
pecking began a short time after the key became blue schedules seem to depend on both the earlier experi¬
and continued until bod presentation. Within the mental history of the organism (extinction versus food
first hour under the chained schedule responding in reinforcement) and the number of components in the
the presence of the red key light had increased from chained schedule.
near mro to about 15 pecks per mmute, and increased Such effects do not appear to depend on the specific
§td! more in the second session. Responding in the training procedures, at least under some schedule
terminal component (blue key light) was unchanged values. A gradual increase to 60 sec in the interval
from the performance during the multiple schedule. durations of chain FI X FI X did not lead to terminal
Ferster and Skinner (1957) also studied chained rates different from those attained by pigeons that
schedule performance after training on multiple were exposed to chain FI 60-sec FI 60-sec immediately
schedules with food reinforcement in both compo¬ after key peck acquisition (Gollub and Vogt, 1970). In
nents, e,g,, a mult VI 3-min FI 1-min schedule. When a related study, Switalski and Thomas (1967) at¬
performances in both components were stable, pecks tempted to trace explicitly the development of stim¬
in the presence of the VI 3-min stimulus produced not ulus control in the last 2 components of chain FR 3
food, but rather the stimulus associated with the FI 1- (VI 40-sec) in pigeons. An unlit key marked the initial
min schedule [chain VI S-min FI l-min). Both birds component, and a monochromatic light (550 nm) and
exposed to this training regimen “substantially lost a line were presented in the last 2 components, with
the performance on VI 3 formerly prevailing under counterbalanced orders for the two groups of pigeons.
the multiple schedule before developing a chained Stimulus generalization gradients for pecking were
performance” (p. 660). In brief, when the stimulus determined for line tilt, and wavelength, after the
presented in the initial component of the chained fifth and twelfth training sessions respectively. At both
2 For definitions of these simple schedules of reinforcement, points in training, typical performance for a chained
and descriptions of performances they generate, see Chapter 8. schedule was demonstrated. At the first generalization
Lewis Gollub 291

test (after five training sessions) the generalization nent. This relation holds both when food delivery is
gradients were steeper for the terminal component dependent on responding in the terminal component
than for the penultimate component. By the 12th (Findley, 1962) and when food is presented indepen¬
session, about equal degrees of control were shown in dently (Kaufman 8c Baron, 1969).
each component by the chain. To paraphrase Swital- In chained schedules with more than two compo¬
ski and Thomas (1967), stimulus control developed nents (“extended chained schedules,” Kelleher 8c
sequentially from the end of the chain forward. Gollub, 1962) the extent to which responding is sus¬
Less attention has been paid to the temporal pat¬ tained in the initial components also depends on the
terns of responding that develop under chained sched¬ time that elapses from the end of the components to
ules. The evidence presented by Ferster and Skinner food reinforcement.
(1957), as well as in later reports, shows that although The profound effect on responding of the number
the temporal patterns within components of chained and duration of fixed-interval components is illus¬
schedules resemble the patterns generated by the cor¬ trated in the following two experiments. In one
responding schedules of food reinforcement, there are experiment Gollub (1958) studied key pecking under
also qualitative and quantitative differences. Respond¬ chained schedules with 2, 3, 4, and 5 components, each
ing to the initial components of chained schedules is of which was FI 30 sec. Responding in a given com¬
more irregular, so that, for example the steady rate ponent was lower the farther the component was from
typically seen under VI schedules of food reinforce¬ food presentation. Figure 1 shows the mean rates of
ment is replaced by alternating periods of responding responding in each component as a function of the
and pausing (rough grain). There are pauses at the be¬ number of components intervening before food. A
ginning of both FI and VI initial components that are single curve connects the mean values. It can be seen
longer than those typically observed with food rein¬ that maintained response rate decreased steeply as
forcement. These effects have, unfortunately, not been
subjected to experimental analysis.

Maintained Responding Under Chained Schedules

A major concern in research on chained schedules

is the effect of the number, type, and quantitative re¬
quirements of the component schedules. Two effects
are commonly reported. First, response rate tends to
increase from earlier components to later components.
Second, the rate of responding in a given component
is determined by its separation from food presenta¬
tion, and is higher when the component is followed by
fewer components or when later components are short
or require few responses. To some extent, number of
components and the schedule requirement in each
component appear to be interchangeable in deter¬
mining temporal separation from food.

Chained Interval Schedules

Two experimental paradigms have been used to

investigate the effects of duration of interval schedule.
In one paradigm, only the terminal component is
varied. In the second paradigm, all components are
varied simultaneously. COMPONENTS BEFORE FOOD
On the basis of the data then available, Kelleher .
Fig. 1 Mean response rate in each component of chained sched¬
and Gollub (1962) concluded that the rate of respond¬ ules with 2, 3, 4, and 5 components, each FI 30-sec, plotted as a
ing in the initial component of 2-component chained function of the number of components that intervene until the
terminal component. The solid line connects the mean rates.
schedules varied positively with the frequency or Note that a logarithmic Y-axis is used. (Adapted from Gollub,
probability of food delivery in the terminal compo¬ 1958.)
292 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

a function of the increasing length of the chain. matching to sample in pigeons under chain FR 3 (FI
Maintained response rates under three-, four-, and X), where X was varied from 16 to 128 sec. The rate of
five-component chained schedules with different FI matches (pecks to a side key whose color matched the
schedules in the components also were controlled previously displayed color on the center key) increased
largely by their separation from food reinforcement from beginning to end of chain, and showed an accel¬
(Gollub, unpublished data). Figure 2 shows rates in erating pattern (scallops) within components.
each component of the following schedules, chain The effects of a given temporal separation are
FR 3 (FI 45-sec), chain FR 4 (FI 30-sec), and chain FR greater for FI component schedules than for compar¬
5 (FI 15-sec), plotted as a function of the mean time able VI schedules. For example, Gollub (1958) showed
that the midpoint of each component preceded food that a 5-component chained schedule with a FI 1-min
presentation. To a first approximation, rate was a de¬ in each component generated extensive pauses,
creasing exponential function of time from reinforce¬ especially in the first two components; with VI 1-min
ment. These experiments show that response rate de¬ components, steady responding was sustained even
creases rapidly as components are separated from food though the components were as distant from food as
presentation by even modest amounts. Thomas (1967) with FI components.
also found that the earlier the component, the more An experiment on 2-component chained schedules
profound the rate change as schedule value was varied by Kendall (1967) suggests that the occasional presen¬
in chain FR 3 (FI X) from 0.25 min to 2 min. tation of short intervals in VI schedules constitutes
Complex behaviors can also be maintained under the important difference between VI and FI compo¬
chained schedules. Boren and Gollub (1972) studied nents (cf. Catania 8c Reynolds, 1968). Kendall (1967)
found higher rates in the first component of chain
VI 1-min VI 1-min when the terminal VI 1-min sched¬
ule provided the first food delivery after 0.25 min
than when the first food presentation was after 1.75
min.

Chained Ratio Schedules

Two experimental paradigms have been used to

study chained ratio schedules. These paradigms
parallel those used with chained interval schedules. In
one paradigm, changes in the response requirement
are made in only one FR component, and all other
components are held constant. The other components
may be fixed ratio, or another schedule. In the second
paradigm, in which all components are fixed ratio, all
of the components are changed simultaneously.
An experiment of the first type was reported by
Hanson and Witoslawski (1959). They showed that
responding in the initial component of chain FI FR
decreased as the number of responses required to pro¬
duce food delivery in the second component was in¬
creased from 5 to 120 responses. Since increases in FR
requirement produced correlated increases in the
0 .5 1.0 1.5 2.0 2.5 duration of the terminal component, changes in the
initial component rate may reflect changes in the
TIME BEFORE FOOD (min)
duration of the terminal component rather than in
response requirement (cf. Killeen, 1969). Findley
Fig. 2. Mean response rate in each component of chain FR 3
(FR 45-sec), plotted as filled circles, chain FR 4 (FI 30-sec),
(1962) showed that changing the ratio requirement in
plotted as triangles, and chain FR 5 (FI 15-sec), plotted as open only one component of chain FR FR FR affects re¬
circles. Each value is plotted as a function of average time at sponding in only the changed component and those
which the midpoint of the component preceded food delivery.
Note that a logarithmic Y-axis is used. (Gollub, unpublished
preceding it, with quantitatively greater effects earlier
data.) in the chain.
Lewis Go/lub 293

An example of the second paradigm for studying component is lowest and the rate increases throughout
chained FR schedules, in which the distribution of a the chain until food presentation occurs. The response
given response requirement in different components rate in a component is controlled primarily by the
is varied, was reported by Jwaideh (1973). She com¬ separation in time of that component from food
pared responding under 3- and 5-component chained presentation, so that number of components and the
FR schedules in which the total number of responses schedule requirement in each are interchangeable
required for food presentation was constant. For when these temporal relationships are constant, as
different pigeons, the total number of pecks required under interval schedules. Changes in overall response
was 60, 120, or 180. For example, one pigeon was rate are, however, modulated by the component sched¬
studied under a chain FR 3 (FR 20) and chain FR 5 ules so that temporal patterns of responding in a
(FR 12). Post-reinforcement pauses were longer under component resemble those typically found under the
the five-component chains than under the three- same isolated schedule of food presentation.
component chains. Response rates following the pause Previous accounts of chained schedules (Ferster 8c
did not, however, vary systematically with different Skinner, 1957; Kelleher 8c Gollub, 1962; Kelleher,
chained schedules. In a similar experiment, Ferster 1966a) have appealed to a dual function of component
and Skinner (1957) found longer pauses before peck¬ stimuli as both discriminative stimuli that control a
ing began under chain FR 4 (FR 30) than under rate and pattern of responding appropriate to the
chain FR 15 FR 105. Segmenting a total number of component schedule, and conditioned reinforcing
responses into differing numbers of components seems stimuli for behavior in a preceding component. This
to have a greater effect on responding than corre¬ interpretation has proved to be exceedingly difficult to
sponding segmentation of an interval (cf. Figure 2). analyze experimentally, at least in part because un¬
ambiguous and independent measures of stimulus
Other Component Schedules control and reinforcement strength are not currently
Experiments on chained FI or chained FR compo¬ available. The next section of this chapter reviews
nents generally reveal a monotonic increase in re¬ some of the ways in which stimulus factors have been
investigated.
sponse rate from beginning to end of the chained
schedule sequence. Such rate patterns are due to the
specific component schedules, and are not necessary
Stimulus Functions in Chained Schedules
consequences of the chaining operation per se. Re¬
quiring a low rate in the terminal component can Four types of manipulation have been used to
reverse the overall rate increase. Ferster and Skinner study the role of stimuli in chained schedules: (1)
(1957) scheduled a chain FR 95 DRL 6-sec in which Omitting stimulus changes with otherwise identical
the ninety-fifth peck changed the key from red to schedule contingencies (tandem schedule): (2) Chang¬
purple; the first peck that was spaced six seconds or ing the order of presentation of the component
more from a preceding peck while the key was purple stimuli; (3) scheduling the sequence of stimuli inde¬
produced food delivery. High rates of responding fol¬ pendent of responding; and (4) presenting the compo¬
lowed a brief pause during the initial red light compo¬ nent stimuli for brief, response-dependent exposures.
nent, and a very low rate prevailed during purple
until the DRL requirement was met. The entraining Tandem Schedules Compared to
effect of periodic food delivery is thus only one factor Chained Schedules
in controlling regular patterns of responding under
chained schedules. A tandem (tand) schedule is one “in which a single
reinforcement is programmed by two schedules acting
Summary of Schedule Effects in succession without correlated stimuli” (Ferster 8c
Skinner, 1957, p. 733). Thus, for every chained sched¬
Parametric investigations of chained interval and ule there is a corresponding tandem schedule, with the
chained ratio schedules implicate three important fac¬ same response requirements. The system for denoting
tors controlling response rate: the type and the num¬ tandem schedules is similar to that for chained sched¬
ber of component schedules, and the requirement in ules. Dissimilar component schedules are listed after
each. These variables operate in the context of re¬ the schedule abbreviation, thus, tand FI 2-min FR 5.
curring food presentations (cf. Staddon, 1972). When When the same schedule appears in each component,
the component schedules are all of the same type (e.g., a denotation like FR X (FI 1-min) is used. Note in
all fixed ratio), the average response rates in the initial this case that no special term is needed to indicate a
294 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

Fig. 3. Tandem and chain FI 3-min FI 2-min. Record A is the transition from tandem
to chained schedule (at the arrow). Record Ai is the continuation of that session.
Records B, C, and D show further development in the chained schedule: Record B is
from the second, C from the fourth, and D from the eighth session on the chained
schedule. Pips indicate change from one schedule component to the next. Small dots
above the record indicate food deliveries. (From Kelleher & Gollub, 1962. © 1962 by the
Society for the Experimental Analysis of Behavior, Inc.)

tandem schedule since it can be described as a second- results studying multiple chained and tandem FI
order schedule (cf. Kelleher, 1966a). schedules. Thomas (1967) studied 3-component sched¬
Gollub (1958) compared the performances of ules with fixed intervals of .25 to 2.0 min. Generally,
pigeons under tandem FI schedules with later per¬ the rates of responding maintained in the terminal
formances of the same pigeons under the correspond¬ component of chained schedules were higher than in
ing 2- and 5-component chained schedules. In all corresponding tandem schedules with the difference
cases, the response rate in the terminal component of decreasing with increasing values of the fixed-inter¬
the chained schedule increased over that previously vals. The rates in the initial and middle components
maintained in the terminal component of the tandem of the tandem schedules were higher than those tinder
schedule. The rate in the initial component increased chained schedules, with the difference increasing with
in two-component chains, but decreased with five increasing interval duration.
components. The temporal pattern of responding also Jwaideh (1973), in the experiment described above
changed. Figure 3 shows the transition from tand FI on chained and tandem FR schedules found that the
3-min FI 2-mm to chain FI 3-min FI 2-min, A similar pause after food presentation was generally longer for
result for chain FR 3 (FI 1-min) components was re¬ chained than for tandem schedules (cf. Ferster & Skin¬
ported by Kelleher and Fry (1962). ner, 1957). In addition, the rate after responding be¬
In some experiments, however, rates in tandem and gan (the running rate) in the first component was
chained schedules have not differed substantially. For lower in chained than in tandem schedules in most
example, Malagodi, DeWeese, and Johnston (1973) cases, and the mean time to complete the required
studied pigeons under chain FR 2 (FI 2-min) and response number was greater for chained than tandem
tand FR 2 (FI 2-min), and found comparable average schedules, with the difference increasing with increas¬
rates in both schedules. Appropriate temporal pat¬ ing total response requirement. Thomas (1964) also
terns of responding occurred under the chained found lower average rates in the initial chain compo¬
schedule. nent of multiple chained and tandem FR schedules.
Performances can be studied under both tandem In summary, experiments comparing performance
and chained schedules in each session if there is one under tandem and chained schedules have yielded
stimulus for the tandem schedule and additional complex results. For two-component chains of FI
stimuli for the chained schedule components (multi¬ schedules, the rate in the first component under chain
ple chained and tandem schedules, cf. Thomas, 1964). was generally higher than tandem (Gollub, 1958), but
Gollub (1965) and Thomas (1967) obtained similar not always (Malagodi, DeWeese, & Johnston, 1973).
Lewis Gollub 2 95

Higher chained schedule rates have been interpreted their original chained schedule components. For
as showing a conditioned reinforcing effect: respond¬ example, the reverse order from the chained schedule
ing that produces the terminal stimulus occurs at a produced a negatively accelerated pattern of respond¬
higher rate than responding the same distance from ing. With continued exposure to the variable order
food that does not change the stimulus conditions. of stimuli the pigeons responded at a more or less
Chains with more than two FI components have con¬ constant rate throughout each schedule sequence, with
sistently shown lower rates in the initial component only brief pauses, if any, in the initial component.
than occur under corresponding tandem schedules This pattern resembled that under the tandem sched¬
(Gollub, 1958; Kelleher 8c Fry, 1962). Similarly, rates ule, with a single stimulus presented throughout the
were lower and pauses were longer in the initial com¬ chain. After still further training, a new pattern of
ponent of three- and five-component chained FR responding developed. Pauses after food were seldom
schedules than in corresponding tandem schedules longer than the scheduled interval (1.5 min), and
(Ferster 8c Skinner, 1957; Jwaideh, 1973; Thomas, miniature scallops comprised of a pause followed by
1964). accelerated responding typically occurred in both the
The comparison of chained and tandem schedules middle and terminal components. Similar effects of
is more complicated, however, than might appear at reversing the order of component stimuli of chained
first glance. Although schedules that are formally FR schedules were reported by Jwaideh (1973) and by
identical can be arranged, whether they are sufficiently Ferster and Skinner (1957) with a continuously vary¬
comparable is a moot question. For example, in what ing stimulus (an added counter).
sense is a FI component of a chained schedule in The preceding experiments as well as others (Find¬
which its duration is marked precisely by a component ley, 1962; Marr, 1971) on the effects of varying the
stimulus comparable to a FI component of a tandem stimulus sequence and the relationship of stimulus
schedule without accompanying time markers? The order to food presentation demonstrate that exposure
concept of reinforcement contingencies implies that of to chained schedules develops strong discriminative
discriminative stimuli (Skinner, 1969). Comparisons control of responding by the component stimuli. This
among schedules which are as different as tandem and control is demonstrated by the maintenance of charac¬
chained may involve too many differences to illu¬ teristic rates by each stimulus component when the
minate specific aspects of control by either schedule. stimulus is presented in a new sequence. These effects
demonstrate little, however, about conditioned rein¬
Varying the Order of Stimuli
forcing effects of chained schedules. One other manip¬
in Chained Schedules
ulation of stimulus order in chained schedules can
provide relevant data here: scheduling the terminal
The discriminative and reinforcing functions of the stimulus in two or more different components of a
component stimuli of chained schedules have been chained schedule. Responding during the early presen¬
investigated by changing their order of presentation. tation of that stimulus would show discriminative
Several experiments have used the following tech¬ effects, independently of time of presentation. Re¬
nique. Responding is first stabilized under a chained sponding in the presence of the stimulus component
schedule. The order of stimuli is then changed, and that precedes an early presentation of the terminal
the transition and stable performances under the new stimulus component would reflect the reinforcing
order are studied. In general, the results have shown effects of that stimulus as a consequence of re¬
strong control of response rate by the prevailing sponding.
stimulus, so that high rates can be produced in early Byrd (1971) studied the performances of pigeons
components of the chained schedule when a stimulus on chained schedules with 3, 5, or 7 FI 1-min compo¬
that has been presented previously toward the end of nents in which the same key color was presented in
the chain is presented early. In addition, a stimulus the terminal component as in some earlier compo¬
that used to occur near the end of a chain can increase nents (e.g., in the 1st, 3rd, and 5th, as well as in the
responding when it is presented as a consequence of 7th component). Different colors were presented in the
responding in an earlier component. other components. Strong rate-controlling effects of
After establishing performances under chain FR 3 such repeated stimuli were demonstrated. In all cases,
(FI 1.5-min), Kelleher and Fry (1962) presented the higher rates occurred during the early presentation of
three stimulus components in different orders on suc¬ the repeated color than in the component that fol¬
cessive exposures to the chain. At first, the three colors lowed it. In one phase of the experiment Byrd (1971)
controlled rates very similar to those maintained in presented one color (amber) in the 1st, 3rd, and 5th
296 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

components of a 7-component chain, and unique line presented immediately after food presentation
colors in each of the other components. The rates in and shrinking to a spot at the end of the FI period,
those components preceding amber (2nd and 4th) response rates were maximal at the beginning of the
were compared with the rates previously maintained interval, and decreased to zero as time passed. These
when the repeated stimulus also was paired with food, results resemble effects reported by Jwaideh (1973)
i.e., was presented in the 7th component. Rates were and Kelleher and Fry (1962) of comparable manipula¬
not higher under the latter condition. Byrd (1971) tions of chained schedule component stimuli. (See also
concluded that these results did not reveal a con¬ Segal, 1962; Boren 8c Gollub, 1972.)
ditioned reinforcing effect of the component stimuli. With pigeons on chained FI schedules, the early
An unpublished experiment by Gollub indicates components frequently exceed their scheduled dura¬
that chain component stimuli presented out of order tions, since low rates with extensive pauses prolong
can increase responding that produces these stimuli, them. The discrete-value “clock” of the chained sched¬
but that these effects may be transient. Responding ule thus has settings with unequal durations. Tallen
was maintained under chain FR 4 (FI 1.5-min). The and Dinsmoor (1969) produced such a 3-valued clock
color presented in the terminal component was then by presenting the key colors to one pigeon that were
also presented in the second component. Responding simultaneously produced by a second pigeon under a
increased immediately in both the initial and second chain FR 3 (FI) with FI values of 20-sec, 30-sec, or
components. Over the succeeding 27 sessions, how¬ 45-sec (yoked-box technique). Responding under the
ever, responding in the initial component returned to clock schedule was confined almost entirely to the
its previous value, and responding in the second com¬ final stimulus (97.6-100% of all pecks), whereas pi¬
ponent also decreased markedly, Thus, an initial rate- geons under the chained schedule responded through¬
enhancing effect of a chain component stimulus can out the chain, with only 41.3-72% of their pecks in
disappear with continued training. With continued the terminal component, and the remainder of their
training, white after orange (or 1.5 min after food) pecks in the first two components.
gains different discriminative control from while after In summary, responding occurs at very low rates in
bins (or, 4A min after food). Byrd's (1971) pigeons had the presence of stimuli that are never associated with
been e^ps>sed te presentation of the terminal stimulus food. When responding in the presence of such stim¬
at multiple locations in the chain for 115 sessions be¬ uli is required for progression through the chain, re¬
fore the comparison mentioned above. It would not sponse rates increase substantially. Such comparisons
be surprising if Control by tbe complex stimulus bad do not, however, distinguish the effects on responding
developed by this time. of the response-dependent presentation of subsequent
chain stimuli from the effects of the response require*
ment for the ultimate presentation of food. More
Clock Schedules
comprehensive designs incorporating comparable tan¬
A major effect of the initial stimulus, especially in dem, chained, and clock procedures would help com¬
chains with more than two components, is discrimina¬ pare the discriminative effects of component stimuli
tive. It controls very low rates. By definition, the ini¬ (clock vs. chain) and the effects of the response de¬
tial stimulus component occurs furthest from the next pendencies (chain vs. tandem).
food presentation, and responses in its presence are
never followed promptly by food. Outside the chain¬
Brief Presentation of Component
ing situation, such a stimulus would be referred to as
and Clock Stimuli
SA or S~, and would normally control very low re¬
sponse rates (see Chapter 15). If component stimuli are reinforcers, a response
An interesting comparison to chained schedules is that produces them even outside a chaining situation
the “added clock” of Ferster and Skinner (1957). They should be maintained. Hendry and Dillow (1966)
presented a spot of light on the key that grew pro¬ found that clock stimuli from FI 3-min and FI 6-min
gressively longer in a FI period reaching a maximum maintained pecks on a second key, with presentation
length when the next peck would produce food. The of the terminal stimulus maintaining the highest rate.
effects were quite dramatic. Zero response rates oc¬ Whether each clock stimulus alone could maintain
curred in early parts of the interval, and were fol¬ pecking was not directly assessed.
lowed, often with a relatively rapid transition, by Kendall (1972) investigated this question with a
very high rates, sustained until food presentation. similar procedure. Pecks on one key produced food
When the dock was “run backwards,” with the long under FI 3-min. Pecks on a second key produced a
 Lewis Gollub 297

0.2-sec presentation of white, green, or red key-light stimuli. Stimuli that provide the same “information”
in the first, second, and third minutes of the interval, concerning elapsed time in the interval, but that have
respectively. In two later conditions of the experi¬ no close behavioral contiguity with food, did not sus¬
ment, pecks on the second key produced red during tain behavior (cf. Chapter 11).
the last minute, and had no consequences during the Marr (1969) found the corresponding result for
first two minutes, or produced white and green during stimulus components of chained schedules with a com¬
the first and second minutes, respectively. Figure 4 plex one-key procedure. Briefly, higher rates were
shows that pecks that produced time correlated stim¬ maintained by brief presentations of the terminal
uli were maintained only when red, the terminal stimulus than by presenting the initial stimulus.
minute stimulus, could be produced either as one of In summary, these experiments show that stimuli
three stimuli, or as the only stimulus. When only the that are serially correlated with elapsed time in the
first two stimulus components could be produced, interreinforcement interval of either fixed-interval or
responding was not maintained. Whether green pre¬ chained schedules control the rate of responses that
sented during the middle minute would alone sustain produce them. Brief presentation of the terminal
responding was not assessed. White, appearing when stimulus of a 3-component chain will maintain con¬
food was remote, might have suppressed pecks mask¬ siderably higher rates than presentations of the first
ing any rate-enhancing effect of response-dependent stimulus; presentation of the terminal stimulus alone
presentations of green (cf., Mulvaney, Dinsmoor, will maintain as much responding as presenting all 3
Jwaideh, 8c Hughes, 1974, for a direct demonstration clock stimuli; and presenting only the first 2 of 3 clock
of response suppression by S~~). The experiment does stimuli will not maintain responding.
show clearly that responses must produce the terminal
clock stimulus at least part of the time to be main¬
Concurrent Chained Schedules
tained. Since the “information” concerning the occur¬
rence of the terminal minute is the same whether a A discussion of chained schedules would not be
unique positive signal (red) is given or the lack of a complete without some consideration of concurrent
negative signal (white or green) occurs, this experi¬ chained schedules. (See Chapter 11 for a more com¬
ment strengthens the line of argument that association prehensive discussion of this topic.) This procedure
with food or other positive reinforcers is necessary for has a basic appeal both for examination of the param¬
the development of response maintaining effects by eters of reinforcement in chained schedules (Kelleher
8c Gollub, 1962) and for scaling diverse reinforcement
variables (cf. Baum 8c Rachlin, 1969).
The procedure was developed by Autor in his doc¬
toral dissertation in 1961, and later published (Autor,
1969). The schedule can be conveniently divided into
two parts. In the first (concurrent) part, two response
keys are lighted. Equal VI schedules are associated
with each key so that a peck occasionally produces the
terminal component associated with that key. In the
second (terminal component) part of the schedule, the
color of the key changes, and the other key is not lit.
Pecks in the terminal component produce food ac¬
cording to a schedule associated with that key. After
food, the concurrently lit keys are again available.
Autor (1969) showed that the relative rates of re¬
sponding during the concurrent part of the schedule
were similar to the relative frequencies of reinforce¬
ment during the terminal components. This matching
relationship occurred not only when food presentation
Fig. 4. Fixed-interval and observing response rates for each of in the terminal components was response-dependent
the three birds in each of the three conditions. Fixed-interval under VI schedules, but also when food was presented
rates are plotted in the upper panel and observing response
occasionally and pecking was not permitted. Herrn-
rates in the lower panel for each bird. (From Kendall, 1972. ©
1972 by the Society for the Experimental Analysis of Behavior, stein (1964) extended these results by scheduling food
Inc.) presentations under variable-ratio (VR) schedules. He
298 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

found that response proportions in the concurrent components. Over the experiment these were VI 40-
components tended to match more closely the relative sec, VI 120-sec, and VI 600-sec. The mean relative
temporal frequencies of food presentation than rela¬ response frequency on the key associated with the VI
tive number of responses per food presentation. Be¬ 30-sec terminal component was 0.95, 0.81, and 0.60,
cause of the growing interest in studying the effects of respectively, as the initial schedules were varied. Thus,
reinforcement variables on relative measures of re¬ response distribution in the initial components de¬
sponding (Herrnstein, 1961), many papers followed pends in part on the absolute value of the equal, con¬
these initial studies. Relative response rate was stud¬ current schedules. Alternative quantitative formula¬
ied when the terminal components differed in: num¬ tions of relative response rates in the concurrent
ber of food presentations (Fantino 8c Herrnstein, 1968); components as a function of reinforcement variables in
FR versus VR schedules (Fantino, 1967); schedules of the terminal components must include some weight¬
differential reinforcement of response rate (Fantino, ing for the concurrent schedule values (Davison 8c
1968); amount and frequency of food presentation Temple, 1973; Fantino, Chapter 11 in this Volume;
varying jointly (Ten Eyck, 1970); and FI versus VI Squires 8c Fantino, 1971).
schedules (Davison, 1969). In these and related studies, Quantitative models to describe behavior under
the relative rate of responding in initial concurrent concurrent chained schedules have grown increas¬
links was used to scale preference or value (Baum 8c ingly complex as experimental variables have been
Rachlin, 1969; Killeen, 1972) of the terminal links. extended beyond the initial limited values (see Fan¬
Concurrent chained schedules have even been used to tino, Chapter 11 in this volume). In part this may
study reinforcement variables in simple chained sched¬ reflect a terminological problem. It is a misnomer to
ules themselves. In these experiments, two concurrent call this experimental paradigm concurrent chains.
initial components lead either to a chained schedule Concurrent scheduling occurs for only part of the
or to a uniform stimulus providing food after about schedule. (For one exception, which was characterized
the same delay as the chained schedule. Schneider by perhaps the most complex quantitative relation¬
(1972) compared cham VI VI and chain FI FI sched¬ ships in this field of research, see Fantino & Duncan,
ules with corresponding tandem schedules. Duncan 1972.) This inaccurate terminology may have also
and Fantino (1972) compared chain FI FI with FI, distracted attention from what appears to be the crux
and chain FR $ (FI) with chain FR 2 (FI), Unfortu¬ of the matter: the organism is enmeshed in a com¬
nately, the two studies found strikingly different re¬ plex set of contingencies, where, among other things,
sults. Schneider (1972) found essentially equal respond¬ termination of the concurrent components not only is
ing leading t6 chained and tandem schedules, whereas followed by one of the terminal components, but also
Duncan and Fantino (1972) found more responding delays the presentation of the alternative terminal
leading to the less segmented schedule (FI versus chain component. Only recently has this point been appre¬
FI FI, and 2-component versus 3-component chained ciated (Duncan 8c Fantino, 1972)- It is naive to con¬
schedules). Since only a small range of parameters was sider concurrent chained schedules and related pro¬
explored in terms of schedule type and schedule val¬ cedures as a simple technique for preference scaling of
ues, it would be premature to speculate on which pro¬ the terminal components.
cedural differences were responsible for the different In summary, the concurrent chains procedure, un¬
results. As with many questions about chained sched¬ derstood as a name and not a description, is a com¬
ules, further research is necessary to clarify the results. plex behavioral situation in which responding in the
Surprisingly, in view of the theoretical weight be¬ initial components is a function of the absolute and
ing placed on the relative rate data from concurrent relative values of both the initial and terminal com¬
chains the constraints in the basic paradigm have ponents. Much additional research is required to elu¬
been little explored until recently. For example, in all cidate the individual sources of control in this situa¬
seven studies on concurrent chained schedules men¬ tion.
tioned earlier the schedules for terminating the con¬
current components were equal (identical VI 1-min).
Conditioned Reinforcement in Chained Schedules
As it turns out, the schedules in the concurrent
components are important determinants of the rela¬ The maintenance of responding under chained
tive concurrent response rates. Fantino (1969) sched¬ schedules has often been interpreted in terms of con¬
uled food presentations in terminal components ac¬ ditioned reinforcement (Ferster 8c Skinner, 1957; Gol-
cording to VI 30-sec and VI 90-sec schedules. Identical lub, 1958; Kelleher 8c Gollub, 1962). The preceding
pairs of schedules were used in the concurrent initial review indicates, however, that the strongest stimulus
Lewis Goliub 299

effects that are demonstrable in chained schedules are presentation of food. These are second-order schedules
discriminative, i.e., the modulation of responding by (Kelleher, 1966a, b). In another procedure, the brief
prevailing stimulation due to previous reinforcement stimulus is presented according to an independent
history in the presence of these stimuli. If reinforce¬ schedule that is concurrent with the schedule of food
ment implies enhancement of responding by response- presentations (J. Zimmerman, 1963).
dependent stimuli, then only in certain circumstances
(e.g., when land FI FI is changed to chain FI FI) does
Second-order Schedules of Brief
response rate increase following response dependent
Stimulus Presentation
presentation of temporally sequential stimuli. The
difficulty in demonstrating clear reinforcement effects Kelleher (1966a) defined a second-order schedule in
of stimulus components of chained schedules has led this way: “A second-order schedule is one in which the
some writers to conclude that these stimuli are not behavior specified by a schedule contingency is treated
reinforcers (e.g., Schuster, 1969). But response en¬ as a unitary response that is itself reinforced according
hancement has been reported under certain condi¬ to some schedule of primary reinforcement" (p. 181).
tions: in acquisition following tandem schedules, in This comprehensive definition applies to tandem
repeated presentation of stimulus components, in com¬ schedules (no additional stimuli), chained schedules
parisons of chained and equivalent serial stimulus (different stimuli during each separate schedule), and
(clock) schedules, and in a direct examination of how schedules of brief stimulus presentation, in which a
much responding is maintained by response-dependent brief stimulus is presented according to the unit sched¬
presentations of component stimuli outside the chained ule contingency. For example, Findley and Brady
schedule. Whether responding is enhanced, suppressed, (1965) presented food when a chimpanzee pressed a
or unchanged in a chained schedule compared to be¬ key 4000 times (FR 4000) in the presence of a red
havior under some procedurally related condition light. In the presence of a green light, food was also
depends on the specific situations compared. Stimulus presented on the 4000th response; m addition, the
components of chained schedules may have complex food hopper was lighted (designated as S) for 0.5 sec
behavioral effects that prevent an unambiguous pre¬ after every 400 responses (FR 400:S). In the terminol¬
diction. ogy of second-order schedules, the schedule of food
presentation is denoted as FR 10 (FR 400:S). That is,
food is delivered upon the tenth repetition of the FR
SCHEDULES OF BRIEF STIMULUS 400 schedule. To facilitate description of these sched¬
PRESENTATION ules, the schedule requirement for stimulus presenta¬
tion will be referred to as the unit schedule.3 The
Extinction tests of conditioned reinforcement fre¬ following review will consider the effects of different
quently involved brief presentation of a stimulus that types of unit schedules on response rate and response
had previously been associated with food, such as a pattern, and will then analyze some of the controlling
magazine sound. As discussed above, interpretation of variables.
these experiments was often ambiguous. Few responses
were maintained during extinction, at least in part Overview

because the association between stimulus and food was

Stimuli that are paired with food or other reinforc¬
broken, and the reinforcing effectiveness of the stim¬
ers can maintain long, orderly sequences of responding
ulus was itself undergoing extinction.
when they are presented according to second-order
More recently, techniques have been developed to
schedules. The pattern of responding is appropriate
study the effects on behavior of brief stimuli where
to the unit schedule, and the overall rate of respond-
food presentation continues. These techniques pro¬
vide a chronic situation for the study of brief stimuli 3 No standard vocabulary has emerged to describe the dif¬
ferent parts of second-order schedules. It has been suggested
as maintaining events with some of the same ad¬
that the schedule according to which the brief stimulus is
vantages over extinction test procedures that the study presented (FR 400 in this present example) be called the sub¬
of schedules of reinforcement has over the traditional ordinate schedule, and the schedule for arranging food (FR 10),
the super ordinate. The priority implied in such a nomenclature
extinction tests of partial reinforcement.
may not always apply, especially when schedules of the type
Two major classes of procedure have been developed FR (FI) are studied. In that case, the superordinate schedule
in the study of response maintenance with brief stim¬ would bear no important relationship to the ensuing behavior,
which more closely resembles that under FI. Similarly, some have
uli. In one, presentations of the brief stimulus are
suggested food schedule, instead of superordinate. Again, it is the
scheduled as part of the schedule that arranges the entire schedule, FR (FI), that arranges food presentation.
 300 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

ing is frequently higher than that maintained by the and FR 15 (FR 4-min :S) where S was a change in key
corresponding second-order schedule without a brief color from blue to white for 0.7 sec. Under both sched¬
stimulus (tandem schedule). The interpretation of ules, the minimum time between food presentations
these results is, however, subject to controversy. In¬ was 60 min. For comparison, behavior was also stud¬
creased rate under some circumstances can be attrib¬ ied under the corresponding tandem schedules. Three
uted to discriminative effects of the brief stimulus aspects of the results are important. First, under all
with respect to food delivery. Other experiments have four schedules (15 and 30 unit schedules, with and
examined the relationship of brief stimulus to food without brief stimulus presentations) behavior was
presentation: no single temporal relationship seems well maintained over the one-hour periods separating
necessary. food presentations. This is in marked contrast to the
effects of chained schedules, in which substantial
Patterns of Responding Under pauses usually occur when even 5 components of FI
Second-order Schedules of 1-min are scheduled (cf. Squires, Norborg, 8c Fantino,
Brief-stimulus Presentation 1975). Second, the average rate of responding was low
for first few unit schedule intervals, and then in¬
Fixed- and Variable-interval Unit Schedules. Long,
creased. This is illustrated by the cumulative records
orderly sequences of behavior can be maintained un¬
shown in Figure 5. Third, the brief stimulus con¬
der second-order schedules of brief-stimulus presenta¬
trolled the temporal pattern of responding. There
tion with FI unit schedules. Kelleher (1966b) scheduled
was usually a pause after each brief stimulus, fol¬
food presentation to pigeons under FR 30 (FI 2 min;S)
lowed by either a gradual or an abrupt acceleration to
a moderately high rate until the next stimulus presen¬
tation. The maximal rate in each unit of the sched¬
BIRD 128
ule sequences with the brief stimulus was generally
higher than the maximal rate reached under the
schedule without brief stimuli (11 out of 12 com¬
parisons). This difference is illustrated in Figure 6
which shows mean response rates in successive quar-

BIRD 128

Fig. 5. Effects of omitting presentations of the white light at QUARTERS OF FIXED INTERVAL
the end of each FI 4-min component (Bird 128). Each cumulative Fig. 6. Effects of omitting presentations of the white light on
response record shows a sequence of 15 consecutive FI 4-min mean rates of responding in each quarter of the fixed-interval
components; each sequence terminated with food reinforce¬ components (Bird 128). Each point is the median of the mean
ment. Short diagonal strokes on the records designated FR 15 rates of responding in the last five sessions under each pro¬
(FI 4-min:W) indicate 0.7-sec presentations of white light. Under cedure. Solid circles: white light presented at termination of
the FR 15 (FI 4-min) schedule, there were no exteroceptive each fixed interval component; triangles: no exteroceptive
stimulus changes during the sequence; short diagonal strokes stimulus change at termination of fixed-interval components;
indicate the end of each FI 4-min component. (From Kelleher, open circles: redetermination of effects of presenting white light.
1966b. © 1966 by the Society for the Experimental Analysis of (From Kelleher, 1966b. © 1966 by the Society for the Experi¬
Behavior, Inc.) mental Analysis of Behavior, Inc.)
Lewis Gollub 301

ters of the unit schedule interval under schedules each food presentation typically follows between 2
with (circles) and without (triangles) brief stimulus and 4 unit schedule sequences, compared to as many
presentations. Figure 6 also shows that the overall as 64 in experiments on second-order schedules. Also,
rate of responding, i.e., the mean number of re¬ response rates in these experiments are generally com¬
sponses per food presentation, was generally greater pared to rates under a simple FI, or involve compari¬
under the schedule of brief stimulus presentation than sons of performance following food with those follow¬
under the comparable tandem schedule. Boren (1973) ing time out, whereas in Kelleher (1966a, b) and most
found similar patterns of responding when pecks on a other experiments, comparisons are made to tandem
matching-to-sample procedure produced food under schedules. The two research paradigms thus cover
FR (FI:S) schedules. different values of schedule, and make comparisons
Reinforcement omission, or percentage reinforce¬ with different baselines. The research on reinforce¬
ment of FI schedules, also involves second-order FI ment omission has, however, contributed to the analy¬
schedules. In this paradigm, responding is stabilized sis of interpretation of the behavioral function of the
under a FI schedule, and food delivery is then omitted brief stimulus, to be discussed later.
at the end of some percentage of the intervals. A time Short, response-initiated fixed-interval unit sched¬
out (a period in which all lights in the experimental ules were studied by Neuringer and Chung (1967),
chamber are turned off) or other brief stimulus oc¬ Chung and Neuringer (1967), and Neuringer (1968).
curs instead. The sequence of intervals ending with Chung and Neuringer (1967) studied a procedure in
and without food is irregular. The total schedule com¬ which a key peck started a fixed interval of 1 sec to
prises a VR (FI:S), in that all intervals end in a brief 30 sec duration. The first peck after the interval
stimulus (S) and after a variable number of these elapsed produced either a 1-sec blackout (S) or, after
intervals, food also follows.4 Staddon and Innis (1969) variable intervals averaging one minute, food. The
found that responding began much sooner in those schedule can thus be notated VI 1-min (Land FR 1 FI
2-min fixed intervals which were preceded by time out X:S), with S unpaired with food. The pause before
than in those preceded by food. Mean response rates the first peck after blackout increased linearly with
were therefore higher under the second-order schedule interval duration; the mean rate of responding during
than under fixed-interval. The pause at the beginning the fixed-interval period decreased as a negatively ac¬
of the interval varied with the duration of time out, celerated function of interval duration (cf. Starr 8c
which ranged from 2 to 32 seconds. Similar effects of Staddon, 1974). In some parts of the experiment, food
omitted food delivery were demonstrated by Staddon was scheduled after every unit schedule, a FR 1 (tand
(1970, 1972, 1974) and Kello (1972). FR 1 FI X). Response rates under different values of
In a related experiment, Zeiler (1972) found that the fixed interval were similar to those with VI 1-min
the smaller the percentage of intervals ending with in the second-order schedule, and the pause in these
food, the greater the responding earlier in the inter¬ intervals was lower than when VI was scheduled.
val. As in a number of other experiments with this Chung and Neuringer (1967) reported similar rates
paradigm, Zeiler (1972) found that time out was an for tand FR 1 FI schedules as for fixed ratio (FR 11).
important factor in obtaining temporal control Thus, the contingencies of the short response-initiated
through the interval. In a related experiment, Zeiler interval schedules may make them more comparable
(1972) studied the effects of presenting a 10-sec black¬ to ratio than interval schedules.
out after every interval in addition to either a 4-sec
presentation of food or 4-sec time out that substituted
for food. He found that responding was approx¬ Fixed-ratio Unit Schedules. Second-order schedules
imately equal in all intervals, whether they ended in with fixed-ratio unit schedules of the form FR (FR:S)
food or not. offer special circumstances for examining the behav¬
Response rate increases in the reinforcement omis¬ ioral properties of the brief stimulus. In such sched¬
sion experiments are not necessarily related directly to ules, food is presented after a fixed number of re¬
the occurrence of higher rates with schedules of brief sponses, and performance can thus be compared to
stimulus presentation. In the former experiments, FR schedules with the same total response require¬
ment. The added brief stimulus does not contribute
4 To be precise, this is a second-order schedule with unpaired additional response or temporal requirements, as with
stimulus, since time out does not precede food when the latter fixed-interval unit schedules, or with chained sched¬
was presented. Some differences between paired and unpaired
ules.
brief stimuli, and between blackout and other events as the
brief stimulus, will be discussed later. The facilitatory effects of an added brief stimulus
 302 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

under FR 10 (FR 400:S) in the chimpanzee and stimulus may be unwarranted, since performance de¬
monkey were described earlier (Findley & Brady, 1965). pends on the specific scheduling of the brief stimulus.
Thomas and Stubbs (1967) found a similar effect with Parametric variation of schedule values is clearly
pigeons responding under FR 5 (FR 30 :S) compared necessary to determine functional properties in this
with FR 150. complex situation.
Lee and Gollub (1971) studied second-order FR The effects of different FR unit sizes was also in¬
schedules of the form FR X (FR Y:S), with X • Y = vestigated by Shull, Guilkey, and Witty (1972). They
256. Values of X equal to 1, 2, 4, 8, 32, and 128 were studied FI (FR) schedules in which 10 or 20 key pecks
presented in both ascending and descending orders. turned off the key light for 0.7 sec; pigeons were
The brief stimulus was a 0.5-sec change in key light studied under FI 3 min, FI 6-min, and FI 12-min
from red to green. Although 256 responses were re¬ schedules, and corresponding second-order schedules.
quired for food presentation under all conditions, the The FR unit schedules generated the pause-run re¬
size of the unit schedule (or, conversely, the number sponse pattern typical of FR schedules of food presen¬
of unit schedules and light flashes per food presenta¬ tation. Although Shull et al. (1972) did not present
tion) had dramatic effects on responding, as shown in quantitative comparisons of rate, cumulative response
Figure 7. The FR 256 schedule is represented by the records showed more responses per interval under FR
extreme left point of each graph. Note that the high¬ 10 unit schedules than under simple FI, and more still
est median overall rates were obtained when the unit under FR 20. Remarkably, despite these changes in
schedule was FR 64 or FR 128 (corresponding to 4 or rate and temporal organization of responding, the
2 unit schedules per food presentation). Under the percent of the interval to the first response was not
FR 2 unit schedule responding under the second-order affected by interval duration, and increased only
schedule occurred at lower or equal rate to that under slightly as the behavior required for food presentation
FR 256. increased from 1 peck to 20 pecks. This result shows
This parametric study indicates that global gen¬ the strong temporal control of some features of re¬
eralizations based on comparisons between a single sponding by periodic food presentation, as well as
second-order schedule of brief stimulus presentation control by the unit schedule.
and a corresponding FR schedule without a brief Detailed measurement of responding within FR
unit schedules shows the same pattern of interre¬
sponse times (IRT) within the unit schedule as that
shown when food is presented under FR schedules (cf.
Gott 8c Weiss, 1972). Kelleher (1966a) studied pecking
in pigeons under a FI 10-min (FR 20:S) schedule. The
time to the first peck in each group of 20 pecks was
generally longer than other IRTs, and approximately
equal IRTs characterized the next 19 pecks. Davison
(1969), in an experiment with rats, only partly con¬
firmed these results. Many differences in procedure
between these studies makes a detailed comparison
impossible.

The Analysis of the Behavioral

Functions of Brief Stimuli

Two alternative behavioral functions have been

CONDITION (FRUNIT)
proposed for the brief stimulus, reinforcing and dis¬
Fig. 7. Median overall response rates under each condition for criminative. Compared to responding under identical
Birds 82 and 80. Each pigeon was studied under FR X (FR Y:S) schedules of reinforcement without brief stimulus
second-order schedules of brief stimulus presentation. A total
presentation, increases in overall or local rates of a
of 256 responses was required for each food presentation such
that X*Y = 256. The values shown are medians of the median response that produce the brief stimulus, as in Kelle¬
rate in the last five sessions. Median rates within each session her (1966a, b), Findley and Brady (1965), and Lee and
were calculated from a printed record of the time elapsing be¬
Gollub (1970), directly implicate a reinforcing func¬
tween food reinforcements. Arrows indicate the order in which
the conditions were varied. (From Lee & Gollub, 1971. © 1971 by tion.
the Society for the Experimental Analysis of Behavior, Inc.) The brief stimulus also controls the temporal pat-
Lewis Goliub 303

tern of responding. Unit schedules that terminate in shown in Figure 8. Mean response rate in each quar¬
a brief stimulus engender patterns similar to equiv¬ ter of the FI in three replications with white key light
alent schedules of food presentation. The brief stim¬ paired with food are shown by circles. The rates with
ulus thus serves a discriminative function by control¬ the other (unpaired) dark key are shown in the left
ling responding subsequent to its presentation. panel, and those with (unpaired) red key, in the right
Experimental analysis of the discriminative and panel. Both overall rates and rate changes within the
reinforcing functions of brief stimuli has been at¬ FI period were different under the three conditions.
tempted with several different techniques. The rein¬ Highest overall and terminal rates were obtained
forcing effect has been examined with the concurrent when the light that was paired with food was pre¬
chains procedure. Schuster (1969) scheduled as the sented. Lowest mean rates, with little temporal con¬
terminal components of concurrent chained schedules, trol by the FI unit schedule, were obtained when a
VI (FR 11:S) and VI, arguing that if S served a rein¬ dark key was scheduled. The effects of the red light
forcing function, its presentation in only one terminal were intermediate: moderate mean rate, with some
component should enhance the total reinforcing temporal control, although terminal rates in the inter¬
effectiveness of that component. Instead, the pigeons val were not as high as with the paired stimulus.
had slightly higher rates in the initial component that One of three pigeons in the experiment responded al¬
did not produce the second-order schedule. This re¬ most as much with red as with white brief stimuli.
verse preference may result from the fact that dis¬ Although Kelleher (1966b) interpreted these results as
parate response rates were generated in the terminal showing “that it may be necessary to present a
components (cf. Goliub, 1970; Moore 8c Fantino, 1975). stimulus in temporal contiguity with a reinforcing
Another type of analysis of the reinforcing func¬ stimulus if the former stimulus is to become an effec¬
tion of brief stimuli is related to the process by which tive conditioned reinforcer” (p. 484), he also indi¬
conditioned reinforcers are established. Kelleher and cated that the specific stimulus used and its previous
Goliub (1962) argued that a conditioned reinforcer associations with food, as well as the amount of train¬
was established by simple contiguity of a stimulus ing with an unpaired stimulus (here, 17 and 7 sessions,
with an effective reinforcer. (Alternatively, Keller & respectively) might be important parameters of the
Schoenfeld, 1950, and others, have maintained that effect.
the conditioned reinforcer must be a discriminative Subsequent experiments have confirmed some
stimulus for an operant, and still others have indi¬ aspects of these results but have been equivocal about
cated an informational function, discussed at greater others. Over 20 studies have been concerned with the
length in Chapter 11.) The pairing hypothesis, and pairing operation of brief stimuli in second-order
to a lesser extent the discriminative stimulus hypoth¬ schedules. While the majority of comparisons have
esis, have been tested in a series of studies that shown that a stimulus paired with food produced
examined the effect of pairing the brief stimulus with higher rates, or more pronounced control of rate
food. changes by the unit schedule, or both, many of these
studies have suffered from a serious methodological
Pairing the Brief Stimulus with Primary Rein¬
flaw: the stimulus paired with food was a physically
forcers. Kelleher (1966b) compared behavior under a
different event from the unpaired stimulus. In some
second-order schedule of a brief stimulus paired with
studies the difference was small (e.g., in Kelleher,
food with behavior under a second-order schedule of
1966b, white key light vs. red key light) and the effect
a stimulus not paired with food. In the former case, a
could reasonably be attributed to the pairing opera¬
white light was presented for 0.7 sec under a FR 15
tion. In other studies the difference was considerable
(FI 4-min:S). Comparison conditions consisted of
(presentation of a light in the feeder vs. a light on the
scheduling for 0.7 sec either an unlit key (D) or red
key, de Lorge, 1969; Stubbs, 1969) and differences in
key light (R) 14 times, with food alone presented after
the fifteenth interval. In this case S was an unpaired
stimulus.5 The results for one of the three pigeons is some experiments, e.g., Kelleher (1966b), food presentation under
unpaired conditions is delayed by the duration of the brief
stimulus, to match the key peck-food relation of the paired
5 It should be noted that the precise notation for the schedule. Another procedure, which would have certain formal
schedule of presentation of the unpaired stimulus is tand FR 14 advantages over those described previously, would be denoted
(FI 4-min:S) FI 4-min. This notation emphasizes the existence tand FR 14 (FI 4-min:Si) FI 4-min: S2, where Si and S2 refer to
of two differences between schedules of paired and unpaired two different brief stimuli. That is, every interval would termi¬
stimuli: not only is the unpaired stimulus never followed directly nate with a brief stimulus, but the stimulus presented under
by food, but food is delivered immediately after a response the early components of the schedule would be different from
(FI 4-min), versus after a brief delay with a paired stimulus. In the stimulus paired with food.
 304 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

schedules, deprivation conditions, food presentation

parameters, prevailing stimuli and briefly presented
stimuli show extensive overlap among these studies
(cf. Stubbs, 1971). No single variable distinguishes
studies in which the pairing operation produced
differential effects from those in which it did not.
However, it is possible that a combination of some of
these variables may be responsible for the differences.

Physical Properties of the Briefly Presented Stimu¬

lus. Stubbs (1971) demonstrated that the greater the
number and type of events used as the brief stimulus,
the greater the effect of the brief stimulus on temporal
pattern of responding in FI unit schedules. He pre¬
sented either a red key light, a white houselight, both,
or a blackout, under nonpaired conditions. Greatest
QUARTER OF FIXED INTERVAL temporal rate changes were found with key light and
Fig. 8. Effects of presenting a stimulus that was not paired with houselight changes together, and least with blackout
primary reinforcement on mean rates of responding in each alone. Hughes (1973) found that a paired stimulus
quarter of the fixed interval (Bird 149). Solid circles: white light gave greater temporal rate changes (scallops) with FI
terminated each component; triangles: dark key (left panel)
or red light (right panel); open circles: final determination of unit schedules than did an unpaired stimulus, but
points when white light terminated each component. (From found equivalent performances with a more intense
Kelleher, 1960b. © 1966 by the Society for the Experimental stimulus. An electric shock to the pubis of pigeons
Analysis of Behavior, Inc,)
likewise had identical effects whether paired with food
or unpaired (Stubbs & Silverman, 1972). Similarly,
Kello (1972) found greater control of pausing after
effect could be due to any number of differences in the omitted food presentations when a food magazine
stimuli such as modality, location, intensity, previous light accompanied blackout than with blackout alone.
exposures, and so on. The use of blackout as the brief stimulus has
Three experiments reporting differences between yielded varied results. In most experiments, blackout
paired and unpaired stimuli used the same stimulus has been used as an unpaired stimulus, in the sense
in each condition, presented in different phases of the that food was presented immediately after the ter¬
experiment. Hughes (1973) scheduled changes in key minal peck under the second-order schedule. It can be
color and houselight for pigeons; D, W. Zimmerman argued, however, that a blackout is scheduled like a
(1969) scheduled a tone and a light for rats under con¬ simultaneously paired stimulus, in the sense that key
joint schedules; Byrd (1972) scheduled a light for light and houselight are generally turned off during
squirrel monkeys under a second-order schedule of food presentation. This ambiguity in classification is
electric-shock presentation. In four experiments when possibly related to the inconsistent effects. Kelleher
similar and presumably equivalent stimuli were pre¬ (1966b) found little patterning induced by a 0.7 sec
sented, either paired or unpaired with food, higher change from blue key to dark key in an otherwise
rates were obtained with paired stimuli (de Lorge, unlit chamber. Stubbs and Cohen (1972) found that a
1967, 1969, 1971; Kelleher, 1966b). 2-sec blackout that was scheduled as a paired stimulus
Six studies reported equivalent effects using the produced less patterning in FI 48-sec unit schedules
same stimulus, both paired and unpaired, though of than key light and houselight changes that were either
necessity the conditions were studied in separate paired or unpaired. On the other hand, Neuringer
phases of the experiment (Kelleher, 1966b, who found and Chung (1967) got effective control of responding
equivalent effects in one of three pigeons; Cohen 8c with 0.25 to 7-sec blackouts under FR 11 or brief,
Stubbs, 1976; Stubbs, 1971, who reported 6 experi¬ response-initiated FI schedules. The effects of black¬
ments; Hughes, 1973, with bright houselight as S; outs as brief stimuli under second-order schedules may
Stubbs 8c Cohen, 1972; Stubbs 8c Silverman, 1972, who thus depend on the unit schedule that is studied (cf.
used electric shock as the brief stimulus.) Starr 8c Staddon, 1974).
What accounts for these conflicting results? A com¬ A second parameter that determines the effect of a
parison of schedule types and schedule values, unit brief stimulus is its duration. Byrd (1972) studied
Lewis Gollub 305

second-order schedules in squirrel monkeys in which paired with food under four procedures. Its presenta¬
the terminal maintaining event was a brief electric tions had minimal effect. After six procedures in
shock (cf. Morse 8c Kelleher, 1970). A blue light was which the stimulus was paired with food presentation,
scheduled under FR 4 (FI 4-min:S) for 0.1 to 10 sec. it was again scheduled unpaired with food. It now
As duration of the brief stimulus increased, mean controlled responding nearly as well as when it was
response rate decreased, with lowest rates early in the paired. Thomas and Blackman (1974) compared VI
interval and least variability at the 1-sec duration. 66-sec (FI 10-sec:S) and VI 66-sec (VI 10-sec:S), where
Cohen, Hughes, and Stubbs (1973) varied stimulus S was a change in key light from white to red for 3
duration from 0.5 to 8 sec on VI 240-sec (FI 48-sec:S). sec, with comparable tandem schedules. Response
There was relatively less responding early in the inter¬ rates were higher under both schedules- of brief stim¬
vals under longer durations, and a clear trend of de¬ ulus presentation, but pauses after the stimulus oc¬
creasing rate with increasing duration of the stimulus. curred only with the FI unit schedule. This result is
A third effect requires special consideration. consistent with the fact that food could occur at any
Cohen, Hughes, and Stubbs (1973) found that in time after the stimulus under the VI, whereas it could
both pigeons responding was more positively accel¬ not occur for 10 sec under the FI.
erated within FI units during the second exposure to The critical analysis of the behavioral function of
durations of 0.15 and 2 sec than during the first. This the brief stimulus resembles the inconclusive debates
implies the existence of a partial irreversibility due that characterized the analysis of conditioned rein¬
either to extensive exposure to brief stimulus proce¬ forcement in extinction procedures. Although dis¬
dures or prior exposure to a long value (8 sec). Effects criminative effects were demonstrated in old response
in other studies have been reversible. Hughes (1973) procedures, there has been no agreement on whether
found pairing effects even when the order of presenta¬ response acquisition on new response procedures can
tion of paired and nonpaired conditions was con¬ be ascribed unambiguously to conditioned reinforce¬
trolled. Thus, history alone may not determine the ment (Wike, 1969, but see Schuster, 1969; Longstreth,
effects of pairing, but history in combination with 1971).
some other variables may be crucial (cf. Marr & Zeiler, Second order schedules of brief stimulus presenta¬
1974). tion were developed, among other reasons, “to deter¬
mine whether a brief stimulus that was occasionally
Discriminative Effects of the Bfief Stimulus. Under contiguous with food delivery would maintain re¬
second-order schedules with FI and FR unit schedules, sponding7’ (Kelleher, 1966b), Such a possibility would
the brief stimulus can be expected to serve a discrim¬ permit chronic investigation of conditioned reinforce¬
inative function: responses immediately after it are ment, since food presentation is continued in these
never reinforced. The stimulus thus controls a low schedules. The possibility that at least some rate in¬
or zero rate after its occurrence, as does food in a FI or creases under second-order schedules arise from dis¬
FR schedule with primary reinforcement (cf. Staddon, criminative effects of the brief stimulus (Fantino,
1974). Paired and unpaired stimuli are equivalent Chapter 11 of this volume) also challenges an explana¬
with respect to nonreinforcement of immediately sub¬ tion in terms of reinforcement. Thomas and Black¬
sequent responses. man (1974) concluded that it was “difficult to set how
In the balance of response-dependent rate-enhanc¬ the reinforcement hypothesis is to be differentiated
ing effects and response-independent rate-decreasing from the discriminative hypothesis in terms of the
effects of brief stimuli, the latter are often predomi¬ performances maintained by brief stimuli in second-
nant in the stable state. The former are often stronger order schedules. Both hypotheses suggest that the way
either early in training, or when the stimuli are less in which brief stimuli are scheduled may be crucial
intense and therefore less effective as discriminative and both hypotheses are consistent with data” (p.
stimuli. In fact, all the variables that are relevant to 105).
the development of stimulus control (stimulus salience
Brief Stimuli Scheduled
as determined by modality, intensity, and previous
During Extinction
training conditions) would be expected to affect the
control by scheduled brief stimulus presentations (cf. One of the classic methods for testing the effective¬
Starr 8c Staddon, 1974). ness of a stimulus as a conditioned reinforcer is to
Two recent experiments have shown changes in make its presentation dependent on a response in the
control by brief stimuli with repeated training. Marr absence of food (an extinction test). Because the stim¬
and Zeiler (1974) first scheduled a brief stimulus un¬ ulus was no longer paired with food, its effects even-
306 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

tually disappeared, although intermittent scheduling In earlier experiments (Kelleher, 1957a,b, 1958)
during the training phase could extend its effectiveness chimpanzees received poker chips which later were
(D. W. Zimmerman, 1959). Both Zimmerman (1959) exchanged for foods and liquids. Similar procedures
and Kelleher (1961) showed that the schedule under have recently been developed for use with rats (Mala-
which the brief stimulus was produced, with food godi, 1967 a,b; Waddell, Leander, Webbe, 8c Mala-
completely omitted, determined rate and temporal godi, 1972). As in second-order brief-stimulus sched¬
pattern of responding. ules, the schedule of token delivery engenders
Several additional experiments have extended this consistent patterns of responding. Substantial rates are
paradigm. Thomas (1969) prolonged the effect of a sustained at times considerably removed from food
briefly presented stimulus complex associated with presentation. As in chained schedules, however, ex¬
food delivery. He scheduled brief presentations in the tended pauses occur when the stimulus conditions
presence of one key color and longer presentations in (i.e., number of tokens) are those correlated with zero
the presence of a second color. The two colors alter¬ probability of food presentation. Responding can be
nated during sessions. For example, when a triangle instated almost immediately by presenting, indepen¬
was projected on the key, food was presented for 4 dently of responding, a large number of tokens
sec under FR 120. When the key was green, food was (Kelleher, 1958). The stimulus conditions then re¬
presented for 0.3 sec, a duration too short to allow eat- semble those typically prevailing closer to the end of
mg, Substantial amounts of responding occurred when the schedule sequence.
the key was green, with pauses after each presentation Malagodi (1967a,b) developed token reinforce¬
that were shorter than the pauses when pecks produced ment procedures with rats, Bar presses produced glass
accessible food. When responses during green had no marbles under FR or VI schedules. According to an
effect (EXT) or when they produced a 0.3 sec change exchange schedule, in the presence o£ a characteristic
m key color to red, a stimulus that was not associated stimulus, insertion of each marble into a receptacle
with food presentation, very little responding was sus¬ was followed by delivery of a food pellet. Moderately
tained. When 4 sec of food access was scheduled only low rates were maintained when food was available
when no pecks had occurred for 20 §ec (DRQ 20 §ee) after 2 or 10 marbles had been delivered, a FR (VI)
and brief (0*3 see) operations of the feeder occurred schedule. When a marble was produced under FR 20,
on every tenth peck (FR 10), the pigeon responded a brief pause followed by high rates of bar pressing
faster when not receiving food than when it did. followed each marble delivery. Waddell et al. (1972)
Thomas and Johansen (1970) obtained similar effects also scheduled marble delivery under a FR 20 unit
with a key-light change as the brief stimulus, rather schedule; the opportunity for food-reinforced marble
than stimuli intimately associated with food. Thus, in insertion followed a marble delivery at a fixed time
both these experiments, responding was maintained after the last food delivery, under a FI (FR) schedule.
by presentation of stimuli paired with food in the Even though every marble had always been exchanged
presence of a key color that was never directly asso¬ for one food pellet, as the FI duration increased, the
ciated with food (cf, Herrnstein 8c Loveland, 1972). response rate, and therefore the number of marbles
delivered, decreased. Cumulative records of respond¬
Token Reinforcement. A token is a small physical ing under FI values of 1.5 min, 4.5 min, and 9.0 min
object that is delivered to an organism under some show high local rates alternating with pauses after
schedule. In the presence of another stimulus, a each marble delivery or food presentation, with longer
specified response involving the token, such as insert¬ pauses typically following the latter. Overall response
ing it into a receptacle, is followed by presentation of rate between food presentations had an increasing
food, juice, etc. This procedure resembles both trend, so that responding showed effects of both FI
chained schedules and second-order schedules of brief- and FR schedules. A direct comparison of correspond¬
stimulus presentation. The schedule of token delivery ing chained schedules, schedules of brief stimulus
is the unit schedule, and the schedule for exchange presentation, schedules of token delivery, and tandem
specifies when behavior under the unit schedule is schedules should be informative.
followed by food. The accumulating number of
tokens is also a stimulus that is correlated with the
Concurrent and Conjoint Schedules
possibility of food presentation, especially when food
of Brief Stimulus Presentation
is scheduled after a given number of tokens. The
number of accumulated tokens could have effects The analysis of the role of the added stimulus in
similar to those of chained schedule (or clock and second-order schedules of brief stimulus presentation
counter) component stimuli. is complicated because a fixed relationship of the
Lewis Gollub 307

stimulus to food presentation may establish discrim¬ In addition to the parametric investigation of fre¬
inative control. Reinforcing effects of a stimulus quency of presentation of brief stimuli, the effects of
paired with food may be revealed less ambiguously in different schedules on temporal pattern of responding
concurrent and conjoint schedules of brief stimulus have been investigated. The schedules used include
and food presentation in which the temporal relation FI, VI, FR, DRL, and extinction. In general, the
of the stimulus to food is irregular, since the two temporal pattern under each schedule resembled that
events are scheduled independently. When the sched¬ typically generated by the same schedule of accessible
ules are associated with different operanda, they will food, although at considerably lower rates. Thus,
be called concurrent schedules, and when they are schedules of response-produced stimuli paired with
associated with a single operandum, conjoint (Catania, food appear to engender responding in a similar fash¬
1968). ion to that maintained by traditional positive rein¬
Sustained responding by brief stimulus presenta¬ forcers, such as food and water.
tions that are scheduled concurrently with food has Response maintenance under concurrent and con¬
been demonstrated under several conditions. In one joint schedules of brief stimulus presentation could
group of experiments, by J. Zimmerman and his col¬ be due to a direct effect of delayed presentation of
leagues, pecking on one key produced for 0.5 sec the accessible food. Two types of results argue against this
same stimulus complex that ordinarily accompanied possibility, however. First, multiple schedules of brief
food delivery: the key became dark, the houselight stimulus presentation control appropriate response
was turned off, a light in the feeder compartment was rates. If pecks in the presence of one key color do not
turned on, and a solenoid operated and raised a tray produce the brief stimulus while pecks in a different
of food. A mechanical shutter covered the feeder open¬ key color do (J. Zimmerman, 1963; }. Zimmerman 8c
ing in later experiments, preventing access to food. Hanford, 1966), the latter stimulus controls higher
Accessible food, presented for 3 to 4 sec, was either rates, even though responses in the presence of both
scheduled concurrently, for pecks on a second key, or colors are followed by delayed food. Moreover, this
conjointly, under a variable-time (VT) schedule. In
effect readily reverses when the correlation of schedule
most of the experiments pecks on the key that pro¬
and stimulus is reversed (J. Zimmerman & Hanford,
duced the brief stimulus postponed the delivery of
1967; Hamm and Zimmerman, 1972).
accessible food for 6 sec. This delay presumably
Second, the food-paired brief stimulus complex
attenuated the direct effects of food on pecking.
maintains higher rates than a nonpaired stimulus
A series of five publications showed that rates of
(Hamm 8c Zimmerman, 1972; J. Zimmerman 8c Han¬
pecking between 3 and 10 min were maintained by the
ford, 1966, 1967). This comparison is compromised,
0.5 sec magazine stimulus complex. J. Zimmerman
however, because the nonpaired stimulus differed in
and Hanford (1966) found considerably higher rates
many ways from the food-paired stimulus. A compari¬
in the presence of a blue key light where pecks pro¬
son in which both stimuli were arbitrary would per¬
duced the food-paired stimulus complex on FI 1-min
mit a less ambiguous test of the role of association
than in a yellow key light, where pecks had no conse¬
with food.
quences. Accessible food was presented independently
of pecking under a VT 3-min schedule. J. Zimmer¬ Other experiments have opposed the effects of food
man, Hanford, and Brown (1967) found that the rate presentation under one schedule with brief-stimulus
of pecking increased as the frequency of pecks pro¬ presentation under another schedule presented con¬
ducing the food-paired stimulus increased. jointly. The behavior controlled by the conjointly
The food-paired stimulus also sustains responding presented brief stimulus is chosen such that it reduces
for considerable periods of time after its pairing with the frequency of food presentation. Randolph and
food is eliminated. J. Zimmerman (1969) found that Sewell (1965) scheduled food under DRL 20-sec and
pecking was maintained in two pigeons by presenta¬ DRL 30-sec, and conjointly scheduled brief presenta¬
tion of the brief stimulus for 24 and 32 50-min ses¬ tion of the feeder light and offset of key and house-
sions. J. Zimmerman and Hanford (1967) found light under FR 10. Response rate under the conjoint
similar maintenance for as long as 16 sessions. schedule was higher with brief stimulus presentations
Together with Thomas’s (1969) and Kelleher’s (1961) than without it, especially on the DRL 30-sec sched¬
demonstrations of extensive maintenance of respond¬ ule. Stubbs (1967) also found greater rate increases
ing by food magazine stimuli, these results indicate with longer DRL values. Similarly, when Clark and
that stimuli with extensive histories of contiguous Sherman (1970) arranged for non-matching-to-sample
association with food gain powerful maintaining con¬ responses to produce a food-paired stimulus, the fre¬
trol when presented as behavioral consequences. quency of matching decreased by 30-40%, even
308 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

though only matches produced food. Responses early Conditioned Reinforcement in

in the fixed-interval period were also increased when Second-Order Schedules of Brief
bar pressing by rats produced a stimulus paired with Stimulus Presentation
water under a VI schedule (D. W. Zimmerman, 1969),
The preceding review has shown that second-order
or under a shorter FI than the one that arranged
schedules of brief stimulus presentation engender con¬
water delivery (D. W. Zimmerman, 1971). When the
sistent and well maintained patterns of responding.
brief stimulus was presented independently of re¬
Such patterns depend on the presentation of brief
sponding under a VT schedule, only small changes in
stimuli, but may reflect primarily their discriminative
responding were observed (D. W. Zimmerman, 1971).
effects due to a consistent relation of the stimulus to
As previously mentioned, some experiments on con¬
subsequent food presentation. Whether a reinforcing
joint schedules have found minimal effects of brief
(rate enhancing) effect is obtained depends on the
stimuli. Neuringer and Chung (1967) found no rate
baseline performance and schedule for presenting the
increases with conjoint FR 10:S VI l-min:food.
brief stimulus. Response maintenance or enhancement
Similarly, a 0.7-sec blackout paired with food en¬
can be observed most reliably during initial exposure
hanced responding only slightly under conjoint FR:S
to certain schedules of stimulus presentation. Since
FI:food schedules according to Shull, Guilkey, and
the focus of many experiments with these procedures
Witty (1972). Stubbs (1971) also found no significant
has been steady state performance, some of these
effects of a brief-stimulus unpaired with food pre¬
experiments provide poor evidence for a maintained
sented under FI 60-sec, with food presented under VI
conditioned reinforcing effect. When the relation of
240 see. A second-order schedule with the same compo¬
the stimulus to food is less regular, or when control of
nents engendered substantial patterned responding
the response by food presentation is weak, as in the
whether the brief stimulus was paired or unpaired
presence of stimuli that control low rates, rate en¬
with food.
hancing effects can be observed, and are sustained.
The results of experiments on concurrent and con¬
Striking instances are during extinction (Thomas,
joint schedules of brief-stimulus presentation indicate 1969), or when a concurrent or conjoint response does
that stimuli that precede and/or accompany food
not produce food (J. Zimmerman, 1963, and related
presentation can often increase the rate of responses
papers), and during periods of low rate in FI (D. W.
that produce them, Although the effects with concur¬ Zimmerman, 1971) and FR schedules (Findley &
rent schedules are often quantitatively small, they are Brady, 1965),
sustained over long periods of time and can continue
when food i§ no longer available,
One inference from these experiments has impor¬
CONCLUDING REMARKS
tant implications for the further study of rate enhanc¬
ing effects of brief stimuli. It appears that such effects
can best be obtained when the response that produces The concept of conditioned reinforcement has been
used as both an explanatory term in analyses of oper¬
the brief stimulus is under minimal control by food
ant behavior in general, and as a concept to organize
reinforcement. Thus, the clearest sustained effects are
a variety of behavioral procedures. A reflex orienta¬
obtained when the response never produces food., as in
tion which emphasized close temporal relationships
the concurrent schedules, or is less strongly controlled
predisposed early workers to interpret response acqui¬
by the schedule of food reinforcement, as in long
sition or maintenance with delayed or intermittent
DRL schedules or early in the fixed-interval period.
reinforcement in terms of immediate consequences of
Stated another way, the effects of the brief stimulus
behavior. Experimental paradigms were arranged to
can be masked by the effects of food reinforcement, or
demonstrate these effects. Stimuli associated with effec¬
by ongoing high response rates. When food delivery
tive reinforcers (e.g., food) were shown to prolong per¬
controls responding rather strongly, only discrimina¬
formance when primary reinforcers were omitted, to
tive effects of lower rate after the stimulus may ap¬
reduce the decrement caused by a delay between be¬
pear. This seems the most frequent result in the havior and reinforcer, or even to serve as the sole
majority of studies on second-order schedules, and it is consequence in acquisition of a new operant (cf. Mil¬
an important consideration for experiments designed ler, 1951; Myers, 1958). More detailed analyses have
to test whether a stimulus paired with food delivery is revealed, however, that these stimuli can also be serv¬
a more effective reinforcer than one that is unpaired ing discriminative function.
(cf. Cohen 8c Stubbs, 1976). More recent results with chained schedules and
Lewis Gollub 309

second-order schedules of brief-stimulus presentation ules of food presentation: The repetition of a stimulus
also reveal strong discriminative effects of putative during alternate components. Journal of the Experi¬
conditioned reinforcers. This has led some writers to mental Analysis of Behavior, 1971, 16, 31-38.
Byrd, L. D. Responding in the squirrel monkey under
be extremely skeptical about a rate-enhancing func¬
second-order schedules of shock delivery. Journal of the
tion of such stimuli (e.g., Schuster, 1969; Longstreth, Experimental Analysis of Behavior, 1972, 18, 155-167.
1971). Catania, A. C. (Ed.) Contemporary research in operant be¬
It is hardly a novel discovery, however, that stimuli havior. Glenview, Ill., Scott, Foresman, 1968.
Catania, A. C., 8c Reynolds, G. S. A quantitative analysis
have multiple functions (cf. Skinner, 1938). Different
of the behavior maintained by interval schedules of
functions may be revealed differentially by various
reinforcement. Journal of the Experimental Analysis of
experimental procedures or under parametric manip¬ Behavior, 1968, 11, 327-383.
ulation. The fact that a behavioral effect depends on Chung, S.-H., 8c Neuringer, A. J. Control of responding by
the schedule of presentation of a stimulus is not a percentage reinforcement schedule. Psychonomic Sci¬
ence, 1967, 8, 25-26.
unique to conditioned reinforcers. The behavioral
Clark, H. B., 8c Sherman, J. A. Effects of a conditioned
effects of drugs similarly depend on schedules and
reinforcer upon accuracy of matcli-to-sample behavior
baseline performance (Dews, 1955; Kelleher & Morse, in pigeons. Journal of the Experimental Analysis of
1968). Behavior, 1970, 12, 375-384.

The concept of conditioned reinforcement has Cohen, S. L., Hughes, J. E., & Stubbs, D. A. Second-order
schedules: Manipulation of brief-stimulus duration at
played an important role in the development of prac¬
component completion. Animal Learning and Behavior,
tical techniques of behavioral control (Ayllon & Azrin, 1973, 1, 121-124.
1968; O’Leary & Drabman, 1971). The detailed anal¬ Cohen, S. L., & Stubbs, D. A, Discriminative properties of
ysis of some of these procedures reveals that relevant briefly presented stimuli. Journal of the Experimental
stimuli have other, and sometimes more important Analysis of Behavior, 1976, 25, 15-25.
Davison, M. C. Successive interresponse times in fixed-ratio
effects than response enhancement. Such results
and second-order fixed-ratio performance. Journal of the
should not, however, lead us to neglect rate-enhancing
Experimental Analysis of Behavior, 1969, 12, 385-389.
effects under appropriate conditions. It would in¬ Davison, M. C,, & Temple, W. Preference for fixed-interval
deed be ironic if, at the same time that use of con¬ schedules: An alternative model. Journal of the Experi¬
ditioned reinforcement techniques becomes a com¬ mental Analysis of Behavior, 1973, 20, 393-403.
DE LORGE, J. Fixed-interval behavior maintained by condi¬
monplace in the applied analysis of human behavior
tioned reinforcement. Journal of the Experimental
the results of limited experimental paradigms were Analysis of Behavior, 1967, 10, 271-276.
taken as disproving the existence of conditioned rein¬ de Lorge, J. The influence of pairing with primary rein¬
forcement. The trends in experimental analysis re¬ forcement on the maintenance of conditioned reinforce¬
viewed here indicate that schedules can greatly mod¬ ment in second-order schedules. In D. P. Hendry (Ed.),
Conditioned reinforcement. Homewood, Ill.: Dorsey
ulate conditioned reinforcing effects.
Press, 1969.
de Lorge, J. The effects of brief stimuli presented under a
multiple schedule of second-order schedules. Journal of
REFERENCES the Experimental Analysis of Behavior, 1971, 15, 19-25.
Dews, P. B. Studies on behavior. I. Differential sensitivity
to pentobarbital of pecking performance in pigeons
Autor, S. M. The strength of conditioned reinforcers as a depending on the schedule of reward. Journal of Phar¬
function of frequency and probability of reinforcement. macology and Experimental Therapeutics, 1955, 113,
In D. P. Hendry (Ed.), Conditioned reinforcement. 393-401.
Homewood, Ill.: Dorsey Press, 1969. Duncan, B., 8c Fantino, E. The psychological distance to
Ayllon, T., 8c Azrin, N. H. The token economy: A moti¬ reward. Journal of the Experimental Analysis of Behav¬
vational system for therapy and rehabilitation. Engle¬ ior, 1972, 18, 23-34.
wood Cliffs, N.J.: Prentice-Hall, Inc., 1968. Fantino, E. Preference for mixed- versus fixed-ratio sched¬
Baum, W. M., 8c Rachlin, H. C. Choice as time allocation. ules. Journal of the Experimental Analysis of Behavior,
Journal of the Experimental Analysis of Behavior, 1969, 1967, 10, 35-43.
12, 861-874. Fantino, E. Effects of required rates of responding upon
Boren, M. C. P. Fixed-ratio and variable-ratio schedules of choice. Journal of the Experimental Analysis of Be¬
brief stimuli in second-order schedules of matching to havior, 1968, 11, 15-22.
sample. Journal of the Experimental Analysis of Be¬ Fantino, E. Conditioned reinforcement, choice, and the
havior, 1973, 20, 219-233. psychological distance to reward. In D. P. Hendry (Ed.),
Boren, M. C. P., 8c Gollub, L. R. Accuracy of performance Conditioned reinforcement. Homewood, Ill.: Dorsey
on a matching-to-sample procedure under interval sched¬ Press, 1969.
ules. Journal of the Experimental Analysis' of Behavior, Fantino, E., 8c Duncan, B. Some effects of interreinforce¬
1972, 18, 65-77. ment time upon choice. Journal of the Experimental
Byrd, L. D. Responding in the pigeon under chained sched¬ Analysis of Behavior, 1972, 17, 3-14.
310 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

Fantino, E., 8c Herrnstein, R. J. Secondary reinforcement panzees. Journal of Comparative and Physiological Psy¬
and number of primary reinforcements. Journal of the chology, 1957, 50, 571-575. (a)
Experimental Analysis of Behavior, 1968, 11, 9-14. Kelleher, R. T. A multiple schedule of conditioned rein¬
Ferster, C. B. Sustained behavior under delayed reinforce¬ forcement with chimpanzees. Psychological Reports,
ment. Journal of Experimental Psychology, 1953, 45, 1957, 3, 485-491. (b)
218-224. Kelleher, R. T. Fixed-ratio schedules of conditioned rein¬
Ferster, C. B., 8c Skinner, B. F. Schedules of reinforcement. forcement with chimpanzees. Journal of the Experi¬
Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1957. mental Analysis of Behavior, 1958, 1, 281-289.
Findley, J. D. An experimental outline for building and Kelleher, R. T. Schedules of conditioned reinforcement
exploring multi-operant behavior repertoires. Journal of during experimental extinction. Journal of the Experi¬
the Experimental Analysis of Behavior, 1962, 5, 113-166. mental Analysis of Behavior, 1961, 4, 1-5.
Findley, J. D., 8c Brady, J. V. Facilitation of large ratio Kelleher, R. T. Chaining and conditioned reinforcement.
performance by use of conditioned reinforcement. Jour¬ In W. K. Honig (Ed.), Operant behavior: Areas of re¬
nal of the Experimental Analysis of Behavior, 1965, 8, search and application. Englewood Cliffs, N.J.: Prentice-
125-129. Hall, Inc., 1966. (a)
Gollub, L. R. The chaining of fixed-interval schedules. Kelleher, R. T. Conditioned reinforcement in second-
Unpublished doctoral dissertation, Harvard Elniversity, order schedules. Journal of the Experimental Analysis
1958. of Behavior, 1966, 9, 475-485. (b)
Gollub, L. R. Responding maintained by chained and Kelleher, R. T., 8c Fry, W. T. Stimulus functions in
tandem fixed-interval schedules of reinforcement. Amer¬ chained fixed-interval schedules. Journal of the Experi¬
ican Psychologist, 1965, 20, 554. mental Analysis of Behavior, 1962, 5, 167-173.
Gollub, L. R. Information on conditioned reinforcement: Kelleher, R. T., 8c Gollub, L. R. A review of conditioned
A review of Conditioned reinforcement, edited by Derek reinforcement. Journal of the Experimental Analysis of
P. Hendry. Journal of the Experimental Analysis of Behavior, 1962, 5, 543-597.
Behavior, 1970, 14, 361-372. Kelleher, R. T., 8c Morse, W. H. Determinants of the
Gollub, L. R., 8c Vogt, C. P. Acquisition of responding on specificity of behavioral effects of drugs. Ergebnisse der
chained schedules: Gradual compared to abrupt intro¬ Physiologie: Biologischen Chemie und Experimentellen
duction of the chain. Psychonomic Science, 1970, 18, Pharmakologie. 1968, 60, 1-56.
299-301. Keller, F. S., 8c Schoenfeld, W. N. Principles of psy¬
Gott, C. T., 8c Weiss, B. The development of fixed-ratio chology. New York: Appleton-Century-Crofis, 1950.
performance under the influence of ribonucleic acid. Kello, J. E. The reinforcement-omission effect on fixed-
Journal of the Experimental Analysis of Behavior, 1972, interval schedules: Frustration or inhibition? Learning
18, 481-497. and Motivation, 1972, 5, 138-147.
Hamm, H. D. 8c Zimmerman, J. Differential maintenance of Kendall, S. B. Some effects of fixed-interval duration on
concurrent operants with conditioned reinforcement in response rate in a two-component chain schedule. Jour¬
a free feeding situation. Psychological Record, 1972, 22, nal of the Experimental Analysis of Behavior, 1967, 10,
497-508. 341-347.
Hanson, H, M,, 8c Witoslawski, J. J, Interaction between Kendall, S. B. Some effects of response-dependent clock
the components of a chained schedule. Journal of the stimuli in a fixed-interval schedule. Journal of the Ex¬
Experimental Analysis of Behavior, 1959, 2, 171-177. perimental Analysis of Behavior} 1972, 17y 161-168.
Hendry, D. P., k Dillow, P. V. Observing behavior during KlLLEEN, P. Reinforcement frequency and contingency as
interval schedules. Journal of the Experimental Analysis factors in fixed-ratio behavior. Journal of the Experi¬
of Behaviory 1066, 9} 337-340, mental Analysis of Behavior, 1969,12, 391-395.
HERRNSTEIN, R. J. Relative and absolute strength of re¬ Killeen, P. The matching law. Journal of the Experi¬
sponse as a function of frequency of reinforcement. mental Analysis of Behavior, 1972, 17y 489—495.
Journal of the Experimental Analysis of Behavior, 1961, Lee, J. K., 8c Gollub, L. R. Second-order schedules with
4} 267-272, fixed-ratio components: Variation of component size.
Herrnstein, R. J. Secondary reinforcement and the rate
Journal of the Experimental Analysis of Behavior, 1971,
of primary reinforcement. Journal of the Experimental 15, 303-310.
Analysis of Behavior, 1964, 7, 27-36. Longstreth, L. E. A cognitive interpretation of secondary
Herrnstein, R. J., 8c Loveland, D. H. Food-avoidance in reinforcement. In J. K. Cole (Ed.), Nebraska Symposium
hungry pigeons, and other perplexities. Journal of the on Motivation (Vol. 19). Lincoln: University of Nebraska
Experimental Analysis of Behavior, 1972, 18, 369-383. Press, 1971.
Hughes, J. E. Second-order schedules: A new procedure. Lott, D. F. Secondary reinforcement and frustration: A
Paper presented at the Eastern Psychological Association, conceptual paradox. Psychological Bulletin, 1967, 67,
Washington, 1973. 197-198.
Jwaideh, A. R. Responding under chained and tandem Malagodi, E. F. Acquisition of the token reward habit in
fixed-ratio schedules. Journal of the Experimental Anal¬ the rat. Psychological Reports, 1967, 20, 1335-1342. (a)
ysis of Behavior, 1973, 19, 259-267. Malagodi, E. F. Fixed-ratio schedules of token reinforce¬
Kaufman, A., 8c Baron, A. Conditioned reinforcing and ment. Psychonomic Science, 1967, 8, 469-470. (b)
aversive aspects of the stimuli defining the components Malagodi, E. F., DeWeese, J., 8c Johnston, J. M. Second-
of a two-component chain. Genetic Psychology Mono¬ order schedules: A direct comparison of chained, brief-
graphs, 1969, 80, 151-201. stimulus, and tandem procedures. Journal of the Experi¬
Kelleher, R. T. Conditioned reinforcement in chim- mental Analysis of Behavior, 1973, 20, 447-460.
 Lewis Gollub 31 1

Marr, M. J. Second-order schedules. In D. P. Hendry (Ed.), the Experimental Analysis of Behavior, 1971, 15, 27-38.
Conditioned reinforcement. Homewood, Ill.: Dorsey Squires, N. K., Norborg, J., 8c Fantino, E. Second-order
Press, 1969. schedules: Discrimination of components. Journal of the
Marr, M. J. Sequence schedules of reinforcement. Journal Experimental Analysis of Behavior, 1975, 24, 157-171.
of the Experimental Analysis of Behavior, 1971, 15, Staddon, J. E. R. Temporal effects of reinforcement: A neg¬
41-48. ative “frustration” effect. Learning and Motivation,
Marr, M. J., & Zeiler, M. D. Schedules of response-inde¬ 1970, 1, 227-247.
pendent conditioned reinforcement. Journal of the Ex¬ Staddon, J. E. R. Reinforcement omission on temporal
perimental Analysis of Behavior, 1974, 21, 433-444. go-no-go schedules. Journal of the Experimental Analysis
Miller, N. E. Learnable drives and rewards. In S. S. of Behavior, 1972, 18, 223-229.
Stevens (Ed.), Handbook of experimental psychology. Staddon, J. E. R. Temporal control, attention, and mem¬
New York: Wiley, 1951. ory. Psychological Review, 1974, 81, 375-391.
Moore, J., 8c Fantino, E. Choice and response contin¬ Staddon, J. E. R., 8c Innis, N. K. Reinforcement omission
gencies. Journal of the Experimental Analysis of Be¬ on fixed-interval schedules. Journal of the Experimental
havior, 1975, 23, 339-347. Analysis of Behavior, 1969, 12, 689-700.
Morse,W. H., 8c Kelleher, R. T. Schedules as fundamental Starr, B. C., 8c Staddon, J. E. R. Temporal control on
determinants of behavior. In W. N. Schoenfeld (Ed.), periodic schedules: signal properties of reinforcement
The theory of reinforcement schedules. Englewood Cliffs, and blackout. Journal of the Experimental Analysis of
N.J.: Prentice-Hall, Inc., 1970. Behavior, 1974, 22, 535-545.
Mulvaney, D. E., Dinsmoor, J. A., Jwaideh, A. R., 8c
Stubbs, D. A. Competitive conditioned reinforcement and
Hughes, L. H. Punishment of observing by the negative
efficient differential reinforcement of low rate perfor¬
discriminative stimulus. Journal of the Experimental mance. Psychonomic Science, 1967, 8, 299-300.
Analysis of Behavior, 1974, 21, 37-44. Stubbs, D. A. Contiguity of briefly presented stimuli with food
Myers, J. L. Secondary reinforcement: A review of recent
reinforcement. Journal of the Experimental Analysis of
experimentation. Psychological Bulletin, 1958, 55, 284- Behavior, 1969, 12, 271-278.
301.
Stubbs, D. A. Second-order schedules and the problem of
Neuringer, A. J. Varying reinforcement frequency on a conditioned reinforcement. Journal of the Experimental
percentage-reinforcement schedule. Psychonomic Science, Analysis of Behavior, 1971, 16, 289-313.
1968, 11(5), 163-164. Stubbs, D. A., 8c Cohen, S. L. Second-order schedules:
Neuringer, A. J., 8c Chung, S.-H. Quasi-reinforcement: Comparison of different procedures for scheduling
Control of responding by a percentage reinforcement paired and nonpaired brief stimuli. Journal of the Ex¬
schedule. Journal of the Experimental Analysis of Be¬ perimental Analysis of Behavior, 1972, 18, 403-413.
havior, 1967, 10, 45-54. Stubbs, D. A., 8c Silverman, P. J. Second-order schedules:
O’Leary, K. D., 8c Drabman, R. S. Token reinforcement
Brief shock at the completion of each component. Jour¬
programs in the classroom: A review. Psychological Bul¬ nal of the Experimental Analysis of Behavior, 1972, 17,
letin, 1971, 75, 379-398. 201-212.
Premack, D. Reinforcement theory. In D. Levine (Ed.),
Switalski, R. W., 8c Thomas, D. R. The development of
Nebraska Symposium on Motivation Vol. 13. Lincoln: stimulus control in a three-member chained schedule.
University of Nebraska Press, 1965. Psychonomic Science, 1967, 9, 391-392.
Randolph, J. J., 8c Sewell, W. R. Competitive conditioned
Tallen, R. R., 8c Dinsmoor, J. A. Conditioned reinforce¬
reinforcement during differential reinforcement of low ment in chain fixed-interval schedules. Proceedings of
rates. Psychonomic Science, 1965, 3, 411-412. the 77th Annual Convention of the American Psycho¬
Schneider, J. W. Choice between two-component chained logical Association, 1969, 4, 823-824.
and tandem schedules. Journal of the Experimental Ten Eyck, R. L., Jr. Effects of rate of reinforcement-time
Analysis of Behavior, 1972, 18, 45-60. upon concurrent operant performance. Journal of the
Schuster, R. H. A functional analysis of conditioned rein¬
Experimental Analysis of Behavior, 1970, 14, 269-274.
forcement. In D. P. Hendry (Ed.), Conditioned rein¬ Thomas, G., 8c Blackman, J. Quaisreinforcement: Control
forcement. Homewood, Ill.: Dorsey Press, 1969. of behavior by second-order interval schedules. Learning
Segal, E. F. Exteroceptive control of fixed-interval respond¬ and Motivation, 1974, 5, 92-105.
ing- Journal of the Experimental Analysis of Behavior, Thomas, J. R. Multiple baseline investigation of stimulus
1962, 5, 49-57.
functions in an FR chained schedule. Journal of the
Shull, R. L., Guilkey, M., 8c Witty, W. Changing the
Experimental Analysis of Behavior, 1964, 7, 241-245.
response unit from a single peck to a fixed number of Thomas, J. R. Chained and tandem fixed-interval schedule
pecks in fixed-interval schedules. Journal of the Experi¬ performance and frequency of primary reinforcement.
mental Analysis of Behavior, 1972, 17, 193-200. Psychological Reports, 1967, 20, 471-480.
Sidman, M. Tactics of scientific research. New York: Basic
Thomas, J. R. Maintenance of behavior by conditioned
Books, 1960.
reinforcement in the signaled absence of primary rein¬
Skinner, B. F. The behavior of organisms: An experimental forcement. In D. P. Hendry (Ed.), Conditioned rein¬
analysis. New York: Appleton-Century-Crofts, 1938. forcement. Homewood, Ill.: Dorsey Press, 1969.
Skinner, B. F. Contingencies of reinforcement: A theoreti¬
Thomas, J. R., 8c Johanson, C. Maintenance of fixed-
cal analysis. Englewood Cliffs, N. J.: Prentice-Hall Inc interval responding by conditioned reinforcement in
1969.
multiple schedules. Psychonomic Science, 1970, 19(3)
N. K., 8c Fantino, E. A model for choice in simple
Squires, 135-136.
concurrent and concurrent-chains schedules. Journal of Thomas, J. R., 8c Stubbs, A. Stimulus control of temporally
312 CONDITIONED REINFORCEMENT: SCHEDULE EFFECTS

spaced responding in second-order schedules. Journal of Zimmerman, D. W. Rate changes after unscheduled omis¬
the Experimental Analysis of Behaviors 1967, 10, 175- sion and presentation of reinforcement. Journal of the
183. Experimental Analysis of Behavior, 1971, 75, 261-270.
Waddell, T. R., Leander, J. D., Webbe, F. M., 8c Malagodi, Zimmerman, J. Technique for sustaining behavior with
E. F. Schedules interactions in second-order fixed-inter¬ conditioned reinforcement. Science, 1963, 772, 682-684.
val (fixed-ratio) schedules of token reinforcement. Learn¬ Zimmerman, J. Meanwhile . . . back at the key: Mainte¬
ing and Motivation■, 1972, 3, 91-100. nance of behavior by conditioned reinforcement and
Wire, E. L. Secondary reinforcement: Selected experiments. response-independent primary reinforcement. In D. P.
New York: Harper 8c Row, 1966. Hendry (Ed.), Conditioned reinforcement. Homewood,
Wire, E. L. Secondary reinforcement: Some research and Ill.: Dorsey Press, 1969.
theoretical issues. In W. J. Arnold 8c D. Levine (Eds.), Zimmerman, J., 8c Hanford, P. V. Sustaining behavior with
Nebraska Symposium on Motivation (Vol. 17). Lincoln: conditioned reinforcement as the only response-pro¬
University of Nebraska Press, 1969. duced consequence. Psychological Reports, 1966, 19, 391—
Zeiler, M. D. Fixed-interval behavior: Effects of percen¬ 401.
tage reinforcement. Journal of the Experimental Analy¬ Zimmerman, J., 8c Hanford, P. V. Differential effects of
sis of Behavior, 1972, 77, 177-189. extinction on behavior maintained by concurrent sched¬
Zimmerman, D. W. Sustained performance in rats based on ules of primary and conditioned reinforcement. Psycho-
secondary reinforcement. Journal of Comparative and nomic Science, 1967, 8, 103-104.
Physiological Psychology, 1959, 52, 353-358. Zimmerman, J., Hanford, P. V., 8c Brown, W. Effects of
Zimmerman, D. W. Concurrent schedules of primary and conditioned reinforcement frequency in an intermittent
conditioned reinforcement m rats. Journal of the Ex¬ free feeding situation. Journal of the Experimental
perimental Analysis of Behavior j 1969, 72, 261- 268. Analysts of Behavior} 1967, 10} 331—340.
 11

Conditioned Reinforcement
choice and information*

Edmund Fantino

INTRODUCTION Wyckoff, 1952, 1969), which has evaluated the rein¬

forcing strength of stimuli that signal the availability
Three Conceptions of Conditioned Reinforcement
of, or provide information about, primary reinforce¬
ment; and the study of choice for stimuli associated
Kelleher and Gollub (1962) and Kelleher (1966) with schedules of primary reinforcement (after Autor,
have reviewed the important techniques that have 1960, 1969). In this chapter I discuss research with
been used to study conditioned reinforcement. these techniques and the important implications of
Chained schedules and other second-order schedules this research for the theory of conditioned reinforce¬
occupy large portions of these reviews, and their con¬ ment. I begin by introducing the theories and dis¬
tinuing importance in the study of conditioned rein¬ tinguishing among them on conceptual grounds and
forcement has again been emphasized in the preceding in terms of empirical predictions. The three most
chapter by Gollub. Over the past decade two addi¬ viable conceptions of how a neutral stimulus acquires
tional techniques have become increasingly popular strength based on its relationship to primary reinforce¬
and important in the assessment of conditioned rein¬ ment appear to be the following: (1) the pairing
forcement: the study of observing responses (after hypothesis, which states that the simple pairing of a
stimulus with a primary reinforcer imparts condi¬
* The unpublished research mentioned herein and the
tioned reinforcing strength to that stimulus; (2) the
preparation of this chapter were supported by National Institutes
of Health Grant No. 20752-4 to the University of California, San delay reduction hypothesis, which states that the
Diego. Several individuals made detailed constructive suggestions strength of a stimulus as a conditioned reinforcer is a
of a prior draft, especially the editors and James Dinsmoor, as
function of the reduction in time to reinforcement
well as Bill Baum, David Case, Mike Davison, Joe Farley, Steve
Hursh, Jed Rose, Ben Williams, and Mike Zeiler. Finally, I correlated with the onset of that stimulus; (3) the un¬
thank the members of my seminar on conditioned reinforcement certainty reduction hypothesis, which states that the
early in 1974: Steve Buck, Mark Fridovich, John Hale, Cheryl
Logan, Jay Moore, Jim Norborg, Nancy Squires, and especially strength of a stimulus is a function of its informative¬
Tibor Safar. ness about primary reinforcement, i.e., how much un-

313
314 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

certainty reduction it provides about reinforcement.1 related to the rate of reinforcement in the presence of
When applied to observing responses, the pairing the stimulus. More generally, a pairing view will fare
and delay reduction hypotheses are both forms of a best if it is couched in terms of reinforcement density
more general hypothesis, the conditioned reinforce¬ (i.e., number of reinforcements/unit time) in the
ment hypothesis, which may be most clearly distin¬ presence of a given stimulus. A pairing hypothesis
guished from the uncertainty reduction hypothesis by based on reinforcement density states that conditioned
the assertion that only stimuli having positive associa¬ reinforcing strength will be determined by the rate of
tions with primary reinforcement should reinforce primary reinforcement in its presence. Such a pairing
observing responses (Dinsmoor, Browne, 8c Lawrence, hypothesis, which bears only superficial similarity to
1972). The uncertainty reduction hypothesis, on the the traditional pairing hypothesis, I shall call the
other hand, states that stimuli associated with negative reinforcement density hypothesis.2
outcomes should also be reinforcing; the conditioned The delay reduction hypothesis also states that the
reinforcement hypothesis requires that these stimuli reinforcing strength of a stimulus is determined, in
be aversive—or at least not positively reinforcing. part7 by the length of the interval between the onset
Much of the work on observing responses concerns of the stimulus and the onset of the primary rein¬
these opposing predictions. As we shall see7 virtually forcer. But this interval length must be considered
all of the evidence supports the conditioned reinforce¬ relative to the length of the interval measured from
ment hypothesis. the onset of the preceding stimulus to the onset of the
The pairing and delay reduction versions of the same primary remforcer. In other words, the contribu¬
conditioned reinforcing hypothesis may be distin¬ tion of contiguity to the conditioned reinforcing
guished m terms of temporal factors relating the con¬ strength of a stimulus must be considered in the con¬
ditioned reinforcing stimulus and the primary rein¬ text of how remote primary reinforcement had been
forcer. According to the pairing hypothesis, the degree prior to the onset of the stimulus. The greater the per¬
of contiguity between the stimulus and the primary centage improvement, in terms of contiguity, to
reinforcer determines the strength of that stimulus as primary reinforcement correlated with the onset of
a conditioned reinforcer. Contiguity has been mea¬ the stimulus, the greater its conditioned reinforcing
sured often as the interval between the offset of the strength, Thus a stimulus associated with an FI 30-sec
stimulus and the onset of the primary reinforcer. By schedule should be a stronger reinforcer if it is pre¬
this measure any stimulus that is perfectly contiguous ceded by a 60-sec period of nonreinforcement than if
with the primary reinforcer—i.e.? with a 0-sec interval it is preceded by a 10-sec period of nonreinforcement,
between the offset of the stimulus and the onset of the since in the first case the onset of the 30-sec interval is
reinforcer—should be maximally effective as a con¬ correlated with a 24 reduction in time to primary
ditioned reinforcer. As many studies, including those reinforcement (of an original waiting time of 90 sec,
of observing and choice, suggest, a stimulus associated only 30 sec—or i/3—remains once the stimulus corre¬
with a higher rate of primary reinforcement—as in lated with the interval schedule appears), but in the
a fixed-interval (FI) 10-sec schedule—is generally a second case only with a 1^ reduction in time to pri¬

more effective conditioned remforcer than one asso¬ mary reinforcement (of an original waiting time of 40
ciated with a lower rate (as in an FI 60-sec schedule), sec, 30 sec—or still remains once the stimulus
despite the fact that both stimuli are perfectly con¬ correlated with the interval schedule appears). Neither
tiguous with the primary reinforcer in the sense noted. the traditional pairing hypothesis nor the reinforce¬
Thus a pairing hypothesis based on this view of con¬ ment density hypothesis distinguishes between these
tiguity, henceforth the traditional pairing hypothesis, two cases. While there is no direct evidence bearing
is inadequate. A more viable measure of contiguity in on this prediction, the results of Taus and Hearst
these cases, therefore, is the interval between the onset (1970) and of Byrd (1971) suggest that the delay reduc¬
of the stimulus and the onset of the primary rein¬ tion hypothesis would be supported. The data from
forcer. This measure (time/reinforcement) is closely each of these studies show that the discriminative
strength of a stimulus (in terms of rate of responding
in its presence) increases with the duration of a pre¬
1 Two of the more important traditional hypotheses which
ceding period of nonreinforcement even though the
have proven less viable than the pairing hypothesis are the
discriminative stimulus hypothesis (Keller & Schoenfeld, 1950) temporal relation of the stimulus to reinforcement is
and the cue strength hypothesis (Wyckoff, 1959). These have been unchanged. Since the conditioned reinforcing strength
ably reviewed in a previous volume (Kelleher, 1966) and will
not be discussed here. The discriminative stimulus hypothesis
has also been reviewed in the preceding chapter. 2 A phrase suggested by Dr. James Dinsmoor.
Edmund Fantino 315

of a stimulus often covaries with its discriminative example, Rescorla (1967, 1968, 1972) has shown that
strength, the prediction made above by the delay re¬ in Pavlovian fear conditioning when the probability
duction hypothesis would likely be borne out. of an unconditioned stimulus (US) in the presence
I should make clear that none of the hypotheses re¬ of a conditioned stimulus (CS) is held constant, the
quires that production of the stimuli affect the occur¬ degree of conditioned suppression may be sharply in¬
rence of reinforcement. In terms of the delay reduc¬ fluenced by manipulating the probability of the US
tion hypothesis, for example, a stimulus correlated in the absence of the CS; when the probability of a
with a reduction in time to primary reinforcement US presentation is equal in both the presence and
should be a conditioned reinforcer, i.e., it should absence of the CS, no suppression occurs despite the
maintain responses (such as observing or choice re¬ fact that the number of pairings is kept constant.
sponses) whether or not these responses affect the This result points to a conclusion that will gain sup¬
temporal distribution of reinforcement. port throughout this chapter whether we are discuss¬
The delay reduction hypothesis will be developed ing stimulus-reinforcer pairings, stimulus-reinforcer
more fully as we go on. Research on observing re¬ correlations, or other stimulus-reinforcer relations:
sponses is equally compatible with both the delay The context in which stimulus-reinforcer events are
reduction and the reinforcement density hypotheses. embedded affects the strength imparted to the stim¬
As we shall see, however, research on choice clearly ulus by the reinforcer. For example, in both the ob¬
favors the delay reduction hypothesis. Thus, for serving response and concurrent chains paradigms we
simplicity, I shall stress the delay reduction hypothesis shall see that a stimulus will reinforce behavior (ob¬
when discussing observing, though the reader should serving responses in the first case, choice in the second)
be aware that similar experimental outcomes are re¬ only when it is correlated with a reduction in the
quired by the reinforcement density hypothesis. Re¬ average time to primary reinforcement, regardless of
search on observing responses shows that the uncer¬ the absolute temporal relation between the stimulus
tainty reduction hypothesis is untenable. Thus it will and primary reinforcement.
be seen that only the delay reduction hypothesis of The most striking evidence for the pairing hypoth¬
conditioned reinforcement is consistent with what is esis has come from studies investigating three types
known about observing and choice. of second-order schedules, considered in the previous
chapter: tandem, brief-stimulus, and chained sched¬
ules. On a tandem schedule, the same stimulus is
The Pairing Hypothesis
present throughout. A brief-stimulus schedule is the
The pairing hypothesis, which is more parsimo¬ same as a tandem schedule, except that the end of
nious than either the delay reduction or uncertainty each component is signaled by the brief presentation
reduction hypotheses, even when formulated in terms of a second stimulus. Each component of a chained
of reinforcement density, has a rich history. We shall schedule is associated with a different stimulus. The
consider it and some of its inadequacies before dis¬ differences in the responding maintained by these
cussing the work on observing and on choice, at which three types of second-order schedules have been ex¬
time the two newer hypotheses will be more fully plained by the conditioned reinforcing properties of
developed and evaluated. As the reviews of Kelleher the brief stimuli or the stimuli comprising the chained
and Gollub (1962), Kelleher (1966), and Nevin (1973) schedule (e.g., Kelleher, 1966). According to this view,
have concluded, the pairing hypothesis (after Hull, the brief-stimulus presentations occurring at the end
1943) is the most viable of the traditional viewpoints of each schedule component are effective conditioned
of conditioned reinforcement. By the time of Nevin’s reinforcers because they are intermittently paired with
(1973) review, however, it was clear that the pairing primary reinforcement. Similarly, the terminal-link
hypothesis had serious shortcomings, as Nevin himself stimulus of a chained schedule is an effective condi¬
pointed out. Some of these shortcomings also apply to tioned reinforcer because it is contiguous with pri¬
the reinforcement density version of the pairing mary reinforcement. Studies of second-order schedules
hypothesis—as I shall note below. have demonstrated increments in responding relative
The pairing hypothesis is supported by the com¬ to that maintained on tandem control schedules. It
mon observation that a stimulus paired with uncondi¬ appeared, then, that pairing sufficed to create effective
tioned reinforcement acquires the properties of a rein¬ conditioned reinforcers.
forcer (e.g., Nevin, 1973). It now appears that pairings The relevance of the second-order schedule data for
or contiguity is effective only so long as a correlation the pairing hypothesis of conditioned reinforcement
exists between the stimulus and reinforcer. For (and indeed for conditioned reinforcement in general)
316 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

has been called into question more recently by the which the completion of a fixed number of FI compo¬
experiments of Stubbs (1971), Stubbs and Cohen nents (on the “main” response key) was required for
(1972), and Squires, Norborg, and Fantino (1975). primary reinforcement. For example, in Experiment
Stubbs showed that brief stimuli presented at the end 1, brief (2-sec) stimulus presentations on a second key
of each component of a second-order schedule en¬ (the “brief-stimulus key”) followed the completion of
hanced responding even when they were always omit¬ each FI component (on the main key). In addition,
ted at the end of the component preceding primary during the final brief-stimulus presentation preceding
reinforcement. Despite the fact that these brief stimuli primary reinforcement, a response was required on the
were never paired with primary reinforcement, they second key in order to produce primary reinforce¬
were just as effective in maintaining behavior as were ment. Prior to the end of the final component, re¬
paired brief stimuli. This result is also inconsistent sponses to the brief-stimulus key had no consequences.
with the reinforcement density version of the pairing Such responding did serve as a measure of the extent
hypothesis since primary reinforcement occurs fre¬ to which the brief-stimulus components were discrim¬
quently, i.e., with high density, in the presence of the inated from one another. Squires et al. examined be¬
paired but not the unpaired brief stimulus. In havior on the following second-order schedules: FR 1
Stubbs’s experiment the pairing procedure consisted (FI 120-sec), FR 2 (FI 60-sec), FR 4 (FI 30-sec) FR 8
of the simultaneous pairing of the brief stimulus and (FI 15-sec). They found that responding occurred on
food. Stubbs and Cohen (1972) found comparable re¬ the brief-stimulus key on virtually every brief-stimulus
sults with pairing procedures in which the brief stim¬ presentation. As Figure 1 shows, even when inappro¬
ulus preceded food. Specifically, they found similar priate responding on the brief-stimulus key (i.e., re¬
behavioral effects with (1) a simultaneous pairing pro¬ sponses to the brief stimulus prior to the final com¬
cedure, (2) a procedure in which the brief stimulus ponent) produced a 15-sec blackout and returned the
preceded and overlapped food, and (3) a procedure in subject to the beginning of the second-order schedule,
which the brief stimulus preceded but did not overlap none of the pigeons learned to withhold these re¬
food. sponses; consequently, subjects received food only
Squires et al. (1975) tested the following alternative rarely in this condition. Results were different when
explanation of the difference in responding main¬ different key colors were associated with each compo¬
tained by tandem, chain, and brief-stimulus schedules: nent of the second-order schedule. In such a chain
Stimuli which signal the relative unavailability of schedule, brief-stimulus key pecks were confined to the
reinforcement suppress responding in the early compo¬ last component, i.e., to the only component in which
nents of second-order schedules in which primary rein¬ they were effective.
forcement is never available. Thus responding in the These results suggest that pigeons do not discrim¬
early portions of a tandem schedule is better main¬ inate between the components of second-order sched¬
tained than responding on a comparable chain sched¬ ules in the absence of differential cues. Squires et al.
ule because the visual stimuli in the chain are more (1975, p. 170) note:
effective cues for nonreinforcement (and hence nonre¬
sponding) than are the temporal cues present in the Another possibility is that the mechanism under¬
tandem schedule. On the other hand, brief-stimulus lying conditioned reinforcement is stimulus gen¬
presentations should be reliable cues for nonreinforce¬ eralization, so that the more similar are the
ment in the subsequent component only if the subject conditioned and primary reinforcers, the more
effective will be the conditioned reinforcer. Since
is sensitive to the number of brief-stimulus presenta¬
the paired and unpaired stimuli were identical
tions that have occurred since the previous primary
in their similarity to primary reinforcement,
reinforcement. Since number may be a much less effec¬ their effects should have been the same. When
tive cue than color, at least in pigeons, Squires et al. the paired stimulus more closely resembles the
(1975) reasoned that a brief-stimulus schedule may be primary reinforcer than does the unpaired
one in which pigeons have difficulty discriminating stimulus it will, by this hypothesis, differentially
among the components of the schedule, making the enhance response rates. In support of this hypoth¬
brief-stimulus schedule functionally a schedule of rein¬ esis, a study by Malagodi, DeWeese, and Johnston
forcement in which each component is reinforced a (1973) demonstrated the clear superiority of
paired brief stimuli over unpaired brief stimuli
certain percentage of the time (this position has much
(each added at the end of each component of a
commonality with that of Neuringer 8c Chung, 1967).
chain schedule as in our Experiment 3) when the
In order to test this proposition, Squires et al. ex¬ paired brief stimuli were brief hopper presenta¬
posed pigeons to a series of second-order schedules in tions. J. Zimmerman and his co-workers (Zim-
Edmund Fantino 317

PUNISHMENT NO BRIEF long [to permit ingestion of grain]. The latter ex¬
BASE (15 sec b.o.) STIMULUS PUNISHMENT planation would be similar to the failure-to-dis-
criminate hypothesis suggested above. . . . The
differences in these two explanations (conditioned
reinforcement vs generalization) is crucial because
the latter explanation obviates any need for a
separate conditioned reinforcement concept un¬
der these circumstances. The utility of the con¬
cept of conditioned reinforcement lies in the
prediction that an arbitrary stimulus may be¬
come a conditioned reinforcer. If only those
stimuli that [at the instant they are presented]
cannot be discriminated from primary reinforce¬
ment are effective, a separate concept is no
longer required.
. . . Although it may still be possible to in¬
voke conditioned reinforcement as an explana¬
tory mechanism for the behavior on different
second-order schedules, at the present time . . .
it is more parsimonious to explain the behavior
in terms of the discriminative properties of the
stimuli and of the salience of the stimuli for the
particular organism.

The present analysis and the results of Squires et al.

(1975) suggest that behavior is well maintained on
second-order schedules because of “conditioned con¬
fusion”3 rather than the effectiveness of the paired
stimulus as a conditioned reinforcer. Thus what had
been the most impressive support for the pairing
hypothesis—behavior on second-order schedules—may
turn out not to be support for it at all. Additional
results, which will be described in the context of
choice (e.g., Schuster, 1969; Squires, 1972) are also in¬
Fig. 1. Probability of a brief-stimulus key response during the consistent with the pairing hypothesis. While the pair¬
first brief-stimulus presentation following primary reinforce¬
ing hypothesis may have served us well, it appears to
ment for the last three sessions of base line (no punishment), 20
sessions in which those responses were followed by a 15-sec be time to discard or modify it in the light of recent
blackout and return to the start of the first component (“pun¬ research.
ishment”), 10 sessions in which the brief-stimulus key was not
While most of the enhanced responding on second-
illuminated during the usual time of the first brief-stimulus
presentation, and 10 sessions more of the punishment condition. order schedules may be due to a discrimination failure,
Data for each of four pigeons. (From Squires et al., 1975. © as Squires et al. maintain, the high rates of respond¬
1975 by the Society for the Experimental Analysis of Behavior,
ing engendered by this failure probably mask some
Inc.)
real, albeit small, effect of pairing. The effects of pair¬
merman, 1969; Zimmerman and Hanford, 1966, ing may become manifest when sufficiently sensitive
1968) have also maintained considerable respond¬ procedures, such as multiple schedules (as in de
ing when consequence of responding was the pro¬ Lorge’s work, 1971), are employed. Nonetheless, most
duction of a short hopper presentation (too short of the response enhancement that sometimes occurs
to allow eating) and delay of longer hopper pre¬ on second-order schedules now appears to depend
sentations. The apparent superiority of a brief on pairing only in the following complex sense:
hopper presentation to other paired brief stimuli When the subject cannot discriminate either among
may be attributed to its resemblance to the pri¬ components of a second-order schedule or between
mary reinforcer [i.e., presentation of grain]. This
brief stimuli and primary reinforcement, poor
effectiveness may either be due to the consequent
temporal control results and response rates are en-
conditioned reinforcing effects of those stimuli,
or due to the failure to discriminate whether the
hopper presentations will be short or [sufficiently] 3 I thank Dr. Jim Norborg for turning this phrase.
318 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

hanced; paired brief stimuli may impair such discrim¬ strength of the stimulus associated with the more
inations. Depending on the stimuli selected and on positive outcome depends upon an informative func¬
the schedule of reinforcement, discrimination (and tion. Consider two possible hypotheses about condi¬
hence temporal control) on unpaired brief-stimulus tioned reinforcement strength: One, the traditional
schedules may be superior to that on paired brief- pairing hypothesis, states that the conditioned rein¬
stimulus schedules. If so, response rates on the paired forcing strength of the stimulus derives from the pair¬
schedule would exceed that on the unpaired schedule. ing of the stimulus with primary reinforcement irre¬
If discrimination among components or between un¬ spective of any informative function; a second
paired stimuli and primary reinforcement is poor, describes the conditioned reinforcing strength of the
however, response rates on the paired and unpaired stimulus in terms of its informativeness about the
schedules would be equivalent. immediacy of primary reinforcement. This second
statement is more compatible with the uncertainty
reduction hypothesis, though it remains distinct from
OBSERVING RESPONSES AND it in at least one important respect, as we shall see
CONDITIONED REINFORCEMENT below. The uncertainty reduction hypothesis may or
may not be described in terms of conditioned rein¬
In Wyckoff’s (1952) observing response procedure, forcement. In principle, it is compatible with condi¬
periods during which key pecking was reinforced with tioned reinforcement, since one could say: “A stim¬
food according to FI schedules alternated with periods ulus which becomes a conditioned reinforcer does so
of extinction. The response key remained white because it reduces uncertainty.” Or one may eschew
throughout both periods, unless the pigeon pressed a the term conditioned reinforcement, stating instead
pedal which turned the key red or green. When the that information about reinforcement—i.e., uncer¬
two colors were correlated with the schedule in effect, tainty reduction per se—is a primary reinforcer.
much more pedal pressing was maintained than when Whatever the form of the conditioned reinforce¬
the stimuli and schedules were uncorrelated. Wyckoff ment or uncertainty reduction hypotheses, however,
referred to the pedal presses as “observing responses/’ the two hypotheses may be distinguished, as Dinsmoor
i.e., responses resulting in the presentation of a pair et al. (1972), Jenkins and Boakes (1973), and others
of discriminative stimuli. Other early studies of ob¬ have pointed out: “The assertion that the negative
serving responses include those of Prokasy (1956) and value as well as the positive value of the informative
Kelleher (1958). stimulus variable reinforces the observing response is
One important explanation of observing responses what distinguishes the uncertainty reduction hypoth¬
stresses the information obtained from them and esis from the conditioned reinforcing hypothesis of
stipulates that uncertainty reduction reinforces ob¬ observing behavior” (Jenkins & Boakes, 1973, p. 198).
serving behavior. A second explanation stresses the In this section we shall first consider evidence on
conditioned reinforcing strength of the stimulus corre¬ the question whether information about negative out¬
lated with the more positive outcome. Some form of comes is reinforcing. We shall then summarize evi¬
the first interpretation—which has been called the un¬ dence from observing response studies which have
certainty reduction hypothesis—has been favored by varied the probability of the positively valued alterna¬
Berlyne (1957, 1960), Bloomfield (1972), Hendry tive in order to develop a quantitative formulation of
(1969b), Lieberman (1972), Schaub (1969), and Schaub observing behavior. The resultant hypothesis of ob¬
and Honig (1967), among others, while some form of serving behavior will be a form of the conditioned
the second interpretation—which has been called the reinforcement hypothesis consistent with the theory
conditioned reinforcement hypothesis—has been fa¬ of conditioned reinforcement suggested by studies of
vored by Dinsmoor, Browne, and Lawrence (1972), choice behavior to which I shall then turn.
Jenkins and Boakes (1973), Kelleher and Gollub
(1962), Kendall (1973a), and Mulvaney, Dinsmoor,
Jwaideh, and Hughes (1974), among others. Are Negative Stimuli Reinforcing?
There are many differences among these general
As Bloomfield (1972) has stated:
interpretations, not all of which can concern us here.
We should point out, however, that the uncertainty
Another way in which information-transmission
reduction and conditioned reinforcement hypotheses
theory confronts reinforcement-contiguity theory
can have much in common. In particular, a condi¬ in the field of secondary reinforcement is
tioned reinforcement hypothesis can specify that the through the function assigned to signals that
Edmund Fantino 319

precede a negatively valued environmental light changes whenever the experimental subjects
event. According to contiguity theory, a signal produced them). In a later portion of his experiment,
of that kind must become aversive, or at least Schaub provided response-independent stimuli which
much less reinforcing than the other cues avail¬ made the response-produced stimuli redundant for the
able. The information hypothesis, on the con¬ experimental birds. This eliminated performance
trary, requires that “bad news” be just as much
differences between the experimental and control sub¬
“news” as “good news” and so does not differ¬
jects. When the response-independent stimuli were
entiate these two cases, (p. 194)
then eliminated, response rates during extinction in¬
There is also the position that predicts intermediate creased for the experimental birds, suggesting that re¬
results: the negative stimuli should be reinforcing, but sponding was reinforced by the response-produced
less so than the positive stimuli (Hendry, 1969b; negative stimuli when they were no longer redundant.
Schaub, 1969). In either case, the negative stimuli As Dinsmoor (Dinsmoor, Flint, Smith, 8c Viemei-
should be reinforcing, not aversive. It should be added ster, 1969; Dinsmoor et al., 1972), Wilton and Clements
that the conditioned reinforcement hypotheses would (1971b), and Bloomfield (1972) have pointed out, how¬
also assign conditioned reinforcing value to “bad ever, the Schaub studies are not completely convinc¬
news” if the news could be acted on to increase the ing. In Schaub and Honig (1967), for example, re¬
likelihood of negative reinforcement (in an escape or sponding in the procedure with only negative stimuli
avoidance procedure) or of positive reinforcement (in may have been maintained by the occasional presenta¬
an alternative response procedure). As far as I know, tion of the positive stimulus after the period of ex¬
the experiments suggested by these latter predictions tinction terminated (a possible interpretation of the
have not been carried out. results raised first by Schaub and Honig). Moreover, in
Three studies have been widely cited as evidence either of the Schaub studies the absence of stimulus
that “bad news” is reinforcing (Lieberman, 1972; change after three responses may have signified that
Schaub 8c Honig, 1967; Schaub, 1969). Schaub and the positive component was in effect. Dinsmoor et al.
Honig’s procedure consisted of alternation between (1972) made observing behavior effective on an
periods of reinforcement and extinction for both aperiodic schedule of reinforcement (either VI 1-min
“master” pigeons and yoked control pigeons. In rein¬ or VI 2-min schedules) on which the absence of stim¬
forcement periods, pecks at a white key by the master ulus change was the usual consequences of a peck to
subjects were reinforced on a variable-interval (VI) 1- the observing key (and hence could not reduce uncer¬
min schedule and also produced a change in the color tainty). Thus the only significant uncertainty reduc¬
of the key from white to red—for 1.5 sec—on a fixed- tion was provided by stimulus change. In addition,
ratio (FR) 3 schedule. In extinction periods, pecks at observing responses were made on a different key than
the white key were never followed by primary rein¬ food responses. If the uncertainty reduction hypothesis
forcement, but pecks did produce a change in the were correct, observing responding should be main¬
color of the key from white to green (also on an FR 3 tained equally well if its only consequence were
schedule). Yoked subjects received the same cues inde¬ production of the negative stimulus (correlated with
pendent of their responding. Schaub and Honig found an extinction period on the food key) or if its only
that the master pigeons pecked at a high rate during consequence were production of the positive stimulus
extinction periods (although not at as high a rate as (correlated with responding on an interval schedule
in the reinforcement periods), suggesting that the of positive reinforcement). On the other hand, if the
production of the green stimulus (associated with conditioned reinforcement hypothesis were correct,
extinction) reinforced responding during the extinc¬ observing responding should be maintained only if
tion period. The yoked subjects generally responded the positive stimulus were sometimes produced.
only during reinforcement periods. Schaub (1969) Dinsmoor et al. (1972) found that observing be¬
then conducted a more elaborate study with the same havior was well maintained when only the positive
basic procedure, except that responding could produce stimulus could be produced, as shown in the left
only a single stimulus: either the positive stimulus, panel of Figure 2, but that responding was eliminated
associated with the VI schedule, or the negative stim¬ when the only consequence of pecking the observing
ulus, associated with extinction. He found that when key was production of the stimulus signifying ex¬
subjects could produce only positive stimuli (i.e., when tinction periods (the center panel of Figure 2).
responding was completely ineffective during extinc¬ Finally, when both stimuli were available, pigeons’
tion periods), they responded more in extinction than observing behavior was maintained at an intermediate
did yoked control subjects (who again received key rate, raising the possibility that production of the
 320 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

a trial produced colored key lights which signaled

whether the trial would end with response-independent
reinforcement or with no reinforcement. These stim¬
uli were produced on a VI schedule which began
operating at the onset of the trial. A procedure was
employed which permitted the bird to produce S+
on those trials in which reinforcement would be
delivered with or without producing S~ on nonrein-
forced trials. In one condition, only a response
preceded by at least 6 sec of nonresponding could pro¬
Fig. % Rate of observing on successive sessions by one pigeon duce S-, while any response that satisfied the VI
(4144) when observing responses produced S+ only (left panel), requirement produced S + . In another condition, this
S" only (center panel), or both stimuli (right panel). (From
contingency was reversed. Thus the pigeons could
Dinsmoor et al., 1972. © 1972 by the Society for the Experimen¬
tal Analysis of Behavior, Inc.) selectively produce only S+ (or only S~) by adjusting
their interresponse times appropriately. The pigeons
generally produced fewer negative stimuli in the
negative stimulus had actually been punishing. The course of training, which indicates that S~ was punish¬
data shown in Figure 2, for one of Dinsmoor’s ing observing behavior.
pigeons, were typical for each of the five pigeons in Auge (1974) has extended these observations to a
the experiment, except that for one bird results in the situation in which both alternatives involved pri¬
left and right panels were comparable, mary reinforcement, but in which one was more posi¬
Several other studies have supported the contention tive than the other. He studied mixed fixed-ratio,
of Dinsmoor et al. (1972) that negative stimuli do not fixed-interyal schedules and found that observing be¬
reinforce observing behavior (e,g„ Blanchard, 1975; havior is maintained by the occasional presentation
Dinsmoor, Browne, Lawrence, & Wasserman, 1971; of the stimulus signaling the shorter delay to rein¬
Dinsmoor, et al., 1969; Jenkins £ Boakes, 1973; Ken¬ forcement (i.e., the schedule with the shorter inter¬
dall, 1973a; Mulvaney, Dinsmoor, Jwaideh, & Hughes, reinforcement interval). Observing responses produced
1974). For example, Dinsmoor jet al. (1971) used a stimuli signaling whether an FI 30-sec or an FR X
procedure in which pigeons could produce a display schedule was in effect. When the stimulus signaling
of either the positive or negative discriminative stim¬ the FI schedule was eliminated, observing behavior
ulus on a key as long as they stood on a pedal. They was maintained when the FI 30-sec schedule alternated
left the pedal, however, as soon as the negative stim¬ with the low-valued FR (e,g., 20 or 30) but not with
ulus appeared. Mulvaney et al. (197?) studied two large FRs (e.g., 100, 140, or 200). The converse was true
pigeons m an observing procedure with three keys. when the stimulus signaling the FI 30-sec schedule
During alternating periods of unpredictable duration, was the only one that could be produced by ob¬
responding on the center (food) key was reinforced serving responses; observing behavior was only main¬
on a VI schedule or was never reinforced. In the ab¬ tained when the FI 30-sec schedule alternated with
sence of observing, the color of all three keys was large FRs. In other words, only stimuli associated with
yellow. On identical but independent VI observing the shorter delay to primary reinforcement maintained
schedules, responding on either of the two side keys observing behavior (a result consistent with those
produced either a positive stimulus (green) on all from an earlier study by Kendall 8c Gibson, 1965).
three keys, if the VI food schedule were in effect, or a Auge (1973) has also shown that observing responding
negative stimulus (red) on all three keys, if extinction is maintained by the stimulus signaling the larger of
were in effect. In the critical stage of the experiment, two reinforcement magnitudes but not by the stimulus
the negative stimulus could not be produced by re¬ signaling the smaller.
sponding on one of the two side keys; responding on Lieberman’s (1972) study provides more convinc¬
the other side key continued to produce both positive ing evidence for the uncertainty reduction hypothe¬
and negative stimuli. The subjects responded at a sis than Schaub’s (1969). Nonetheless, his results are
higher rate on the key that produced only the positive also interpretable in several ways, some of which have
stimulus, suggesting that the negative stimulus pun¬ been discussed by Dinsmoor et al. (1972) and by Mul¬
ished responding that produced it. vaney et al. (1974). In Lieberman’s basic procedure, a
Blanchard (1975) studied eight pigeons on a single¬ variable-ratio (VR) schedule alternated with extinc¬
key, discrete-trials observing procedure. Pecks during tion and no exteroceptive stimuli were associated with
Edmund Fanfino 321

the two types of components. Hence the schedule was is shown in Figure 3. Lieberman argues plausibly that
mix VR EXT. Observing responses on a different the decline in observing response rates during the
lever produced 6 sec of exposure to a stimulus associ¬ extinction component after the S- is removed sug¬
ated with the component in effect. In one experiment, gests that the S~ had been a conditioned reinforcer.
the VR value was varied between 5 and 100 and the However, the decline is also compatible with the
effects on observing responding were studied. Lieber- notion that observing behavior had been maintained
man reasoned that a conditioned reinforcement view by stimulus change. If responding during the extinc¬
of observing responding required a decrease in the tion component is reinforced by stimulus change, then
rate of observing responding as the VR value in¬ less responding should occur once stimulus change is
creased (and hence the density of primary reinforce¬ no longer available. Moreover, as Dinsmoor et al.
ment associated with the positive stimulus decreased). (1972) have pointed out:
Instead, an increase was noted (particularly between
the conditions in which the VR requirement was 5— To show that the light had been functioning as a
resulting in about three responses per minute—and reinforcer, Lieberman then eliminated it from
25—resulting in about 5 responses per minute). While his procedure and found that this led to a sharp
the result is indeed consistent with an information decline in the frequency with which observing
view of observing behavior, it is also consistent with responses were recorded during the extinction
other possibilities. In the first place, Lieberman’s pro¬ component. But the situation without the light
was not entirely comparable to the situation
cedure was essentially a concurrent schedule of food-
with the light. During the baseline determina¬
reinforced responding and observing responding. From
tion, the presence of the tone or the light in¬
what is known about interactions on concurrent
dicated that an observing response would have
schedules (e.g., Catania, 1966), it would be expected
no consequence; conversely, the absence of the
that the greater the VR value for food-reinforced tone or the light set the occasion for observing.
responding, the more responding should be main¬ When the light was eliminated from the pro¬
tained by any concurrently available schedule (in this cedure, there was no stimulus to prevent the
case, observing responding). For this reason, more animal from responding during the 6-sec period
observing responding was maintained with higher VR when the light would otherwise have been pres¬
values on the food-reinforced lever. In addition, in the ent. To maintain comparability with the base-
VR component, where about three responses per sec¬ line procedure, Lieberman did not record these
responses. But some of them may have been
ond were emitted, an observing response delayed
responses that under the baseline procedure
primary reinforcement as a result of the changeover
would have been postponed until after the light
delay (COD) requirement in effect. A COD was used
had terminated and that would therefore have
to minimize interaction between responding on the
been recorded. Note that Schaub (1969), using
two levers; thus primary reinforcement could not be a different recording procedure, did not find a
obtained for a food lever response until at least U/2 corresponding decline in observing when he
sec after any response on the observing response lever.
This delay was comparable to the time required for
the subject to obtain primary reinforcement on a
VR 5. Since this delay would be more noticeable with
lower VR values, the failure to obtain an increase in
observing responding with such values is not sur¬
prising.
Similar arguments and others may be leveled
against the remaining portions of Lieberman’s ex¬
periment (see also Dinsmoor et al., 1972, p. 80). As a
last example, consider the main result of his final
experiment (which in my opinion is the most inter¬
esting), in which observing responding during the
extinction component of MIX VR 50 EXT no longer
produced the negative stimulus (“S~”) after the first
ten (base line) sessions (although the positive stimulus
Fig. 3. Rate of observing (averaged over five subjects) on suc¬
—the “S+”—could be produced during the VR
cessive sessions before and after removal of the S- in the final
component). The result, averaged over five monkeys. experiment. (From Lieberman, 1972.)
322 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

withheld S— (Experiment 2a). It is difficult to Studies Varying the Likelihood of Positive Stimuli
see any way in which the data can safely be
compared with and without the light in Lieber- Several investigators have tested implications of the
man’s experiment, (p. 80)
uncertainty reduction hypothesis by varying the prob¬
ability that observing responses will lead to positive
Thus while Lieberman’s results are interesting and stimuli (and, subsequently, positive reinforcement).
constitute the only truly provocative support of the According to the uncertainty reduction hypothesis,
notion that bad news is reinforcing, they do not per¬ the amount of observing behavior should be described
mit a reasonably unambiguous conclusion. Moreover, by the symmetrical function shown in Figure 4A. This
in view of the large body of unambiguous evidence inverted U-shaped function describes the average
suggesting that observing behavior is maintained only amount of information transmitted by both the posi¬
by stimuli correlated with positive reinforcement, it tive and negative stimuli as a function of the proba¬
appears best to suspend judgment on the significance bility of the positive stimulus. Since the function is
of Lieberman’s results. symmetrical, observing should be just as strong when
In conclusion, observing behavior is maintained by the probability of the positive stimulus is p as when it
the stimulus associated with any of the following: (1) is 1 — (e.g., when p = .20 and .80). Figure 4B illus¬
the presence of a schedule of positive reinforcement trates a function based only upon the amount of in¬
(as opposed to extinction—e.g., Dinsmoor et al., 1971; formation transmitted by the positive stimulus. In
Wyckoff, 1952); (2) the shorter of two delays to rein¬ intuitive terms, this function says that observing will
forcement (e.g., Auge, 1974; Kendall 8c Gibson, 1965); increase as a function of the uncertainty reduction
(3) the larger of two magnitudes of reinforcement conveyed by stimuli associated with positive out¬
(e.g., Auge, 1973); (4) the absence of a schedule of comes; the more unlikely the good news is, the more
punishment (e.g., Dinsmoor et al., 1969). Observing reinforcing when it does come. This function also
behavior is not maintained when there are no differ¬ implies that negative stimuli do not affect observing.
ential outcomes (e.g., Kendall, 1972; Wilton 8c Clem¬ This implication is responsible for the asymmetry.
ents, 1971a) or when only the stimulus associated with If this asymmetrical function accurately describes
the less valued of the two outcomes may be observed observing, then more observing should occur when
(Dinsmoor et al., 1972). Thus the weight of evidence p = .20 than when p = .80. Wilton and Clements
clearly favors the conditioned reinforcement view of (1971b) tested this prediction. In their experiment,
observing behavior over the uncertainty reduction responding on an FI schedule produced both a stim¬
yiew. Or in terms of our earlier discussion, the uncer¬ ulus signaling whether or not reinforcement was forth¬
tainty reduction hypothesis is tenable only when coming and the delayed outcome of the trial: either
restricted to uncertainty reduction which provides in¬ nonreinforcement or response-independent reinforce¬
formation that primary reinforcement is forthcoming. ment. Thus the same response was required to ad¬
In that case, however, the hypothesis becomes indis¬ vance to the trial outcome stage as well as to produce
tinguishable from a conditioned reinforcement hy¬ stimuli correlated with the specific outcome. The
pothesis that describes the strength of a stimulus in probability of the positive stimulus (and hence rein¬
terms of its relation to primary reinforcement. More¬ forcement) was .80 in one condition and .20 in the
over, as Gollub (1970), Eckerman (1973), and McMil¬ other. For each of the six pigeons in the one com¬
lan (1974) have pointed out, the appeal of the parison made by Wilton and Clements, more observ¬
information hypothesis, expressed in terms of uncer¬ ing occurred when p = .20 than when p — .80. This
tainty reduction, depends upon the quantitative pre¬ result suggests that the symmetrical function is incor¬
dictions of information theory (Shannon & Weaver, rect and supports the asymmetrical function. Note
1949; Wiener, 1948). Once these predictions are shown that this result is consistent with those discussed in
to be incorrect when applied to the study of observ¬ the previous section: stimuli correlated with negative
ing responses, there seems to be little reason to “save” outcomes do not appear to maintain observing. It
a modified version of the uncertainty reduction hy¬ should be noted that Wilton and Clements did not
pothesis in favor of one based on conditioned rein¬ include a control condition in which responses ad¬
forcement. I now turn to studies which have tested vanced the subject to the trial outcome stage (as in
quantitative implications of the uncertainty reduc¬ their actual procedure) but did not produce observing
tion hypothesis by examining the strength of observ¬ stimuli. Nor did they include a condition in which
ing as a function of changes in the likelihood of the the subject advanced to the trial outcome stage
positive stimulus. whether or not it made an observing response. With-
Edmund Fantino 323

A metrical function have been reported by McMillan

(1974), McMichael, Lanzetta, and Driscoll (1967),
Hendry (1965), and Eckerman (1973). In perhaps the
most comprehensive study, McMillan (1974) allowed
pigeons to convert a mix VI EXT schedule of food
reinforcement to a corresponding multiple schedule
(mult VI EXT). McMillan varied the probability that
the VI component was in effect between the following
values: .00, .20, .35, .50, .65, .80, and 1.00. His prin¬
cipal finding is shown in Figure 5. The data for one
pigeon (310) are inconsistent with either of the func¬
tions under discussion. Of the remaining five, the data
from four (all except 309) suggest an asymmetrical
function similar to that in Figure 4B. Certainly, Mc¬
Millan’s data do not support a symmetrical function.4
Eckerman (1973) and McMillan (1974) have raised
an intuitive objection to the Wilton-Clements hy¬
pothesis. They question the significance of consider¬
ing the informativeness of positive signals in isola¬
tion. Indeed, it is logically impossible to have only

p
0 .20 .40 .60 .80 1.00 0 .20 .40 .60 .80 1.00
Fig. 4. (A) The amount of information transmitted by both S+
and S~ as a function of the probability of S+. (B) The amount PROBABILITY OF POSITIVE OUTCOME
of information contributed by S+ to the average amount of
information, again as a function of the probability of S+. Fig. 5. Percentage of time spent in the presence of the multiple-
(Adapted from Wilton & Clements, 1971b. © 1971 by the Society schedule stimuli as a function of the probability of the positive
for the Experimental Analysis of Behavior, Inc.) stimulus. (Adapted from McMillan, 1974. © 1974 by the Society
for the Experimental Analysis of Behavior, Inc.)

out these controls it is not clear what maintained

responding in their study. 4 McMillan’s study also reports interesting data arguing
against the discriminative stimulus hypothesis of conditioned
Fortunately, other data consistent with the asym¬ reinforcement.
324 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

positive information. Certainly, a meaningful applica¬ The results reviewed in this section may be ac¬
tion of information theory requires consideration of counted for in terms of a conditioned reinforcement
both positive and negative outcomes. There is also theory of observing behavior (such as that of Dins-
empirical evidence against the appropriateness of an moor’s group), as McMillan has most recently sug¬
asymmetrical observing response function. In the first gested. Our argument begins with the fact that mixed
place, Kendall (1973b) found that observing behavior schedules of reinforcement are less reinforcing, as
was better maintained when the probability of a posi¬ measured in choice procedures, than equivalent multi¬
tive outcome was .25 than when it was .50 or .75. But ple schedules (e.g.. Bower, McLean, 8c Meacham,
according to Figure 4B, the amount of uncertainty 1966; Hendry, 1969c; Hursh 8c Fantino, 1974), what¬
reduction is equal when p = .25 and .50 (in each case, ever the likelihood of the positive component, ex¬
p log [1/p] = .50). Similarly, Steiner (1970) found as cept at the end points: when p = 1.0 and p = 0, the
much or more observing behavior when the probabil¬ multiple and mixed schedules are equivalent. For in¬
ity of a positive outcome was .1 than when it was .5, termediate values, the difference in conditioned rein¬
an outcome opposite to that suggested by the shape of forcing strength between the multiple and mixed
the function of Figure 4B. Most significantly, the schedules should increase as p decreases.5 Thus, con¬
function shown in Figure 4B has a telling empirical ditioned reinforcing strength should increase mono-
shortcoming when dealing with experiments on the tonically as the probability of a positive outcome
effects of negative outcomes. As I discussed above, con¬ decreases, until the function approaches p — 0 (ex¬
vincing evidence that the stimuli associated with tinction), at which point observing should not occur.
negative outcomes actually punish observing behavior At the same time, as the probability of a positive out¬
has been supplied recently by Mulvaney et al. (1974) come becomes very small, observing should decline
and Blanchard (1975), This evidence undermines the since it is rarely reinforced. The observing response
rationale for the asymmetrical function—namely, the function generated by these two factors should resem¬
assumption that negative outcomes do not affect ob¬ ble an asymmetrical inverted U-shaped function.
serving. Wilton (197?) bas modified his own view to The delay reduction hypothesis makes comparable
suggest that the appearance of negative stimuli pun¬ qualitative predictions. For example, consider one of
ishes observing behavior rather than having no effect the schedules used by McMillan (1974), a mixed VI
(as suggested by Wilton and Clements? 1971b). Wilton's 70-sec EXT schedule with equiprobable 40-sec com¬
view is just the opposite of a pure uncertainty reduc¬ ponents, Observing responses changed this mixed
tion view (held by Berlyne, 1960; Bloomfield, 1972; schedule to the equivalent multiple schedule. In the
and others) which specifies that information about presence of the mixed schedule, the average delay to
negative events should he positively reinforcing. Wil¬ reinforcement at the beginning of a trial is 140 sec.
ton does not propose a quantitative theory specifying Whereas production of the negative stimulus is not
hOW ptinishiilg the negative stimuli are7 except to say correlated with reduction in delay tQ reinforcement,
that they ar§ le§§ effective in controlling behavior than and should not maintain observing, the onset of the
are the positive stimuli. In practice, this will have the positive stimulus is correlated with an average reduo
effect of moving the peak of the function in Figure tion of 70 sec (since it is correlated with the VI 70-sec
4B toward a lower probability value by some un¬ component), or one-half the average time correlated
specified amount. This change appears to make Wil¬ with the mixed schedule, and should maintain ob¬
ton's later hypothesis consistent with the results q£ serving. The lower the probability of the VI compo¬
Blanchard (1975), Kendall (1973b), Steiner (1970), nent, the greater the reduction in delay to primary
and Dinsmoor’s group, which are inconsistent with the
5 When the probability of the positive component is high, the
Wilton-Clements hypothesis. Thus while Wilton’s
mixed schedule value is close to that of the positive outcome.
(1972) newer hypothesis may be less valuable than his As the probability of the positive outcome decreases, the mixed
older one in that it is both nonquantitative and more schedule value approaches that of the negative outcome. Many
complex, it does have the advantage of being consis¬ studies of how subjects average positive and negative outcomes
(e.g.. Bower, McLean, & Meacham, 1966; Davison, 1969, 1972;
tent with more of the data. In any case, the new Wil¬ Fantino, 1967; Fantino & Navarick, 1974; Herrnstein, 1964b;
ton theory is not only manifestly inconsistent with the Hursh & Fantino, 1973; Killeen, 1968a; Navarick & Fantino, 1974)
have shown that the positive outcome is weighted far more
uncertainty reduction hypothesis (based on the func¬
heavily than the negative in choice. Since the discrepancy be¬
tion shown in Figure 4A) but, in stressing both the tween the positive outcome on the multiple schedule and the
reinforcing and punishing functions of signals, it has value of the mixed schedule is greater, the more unlikely the
more in common with the conditioned reinforcement positive outcome, it follows that preference for the multiple
schedule should be an inverse function of the likelihood of the
hypothesis than with information views. positive outcome.
Edmund Fantino 32 5

reinforcement correlated with the positive stimulus. with multiple delays of reinforcement; more observing
For example, when the probability of the positive out¬ responding was maintained when one of two 15-sec de¬
come is .1, the delay to reinforcement in the presence lays was increased to 120 sec. While these results are
of the mixed schedule is 700 sec and the reduction in indeed incompatible with the traditional pairing hy¬
time to reinforcement signified by the positive signal pothesis, they are completely consistent with both the
is 630 sec or 630/700 = .9 the average delay signified reinforcement density and the delay reduction hy¬
by the mixed schedule. This particular view of con¬ potheses. When both components are correlated with
ditioned reinforcement is appealing because it is the same reinforcement density, the stimulus of the
consistent with the results of the experiments on mixed schedule is also correlated with the same rein¬
choice that will be discussed in the next section of the forcement density. Since observing does not produce a
chapter. Note that this conditioned reinforcement stimulus correlated with a higher density of reinforce¬
view—like that of Dinsmoor’s group—requires that the ment, observing should not occur. Similarly, neither
probability of observing behavior increase as the like¬ multiple-schedule stimulus is correlated with a reduc¬
lihood of the positive outcome decreases, until some tion in time to reinforcement (relative to the mixed-
unspecified maximum point is reached. The descend¬ schedule stimulus) when the schedules are equal.
ing part of the curve makes good sense, of course: Hence observing should not occur according to the
when p is sufficiently low, observing behavior is rarely delay reduction hypothesis. These hypotheses predict
reinforced and should decline. that only the more positively valued of two stimuli
should maintain observing behavior, as Branch (1970)

Observing Responses; Present Status

has hypothesized, and as the results from Dinsmoor
et al. (1972) and Auge (1974) suggest, since the less
The clearest conclusion that may be drawn from positive stimulus is correlated with an increase, rather
the results and theories that have been discussed thus than a reduction, in average time to primary rein¬
far is that the uncertainty reduction hypothesis of forcement (and a decrease, rather than an increase, in
observing behavior (e.g., Bloomfield, 1972) is incor¬ reinforcement density). Such stimuli should be aver¬
rect. Rather drastic modifications of this hypothesis sive rather than reinforcing according to the delay
which assume that negative signals are ineffective reduction and reinforcement density hypotheses.
(e.g., Wilton & Clements, 1971b) or punishing (Wilton, In addition, these hypotheses predict that when
1972) make the uncertainty reduction view largely each of two stimuli are associated with a reduction in
equivalent to those stipulating that observing behav¬ delay to primary reinforcement (or an increase in
ior is maintained by conditioned reinforcement (e.g., reinforcement density), both should maintain observ¬
Dinsmoor et al., 1969, 1972; Jenkins 8c Boakes, 1973). ing responding, and that more observing responding
One form of the conditioned reinforcement hypothe¬ should be maintained by the stimulus associated
sis is inconsistent with some of the observing response with the greater reduction in time to reinforcement.
data. In particular, a hypothesis that requires a stimu¬ For example, consider a procedure in which observ¬
lus to maintain observing behavior when it is paired ing responses change a mixed FI 20-sec FI 40-sec FI
or contiguous with primary reinforcement would have 180-sec schedule to the equivalent multiple schedule.
difficulty handling Auge’s (1973, 1974) results: pri¬ According to the delay reduction hypothesis, the
mary reinforcement was obtained in both components stimuli associated with both the FI 20-sec and FI 40-
of the multiple schedules studied in his experiments, sec schedules should maintain observing behavior
but observing behavior was maintained in only one (since the average time to reinforcement in the mixed
component. In addition, as Wilton and Clements schedule equals 80 sec) with more observing respond¬
(1971a) have argued, the fact that little observing re¬ ing maintained by the FI 20-sec stimulus. If the
sponding is maintained when reinforcement is avail¬ schedule were changed to MIX FI 20-sec FI 120-sec
able on every trial is also difficult to reconcile with FI 180-sec, however, only the stimulus associated with
the pairing hypothesis: as both stimuli in their con¬ the FI 20-sec component of the multiple schedule
tinuous reinforcement condition were consistently should reinforce observing. (Since the average time to
paired with primary reinforcement, they should be reinforcement in the mixed schedule is now 107 sec,
strong conditioned reinforcers and should maintain the middle-valued stimulus—associated with FI 120-
observing behavior as well as they do in the condition sec—no longer represents a reduction in time to rein¬
in which reinforcement is available in the presence of forcement.) The reinforcement density hypothesis
only one of the two stimuli. Kendall (1972) obtained makes equivalent predictions.
results similar to those of Wilton and Clements (1971a) Whether or not the form of the conditioned rein-
326 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

forcement hypothesis specified by the delay reduction reinforcement—including a reinforcement density hy¬
hypothesis proves correct, it is clear that the condi¬ pothesis, such as that considered when discussing
tioned reinforcement explanation of observing be¬ observing—would require that the subject prefer the
havior (e.g., Blanchard, 1975; Dinsmoor et ah, 1972; lights correlated with the 1-min delay. How much the
Jenkins 8c Boakes, 1973; Mulvaney et al., 1974) is more 1-min delay is preferred, however, depends upon an
consistent with the data than the uncertainty reduc¬ additional factor we have yet to specify: the size of the
tion explanation (e.g., Berlyne, 1957; Bloomfield, equal VI schedules leading to the outcomes. Consider
1972; Lieberman, 1972). We now turn to studies of the following three values for the equal Vis: 1, 6, and
choice behavior. The picture that emerges from the 36 min. With VI 1-min schedules the average interval
choice literature is consistent with the one we have between the onset of a trial and receipt of the dollar
drawn from the study of observing responses. is 3]/2 min as long as the subject is responding on both
levers. It takes y2 min, on the average, for concurrent
VI 1-min schedules to arrange reinforcement; the av¬
CHOICE AND CONDITIONED erage wait in the presence of the flashing lights (i/2 X
REINFORCEMENT 1 min + yz x 5 min) is an additional 3 min. But by
responding exclusively on the lever leading to the
shorter delay, the subject can receive a dollar every 2
The Delay Redu^tieh
mm (1 min on the appropriate VI 1-min plus the
The delay reduction hypothesis was developed 1-min delay). Indeed, for concurrent Vis of 4 min or
(Fantina, 1969b) to integrate data from a series of less, the subject will increase his or her earnings by
experiments on choice and conditioned reinforcement exhibiting exclusive preference for the 1-min delay;
begun by Autor (1960) and continued by many others only that outcome is correlated with a reduction in
in the subsequent sixteen years (cf. Fantino & Nayarick, waiting time to reinforcement. With VI schedules
1974; Hendry, 1969). When applied to choice pro¬ greater than 4 min, however, both outcomes are cor¬
cedures, the delay reduction hypothesis states that (1) related with delay reduction and the subject should
organisms will choese the stimulus correlated with the respond on both levers. For example, with VI 6-min
greatest reduction in time to primary reinforcement schedules it takes an average of 6 min to obtain a dol¬
and (2) preference will be greater the larger the dif¬ lar (an average of 3 min of responding on the levers
ference in the delay reductions correlated with the and 3 min in the presence of the flashing lights) if the
chosen alternatives. Usually, the delay reduction hy¬ subject responds on both levers, but 7 min if re¬
pothesis has been applied only in this sense of im- sponses are made exclusively on the lever with the
provement in temporal proximity to reinforcement. shorter delay. Here the 5-min delay represents a re¬
Obviously, other variables such as punishment and duction of 1 min (i.e., the onset of the flashing lights
the probability and magnitude of reinforcement affect correlated with the longer delay brings the subject 1
conditioned reinforcement. The delay reduction hy¬ mm closer in time to the dollar than he had been
pothesis may be broadened readily to encompass im¬ while in the choice phase) whereas the 1-min delay
provement in reinforcer amount and probability and represents a 5-min reduction. Thus the short-delay
in punishment reduction, as I suggest below. outcome represents a reduction 5 times greater than
First, consider a subject operating a two-levered the alternative outcome. The longer the Vis, the
gambling device with the following payoff structure: smaller this ratio. Hence, with the VI 36 min sched¬
Pulls on either lever are reinforced (according to ules, the delay reductions for the 1- and 5-min out¬
equal VI schedules) by access to one of two sets of comes are 20 min and 16 min, respectively, a ratio of
flashing lights, each correlated with the delivery of a just 5:4.
response-independent dollar bill. The dollar is de¬ In summary, the delay reduction hypothesis states
livered after 5 min in the presence of one set of flash¬ that subjects’ choice for the more preferred of two
ing lights, but after only 1 min in the presence of the alternatives should increase, the shorter the choice
other lights. After the dollar is received, the subject phase: with sufficiently short Vis only one outcome
may again pull the levers leading to the flashing lights represents an improvement in terms of temporal prox¬
and more dollar bills. Pulls on one lever (when effec¬ imity to reinforcement; as the Vis are made progres¬
tive—as determined by the VI schedule) always lead to sively longer, both outcomes represent improvement
the same outcome. Choice is measured by the relative and the subject should shift away from exclusive
rates of lever pulling leading to the two outcomes. preference toward indifference. Note that a reinforce¬
Obviously, many theories of choice or conditioned ment density hypothesis assumes the choice will de-
Edmund Fantino 327

pend only on the relative rates of reinforcement dur¬ chain. The independent variable is some manipula¬
ing the two outcomes and that choice should therefore tion of the relation between the conditioned and the
be independent of the length of the choice phase primary reinforcer. The dependent variable is the
(Autor, 1960, 1969; Herrnstein, 1964a). As we shall relative strength of the two conditioned reinforcers as
see, data instead support the delay reduction hypothe¬ measured in the equal initial links: the number of
sis. choice responses made for one conditioned reinforcer
Since probability of reinforcement is closely analo¬ divided by the total number of choice responses made
gous to frequency (probabilities of obtaining a dollar for both conditioned reinforcers.
may be substituted for waiting times as the outcomes Beginning with Autor (1960, 1969), the concurrent
in the example above) and since punishment is con¬ chains procedure has been used extensively in the
ceptually and empirically symmetrical with reinforce¬ study of both choice and conditioned reinforcement.
ment (though opposite in sign—cf. Fantino, 1973, for While some investigators have used it primarily to
a review) there is no need to illustrate how the delay study choice, others have emphasized its relevance for
reduction hypothesis applies to these variables. We conditioned reinforcement. Indeed, the same depen¬
shall consider choice for reinforcer magnitudes, how¬ dent variable can be taken as both a measure of choice
ever. Assume you are walking in a foreign city and and of conditioned reinforcement. Before outlining
are hungry. You know of two fine restaurants to the procedure we should point out its potential
which you had once been taken. You enjoy both but strengths and weaknesses.
have a preference for one. You anticipate it will take The procedure is a good one for studying choice
about X min to track down each restaurant. Do you between different schedules of reinforcement because,
decide to dine at whichever you find first or will you unlike simple concurrent schedules, the measure of
hold out for the preferred one? I predict your decision choice is not confounded with the rates of responding
would be based, in part, on the size of X (correspond¬ generated by the schedules chosen. For example, re¬
ing to the duration of the choice phase): the shorter sponse rates on VR schedules tend to be much higher
the choice phase, the more likely you are to persist in than on FI schedules. If we took the relative rates of
finding the preferred restaurant. If X is very large, responding on a simple concurrent VR FI schedule a§
you are more likely to behave indifferently to the two. our measure of choice, therefore, we would be stack¬
If this assumption is correct, then temporal context ing the deck in favor of the VR schedule. Such a pro¬
affects choice in the same way whether the outcomes cedure would be more obviously inappropriate if we
differ with respect to rate or magnitude of reinforce¬ were comparing choice between a ratio schedule and a
ment. Doug Navarick and I supported this notion schedule which required low rates of responding. Per¬
recently in an analogous experiment with pigeons haps to avoid such confusion of choice with the re¬
choosing between 4.5 and 1.5 sec access to grain: as the sponse rates generated by the schedules being chosen,
duration of the choice phase decreased—relative to the Autor developed the concurrent chains procedure dia¬
duration of the outcome phase—choice for the larger gramed in Figure 6. In this procedure, the organism
reward increased. (normally a pigeon) responds on two concurrently
Before discussing the delay reduction hypothesis available keys, each illuminated by the same stimulus.
and its implications more fully and rigorously and Responses on each key occasionally produce another
presenting supporting data, we should review briefly stimulus, correlated with entry into the terminal link
the concurrent chains procedure developed to study of the chain on that key. Entry into the two terminal
conditioned reinforcement and used to test the delay links generally occurs at the same rate. Once the sub¬
reduction hypothesis. ject enters one terminal link the other key becomes
dark and inoperative. Responses in the presence of the
The Concurrent Chains Procedure
terminal-link stimuli are reinforced with food. In
most experiments the initial links are reinstated after
This procedure, which is described fully below, in¬ the subject obtains a single reinforcement in a termi¬
volves measuring choice by rate of response during nal link. The independent variable has generally in¬
two equal VI schedules, each leading to a different volved some difference in the conditions arranged
conditioned reinforcer. Since the concurrently avail¬ during the two terminal links. The dependent vari¬
able VI schedules are equal, differences in response able is the measure of choice: the responses in the
rates can be assumed to reflect differences in the rein¬ initial links.
forcing effectiveness of the stimuli being chosen, i.e., Another advantage of the concurrent chains pro¬
the stimuli associated with the terminal links of the cedure is that it keeps the number of primary rein-
328 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

Fig. 6. The concurrent chains procedure. Panel A indicates the

sequence of events when responses on the left key are reinforced:
panel B presents the analogous sequence on the right key.
Responses in the presence of the colored lights (the stimuli of
the terminal links) are reinforced, with food according to some
schedule of reinforcement (generally, the independent variable).
The measure of choice is the relative rate of responding in the
presence of the concurrently available white lights. Typically,
ecjual VI schedules arrange access to the terminal links. (Adapted
from Fantino, 1969b. © 1969 by the Society for the Experimen¬
tal Analysis of Behavior, Inc.)

lor cement s for responding on each key close to the

number intended by the experimenter over a wide
range of preference for pecking one key or the other.
If a subject responded exclusively on one key, for 0 -2 4 .6 ,8 1.0

example, all primary reinforcements would be de¬ relative rate of relative reinforcements
livered on that key. Because of the nature of concur¬ reinforcement per response
rently available VI schedules, however, the subject
Fig. 7. Relative rate of responding as a function of relative
produces a higher rate of entry into the terminal links rate of primary reinforcement (left panels) and the relative
(and typically a higher rate of primary reinforcement) probability of primary reinforcement (right panels). The diag¬
if it responds on both keys. In practice, this assures onal line from the origin to (1.0, 1.0) represents the locus of
perfect matching between relative response rates and relative
that the terminal link of each key will be entered reinforcements. The other line represents the linear regression
equally often, even while the dependent variable- line through the data points. Each graph shows the calculated
linear equation and the standard deviation around the regres¬
relative response rate in the initial links—is varying
sion line. (Adapted from Herrnstein, 1964a. © 1964 by the
over a wide range. Thus the effects of number of Society for the Experimental Analysis of Behavior, Inc.)
reinforcements are not confounded with those of the
intended independent variable. The procedure is also
useful for studying conditioned reinforcement for two tioned reinforcer does not therefore suffer extinction.
additional reasons. Normally, the effects of a condi¬ Secondly, choice situations are known to be very sensi¬
tioned reinforcer are examined in the absence of pri¬ tive to experimental manipulations (e.g., Catania,
mary reinforcement in order to avoid possible con¬ 1963, 1966; Herrnstein, 1961; Rachlin, 1967), and this
founding between the two reinforcing effects. In the favors experimental differentiations among the
terminal link of concurrent chains, a primary rein¬ strengths of different conditioned reinforcers.
forcement schedule can be maintained, and the condi- Most of the work with the concurrent chains pro-
Edmund Fantino 329

cedure has examined the dependence of choice upon

parameters of the interreinforcement interval (IRI),
the interval between the conditioned reinforcement
(i.e., a reinforced choice response) and the uncondi¬
tioned reinforcement (i.e., food). This is also the
material most relevant to our theoretical treatment of
conditioned reinforcement and to the integration of
results from concurrent chains studies with those
from studies of observing responses: both sets of re¬
sults indicate that the strength of a conditioned rein¬
forcer is determined primarily by the increase in
proximity to primary reinforcement correlated with
the conditioned reinforcer (the delay reduction hy¬
pothesis). We shall not, therefore, discuss other vari¬
ables which have been shown to affect preference in
the concurrent chains procedure such as magnitude of
reinforcement (e.g., Schwartz, 1969; Ten Eyck, 1970)
or number of reinforcements (Fantino 8c Herrnstein,
1968; Squires & Fantino, 1971).

Choice with Aperiodic Schedules in Terminal Links Fig. 8. Average linear regression lines for relative rate of re¬
sponding during choice phase as a function of the relative rate
Although all the research we shall summarize sup¬ of reinforcement (solid line crossing the diagonal line) and the
relative probability of reinforcement (dashed line). The diagonal
ports the notion that choice is determined by the
line from the origin to (1.0, 1.0) represents the locus of match¬
extent of reduction in delay to primary reinforce¬ ing between choice and relative reinforcements, The subscripts
ment correlated with the alternatives, the early work of X in the equations (T and ^>) correspond to the independent
variables of relative rate (XT) and relative probability (X ).
was consistent with a somewhat simpler interpretation
(Adapted from Herrnstein, 1964a. © 1964 by the Society for the
which was accepted for nearly a decade. Specifically, Experimental Analysis of Behavior, Inc,)
both Autor (1960, 1969), using VI, VR, and response-
independent schedules in the terminal links, and
Herrnstein (1964a), using VI and VR schedules, found Figures 7 and 8 show Herrnstein’s basic results: the
that the relative rates of choice responding (the num¬ relation of choice responding to both the relative rates
ber of initial-link responses on one key divided by the of reinforcement and the relative reinforcements per
total number of initial-link responses on the two keys) response (in the terminal link) for individual birds
matched the relative rates of reinforcement (the rate (Figure 7) and the averaged data (Figure 8). As Figure
of reinforcement on one key divided by the sum of the 8 shows most clearly, choice responding matched rela¬
two rates of reinforcement) in the two terminal links. tive frequency of reinforcement more closely than
This relation may be summarized by the following relative probability of reinforcement. In this sense,
equation: Herrnstein’s study went beyond that of Autor in show¬
ing that the relative rate of primary reinforcement,
RL_\JhL rather than its relative probability, controlled the
RL + Rr 1/ + 1/ hR
effectiveness of the conditioned reinforcers correlated
with the terminal links of concurrent chains.
in which RL and Rr represent the number of re¬ Fantino (1969a, 1969b) suggested an alternative
sponses during the initial links on the left and right (now called the delay reduction hypothesis) to Equa¬
keys, respectively, and t2L and t2R represent the aver¬ tion 1 which was also consistent with these data.
age durations of the left and right terminal links. Specifically:

R, T - t9
(when hl < T, t2R < T)
Rl + Rr (T ~ hl) + (T — t2B)

= 1 (when t2L < T, t2R > T) (2)

= 0 (when ^2L > T> i2R < T)
330 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

where T represents the average-delay-to-primary rein¬ tion 1). Equation 2 also predicts the circumstances
forcement from the onset of either initial link and under which the organism should respond exclusively
t2L and t2R again represent the average durations of for the shorter terminal link: When entry into one
the left and right terminal links, respectively. Note terminal link fails to bring the organism closer in
that when entry into either terminal link produces an time to primary reinforcement that terminal link will
increase in the average delay to primary reinforce¬ not be entered. In terms of conditioned reinforcement,
ment (either t2L > T or t2R > T), Equation 2 requires a stimulus that is not correlated with a reduction in
the organism to emit all its choice responses to the average delay to reinforcement should not be a con¬
other key. In other words. Equation 2 specifies when ditioned reinforcer regardless of how contiguous the
the organism should respond exclusively on one key. onset of the stimulus may be to primary reinforcement
Of course, the case in which both t2L and t2R are (just as observing is not well maintained by stimuli
greater than T is impossible. correlated with positive reinforcement but with a
Equation 2 implies that the relative rate of re¬ decrease in proximity to reinforcement—e.g., Auge,
sponding in the initial links matches the reduction in 1974). Thus when either value of t is greater than 7\
the average delay to primary reinforcement correlated entry into the longer terminal link actually moves the
with entry into one terminal fink relative to the organism further from primary reinforcement, and the
reduction correlated with entry into the other. In stimulus correlated with entry into that terminal link
other words, the greater the improvement in terms of should not be a conditioned remforcer,
temporal proximity to reinforcement, correlated with Since Autor’s and Herrnstein’s data were consistent
the onset of the stimulus, the more effective it will be with both Equations 1 and 2, Fantino (1969b) tested
as a conditioned reinforcor. Like Equation 1, Equa¬ the two equations by varying T while holding tL and
tion 2 assumes that the conditioned reinforcing effec¬ tR constant. Specifically, Fantino used three different
tiveness of a stimulus will be a function of its prox¬ pairs of identical VI schedules to arrange entry into
imity to primary reinforcement and tQR in the two terminal links which were always VI 30-sec
Equations 1 and 2). Unlike Equation 1, Equation 2 and VI 90-sec. When the pair of initial-link schedules
requires consideration of the temporal context m of intermediate duration wa§ in effect (VI 120-sec), the
which thi§ proximity is embedded. For example, an choice proportions in the initial links matched the
implication of Equation 1 is that the organism’s pref¬ relative rates of reinforcement in the terminal links
erence for one conditioned reinforcer over another (Equation 1), a result consistent with both formula¬
should be invariant regardless of the rate of entry into tions, With shorter or longer initial-link durations,
the terminal links, at least when the initial links are however, the distribution of choice responses no longer
equal. In particular, if and were equal to 30 matched the relative rates of reinforcement, but con¬
and 90 sec, respectively. Equation 1 requires a choice tinued to be well described by Equation 2. In addi¬
proportion of .75 for t2L regardless of the value of the tion, Fantino Utilized two schedules in which both the
initial-link Schedules. But one may question why the initial and terminal links were unequal: chain VI 90-
organism should ever choose to enter the longer termi¬ aec VI 30-sec vs. chain VI 30-sec VI 90-sec. Choice data
nal link (t^n) if it can obtain almost immediate access from this procedure were also consistent with Equa¬
to the shorter one. The more inaccessible the two tion 2.
terminal-link schedules—i.e., the longer the initial The results from this experiment are shown in
links—the more the organism should be indifferent to Table 1. These data show that for 15 out of 16 cases
their difference, since with very long initial links, en¬ in which Equations 1 and 2 describe different choice
trance into either terminal link should be highly rein¬ proportions, Equation 2 provides a closer fit to the
forcing (just, as we saw earlier, observing is more obtained data. Moreover, for each of the 16 points.
frequent the less likely a positive outcome). Specifi¬ Equation 2 accounts for the direction of the devia¬
cally, Fantino (1969a) speculated that with sufficiently tions from Equation 1. Thus for each of the 10 cases
long initial links, organisms should be indifferent be¬ in which Equation 2 requires a higher choice propor¬
tween the different terminal links and that with suffi¬ tion than Equation 1, columns (i) and (iv) in section
ciently short initial links the organism should respond B of Table 1 show that Equation 1 underestimates
exclusively to the side leading to the shorter terminal these proportions. Similarly, for each of the six cases
link. If this is true, then only for a particular band of in which Equation 2 requires a lower choice propor¬
intermediate values should the distribution of choice tion than Equation 1, column (ii) in section B of
responses match the distribution of reinforcements Table 1 shows that Equation 1 overestimates these
obtained in the terminal links (as required by Equa¬ proportions. The data in column (iv) in section A of
Edmund Fantino 331

Table 1 provide partial support for the prediction values are, since the two terminal-link stimuli are
that the organism will cease responding for entrance correlated with the same degree of reduction in aver¬
into the longer terminal link when such entry moves age delay to reinforcement. Instead, one might expect
him further in time from primary reinforcement. This that preference would vary with the relative values of
prediction is supported in that there was a very strong the initial links, since the rates of both primary rein¬
preference for the shorter terminal link and, for two
of the six birds, exclusive preference was reached. forcement — and -———-jand condi-
There is a prediction of Equation 2 that seemed y hl + hl Hr + hR J
doubtful. Whenever the terminal-link durations t2L
and t2B are equal, a choice proportion of .50—i.e., in¬
difference—is required no matter what the initial-link tioned reinforcement y(-
t1L and .
t1R J are different

Table 1*

A. Proportion of choice responses to key providing higher rate of reinforcement in ter¬

minal link for each pigeon in each of four conditions. The average proportion for
each condition and the proportions required by Equations 1 and 2 are listed below
the line. All VI values listed are in seconds.

(**) (Hi) (iv)

(0
Chain Chain Chain Chain
VI90 VI30 VI600 VI30 VI120 VI30 VI 40 VI30
vs. chain vs. chain vs. chain vs. chain
VI30 VI90 VI 600 VI90 VI120 VI90 VI40 VI90

Pigeon 1 .97 .66 .74 .93

Pigeon 2 1.00 .56 .77 1.00
Pigeon 3 .83 .63 .87 1.00
Pigeon 4 .98 .63 .70 .89
Pigeon 5 — .57 .97 .96
Pigeon 6 — .53 .82 .92
Ayg.
Proportion .94 .60 .81 .95
Equation 1 .75 .75 .75 .75
Equation 2 .90 .55 .75 1.00

B. The deviations of the choice proportions above from the proportions required by
Equations 1 and 2 for each pigeon for conditions (i)-(iv). For each of the sixteen
points in which Equations 1 and 2 make different predictions, the smaller deviation is
underscored. Column (v) on each side gives the mean of the absolute deviations for
each pigeon. The means of the absolute deviations for each condition are listed below
the line.

Equation 1 Equation 2
(») (iii) (iv) (") (») (iii) (iv) (»)
(0 (0

Pigeon 1 + .22 -.09 - .01 + .18 .12 + .07 + .ii - .01 -.07 .06
Pigeon 2 + .25 - .19 + .02 + .25 .18 + .10 + .01 + .02 0 .03
Pigeon 3 + .08 -.12 + .12 + .25 .14 -.07 + .08 + •12 0~ .07
Pigeon 4 + .23 - .12 -.05 + .14 .14 + .08 + .08 -.05 - .11 .08
Pigeon 5 — -.18 + .22 + .21 .20 - + .02 + .22 -.04 .09
Pigeon 6 - -.22 + .07 + .17 .15 — -.02 + .07 -.08 .06

Mean of
absolute
deviations .20 .15 .08 .20 .16 .08 4)5 .08 M .06

* (From Fantino, 1969b. © 1969 by the Society for the Experimental Analysis of Behavior, Inc.)
 332
CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

for the two keys when the value of the initial-link VI Squires and Fantino (1971) supplied this test and
schedules (t1L and t1R on the left and right, respec¬ suggested that an additional variable, which takes
tively) are unequal. Therefore, a critical test of the into account the rate of primary reinforcement on
generality of Equation 2 assesses whether indifference each key separately, should be incorporated into
holds when the initial links are unequal for the two Equation 2:
keys with t2L equal to t2R.

Rl tl(T — t2L)
(when t2L < T, t2R < T)
Rl + Rr tl (T — t2L) + rR (T — t2R)

= 1 (when hl < T, t2B > T) (3)

= 0 (when t2L > T, t2R < T)

where rL _ nLj(t1L -f nLt2L), which is the rate of pri¬ binary choice, i.e., whether one schedule will be pre¬
mary reinforcement on the left key (nL is the number ferred to another, when the terminal links consist of
of primary reinforcements obtained during one entry schedules other than Vis. This formulation must be
into the terminal link of the chain on the left key); and restricted largely to VI schedules, however, where
r& — nR/{hs + the rate of primary reinforce¬ precise quantitative predictions are required. These
ment on the right key. Equation 3 has the additional restrictions (which also apply to the reinforcement
advantage of predicting matching m choice behavior density formulation represented by Equation 1) de¬
with concurrent VI schedules (Herrnstein, 1970), i.e., pend, in part, on two characteristics of choice. First,
when UL = taR.
short interreinforcement intervals are weighted more
Figure 9 illustrates the predictions made by Equa¬ heavily than long ones in choice, a fact consistent
tions 1 through 3 in two concurrent chains situations. with the relative contribution to observing behavior
Figure 9a illustrates the case, studied by Autor (1960), made by the more positive stimulus as opposed to
Herrnstein (1964a), and Fantino (1969b), in which the either the less positive or negative stimulus. Thus
terminal-link schedules are unequal (VI 30-sec and VI Herrnstein (1964b), using the concurrent chains pro¬
90-sec in the example). Figure 9a shows the predic¬ cedure, found that pigeons strongly preferred a VI
tions made by each of the three equations as the value schedule with an average interreinforcement interval
of the (equal) initial links increases. Figure 9b illus¬ of 15 sec to an FI 15-sec schedule. Comparable results
trates the type of schedule studied by Squires and were reported by Bower et al. (1966), Davison (1969,
Fantino (1971) in which the two terminal links are 1972), Fantino (1967), Hursh and Fantino (1973),
equal but the initial links are unequal. While Equa¬ Killeen (1968a), and Navarick and Fantino (1975).
tions 1 and 2 require choice proportions of .50 regard¬ Since variable schedules are preferred to fixed sched¬
less of the difference between the two initial links, ules providing the same reinforcement density (and
Equation 3 requires an increase in choice proportions the same mean delay reduction), an appropriate prin¬
on the key with the constant initial-link VI value as ciple of transforming variable schedules into their
the size of the initial-link VI schedule increases on the fixed equivalents, and vice versa, would be necessary
other key,
if a single quantitative model were to describe choice
Squires and Fantino’s results unequivocally sup¬ proportions for both types of schedules. Choice be¬
ported Equation 3. Nonetheless, for the sake of clar¬ tween two schedules could then be described by first
ity, we shall discuss the simpler, unmodified version converting the schedules into their VI equivalents and
in this chapter. This is acceptable since Equations 2 then applying Equation 3. But no adequate general
and 3 make similar predictions with equal initial principle of transformation has been found. In the
links, and since virtually all studies of concurrent second place, when different types of schedules are
chains schedules have used equal initial links. arranged in the terminal links of concurrent chains,
Thus data from these procedures suggest that the choice is likely to fail tests of strong stochastic transi¬
strength of a conditioned reinforcer is a function of tivity (Navarick & Fantino, 1972, 1974, 1975). One
the reduction in average delay to reinforcement cor¬ implication of such intransitivities is that a general
related with one conditioned reinforcer, relative to principle of transformation cannot be found. The
the reduction in average delay to reinforcement cor¬ issues here are thorny, and the interested reader can
related with the other. The delay reduction formula¬ find a complete account in Fantino and Navarick
tion is also useful in making ordinal predictions for (1974). For our present purpose it is sufficient to note
Edmund Fantino 333

would probably affect observing in the same way, as

we shall point out, though in most cases this has not
been assessed. (In general, I predict that if one stim¬
ulus is preferred to another, it should maintain ob¬
serving responses which change a mixed schedule,
composed of both stimuli, to the corresponding multi¬
ple schedule.) We consider these effects in the next
section.

Events During the Terminal Links

Neuringer (1969), in an interesting concurrent

chains experiment, compared choice between rein¬
forcement on an FI schedule and reinforcement after
an interval in which the key was dark and inopera¬
tive. He found indifference between the FI and the
response-independent schedules when the duration of
the terminal links was equal. From his own and re¬
lated data (including Anger, 1956; Autor, 1960; Dews,
1962, 1965; Herrnstein, 1964a; Killeen, 1968b; Neu¬
ringer & Schneider, 1968), Neuringer concluded that
“these studies, together with the present findings, sug¬
gest the following hypothesis; the probability (or
rate, or latency) of a response is controlled by the
interval between that response and reinforcement (a)
independently of the number of other responses inter¬
vening in the interval, and (b) independently of
whether such intervening responses are required or
prohibited77 (p. 382).
Others, including Rachlin and Herrnstein (1969)
and Schneider (1972) have also suggested that or¬
ganisms should be indifferent between two terminal
links whose overall durations are equal. In other
words, given equal initial links, choice proportions
should approximate .50 as long as the durations of
VALUE OF ONE INITIAL LINK IN SEC (X) the terminal links are also equal. This result would
Fig. 9. The predictions of Equations 1, 2, and 3. In (a) the be consistent with all the models of choice we have
choice proportions (for the VI 30-sec terminal-link schedules) considered thus far. For example, in Neuringer’s ex¬
required by the three equations are plotted against the value periment the stimuli associated with each terminal
of the equal initial links (VI X-sec). In (b) the choice propor¬
tions for the key with the VI 60-sec schedule in the initial link
link are correlated with identical reductions in time
required by each of the three equations are plotted against the to reinforcement. It might seem surprising, however,
initial-link value on the other key (VI X-sec). (Adapted from if choice were so simply and directly dependent on the
Squires & Fantino, 1971. (c) 1971 by the Society for the Experi¬
mental Analysis of Behavior, Inc.) size of the terminal links (t2L and t2R in Equations
1-3) irrespective of events occurring during them. In¬
deed, there are important exceptions to this gen¬
that the delay reduction hypothesis is consistently eralization. These are considered in this section.
superior to the reinforcement density hypothesis in
accurately describing choice.
Addition of Stimuli Paired
Up to now we have considered only the temporal
with Reinforcement
duration of the terminal links (i.e., t2L and t2B, or
what we have called the interreinforcement interval Schuster (1969) reported an important study testing
or IRI). Events during the IRI also affect choice and the notion that stimuli paired with primary rein-
334 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

forcement will serve as conditioned reinforcers. Brief- The Effects of Required

stimulus presentations paired with primary reinforce¬ Rates of Responding

ment at the end of each terminal link were made

available (on an FR 11 schedule) during one of two Fantino (1968) demonstrated that high response
terminal links. The brief-stimulus presentations were rate requirements (such as schedules differentially
therefore superimposed on the terminal-link food reinforcing high rates, or DRH) are apparently aver¬
schedule. The main dependent variable was the rate sive, at least when compared with a simple FI that
of choice responding in the concurrently available provides reinforcement at the same rate and on the
initial links. Rate of primary reinforcement in the two same proportion of trials. Other studies, however,
terminal links was identical and was available on VI have failed to demonstrate the effects of response rates
30-sec schedules. Therefore, if the stimuli paired with upon choice (for example, Herrnstein, 1964a; Killeen,
food reinforcement were conditioned reinforcers, the 1968b, 1971; Neuringer, 1969, discussed above). Moore
pigeons should prefer to enter the terminal link with and Fantino (1975) shed some light on these ostensibly
the superimposed schedule of brief stimuli, Schuster conflicting findings. They noted that studies on the
found that while the superimposed brief stimuli in¬ response rate problem fell into three categories: (1)
creased response rates in the terminal link, this termi¬ those using aperiodic schedules in which response
rates were varied but particular response rates were
nal link “was eventually chosen less often. This result
was consistent with a functional analysis, since, on the not required (Herrnstein, 1964a; Killeen, 1968b)—for
key with the superimposed schedules, the brief stimuli example, Killeen’s pigeons were indifferent between a
were correlated more often with nonreinforcement” VI schedule and a comparable response-independent
(p. 234). schedule, although large differences in response rates
were maintained by the two schedules); (2) those using
As Fantino (1969a), Gollub (1970), and Squires
(1972) have observed, Schuster’s conclusion is rendered periodic schedules in which response rates were varied
ambiguous by the potential effects of the brief-stimulus
but particular response rates were not required (Neu¬
ringer; 1969; Killeen, 1971); and (3) those in which
presentations on terminal-link response rates. In par¬
ticular, the high response rates generated by super¬ particular response rates were required on periodic
imposed brief stimuh might have created an aversion
schedules (Fantino, 1968). Moore and Fantino ex¬
amined preference for additional schedules falling
for that terminal link which might have masked any
conditioned reinforcing effect. Squires (1972) repeated into each of the three categories and replicated the
Schuster’s experiment in a way that avoids the con¬ prior results; only when particular response rates
founding factor present in the earlier experiment: were required on periodic schedules did subjects
choose the schedule without the response rate require¬
whereas Schuster had scheduled the superimposed
brief stimuli on an FR schedule. Squires scheduled ment. In addition, choice was unaffected when par¬
them on a VI. Squires found no consistent preference ticular response rates were required on aperiodic
or aversion for the schedule with superimposed brief-
schedules. These results suggested that response re¬
quirements influence choice to the extent that re¬
stimulus presentations. Thus when the high rates of
sponses must be emitted during discriminable periods
terminal-link responding, confounded in Schuster’s
work with brief-stimulus presentations, are eliminated, of nonreinforcement, as in the early portion of a
a preference for the schedule providing the brief stim¬
periodic schedule. Moore and Fantino confirmed this
uli still fails to develop. notion by showing that pigeons preferred a periodic
response-independent schedule to a periodic, response-
It must be noted, however, that the superimposed
dependent schedule that included a requirement to
brief stimuli are conditioned reinforcers only in the
respond early in the terminal link even though re¬
sense of the pairing hypothesis introduced and dis¬
sponding produced reinforcement only later. It is
missed earlier. They certainly are not reinforcers in
likely that observing would also be maintained by the
the informational sense (as Schuster rightly pointed
stimulus of the response-independent schedule, i.e.,
out), nor are they correlated with a reduction in aver¬
that subjects would respond to change a mixed into a
age time to reinforcement. Nor are they likely to
multiple schedule if one component required responses
maintain observing, though this has yet to be as¬
sessed. during discriminable periods of nonreinforcement.

In any case, although Squires’s results are consis¬

The Effects of Stimulus Sequences
tent with Neuringer’s conclusion that choice is un¬
affected by events occurring during the IRI, results Fantino (1969a) suggested that if two schedules of
from three other types of experiments are not. equal duration were segmented differently, the or-
Edmund Fantino 335

ganism would prefer the one composed of fewer dis- cannot be explained in terms of reinforcement rates.
criminable components. In other words, a simple FI Duncan and Fantino (1972) concluded that prefer¬
2X should be preferred to a chain FI X FI X schedule. ence for the simple FI could be attributed either to
Duncan and Fantino (1972) support this suggestion, the additional stimulus or to the additional response
showing that choice for a schedule is substantially requirement associated with the chain FI FI, but that
reduced by the chaining operation. Their procedure whether the results were due to stimulus or response
is schematized and their main finding, preference for aspects of the chaining operation (or both), the results
simple FI schedules over two-link chains with equiva¬ pointed to the insufficiency of reinforcement rates (or
lent durations, is shown in Figure 10. Note that the delay reductions) in determining choice. In order to
obtained terminal-link durations were close to the tease apart the role of stimulus and response aspects
scheduled ones (Xs close to .50 in Figure 10b), espe¬ of the chaining procedure, one portion of an experi¬
cially with the longer terminal links, for which the ment by Wallace (1973) assessed choice between chain
largest preferences were found. Thus these results FT X FT X and a simple FT 2X schedule. If stimulus
aspects were responsible for the large preferences ob¬
tained by Duncan and Fantino, then preferences
should be comparable in this procedure which is
identical except for the absence of a response require¬
ment in the terminal links. On the other hand, if
response factors were sufficient to explain the large
preferences obtained by Duncan and Fantino, then
the results should reveal indifference between the
chain and the simple FT schedules. Wallace found a
clear and consistent preference for the simple FT
schedule. Nonetheless, preferences were far smaller
than they had been in the Duncan and Fantino study.
This result suggests that both stimulus and response
factors were responsible for the large preferences ob¬
tained by Duncan and Fantino (and replicated in a
different portion of the Wallace study). It is possible,
of course, that response factors are effective only inso¬
far as they increase the degree of stimulus control.
Taken together, these results suggest that choice in
the concurrent chains procedure may not be accurately
described by considering simply the relative size of
the terminal links, if the terminal links are segmented
and are arranged according to periodic (as opposed to
aperiodic) schedules of reinforcement.6 Apparently,
the initial-link stimulus of a chain FI FI or chain FT
FT schedule is a more effective signal that the or¬
ganism is temporally distant from primary reinforce¬
ment than is the stimulus associated with an FI or FT
schedule, even though the durations and, therefore,
the average delays to primary reinforcement are
equal. Apparently, the subjective delay (or the “psy¬
Fig. 10. (Top) Pictorial representation of the experimental chological distance”) to primary reinforcement is
procedure used by Duncan and Fantino (1972). The left portion greater in the segmented schedule. It should be ac¬
of the figure indicates the sequence of events when responses
on the left key were reinforced; the right portion indicates the knowledged that these results not only limit the scope
sequence of events when responses on the right key were rein¬ of the delay reduction hypothesis (as well as that of
forced. The terminal links consisted of a simple FI schedule on one
of the keys and a chain FI FI schedule on the other key. (Bot¬
tom) The mean choice proportions for each bird on the FI key 6 Schneider (1972) found no difference between chain and
as a function of the size of the intervals in the terminal links. tandem VI schedules. The difference between his finding and
The x’s indicate the relative rate of reinforcement on the FI that of Duncan and Fantino (1972) and Wallace (1973) is
key. (Adapted from Duncan & Fantino, 1972. © 1972 by the probably due to his use of VI schedules. For a discussion, see
Society for the Experimental Analysis of Behavior, Inc.) Duncan and Fantino (1972).
 336 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

other formulations of choice in the concurrent chains related with differing delays of reinforcement. Such
procedure), but are consistent with the pairing hy¬ preference, and its interpretation in terms of the rein¬
pothesis, dismissed as inadequate earlier, in the fol¬ forcing strength of stimuli correlated with positive
lowing sense: contiguity with primary reinforcement outcomes, is consistent with the data and theory on
may sometimes enhance the effectiveness of a condi¬ observing responses, reviewed earlier.
tioned reinforcer. In terms of observing responses,
these results suggest that subjects would respond to Summary
convert mixed FI 2X (chain FI X FI X) or mixed FT The results considered in the last three sections all
2X (chain FT X FT X) schedules to their multiple- suggest that while terminal-link duration (the inter¬
schedule equivalents. Again, these experiments have reinforcement interval or IRI) is a crucial determi¬
not been done. nant of choice, events during the IRI must also be
considered in any complete account of choice in the
Differential Discriminative Stimuli
concurrent chains procedure. Stated in terms of con¬
in the Terminal Links ditioned reinforcement on periodic schedules, the
results from the section on events during the IRI sug¬
Several experimenters have shown that organisms gest that the strength of a stimulus as a conditioned
prefer multiple schedules of reinforcement to equiva¬ reinforcer depends not only on the reduction in
lent mixed schedules (e.g., Bower et al., 1966; Hendry, average delay to reinforcement correlated with its
1969b; Hursh 8c Fantino, 1974). Both Bower et al. and onset but also on whether response rate is con¬
Hendry found uniformly large preferences for the strained in its presence, on the number of additional
multiple schedules. But responding in the initial links stimuli intervening between it and primary reinforce¬
in their experiments was reinforced on FR schedules ment, and on the presence of differential discrimina¬
which, unlike concurrent VI schedules, are likely to tive stimuli in the terminal links (as in the mixed- vs.
lead to exclusive responding to one of the concurrent multiple-schedule comparison). Results from each of
stimuli; this in turn means that most trials ended the three areas reviewed have been firmly established
with the multiple-schedule terminal link. This bias only with periodic (FR, FI, and FT) schedules, as
may have contributed to the magnitude of the prefer¬ opposed to aperiodic (VR, VI, and VT) schedules,
ence for the multiple schedule. Hursh and Fantino suggesting the following: since clearly discriminable
(1974) obtained reliable but much smaller preferences periods of nonreinforcement are not present in aperi¬
for the multiple schedule with VI 60-sec schedules in odic schedules, differences between aperiodic schedules
the choice phase and showed that the degree of prefer¬ are likely to be less salient and to have less effect upon
ence could be manipulated by varying the size of the choice than comparable differences between periodic
initial-link schedules (i.e., the shorter the choice schedules. Thus while the delay reduction hypothesis
phase, the larger the preference). of conditioned reinforcement is consistent with all of
This preference for multiple over mixed schedules the data from the observing response and concurrent
can be understood in terms of the well-established chains literature when only the size of the IRI is
finding that the presence of short interreinforcement manipulated, some events occurring during the IRI
intervals has a disproportionate influence on choice must also be considered for a complete account of
(Davison, 1969, 1972; Fantino, 1967; Herrnstein, 1964b; choice. It is likely that these same events would have
Hursh &! Fantino, 1973; Killeen, 1968a; Navarick comparable effects on observing responses.
& Fantino, 1975). In mixed schedules of reinforce¬
ment, no stimuli correlated with a short delay occur
at the beginning of the terminal links, even on that CONCLUSIONS
half of the trials in which the shorter delay is sched¬
uled. On multiple schedules, on the other hand, stim¬ The research on observing behavior suggests that
uli correlated with the short delay are immediately information per se is not reinforcing. While drasti¬
available on half of the trials. The occurrence of these cally modified versions of the information hypothesis
stimuli are responsible for preference of the multiple —stressing “good news” only—can be made to fit most
schedule over the mixed schedule. of the observing response data, such accounts are often
The results reviewed here suggest that even when indistinguishable from conditioned reinforcement hy¬
different terminal links provide the same distribu¬ potheses such as the delay reduction hypothesis and
tion of interreinforcement intervals, subjects will the reinforcement density hypothesis. Moreover, the
choose the link in which different stimuli are cor- information hypotheses all seem to lack any specifica-
Edmund Fantino 337

tion of the importance of temporal variables, which Byrd, L. D. Responding in the pigeon under chained
schedules of food presentation: The repetition of a
are critical for the maintenance of operant behavior.
stimulus during alternate components. Journal of the
A more viable possibility is that organisms will re¬ Experimental Analysis of Behavior, 1971, 16, 31-38.
spond to produce stimuli correlated with a reduction Catania, A. C. Concurrent performances: Reinforcement
in delay to primary reinforcement (whether or not the interaction and response independence. Journal of the
observing response actually affects time to reinforce¬ Experimental Analysis of Behavior, 1963, 6, 253-263.
Catania, A. C. Concurrent operants. In W. K. Honig (Ed.),
ment). Studies of choice behavior with the concurrent
Operant behavior: Areas of research and application.
chains procedure are also consistent with this delay Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966.
reduction hypothesis: when an organism chooses be¬ Davison, M. Preference for mixed-interval versus fixed-in¬
tween two stimuli correlated with different reductions terval schedules. Journal of the Experimental Analysis
in delay to primary reinforcement, its choice of either of Behavior, 1969, 12, 247-252.
Davison, M. Preference for mixed-interval versus fixed-
stimulus is a monotonic function of the relative re¬
interval schedules: Number of component intervals.
duction in average time to reinforcement correlated Journal of the Experimental Analysis of Behavior, 1972,
with that stimulus. If a stimulus is not correlated with 17, 169-176.
a reduction in delay to primary reinforcement, it will deLorge, J. The effects of brief stimuli presented under

not maintain a significant amount of responding in a multiple schedule of second-order schedules, Journal of
the Experimental Analysis of Behavior, 1971, IS, 19—25.
either the observing response or concurrent chains
Dews, P. B. The effect of multiple periods on respond¬
procedures—i.e., the stimulus will not be a condi¬ ing on a fixed-interval schedule. Journal of the ExperU
tioned reinforcer. mental Analysis of Behavior, 1962, 5> 369-374.
Dews, P, B, The effect of multiple SA periods on respond¬
ing on a fixed-interval schedule, III: Effect of changes in
pattern of interruptions, parameters, and stimuli. Jour¬
nal of the Experimental Analysis of Behavior, 1965, 8,
REFERENCES
427-435.
Dinsmoor, J. A., Browne, M. 8c Lawrenge, G. E. A test
Anger, D. The dependence of interresponse time upon the of the negative discriminative stimulus as a reinforcer
relative reinforcement of different interresponse times. of observing. Journal of the Experimental Analysis of
Journal of Experimental Psychology, 1956, 52, 145-161. Behavior, 1972, IS, 79-85.
Auge, R. J. Effects of stimulus duration on observing be¬ Dinsmoor, J. A., Browne, M. P., Lawrence, C. E., 8c
havior maintained by differential' reinforcement magni¬ Wasserman, E. A. A new analysis of WyckofFs observing
tude. Journal of the Experimental Analysis of Behavior, response. Proceedings of the 79th Annual Convention
1973, 20, 429-438. of the American Psychological Association, 1971, 679-
Auge, R. J. Context, observing behavior, and conditioned 680. (Summary)
reinforcement. Journal of the Experimental Analysis of Dinsmoor, J. A., Flint, G. A., Smith, R. F., 8c Viemeister,
Behavior, 1974, 22, 525-533. N. F. Differential reinforcing effects of stimuli associated
Autor, S. M. The strength of conditioned reinforcers as a with the presence or absence of a schedule of punish¬
function of frequency and probability of reinforcement. ment. In D. P. Hendry (Ed.), Conditioned reinforce¬
Unpublished doctoral dissertation, Harvard University, ment. Homewood, Ill.: Dorsey Press, 1969.
1960. Duncan, B., 8c Fantino, E. The psychological distance to
Autor, S. M. The strength of conditioned reinforcers as a reward. Journal of the Experimental Analysis of Be¬
function of frequency and probability of reinforcement. havior, 1972, 18, 23-34.
In D. P. Hendry (Ed.), Conditioned reinforcement. Eckerman, D. A. Uncertainty reduction and conditioned
Homewood, Ill.: Dorsey Press, 1969. reinforcement. Psychological Record, 1973, 23, 39-47.
Berlyne, D. E. Uncertainty and conflict: A point of contact Fantino, E. Preference for mixed- versus fixed-ratio sched¬
between information theory and behavior theory con¬ ules. Journal of the Experimental Analysis of Behavior,
cepts. Psychological Review, 1957, 64, 329-333. 1967, 10, 35-43.
Berlyne, D. E. Conflict, arousal, and curiosity. New York: Fantino, E. Effects of required rates of responding upon
McGraw-Hill, 1960. choice. Journal of the Experimental Analysis of Be¬
Blanchard, R. The effect of S~ on observing behavior. havior, 1968, 11, 15-22.
Learning and Motivation, 1975, 6, 1-10. Fantino, E. Conditioned reinforcement, choice, and the
Bloomfield, T. M. Reinforcement schedules: Contingency psychological distance to reward. In D. P. Hendry (Ed.),
or contiguity. In R. M. Gilbert 8c J. R. Millenson (Eds.), Conditioned reinforcement. Homewood, Ill.: Dorsey
Reinforcement: Behavioral analysis. New York: Aca¬ Press, 1969. (a)
demic Press, 1972. Fantino, E. Choice and rate of reinforcement. Journal of
Bower, G., McLean, J., 8c Meacham, J. Value of knowing the Experimental Analysis of Behavior, 1969, 12, 723-
when reinforcement is due. Journal of Comparative and 730. (b)
Physiological Psychology, 1966, 62, 184-192. Fantino, E. Aversive control. In J. A. Nevin 8c G. S.
Branch, M. N. The distribution of observing responses Reynolds (Eds.), The study of behavior. Glenview, Ill.:
during two VI schedules. Psychonomic Science, 1970, Scott, Foresman, 1973.
20, 5-6. Fantino, E., 8c Herrnstein, R. J. Secondary reinforcement
338 CONDITIONED REINFORCEMENT: CHOICE AND INFORMATION

and number of primary reinforcements. Journal of the the Experimental Analysis of Behavior, 1973, 20, 73-83.
Experimental Analysis of Behavior, 1968, 11, 9-14. <b)
Fantino, E., & Navarick, D. Recent developments in Kendall, S. B., Sc Gibson, D. A. Effects of discriminative
choice. In G. H. Bower (Ed.), The psychology of learn¬ stimulus removal on observing behavior. The Psycho¬
ing and motivation (Vol. 8). New York: Academic Press, logical Record, 1965, 15, 545-551.
1974. Killeen, P. On the measure of reinforcement frequency in
Gollub, L. R. Information on conditioned reinforcement. the study of preference. Journal of the Experimental
A review of Conditioned reinforcement (D. P. Hendry, Analysis of Behavior, 1968, 11, 263-269. (a)
Ed.). Journal of the Experimental Analysis of Behavior, Killeen, P. Response rate as a factor in choice. Psycho¬
1970, 14, 361-372. nomic Science, 1968, 12, 34. (b)
Hendry, D. P. Reinforcing value of information. NASA Killeen, P. Response patterns as a factor in choice. Psycho¬
Technical Report 65-1. Space Research Laboratory, nomic Science, 1971, 22, 23-24.
University of Maryland, 1965. Lieberman, D. A. Secondary reinforcement and informa¬
Hendry, D. P. (Ed.). Conditioned reinforcement. Home- tion as determinants of observing behavior in monkeys
wood, Ill.: Dorsey Press, 1969. (a) (Macaca mulatto). Learning and Motivation, 1972, 5,
Hendry, D. P. Introduction. In D. P. Hendry (Ed.), Con¬ 341-358.
ditioned reinforcement. Homewood, Ill.: Dorsey Press, Malagodi, E. F., DeWeese, J., Sc Johnston, J. M. Second-
1969. (b) order schedules: A comparison of chained, brief-stimulus,
HENDRY, D. P. Reinforcing value of information: Fixed- and tandem procedures. Journal of the Experimental
ratio In D. P, Hendry (Ed.), Conditioned
schedules. Analysis of Behavior, 1973, 20, 447-460.
reinforcement. Homewood, Ill,: Dorsey Press, 1969. (c) McMigliael, J. S., Lanzetta, J. T., & Driscoll, J. Infre¬
Herr^sxein, R, J, Relative and absolute strength of re¬ quent reward facilitates observing responses in rats.
sponse as a. function of reinforcement, Journal of the Psychonomic Science, 1967, 8, 23-24.
Experimental Analysts of Behavior, 1961, J, 267-272. McMillan, J. G. Average uncertainty as a determinant of
Herrnstein, R. J. Secondary reinforcement and the rate of observing behavior. Journal of the Experimental Analy¬
primary reinforcement. Journal of the Experimental sis of Behavior, 1974, 22, 401-408.
Analysis of Behavior, 1964, 7, 27—36, (a) Moore, J., 8c Fantino, E. Choice and response contin¬
Herrnstein, R. J. Aperiodicity as a factor in choice, Jour¬ gencies. Journal of the Experimental Analysis of Be¬
nal of the Experimental Analysis of Behavior, 1964, 7, havior, 1975, 23, 339-347.
179-189. (h) Mulyaney, D. E., Dinsmoor, J. A., Jwaideh, A. R., 8c
Herrnstein, R. J. On the law of effect. Journal of the FIughes, L. H. Punishment of observing by the negative
Experimental Analysis o f Behavior, 1970, 1), 243-266. discriminative stimulus. Journal of the Experimental
Hull, G, L, Principles of Behavior. New York: Appleton- Analysis of Behavior, 1974, 21, 37-44.
Century-Crofts, 1943. Navarick, D. J., 8c Fantino, E .Transitivity as a property of
Hursei, S, R., 8c Fantino, E. Relative delay of reinforcement choice. Journal of the Experimental Analysis of Be¬
and choice. Journal of the Experimental Analysis of havior, 1972, 18, 389-401.
Behavior, 1973, 19, 437-450. Navarick7 D. J., & Fantino, E. Stochastic transitivity and
Hursh, S, R,, & Fantino, E, An appraisal of preference for unidimcnsional behavior theories. Psychological Review,
multiple versus mixed schedules, Journal of the Experi¬ 1974, 81, 426-441.
mental Analysis of Behavior, 1974, 22, 31—38. Navarick, D. J., & Fantino, E. Stochastic transitivity and
JENKINS, H. M., 2c BoAKES, R. A. Observing stimulus sources the unidimensional control of choice. Learning and Mo¬
that signal food or no food. Journal of the Experimental tivation, 1975, 6, 179-201.
Analysis of Behavior, 1973, 20, 197-207. Neurinqer, A, J. Delayed reinforcement versus reinforce¬
Kelleher, R. T. Stimulus producing responses in chimpan¬ ment after a fixed interval. Journal of the Experimental
Analysis of Behavior, 1969, 12, 375-383.
zees. Journal of the Experimental Analysis of Behavior,
1958, 1, 87-102. Neuringer, A. J., 8? Chung, S. Quasi-reinforcement: Con¬
Kelleher, R, T, Chaining and conditioned reinforcement. trol of responding by a percentage reinforcement sched¬
In W. K. Honig (Ed.), Operant behavior; Areas of ule. Journal of the Experimental Analysis of Behavior,
1967, 10, 45-54.
research and application. Englewood Cliffs, N.J.: Pren¬
Neuringer, A. J., 2c Schneider, B. A. Separating the effects
tice-Hall, Inc., 1966.
of interreinforcement time and number of interrein¬
Kelleher, R. T., 8c Gollub, L. R. A review of positive
forcement responses. Journal of the Experimental Analy¬
conditioned reinforcement. Journal of the Experimental
sis of Behavior, 1968, 11, 661-667.
Analysis of Behavior, 1962, 5, 543-597.
Nevin, J. A. Conditioned reinforcement. In J. A. Nevin 8c
Keller, F. S., 2c Schoenfeld, W. N. Principles of psychol¬ G. S. Reynolds (Eds.), The study of behavior. Glenview,
ogy. New York: Appleton-Century-Crofts, 1950. Ill.: Scott, Foresman, 1973.
Kendall, S. B. Effects of informative stimuli with multiple Prokasy, W. F. The acquisition of observing responses
delays of reinforcement. Psychonomic Science, 1972, 29, in the absence of differential external reinforcement.
41-42. Journal of Comparative and Physiological Psychology,
Kendall, S. B. Redundant information in an observing- 1956, 49, 131-134.
response procedure. Journal of the Experimental Analy¬ Rachlin, H. The effect of shock intensity on concurrent
sis of Behavior, 1973, 19, 81-92. (a) and single-key responding in concurrent-chain schedules.
Kendall, S. B. Effects of two procedures for varying infor¬ Journal of the Experimental Analysis of Behavior, 1967,
mation transmission on observing responses. Journal of 10, 87-93.
Edmund Fanfino 339

Rachlin, H., 8c Herrnstein, R. J. Hedonism revisited: On conditioned reinforcement. Journal of the Experimental
the negative law of effect. In B. Campbell 8c R. M. Analysis of Behavior, 1971, 16, 289-313.
Church (Eds.), Punishment and aversive behavior. Engle¬ Stubbs, A. D., 8c Cohen, S. L. Second-order schedules:
wood Cliffs, N.J.: Prentice-Hall, Inc., 1969. Comparison of different procedures for scheduling
Rescorla, R. A. Pavlovian conditioning and its proper paired and non-paired brief stimuli. Journal of the
control procedures. Psychological Review, 1967, 74, 71— Experimental Analysis of Behavior, 1972, 18, 403-413.
80. Taus, S. E., 8c Hearst, E. Effects of intertrial (blackout)
Rescorla, R. A. Probability of shock in the presence and duration on response rate to a positive stimulus. Psy¬
absence of CS in fear conditioning. Journal of Compara¬ chonomic Science, 1970, 79(5), 265-267.
tive and Physiological Psychology, 1968, 66, 1-5. Ten Eyck, R. L. Effects of rate of reinforcement-time upon
Rescorla, R. A. Informational variables in Pavlovian con¬ concurrent operant performance. Journal of the Experi¬
ditioning. In G. Bower (Ed.), The psychology of learn¬ mental Analysis of Behavior, 1970, 14, 269-274.
ing and motivation (Vol. 6). New York: Academic Press, Wallace, R. F. Conditioned reinforcement and choice.
1972. Unpublished doctoral dissertation, University of Cali¬
Schaub, R. E. Response-cue contingency and cue effective¬ fornia, San Diego, 1973.
ness. In D. P. Hendry (Ed.), Conditioned reinforcement. Wiener, N. Cybernetics. New York: Wiley, 1948.
Homewood, Ill.: Dorsey Press, 1969. Wilton, R. N. The role of information in the emission of
Schaub, R. E., 8c Honig, W. K. Reinforcement of behavior observing responses and partial reinforcement acquisi¬
with cues correlated with extinction. Psychonomic Sci¬ tion phenomena. Teaming and Alotivation, 1972, 5, 479—
ence, 1967, 1, 15-16. 499.
Schneider, J. W. Choice between two-component chained Wilton, R. N., 8c Clements, R. O. Observing responses and
and tandem schedules. Journal of the Experimental informative stimuli. Journal of the Experimental Analy¬
Analysis of Behavior, 1972, IS, 54-60. sis of Behavior, 1971,15, 199-204. (a)
Schuster, R. H. A functional analysis of conditioned rein¬ Wilton, R. N., 8c Clements, R. O. The role of information
forcement. In D. P. Hendry (Ed.), Conditioned rein¬ in the emission of observing responses: A test of two
forcement. Homewood, Ill.: Dorsey Press, 1969. hypotheses. Journal of the Experimental Analysis of
Schwartz, B. Effects of reinforcement magnitude on pi¬ Behavior, 1971, 16, 161-166. (b)
geon’s preference for different fixed-ratio schedules of Wyckoff, L. B., Jr. The role of observing responses in
reinforcement. Journal of the Experimental Analysis of discrimination learning. Psychological Review, 1952, 59,
Behavior, 1969, 12, 253-259. 431-442.
Wyckoff, L. B., Jr. Toward a quantitative theory of sec¬
Shannon, C. E., & Weaver, W. The mathematical theory of
ondary reinforcement. Psychological Review, 1959, 69,
communication. Urbana: University of Illinois Press,
68-78.
1949.
Wyckoff, L. B., Jr, The role of observing responses in dis¬
Squires, N. K. Preference for conjoint'schedules of primary
crimination learning. In D. P. Hendry (Ed.), Condi¬
reinforcement and brief-stimulus presentation. Unpub¬ tioned reinforcement. Homewood, Ill.: Dorsey Press,
lished doctoral dissertation, University of California, 1969.
San Diego, 1972. Zimmerman, J. Meanwhile . . . back at the key: Mainte¬
Squires, N., 8c Fantino, E. A model for choice in simple nance of behavior by conditioned reinforcement and
concurrent and concurrent-chains schedules. Journal of response-independent primary reinforcement. In D. P.
the Experimental Analysis of Behavior, 1971, 15, 27-38. Hendry (Ed.), Conditioned reinforcement. Homewood,
Squires, N., Norborg, J., 8c Fantino, E. Second-order Ill.: Dorsey Press, 1969.
schedules: Discrimination of components. Journal of the Zimmerman, J., 8c Hanford, P. V. Sustaining behavior with
Experimental Analysis of Behavior, 1975, 24, 157-171. conditioned reinforcement as the only response-produced
Steiner, J. Observing responses and uncertainty reduction, consequence. Psychological Reports, 1966, 19, 391-401.
II: The effect of varying the probability of reinforce¬ Zimmerman, J., 8c Hanford, P. V. Differential effects on
ment. Quarterly Journal of Experimental Psychology, extinction of behaviors maintained by concurrent sched¬
1970, 22, 592-599. ules of primary and conditioned reinforcement. Psycho¬
Stubbs, A. D. Second-order schedules and the problem of nomic Science, 1968, 8, 103-104.
 12

Conditioned Suppression
and the Effects
of Classical Conditioning
on Operant Behavior

Perefc Blackman

INTRODUCTION This chapter considers the effects of procedures of

this nature in which events which signal the delivery
A defining feature of studies of classical condition¬ of a US such as food or a shock are presented inde¬
ing is that the delivery of stimuli and the relationship pendently of an animal s behavior. However, the dis¬
between them are controlled by the experimenter in¬ cussion is confined to experiments in which the effects
dependently of a subjects behavior (Black Sc Prokasy, are measured by the changes they produce in behavior
1972, p. xi), Payloy (1927) was, of course, the first sys¬ which is maintained by response-dependent reinforce¬
tematically to investigate the effects of such pro¬ ment. The dependent variable in such studies is pro¬
cedures on behavior. He presented one event (the vided by a comparison of the frequency with which
conditioned stimulus: CS) regularly m a fixed tem¬ operant responses are emitted during the Pavlovian
poral relationship with a second event (the uncondi¬ CS and in its absence. A typical experimental situa¬
tioned stimulus: US) which reliably elicited a re¬ tion is shown in diagramatic form in Figure 1. The
sponse (the unconditioned response: UR). He found top line shows the presentation of a continuous stim¬
that eventually the CS came to produce behavior (the ulus (e.g., a light) which is associated with a schedule
conditioned response: CR) which was similar to the of reinforcement. The fourth line shows the operant
UR. Pavlov’s measures of this conditioning of an responses (e.g., lever presses) which are emitted during
acquired reflex were simple but adequate. For exam¬ this discriminative stimulus, and the bottom line de¬
ple, when the UR was salivation caused by an irri¬ picts the delivery of resultant response-dependent rein¬
tant placed on the tongue, the CR was measured in forcers according to an intermittent schedule, such as
terms of the number of drops of saliva resulting from a variable-interval schedule. During the discriminative
the presentations of the CS. In this way conditioned stimulus, and therefore while the subject is emitting
responses were measured as they developed from a operant responses, a second stimulus, such as a tone, is
zero value on the first presentations of the CS to an presented: this is shown in the second line of Figure 1.
asymptotic value when the stimulus was repeatedly This second stimulus signals the delivery of a Pav¬
presented with the US. lovian US such as electric shock (third line). The sec-

340
Derek Blackman 341

Discriminative stimulus
review of the truly vast body of literature reporting
experiments in which Pavlovian procedures have been
superimposed on operant behavior. The discussion is
deliberately and strenuously selective, in the hope that
general principles and problems may emerge more
directly and more clearly.

THE ESTES-SKINNER PROCEDURE

AND THE MEASUREMENT OF
ITS EFFECTS
Fig. 1. Diagram representing typical experimental arrangements
in the studies discussed in this chapter.
In 1941 Estes and Skinner reported the results of a
study in which they exposed rats to an intermittent
ond and third lines therefore depict in this case a schedule of food reinforcement which would now be
typical Pavlovian delayed conditioning procedure, described as fixed-interval (FI) 4 min. When the rats’
which is superimposed on a situation in which op¬ lever-pressing behavior had stabilized, a tone was pre¬
erant behavior is maintained by occasional reinforce¬ sented for a period of 3 min (5 min in later condi¬
ment. The only behavioral measure in this experi¬ tions). As each period of tone ended, an unavoidable
ment is provided by the operant responses depicted m and inescapable shock was delivered to the rat§ through
Figure 1. This is the case in most of the experiments the grid floor. The delivery of both tone and shock
discussed in this chapter, although in some studies was programmed independently of the rats’ behavior,
additional measures are taken of autonomic activity and the temporal relation between them makes it
(respondent behaviors) which occur during the CS or possible to term the procedure classical conditioning t
after the US, such as changes in heart rate* the tone is thus a Payloyian CS and the shock is a US,
The experiments to be discussed differ from con¬ Unfortunately, the intensity of the shock was not re¬
ventional classical conditioning experiments, then, in ported in this early paper, but it must have been rela¬
two important ways. First, the behavioral measures are tively mild because Estes and Skinner mentioned that
provided by operant rather than by traditional re¬ it produced no noticeable disturbance in operant be¬
spondent behavior. Second, when the Pavlovian CS is havior when it was first delivered. However, as the
presented, the subject is at that time emitting a pat¬ repeated pairings of tone and shock continued, be¬
tern of behavior which is recognized (and controlled) havior during the tone became disrupted. The rate of
by the experimenter. These differences in emphasis responding during the tone fell until it was about
have not, however, prevented the collection of data one-third the rate during the same period “in control
of considerable importance for classical conditioning. experiments.” This is illustrated by cumulative rec¬
Moreover, such experiments have brought a number ords taken during the experiment, which show clearly
of theoretical issues into sharp focus. the decrease in operant response rate during the period
Clearly, there are many different interactions which of tone in comparison with rates before and after the
may be studied within the general procedure discussed tone. The general finding that food-reinforced operant
above. As Rescorla and Solomon (1967) have pointed behavior decreases in frequency during a preshock
out, the operant behavior may be maintained by a stimulus has since been widely replicated in many
schedule of positive or of negative reinforcement, and different experimental conditions. The effect is
the Pavlovian US may be either appetitive or aversive. sometimes called conditioned suppression (see detailed
These various interactions will be reviewed in this reviews by Davis, 1968, and Lyon, 1968). It is illus¬
chapter. However, most of the research conducted in trated by the segments of cumulative record shown
this area has studied the effects of an aversive US in Figure 2 (from Blackman, 1974) which show the
(specifically, shock) with the Pavlovian procedure in¬ operant behavior of a rat exposed to a variable-
troduced when operant behavior is maintained by interval (VI) 30-sec schedule of food reinforcement,
schedules of food or water reinforcement. It is here the delivery of which is shown by brief hatchmarks in
that the most developed theoretical implications are the usual way. In the middle of successive 7-min peri¬
to be found, and it is this area of research and discus¬ ods, an auditory stimulus was presented for 1 min,
sion to which we therefore turn first. No attempt will during the whole of which time the pen on the cumu¬
be made in this chapter to provide a comprehensive lative recorder was deflected downward, although it
 342 CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

could still be stepped across the paper by an operant rates during the CS. This measurement is achieved by
response. means of a suppression ratio or inflection ratio, but
At the end of each of these 1-min periods, a very unfortunately there has been no general agreement
brief shock was delivered. The record at the top of about the best way to make this calculation. One sim¬
Figure 2 shows that the responding maintained by the ple method is to present the rate during the CS as a
VI schedule was completely eliminated during the direct proportion of a control rate of responding (e.g.,
preshock signal, for the record is horizontal during Stein, Sidman, & Brady, 1958). In this case, complete
the 1-min deflection. Immediately after the shock, the conditioned suppression is depicted by a ratio of 0; if
animal resumed the steady rate of operant responding there is no effect on response rate attributable to the
maintained by the schedule. The lower segment of the CS, the suppression ratio is 1.0, and if the response
record shows that during another two 1-min periods rate increases during the CS, the ratio is greater than
of the noise the rat did not stop responding com¬ 1.0. However, another widely used ratio (e.g., Kamin,
pletely although the response rate was lower during 1965; Rescorla, 1968) expresses the rate during the
the noise than in its absence. Since the VI schedule CS as a proportion of the sum of control and CS re¬
remained in operation during these periods^ responses sponse xates. This results in a figure of 0 for complete
might 5till occasionally be followed by a rcinforccr suppression, .5 for no suppression, and greater than .5
(shown by a brie! upward hatclimark on the record). for acceleration of responding during a CS. The use
Figure % therefor depicts complete conditioned sup¬ of these different calculations, and of others, is a po¬
pression (top) and partial conditioned suppression tential confusion in this area of research (see Lyon,
(bottom) of operant behavior during a CS (noise) 1968). Moreover, as will be discussed later, the use of
terminated by a US (shock). any relative rate measure such as these is not without
In this area of research, it has been generally ac¬ its problems (Lea 8c Morgan, 1972).
cepted that an appropriate measurement is a com¬
parison between response rates during a CS and in its
absence rather than the absolute reduction of response INVESTIGATIONS OP CLASSICAL
CONDITIONING PARAMETERS

Many investigators have found that the amount of

conditioned suppression (however the suppression
ratio is calculated) is a function of conventional
parameters in classical conditioning (Davis, 1968). To
mention two simple examples, Annau and Kamin
(1961) showed that the amount of conditioned sup¬
pression in rats is an increasing monotonic function
of the intensity of the shock US, and Kamin and
Schaub (196?) have shown a similar effect of the in¬
tensity of the CS. Rescorla and Solomon (1967) have
suggested that "it might very well turn out that in¬
strumental responding is as sensitive, or perhaps even
more sensitive, a measure of the effects of Pavlovian
conditioning procedures than are the traditionally
measured conditioned visceral or motor reflexes them¬
selves.” Although Rescorla and Solomon considered
this possibility “somewhat ironic,” it is certainly true
that the conditioned suppression paradigm has been
widely and successfully used in order to develop our
understanding of classical procedures in general. Re¬
viewing a great deal of such work carried out in his
own laboratory, Kamin (1965) has claimed that “we
are measuring respondent behavior indirectly, with a
Fig. 2. Cumulative records illustrating the effects on operant
surprising quantitative sensitivity.” It is not possible
behavior of a stimulus ending with an unavoidable shock (con¬
ditioned suppression). The response pen is offset during the to review here the large body of literature in this tra¬
preshock stimulus; hatchmarks denote reinforcement. dition. However, as one example of current work,
Derek Blackman 343

Rescorla’s investigations of the necessary and sufficient

conditions for a stimulus to become a classical condi¬
tioned stimulus may be cited.
Rescorla’s important work grew out of his theo¬
retical discussion of the appropriate control pro¬
cedures for Pavlovian conditioning (Rescorla, 1967).
In this paper, he suggested that conventional pro¬
cedures did not allow for the measurement of appro¬
priate base lines against which to assess accurately the
strengths of a conditioned response developed by
Pavlovian procedures. For example, in some experi¬
ments the stimulus which becomes the conditioned
Fig. 3. Median suppression ratios for groups of rats as a func¬
stimulus has first been presented in such a way that it tion of the probability of shock in the presence and absence of
is explicitly never paired with the unconditioned the CS. A ratio of 0.00 denotes complete suppression, and .50
shows that the CS has no effect. (From Mackintosh, 1974, after
stimulus. The temporal contingency between the CS
Rescorla, 1968.)
and the US is then introduced in the conditioning
phase of the experiment. Rescorla argued that tradi¬
tional control procedures such as this fail to provide of tone was specified for different groups as .40, .20,
an unconfounded measure of the effects of the experi¬ .10, or 0 (i.e., no shocks were delivered). Within these
mental contingency between the stimuli. He suggested groups, subgroups of animals were exposed to vary¬
that the only way in which this could be achieved was ing probabilities of shock m periods when the tone
by means of what he termed a “truly random” control was absent. Rescorla then measured the effects of the
procedure. With this control procedure, the stimulus tone after these training conditions by superimposing
which is to become the CS and the US are first pre¬ the tone (without shocks) on food-reinforced operant
sented at the frequencies to be used in the condition¬ behavior and measuring its disruptive effects. The de¬
ing phase, but entirely independently; it is therefore pendent variable was expressed as a median suppres¬
possible for them to be occasionally presented together sion ratio, this being calculated by the formula which
by chance. Hence occasional contiguity between the yields a ratio of 0 for complete conditioned suppres¬
two stimuli may occur, but no reliable contingency sion and .5 for no disruptive effect. Figure 3 shows the
exists between them at this stage of the experiment. effects of the tone on the first day on which it was
Rescorla’s empirical work has subsequently devel¬ superimposed on the operant behavior of the various
oped these ideas. He has shown, for example, that groups (four presentations for each rat). If the prob¬
mere contiguity between two stimuli is in fact not ability of a shock was the same both in the presence
sufficient for Pavlovian conditioning to develop. If a and the absence of a tone (Rescorla’s truly random
stimulus is to become a CS and thus elicit a CR, it procedure), the tone had no suppressive effect. Thus
now seems that it must, in simple terms, provide a suppression ratios are consistently at a value of ap¬
subject with “information” about the occurrence of proximately .5 whether the probability of a shock in
the US. In his truly random control, the occurrence of the presence and in the absence of the tone was .40,
a CS provides no information about the occurrence of .20, or .10. On the other hand, if shocks had initially
a US, for the probability of a US is the same both occurred only during a period of tone (so that the
when a CS is presented and when it is not. On the probability of shock in the absence of the tone was
other hand, in traditional delayed conditioning ex¬ 0), the tone caused relatively severe suppression of
periments, the occurrence of a CS provides informa¬ operant behavior. The amount of this suppression in¬
tion that a US is about to be presented; moreover, no creases with greater probabilities of shock during the
US is presented without a preceding CS. One of tone, for when the probability of shock in the tone
Rescorla’s elegant experiments (1968) has used the was .10, .20, or .40, suppression ratios were approx¬
conditioned suppression paradigm to investigate the imately .20, .10, and 0 respectively. Figure 3 shows,
effects of these and of various intermediate relation¬ then, that the tone suppressed operant behavior to
ships between a CS and a US, and its results are the extent that it had been differentially associated
summarized in Figure 3. The CS was a 2-min presen¬ with the occurrence of a shock, that is, in proportion
tation of a tone, and the US was shock. Small groups to the difference between the probability of shock
of rats were exposed to various relationships between during the tone and its probability in the absence of
these stimuli. The probability of a shock in a period the tone. The degree of classical conditioning in-
344 CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

creased as the probability of the US during the CS groups is controlled by an identical procedure, we
became greater than the probability of the US in the have seen that it is indeed possible to further the
absence of the CS, and not simply as the former value analysis of the necessary and sufficient conditions for
increased. classical conditioning to occur and to measure its
Rescorla has developed his account of the neces¬ strength as a function of specified independent vari¬
sary and sufficient conditions for classical conditioning ables. However, the amount of conditioned suppres¬
far beyond the basic idea indicated above (see, for ex¬ sion during a preshock stimulus is not determined
ample, Rescorla, 1969; Rescorla & Wagner, 1972), but solely by such Pavlovian variables as the parameters
it is not necessary to provide a more complete de¬ of the conditioned and unconditioned stimuli and the
scription here. Rescorla’s important work on classical contingencies between them. In this section other
conditioning is based closely on his use of the con¬ important variables are discussed which are related to
ditioned suppression procedure. This procedure makes the maintenance of the operant response on which
it possible to measure behavior (in the form of op¬ the classical conditioning procedure is imposed.
erant response rates) throughout both non-CS and CS Anything which affects the nature or strength of
periods, whatever the probability of the US in cither operant behavior may also affect the amount of dis¬
of them. The demands for a sensitive and reliable ruption produced by a specified conditioned stimulus
dspsndsnt variable to show the effects of the quantita¬ when it is superimposed on that behavior. This is
tive differences in Rescorla’s independent variables is perhaps not surprising, since conditioned suppression
met by the indirect measurement of classical condi¬ can be regarded as the result of pitting classical against
tioning through the frequency of operant responses. operant conditioning effects. An important study
Our understanding of classical conditioning pro¬ which emphasizes this was reported by Stein, Sidman,
cedures has therefore been significantly advanced by and Brady (1958), who investigated the effects of vary¬
studies of their effects on operant behavior. The ex¬ ing the duration of a preshock stimulus through a
amples discussed here are representative of a very range of 30 sec to 50 min and also examined the
large body of research which has been carried out effects of varying the interval between successive stim¬
within this general strategy, and they illustrate its ulus presentations. Considerable variation was found
contemporary impact. The procedure has consistently in the amount of conditioned suppression produced
proved to be robust, reliable, and sensitive, and so by different combinations of these variables. However,
any inherent ironies may surely be readily tolerated neither of them proved to be a critical determinant in
by researchers in the field of classical conditioning. itself: instead, there was a high negative correlation
between the amount of suppression and the relative
duration of the preshock stimulus, i.e., the proportion
INVESTIGATIONS OF OPERANT of time in any session during which the CS was pres¬
CONDITIONING PARAMETERS ent. In considering ways in which this somewhat at>
stract temporal value might control the amount of
In the research discussed in the previous section, conditioned suppression, Stein et al. noted that the
the emphasis was placed on the way in which the behavior of the rats in their study was suppressed only
processes of classical conditioning may be investigated to the extent that they did not thereby miss more
by means of conditioned suppression procedures. The than 10% of the total number of reinforcements which
operant behavior which provides the only dependent could be set up by the VI schedule. So, when a pre¬
variable m these studies is usually maintained by a shock stimulus (of any duration) was present for a
simple schedule of intermittent reinforcement. Typ¬ relatively high proportion of the experimental ses¬
ically, a variable-interval schedule is used for this sion, complete suppression of operant behavior would
purpose, for, of course, such schedules maintain op¬ have produced a substantial reduction in the number
erant behavior over considerable periods of time, and of reinforcements obtained. In these situations, only
the generally moderate and consistent rates of re¬ partial suppression of behavior was observed during
sponding which they generate make it easy for the the preshock stimulus. However, if a relatively short
experimenter to measure any suppressive effect of a preshock stimulus was presented only rarely, the sub¬
CS (as in Figure 2, for example). Also, of course, par¬ jects could “afford” to suppress completely and yet
tial suppression of behavior maintained by a variable- still obtain at least 90% of the total possible rein¬
interval schedule may have only minimal effects on forcers, and indeed complete suppression was observed
the frequency of reinforcement obtained. If the op¬ in such situations.
erant behavior of the subjects in various experimental Carlton and Didamo (1960) reported a study based
Derek Blackman 345

on that of Stein et al. (1958), but they varied the ing are more suppressed during a preshock stimulus
length of their experimental sessions so that the num¬ than are lower rates which obtain the same frequency
ber of reinforcers actually obtained by subjects was of reinforcement. This conclusion was prompted by
constant throughout the various conditions of the suppression ratios, but since these are relative mea¬
experiment. Again it was found that the amount of sures of the responding during a preshock stimulus, it
conditioned suppression decreased as the relative seemed possible that the differences in suppression
duration of the preshock stimulus increased. Carlton ratio might have been merely artifacts of the different
and Didamo suggested that this reduction in the sup¬ base line response rates. For example, if the absolute
pressive effect was due to “changes in response output number of responses emitted during a preshock stim¬
which minimise the decline in reinforcement rate.” ulus was constant whatever the base line response
This suggestion implies that behavior which is rein¬ rates/ then suppression ratios would inevitably sug¬
forced only occasionally will be less resistant to the gest less suppression in the condition in which the
suppressive effects of a preshock stimulus, for a “de¬ preshock stimulus was superimposed on the lower
cline in reinforcement rate” resulting from suppres¬ rate. This was not the case, however. For example, in
sion during a fixed preshock stimulus might not be many conditions the absolute response rates during a
so readily detected. An experiment by Lyon (1964) preshock stimulus were higher when it was superim¬
supports this hypothesis. Using pigeons as subjects, posed on the lower control rate than when it was
Lyon superimposed a preshock stimulus on both com¬ superimposed on the higher control rate (Blackman,
ponents of a multiple schedule in which two fre¬ 1968b, Table 6). So, for example, one subject (rat 1)
quencies of reinforcement were programmed (mult emitted 89 responses per min in the control conditions
VI 1-min VI 4-min). It was found that the pigeons’ of one component (A) of a multiple schedule and 36
behavior was more suppressed during the preshock responses per min in the control conditions of the
stimulus when it was superimposed on the lower fre¬ other component (B). The schedules in both compo¬
quency of reinforcement than when it occurred dur¬ nents provided 2 reinforcements per min on average,
ing the component with the higher reinforcement but different response-pacing requirements were in
frequency. Lyon therefore suggested that behavior operation in each component. During a 1-min period
which is reinforced relatively frequently is more re¬ of tone which ended with a ,5-mA shock the suppres¬
sistant to disruption by a conditioned suppression sion ratios (response rate during CS divided by re¬
procedure than is behavior which is reinforced only sponse rate in absence of CS) for this rat were .04 on
rarely. This has been corroborated by Blackman component A and ,60 on component B. These ratios
(1968b), who used response-pacing procedures (Ferster reflected a mean rate of 5 responses per min in the
Sc Skinner, 1957) which controlled responding at ap¬ preshock stimulus when it was superimposed on com¬
proximately equal rates in two components of a multi¬ ponent A (high control rate) and 22 responses per min
ple schedule, but in which the frequencies of rein¬ during the same preshock stimulus when it was super¬
forcement were controlled by two different VI sched¬ imposed on component B (lower control rate). Hence
ules. This made it possible to identify the effects of the relative differences in control response rates were
reinforcement frequency on conditioned suppression reversed during the preshock stimulus, which supports
more unequivocally than did Lyon’s (1964) study, for the view based on suppression ratios that lower rates
in the latter the different frequencies of reinforcement of responding are more resistant to disruption by a
produced different control response rates, a possible preshock stimulus than are higher rates.
confounding effect. The amount of conditioned suppression depends in
If the conditioned suppression phenomenon is con¬ part, then, on the frequency of reinforcement and on
ceptualized as the outcome of a competition between the rate of operant responding. The effects of classical
a classically conditioned response with a fixed strength conditioning procedures may therefore be dependent
and the tendency to emit operant responses which are on the schedule of reinforcement on which they are
occasionally reinforced, the relative resistance of be¬ superimposed. The importance of schedules in deter¬
haviors which result in frequent reinforcement may mining the behavioral effects of other independent
not seem surprising. Less predictable, however, is the variables requires no emphasis here, for it has been
finding that, when reinforcement frequency is con¬ demonstrated in many other contexts, such as the
trolled, rates of responding are differentially suscepti¬ effects of drugs (e.g., Kelleher 8c Morse, 1968) and the
ble to conditioned suppression during a preshock effects of unsignaled aversive stimuli (e.g., McKearney,
stimulus. Blackman has shown in a series of experi¬ 1972).
ments (1966, 1967, 1968b) that high rates of respond¬ The effects of conditioned suppression procedures
346 CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

have now been investigated with all the principal stimulus was presented. If this occurred just after
schedules of reinforcement. Lyon and Felton (1966a) reinforcement, Lyon observed complete suppression
studied pigeons’ behavior maintained by variable-ratio during the stimulus. If, when the stimulus was intro¬
(VR) schedules. They had expected that as the mean duced, the bird had emitted more than 60 responses
ratio requirement was increased from 50 to 100 to in the required sequence of 150, it continued respond¬
200, the subjects would show more conditioned sup¬ ing until the next reinforcement and then suppressed
pression, because the overall frequency of reinforce¬ completely until the end of the stimulus. If the stim¬
ment would fall. In fact, however, the results of their ulus began when the bird was between 20 and 60
experiment were inconclusive, for they found that the responses into the required sequence, immediate sup¬
behavior maintained by all the VR schedules was pression was sometimes seen and on other occasions
quite insensitive to the conditioned suppression pro¬ the animal continued to respond until the next rein¬
cedure. This may be because reinforcements were forcement was obtained. The initial resistance to sup¬
contingent upon the continued and sustained emission pression when the bird had completed more than 60
of responses with this schedule in a way that is not the responses may perhaps be taken as support for Lyon
case with variable-interval schedules. Fantino (1973) and Felton’s (1966a) report that variable-ratio be¬
has pointed out that partial suppression during a pre¬ havior in pigeons is resistant to suppression, for with
shock stimulus superimposed on a variable-interval the variable schedule the imminence of the next rein¬
schedule can have, within limits, virtually no effect on forcement may always be as close as in those condi¬
the rate of reinforcement. This is clearly not the case tions of Lyon’s fixed-ratio experiment when the
with ratio schedules. Fantino therefore regards the behavior was found to be resistant to suppression. Simi¬
results of Felton and Lyon as being “readily inter¬ larly, with variable-ratio schedules, the birds do not
pretable. ” However, Blackman (1966) reported an show postreinforcement pauses as they do on FR 150.
experiment using rats as subjects in which VR 100 Another study by Lyon and Felton (1966b) found that
behavior was far from resistant to conditioned sup¬ birds exposed to FR 25 (and to a lesser extent FR 50)
pression: all three rats showed virtually complete con¬ did begin to respond again after a reinforcement had
ditioned suppression when the shocks (.5 mA, .2 sec) been obtained during a preshock stimulus. The birds
were introduced. Another three animals were “yoked" therefore often obtained several reinforcements dur¬
to these first three, i.e.7 reinforcements were made ing the stimulus, and so to that extent could also be
available to them by the delivery of reinforcements described as resistant to conditioned suppression.
to the VR animals, so that they were in effect on a VI The crucial relationship between the onset of a
schedule with a mean interreinforcement interval preshock stimulus and the imminence of reinforcement
identical to that of the ratio animals. These VI an¬ has also been suggested with fixed-interval schedules.
imals did not show such severe conditioned suppres¬ For example, Blackman (1968a) discussed the behavior
sion, which emphasizes the susceptibility of the ratio of one subject (rat 1) which was exposed to a FI 20-
animals to conditioned suppression in this experi¬ sec schedule. A 1-min preshock stimulus was presented
ment, in contrast to the resistance shown by Lyon in such a way that the next reinforcement became
and Felton?s pigeons. The reasons for these incon¬ available 5 sec after its onset (and, therefore 25 and
sistencies remain obscure; one hesitates to invoke 45 sec after the onset). When the shock which ended
species differences, especially as pigeons and rats ap¬ this stimulus was of .5 mA for .2 sec, the rat responded
pear to be used interchangeably in other studies of long enough into the stimulus to obtain the first two
conditioned suppression. Perhaps Fantino’s (1973) of these reinforcements (i.e., for 25 sec), and then sup¬
suggestion that “it would have been interesting [in the pressed completely until the end of the stimulus.
Lyon and Felton study] to note whether conditioned When the shock was increased to 1.6 mA, responding
suppression would have been obtained with high continued only long enough for the first reinforce¬
shock intensities” is useful, for it is possible that the ment to be obtained (i.e., for 5 sec). With a shock
shocks used by Lyon and Felton would not have sup¬ setting of 3.0 mA for .5 sec, all responding was sup¬
pressed other patterns of behavior in these pigeons. pressed immediately the preshock stimulus was pre¬
The effects of conditioned suppression procedures sented, even though a reinforcement would become
have also been investigated with fixed-ratio and fixed- available only 5 sec later. A study by Lyon and Millar
interval schedules. Lyon (1964) found that the effects (1969) also suggests that the imminence of reinforce¬
of a preshock stimulus superimposed on FR 150 be¬ ment in an FI schedule may attenuate conditioned
havior in pigeons depended on how far the bird was suppression. In the interreinforcement intervals of an
advanced in its sequence of behavior when the FI 2-min schedule, a preshock stimulus of 30 sec was
Derek Blackman 347

presented 30, 60, or 90 sec after the preceding rein¬ absence and at a higher frequency than in control
forcement. It was found that there was marked sup¬ conditions.
pression of responding during the stimulus when it In an experiment which employed a two-lever situ¬
was presented early in the interval, but no suppression ation (Blackman, 1970a), rats were exposed to a sched¬
when the stimulus occurred late in the interval. ule in which a response on lever B was followed by
Preshock stimuli have also been superimposed on reinforcement if a preceding response on lever A had
behavior maintained by a schedule which differen¬ been made at least 5, 10, or 15 sec before. When a
tially reinforces a very low rate of responding (DRL). preshock stimulus was superimposed on the behavior
In some circumstances it has been shown that re¬ generated by this schedule, it was found that the fre¬
sponding on this schedule increases in frequency quency of timing attempts, i.e., of A-to-B sequences,
during a preshock stimulus. For example, Blackman decreased during the stimulus. This had also been
(1968a) exposed rats to a multiple schedule, of which found in a similar experiment by Migler and Brady
one component was a DRL 15-sec schedule and the (1964). When the delay required was 5 sec, there was
other generated higher response rates. In all condi¬ no change in the distribution of A-to-B times during
tions of the experiment, this second pattern of be¬ the preshock stimulus. Thus although the frequency
havior was suppressed during a preshock stimulus. of timing attempts decreased, their accuracy was not
However, when the stimulus ended with a relatively impaired, again replicating a finding by Migler and
mild shock (.2 mA for .5 or 1.0 sec), the DRL behavior Brady (1964) . However, when the required A-to-B
increased in frequency during its presentation, al¬ delay was 10 or 15 sec, the distribution of A-to-B times
though with higher intensities of shock the more changed during the preshock stimulus, there being
usual suppressive effect was found. The acceleration more shorter intervals. Also noticeable (especially with
of DRL responding during a stimulus which ends the 15-sec-delay requirement) was an increased pro¬
with a mild shock is illustrated in Figure 4, which portion of inappropriate B responses, i.e., B responses
shows the cumulative records of three rats exposed which were made without a preceding A response to
throughout each experimental session to a DRL 15- initiate a timing attempt. The disruption of timing
sec schedule. A tentative suggestion has been made efficiency and the increase in appropriate B responses
(Blackman, 1968a) that the acceleration effect on the led m one case to an overall acceleration of B re¬
DRL responding was attributable to a disruption of sponses in comparison with control conditions, in
the collateral behavior which appeared to mediate the spite of the decreased frequency of appropriate timing
lever-pressing behavior. These stereotyped sequences attempts. This accelerative effect may be analogous to
of behavior were not formally measured in the ex¬ the acceleration reported with a single-lever DRL
periment, but they characterized the DRL behavior in schedule. A generally similar effect was reported by
control conditions. During the preshock stimulus, Blackman and Scruton (1973a), who superimposed a
however, the collateral behaviors seemed to be quickly preshock stimulus on a two-lever counting schedule.
disrupted, and lever pressing then occurred in their In this case, rats were required to make at least a
specified number of successive responses on lever A
before switching to lever B to produce reinforcement,
and there was a shift to shorter sequences of responses
on lever A during the preshock stimulus.
Hearst (1965) has reported a deterioration in dis¬
criminative control as a result of the Estes-Skinner
procedure. He superimposed a preshock stimulus on
periods of intermittent reinforcement, and found that
the operant behavior was suppressed in the usual way
during this stimulus. However, the rats were also
exposed to periods when no reinforcement was possi¬
ble (extinction). The preshock stimulus never occurred
during periods of extinction, but the deterioration in
discriminative control reported by Hearst took the
Fig. 4. The accelerative effects of a preshock stimulus on re¬ form of an increase in responding during these peri¬
sponding maintained by a schedule which differentially rein¬ ods. Hearst related this finding to the Pavlovian con¬
forces a low rate of lever pressing (DRL 15 sec). The response
pen is offset during the preshock stimulus; hatchmarks denote
cept of disinhibition. However, it should perhaps be
reinforcement. (Unpublished data of Sanger & Blackman.) mentioned that two subsequent experiments have
348 CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

failed to replicate his findings (Blackman 8c Scruton, man, 1966, 1967, 1968b). Such a finding offers the
1973b; Weiss, 1968). In the former case, there was no investigator two very different interpretations (Black¬
increase in responding during periods of extinction, man, 1972):
even when the preshock stimulus was subsequently 1. A suppression ratio may always reflect ac¬
presented during extinction periods as well as during curately the strength of a classically conditioned re¬
the periods of intermittent reinforcement. The rea¬ sponse elicited by the conditioned stimulus. In other
sons for these potentially important inconsistencies words, the more severe the disruption of operant be¬
between experiments are not clear as yet. havior (as expressed by a suppression ratio), the greater
It is clear that the amount of suppression produced is the strength of the CR. If this is true, different
by a given preshock stimulus depends crucially on the strengths of CR are developed by a uniform pro¬
nature of the schedule which maintains the operant cedure when it is superimposed on different operant
behavior on which it is superimposed. Indeed, in some response rates. Why this should be so remains un¬
circumstances (albeit limited to fairly stringent timing explained, although it has often been suggested in a
schedules) a preshock stimulus will lead to an in¬ more general context that the effects of any indepen¬
crease in the rate of food-reinforced operant respond dent variable depend on the nature of ongoing be¬
mg. Certain familiar schedules of reinforcement, par¬ havior as controlled by schedules of reinforcement
ticularly variable-interval, provide a base line of (e.g., Dews, 1963),
behavior against which the effects of classical condi¬ 2. A standard classical conditioning procedure may
tioning parameters can be readily assessed. However, always result in a CR of uniform strength. Different
it is also the case that, when classical conditioning suppression ratios describing the effects of a preshock
procedures arc held constant, substantial differences stimulus on different patterns of operant behavior
in the effects of a conditioned stimulus may emerge may then result from the fact that this uniform con¬
as a function of the precise schedules of reinforcement ditioned response interacts differently with these
which maintain the operant behavior, differences not patterns of behavior. According to such a view, the sup¬
only of degree but even on occasion of direction. pression ratio is therefore not an uncontaminated
reflection of the strength of a CR.
It is difficult to decide between these alternatives,
MEASUREMENT OF CONDITIONED and initial preferences may reflect whether one’s basic
SUPPRESSION allegiance is to the study of classical conditioning or
of operant conditioning processes. Given this problem
We have already noted that most workers have of interpreting suppression ratios, however, it would
measured conditioned suppression by comparing the seem only prudent to support inferences based on
response rate during the preshock stimulus with the these ratios by absolute data whenever possible. Thus
control rate of responding, i.e., in the absence of in the case discussed in the previous section, suppres¬
the preshock stimulus. We may continue to regard the sion ratios suggested that high response rates were
different formulae which have been used to make such more disrupted by a preshock stimulus than were
a comparison as no more than tedious and potentially lower rates of responding. Corroborative evidence was
confusing, However, Hoffman (1969) and Millenson provided by the absolute data, which showed that the
and de Villiers (1972) have suggested that the use of stimulus was accompanied by fewer responses (in abso¬
relative measures has not been adequately justified. A lute terms) when superimposed on normally high re¬
relative measure may make it easy to compare condi¬ sponse rates than when superimposed on normally
tioned suppression of different patterns of operant be¬ lower rates. These two forms of data suggested that a
havior, but this entails an arbitrary assumption: differential effect is attributable to the patterns of
“Measurement by relative suppression presupposes operant behavior.
that under constant experimental conditions the warn¬ In many cases of interactions between a classical
ing signal will produce the same relative decrement conditioning procedure and different patterns of op¬
independent of the rate of the responding at the mo¬ erant behavior, the dilemma outlined above can be
ment of the warning signal presentation” (Hoffman, regarded as unimportant, for research interest may
1969, p. 68) . There is now ample evidence that this focus principally on the details of schedule control
assumption is false, as shown in the previous section. and its disruption. The matter can be of great prac¬
Suppression ratios obtained in experiments in which tical significance, however, as may be illustrated by an
the same preshock stimulus is superimposed on differ¬ example. Figure 5 shows data obtained from one rat
ent rates of operant behavior are not identical (Black¬ which was exposed to a FR 10 schedule of food rein-
Derek Blackman 349

appears: does the orderly effect of increasing dosage

on suppression ratios reflect (1) progressive reduc¬
tions in the strength of the underlying conditioned
response, or (2) merely contaminations produced by a
changing control base line, with the strength of the
conditioned response remaining unchanged? Although
it may be difficult to answer such a question, it is
surely clear that informed interpretation of the drug’s
effects demands absolute as well as relative data. Nev¬
ertheless, experimental reports continue to emphasize
suppression ratios and frequently fail to supplement
these by absolute rates in the presence and absence of
a conditioned stimulus.
A further complication arises whenever any simple
measure of conditioned suppression is reported,
whether this be in the form of relative or of absolute
* i_I_I_I
response rates during a preshock stimulus. We have
0 2 4 8 16
noted previously that a stimulus which ends with a
shock is a conditioned stimulus within the Pavlovian
CHLORDIAZEPOXIDE mg/kg delayed conditioning paradigm. With traditional Pav¬
lovian procedures, some form of temporal discrimina¬
Fig. 5. The effects of various dosages of chlordiazepoxide on
conditioned suppression of a rat’s responding on a FR 10 sched¬ tion usually develops within such a CS, the conditioned
ule. Complete suppression =0; no effect =1.0. (Unpublished
data of D. Sanger.)

forcement. When this behavior had stabilized, a con¬

ventional conditioned suppression procedure was in¬
troduced, the details of which are not important here.
The rat was then tested after various injections of the
minor tranquilizer chlordiazepoxide. Figure 5 plots
the suppression ratios which were obtained with the
various doses, these being calculated by Stein, Sidman,
and Brady’s (1958) formula: SR = CS rate/Control
rate. It seems clear that increasing doses of the drug
have an increasingly attenuating effect on conditioned
suppression. However, with a relative measure such as
this, these changes may be produced either by changes
in response rate during the CS or by changes in its
absence. Figure 6 shows the absolute response rates in
these two periods after each dosage of the drug. The
open points display the drug’s effects on response
rates during the preshock stimulus (FR 10/CS), and
the closed points show response rates in its absence.
Clearly, the effects of the drug are by no means as
simple as at first they may have seemed on the basis ■ ■ ■ 1
of the suppression ratios in Figure 5. In fact, the ra¬ 0 2 4 8 16
tios change as a result of changes in response rates
CHLORDIAZEPOXIDE (MG/KG)
both within the CS and in control conditions. With
lower dosages, increases occur in both response rates.
Fig. 6. The effects of chlordiazepoxide on conditioned suppres¬
The differences between the effects of 8 mg/kg and sion. The data are those which are expressed in the form of
16 mg/kg, however, can be seen to be almost entirely suppression ratios in Figure 7, but are here plotted in absolute
terms, i.e., as response rates during the preshock stimulus (FR
due to differences in control rates, with little change
10/CS) and rates in its absence (FR 10). (Unpublished data of
in CS rates. The dilemma outlined earlier therefore D. Sanger.)
 350
CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

response eventually being characteristically elicited this experiment: certainly, there was no evidence of
only toward the end of the CS (i.e., just before the an orderly temporal patterning during the stimulus.
US). A similar temporal discrimination has sometimes Of course, the delivery of shocks at unpredictable
been reported with conditioned suppression. For ex¬ moments during a stimulus is strongly reminiscent of
ample, Hendry and Van Toller (1965) reported that Rescorla’s procedures reviewed earlier, although in
initial sustained suppression throughout a preshock these studies shocks and another stimulus were as¬
stimulus was superseded by a pattern in which the sociated only before operant conditioning occurred,
suppression occurred only in the second half of the and only the CS was subsequently superimposed on
stimulus. In some cases, response rates during the first operant behavior. Nevertheless, a complication even
half of the stimulus in fact increased in comparison in presenting shocks at random moments during a
with control conditions. However, such temporal pat¬ stimulus emerges from one of Rescorla’s studies
terning has not been reported consistently in the (1968). He found that response rates were not con¬
literature, even in studies in which a preshock stim¬ sistently suppressed even when the conditioned stim¬
ulus has been superimposed on operant behavior re¬ ulus was superimposed on variable interval behavior.
peatedly. For example, Stein et al. (1958) remark that There was greater suppression during the initial parts
the type of response patterning within a fixed pre¬ of the stimulus, with less in the later parts (i.e., the
shock stimulus was "'not necessarily invariant from opposite of Hendry 8c Van Toller’s 1965 results using
one stimulus presentation to the next,” This is a a conventional preshock stimulus). Rescorla suggested
further example of an inconsistency in the literature that this effect may reflect the fact that the onset of a
which has not yet been resolved. The development of CS is more discriminable than its continued presence.
temporal patterning may depend on a number of A second possibility mentioned by Rescorla, however,
variables, such as the relative duration of the preshock brings us full circle, for he suggests that his differen¬
stimulus, the intensity of the shock, the number of tial suppression within a CS may be
conditioning trials, and the nature of the schedule
which maintains the operant behavior. an artifact of the measuring technique. A VI
Unrecorded temporal patterning within a preshock schedule of reinforcement is such that the longer
stimulus could have a considerable contaminating [a subject] has refrained from pressing, the
effect on reported results. Millenson and Leslie (1974), higher the probability that its next press will
for example, argue that a drug which appears to be reinforced. Thus the longer [the subject]
alleviate or enhance conditioned suppression might suppresses, the more “pressure” the base-line
do so principally by affecting the nature of any such operant schedule places on it to respond. (Res¬
corla, 1968, p. 5)
temporal discrimination, There would appear to be
two ways of counteracting this possible contamina¬
Rescorla therefore goes on to suggest that the strength
tion, The first is to vary the duration of the preshock
of the classically conditioned response may be con¬
stimulus from trial to trial, although still ending each
stant throughout the CS: only the tendency to emit an
stimulus presentation with a shock. Millenson and
operant response changes. This suggestion is clearly
Hendry (1967) found that such a procedure did result
based on the second interpretation of conditioned sup¬
m consistently suppressed responding during the stim¬
pression discussed toward the beginning of this section.
ulus. An alternative expedient is to use a conditioned
The measurement of conditioned suppression is
stimulus of fixed duration in the usual way; but to
fraught with difficulties, some of which pose interest¬
deliver shocks at unpredictable moments throughout
ing dilemmas. There is scope for ambiguity even
the Stimulus and not merely as the stimulus ends.
when responding is totally suppressed during a pre¬
This procedure has been used occasionally. For exam¬
shock stimulus. For example, Lyon (1965) claimed
ple, Azrin (1956) included it (termed by him “VI
that a change in base line response rate is not suffi¬
uncorrelated shock”), and his cumulative records re¬
cient to change the amount of conditioned suppres¬
veal consistent suppression throughout the stimulus
sion. However, in the first phase of his experiment,
associated with shock. More recently, Bond, Black¬
Lyon used a procedure which resulted in complete
man, and Scruton (1973, Experiment 2) used the pro¬
conditioned suppression, and he then found that com¬
cedure. In this experiment the response rates were not
plete suppression still occurred when the base line
entirely consistent throughout the stimulus associ¬
response rate was increased. Subsequent research (e.g.,
ated with shock, but the inconsistencies may have re¬
Blackman, 1968b) has shown that increases in base
sulted from the suppressive effects of the procedure on
line response rate lead to an increase in the amount
adjunctive licking which had reliably developed in
of conditioned suppression. Since this effect could not
Derek Blackman 351

be shown in Lyon’s experiment because of a “ceiling three accounts separate, and any attempt to do so
effect,” he was therefore led to a general conclusion results in some arbitrary decisions. The discussion
which was false. Problems of measurement in studies continues to be confined to the effects of a preshock
of conditioned suppression must therefore be borne stimulus on food-reinforced behavior. The extension
in mind constantly in this area of research. Sometimes of the theories to other examples of classical condi¬
a simple suppression ratio in one sustained experi¬ tioning effects on operant base lines will be consid¬
mental situation may not be the most useful measure. ered subsequently.
For example, Fleshier and Ffoffman (1961) investi¬
gated the stimulus generalization of conditioned sup¬
The Interference Hypothesis
pression with pigeons. First, complete conditioned
suppression was obtained during a 1000-Hz tone which The possibility that other behaviors interfere with
preceded a shock. Then tones of different frequencies ongoing operant behavior to produce conditioned
were presented in a generalization test in extinction suppression has been suggested in terms both of com¬
conditions (i.e., no tone ended with shock). At first, the peting respondent behavior and of competing op¬
stimulus generalization gradient, which was measured erants, although the former has received by far the
by suppression ratios, was flat, there being almost more attention.
complete conditioned suppression during all the tones. We have seen in a preceding section of this chapter
However, as testing proceeded, the gradient sharp¬ that conventional Pavlovian conditioning parameters
ened, the suppression ratios during the tones which such as the intensity of the CS and the US determine
were most different from the previous CS showing the severity of conditioned suppression, so that the
that these stimuli were the first to lose their control phenomenon has been frequently studied as an ex¬
over behavior. Thus the flat gradient first obtained ample of Pavlovian conditioning. Kamin (1965) has
did not reflect uniform effects of the different test expressed a widely held view that “the most obvious
stimuli on behavior, and Fleshier and Hoffman’s ex¬ assumption has been that the interference with be¬
tinction procedure made it possible to identify their havior, which serves as our measure, is largely the
different degrees of behavioral control in spite of an result of incompatibility between respondents elicited
initial “ceiling effect.” by St [the pre-shock stimulus] and the ongoing be¬
Although no simple measure of conditioned sup¬ havior/7
pression is entirely satisfactory, many problems of in¬ The empirical status of this interfering respondents
terpretation may be overcome by using measures of hypothesis is open to some doubt. First, it is necessary
absolute response rates during a preshock stimulus as to specify the behaviors said to be conditioned during
well as the relative measures provided by suppression the preshock stimulus which are supposed to inter¬
ratios, and in some circumstances by using repeated fere with the operant behavior. Second, it remains
tests in changing conditions (as in Fleshier & Hoff¬ necessary to show why and how any such behaviors
man’s 1961 experiment). The most important conclu¬ are incompatible with the emission of an operant
sion to be prompted, however, is that the most appro¬ such as pressing a lever. There are several obvious
priate measure of conditioned suppression in any contenders in answer to the first of these questions,
experiment should always be considered carefully. but surprisingly little systematic work has been car¬
ried out in an attempt to monitor changes in auto¬
nomic activities to see if their intensities vary with the
SOME INTERPRETATIONS OF amount of suppression of operant behavior. On a
CONDITIONED SUPPRESSION gross level, traditional signs of autonomic activity
such as defecation, urination, piloerection, and freez¬
Why is positively reinforced operant behavior usu¬ ing of motor activity have frequently been discussed
ally suppressed during a stimulus which is associated in the context of conditioned suppression. In an early
with shock? Three major explanations for this phe¬ experimental program by Brady and his associates
nomenon will be considered here: operant behavior is (see, for example, Brady, 1951) the effects of a pre¬
suppressed because (1) other behaviors resulting from shock stimulus were measured either in terms of the
the procedure interfere with it; (2) the procedure gen¬ suppression of operant behavior with one group of
erates an emotional state which affects the underlying animals or in terms of gross changes in these auto¬
motivational state of the subject; (3) the procedure nomic activities. Similarly, Hunt and Brady (1955)
allows for occasional adventitious punishment of the commented on such activity during a stimulus which
operant behavior. It is not always easy to keep these precedes an unavoidable shock. There seems little
352 CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

doubt that signs of autonomic activity such as these do eventually took the form of only slight suppression in
characteristically accompany the early stages of many the first minute of the stimulus, with almost total
conditioned suppression experiments. However, Mil- suppression in the second and third minutes. How¬
lenson and de Villiers (1972) have suggested that these ever, measures of autonomic activity fail to show this
signs seem to decrease with continued testing, al¬ biphasic pattern. It is also worth noticing that on
though their comment is based on informal observa¬ some trials (e.g., trials 18 and 31 of those shown) lever
tions which deserve to be quantified systematically. pressing occurred more frequently in the first minute
Certainly in later stages of experiments suppression of the stimulus than in control conditions—the effect
of operant behavior does appear to persist when gross reported by Hendry and Van Toller (1965) and dis¬
signs of autonomic arousal are minimal or nonexis¬ cussed earlier. Again, there is no characteristic pat¬
tent. terning of autonomic activity which reflects this dis¬
There have been many studies of other more con¬ tribution of operant responses during the preshock
strained respondent changes resulting from the de¬ stimulus.
livery of a preshock stimulus (see, for example, the On the basis of their data, Brady and his associates
review by Weiskrantz, 1968). Two experimental pro¬ concluded that the operant and autonomic effects of
grams which have related such changes to simultane¬ their experiment were causally independent, although
ous suppression of operant behavior are particularly doubtless related in complex ways. In the most gen¬
interesting. In the first of these (de Toledo & Black, eral terms, they suggested that their finding “reflects
1966) the heart rates of rats were recorded, It was unfavorably upon theoretical formulations that em¬
found that changes in heart rate did occur during the phasize either the causal interdependence of behav¬
preshock stimulus, but they developed less quickly ioral and physiological events or the primacy of either
than did the suppression of operant responding. one.”
Moreover, the changes in heart rate were much more
variable and of shorter duration than the operant
suppression. This finding has been supported in a
study by Brady, Kelly, and Plumlee (1969), in which
+ 40
the heart rate and blood pressure (both systolic and
j= + 20 _ Systolic
diastolic) of rhesus monkeys were monitored through¬
03 CT 0
out the development and maintenance of conditioned ^X
L0
v?
j-
£
-20
U ff U fl
suppression. During the preshock stimulus, there were +40 _
£ £
certainly changes in these autonomic indicators. + 20 - Diastolic
E
Again, however, suppression of the operant behavior 0 « f* ^
CD
developed before any detectable and reliable changes -20
in heart rate and before changes in blood pressure. It
+ 40
was impossible to identify any consistently similar
6?
variations in the dependent variables
with one subject changes in heart rate
in this study:
even appeared IX
*- o
£
+ 20

0 A 7= <- rvvV
lAv\v
to be inversely related to the amount of conditioned a qo

suppression of operant behavior. On frequent occa¬ X -20

sions the two measures of blood pressure showed di¬ + 100

vergent patterns of conditioned reactions. The results MONKEY "B"
of this experiment are illustrated by data from one + 50

subject in Figure 7. This shows the percentage changes

in each of the four behavioral measures, expressed for 0
each of the successive minutes of the 3-min preshock
stimulus. Selected conditioning trials are shown. Reli¬
-50
/
able suppression of operant behavior developed be¬ -100 L U
fore any consistent disruption in autonomic activities. 1 2 3 4 5 6 7 8 9 10 16 17 18 19 20 31 40 50

The lack of consistent covariation between the mea¬ Conditioning Trials

sures can also be seen. This monkey also shows the Fig. 7. Changes in blood pressure, heart rate, and lever-pressing
development of a temporal discrimination in the con¬ response rate of a rhesus monkey during the S min of a pre¬
ditioned suppression of operant behavior, as discussed shock stimulus. The zero points represent control values in the
absence of the preshock stimulus. (From Brady, Kelly, & Plum¬
earlier. This began to develop by the 16th trial, and lee, 1969.)
Derek Blackman 353

Of course, in the context of the interference hy¬ the aversiveness of a shock when it is delivered, as
pothesis of conditioned suppression it would always Weiskrantz (1968) has suggested.
be possible to suggest that experimenters have failed “Freezing” behavior during a preshock stimulus has
to measure those respondents which do have a disrup¬ been investigated with pigeons by Stein, Hoffman, and
tive effect on operant behavior. Nevertheless, the evi¬ Stitt (1971). They used ethological recording tech¬
dence at present points unequivocally to the conclu¬ niques to measure general behavior which occurred
sion that conditioned respondents may accompany in addition to operant key pecking and found that
but do not cause conditioned suppression. In any case, there was a marked decrease in all overt activity (in¬
even if some respondent were to be identified which cluding key pecking) during the stimulus. In this
varied in direct proportion to variations in operant experiment it is unlikely that such a general in¬
responding, it would be by no means clear why it hibitory effect in behavior was maintained by an
should be physically incompatible with that respond¬ unprogrammed instrumental contingency, since the
ing, the second necessary step if the interference the¬ shock was delivered through wing bands.
ory is to be convincing. Further difficulties for this Whether any “freezing” responses during a pre¬
hypothesis are presented by the differential disruption shock stimulus should be regarded as competing re¬
of various frequencies and patterns of operant re¬ spondents or competing operants, this general inter¬
sponding which was reviewed in an earlier section, for pretation of conditioned suppression is open to the
it is not obvious why any interfering respondent be¬ objections discussed above. As with other putative
havior should be more incompatible with some pat¬ competing responses, even if they occur reliably and
terns and frequencies of operant responding than consistently, it is not clear whether they interfere
with others. This is, of course, particularly true when with recorded operants and thereby cause their sup¬
operant response rates are similar but reinforcement pression or are merely a reflection of the same proc¬
frequencies differ. Nevertheless it would certainly be ess which causes such suppression.
of great interest to monitor autonomic changes when
a preshock stimulus results in differential suppression
Motivational Explanations
of operant behavior. For example although operant
and respondent behavior may not be functionally re¬ Many researchers have suggested that a preshock
lated on a 1:1 basis during a preshock stimulus, it stimulus produces a change in the motivational state
would be interesting to discover whether the two of a subject, which in turn leads to conditioned sup¬
classes of behavior are relatively resistant to disrup¬ pression. In recent years, Estes (1969 p. 80) has sug¬
tion in the same circumstances (e.g., in situations gested that
which generate low operant response rates or with
high reinforcement frequencies). Experiments such as a stimulus which has preceded a traumatic
this might have the greatest relevance in the general event, e.g., shock, . . . acquires the capacity of
inhibiting the input of amplifier elements from
study of the relationships between autonomic proc¬
sources associated with hunger, thirst and the
esses and operant behaviors and of the relationship
like. If then, while the animal is performing an
between physiological events and directly observable
instrumental response for, say, food reward, this
behaviors. conditioned stimulus is presented, the facilita-
There remains one pattern of behavior not yet tive drive input will be reduced and so also the
fully discussed but whose occurrence during a pre¬ probability or rate of the instrumental response.
shock stimulus might certainly be physically incom¬
patible with lever pressing. Rats frequently crouch or A preshock stimulus may therefore be said to produce
“freeze” when shocked, and such behavior might oc¬ anxiety, which can be regarded as a motivational force
cur during a preshock stimulus. Discussion of this which reduces positive motivation for reinforcement
possibility has been delayed, since it would be diffi¬ and thereby decreases the frequency of operant be¬
cult to assert that this would necessarily be an exam¬ havior.
ple of a competing respondent. Leaving aside the A similar argument may also be developed from
question of whether such skeletal behaviors can be the description of conditioned suppression as result¬
classically conditioned (see Chapter 3), it is possible ing from a “conditioned emotional response” (CER).
that they might develop or be maintained as a result Thus Hunt and Brady (1951) hypothesized “an inter¬
of their consequences, and hence should be regarded nal state underlying the behavioral reaction,” and
as competing operant behavior. In other words, adopt¬ Kamin also used the term “CER” frequently (e.g.,
ing certain postures such as “freezing” might minimize 1965). However, it is not consistently clear whether
 354
CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

either Hunt and Brady or Kamin wish to emphasize posed to a random-interval 60-sec schedule of food
the adjective emotional sufficiently to demand that reinforcement in two conditions on each day: first
their theories be considered under the present head¬ when 9 hr food-deprived (prefeeding) and then after
ing rather than the previous one; for Kamin, at least, being given 8-15 g of free food (postfeeding). In both
has consistently conducted research which could be conditions, a stimulus of variable length (Millenson &
said rather to emphasize the “conditioned ... re¬ Hendry, 1967) ended with an unavoidable shock. The
sponse” as the behavioral outcome of Pavlovian con¬ panel on the left in Figure 8 shows that mean rates of
ditioning rather than as a motivational state. In their responding in the safe periods (i.e., in the absence of
interesting paper, Millenson and de Villiers (1972) the preshock stimulus) were consistently higher in the
suggest that this “failure to consider that the CER is prefeeding condition that in the postfeeding condi¬
an emotional phenomenon” has been a barrier to the tion. The preshock stimulus suppressed both these
adequate understanding of the effects we have been patterns of behavior, the effect being dependent on
discussing. These writers seek to develop Skinner’s the intensity of the shock delivered in the various
(1938) statement that emotion is “a state of strength phases of the experiment. The absolute decrease in
comparable in many respects with a drive” (p. 407) response rate was greater in the prefeeding condition.
and to argue that conditioned suppression results However, the panel on the right in Figure 8 shows
fr<?m “a negative drive activity,” a view similar to that that in terms of a suppression ratio (CS/Control rate),
of Estes. Thus “when the signal for shock [is] pre¬ the postfeeding condition appears to show the greater
sented the rads hunger might be temporarily sus¬ relative suppression at all shock intensities, the effect
pended and 'suppression’ is the natural consequence being clearer at .2 and ,1 mA, where it is less con¬
of food (as well as all other positive remforcers) hay¬ taminated by a ceiling effect produced by severe
ing temporarily lost its reward value” (Millenson, disruption of behavior. The data of this study are pre¬
1971, p. 229). sented m terms of both absolute and relative response
A motivational decrement theory such as this di¬ rates, and it can therefore be seen that the lower
rects research attention to questions somewhat differ¬ control response rates (postfeeding) are the more sup¬
ent from those discussed so far. Clearly, a stimulus pressed in terms of suppression ratios. Since, with pac¬
which precedes a shock of a given intensity will have ing procedures in which the deprivation conditions
a greater suppressive effect, according to this theory, are held constant, lower rates of responding are the
if it is superimposed on behavior which is relatively /ew disrupted (Blackman, 1968b), it seems reasonable
weakly motivated. In this context, Millenson and de to conclude with Millenson and de Villiers that the
Villiers (1972) discuss experiments in which they suppressive effects of their preshock stimulus are
varied the deprivation conditions for subjects ex¬ related directly to the value of the reinforcers. Thus
posed to a preshock stimulus. The results for one of “emotion” has a greater disruptive effect on behavior
these are shown in Figure 8. Groups of rats were ex¬ which is less strongly motivated.

Fig. 8. The effects of a preshock

stimulus on random-interval be¬
havior in rats. The rats were
tested when 9 hr food-deprived
(prefeeding) and shortly after
8—15 g of free food (postfeed¬
ing). On the left are shown re¬
sponse rates during the CS and
in its absence (safe) in both
conditions. On the right are
shown the resulting suppression
ratios. (From Millenson & de
Villiers, 1972.)
Derek Blackman 3 55

Millenson and de Villiers (1972) also reported an motivation generated by different schedules which is
interesting change in relative preference when a pre¬ clearly required to support the motivational theory of
shock stimulus is superimposed on behaviors which conditioned suppression. However, on this basis Fan-
are maintained by a concurrent schedule of reinforce¬ tino’s experiment does not suggest a reason why low
ment. Rats were exposed to a situation in which they rates of responding are even more resistant to condi¬
could always press one lever for occasional access to tioned suppression than unpaced moderate rates, since
1.5-sec milk reinforcement or press another lever for he found that the latter are preferred in choice situ¬
access to the same reinforcer for 4.5 sec. In control ations.
conditions, an asymmetric performance was observed, The idea that conditioned suppression results from
rats showing some preference for the second lever. an underlying emotional state has proved attractive
During a preshock stimulus which was superimposed in psychopharmacology. It has been argued that this
on this concurrent schedule, the preference was en¬ behavioral manifestation of anxiety or of a condi¬
hanced, there being more suppression (in relative tioned emotional response may prove useful in the
terms) of the responding on the 1.5-sec lever than on analysis of drugs which are presumed to act spe¬
the 4.5-sec lever. This is further support (it is argued) cifically on such states. Hence the effects on condi¬
for a drive decrement theory of conditioned suppres¬ tioned suppression have been reported of many drugs
sion, since the increase in preference results from the such as “tranquilizers” and barbiturates which have
greater suppressive effect of a conditioned emotional been used in clinical practice in an attempt to allevi¬
response on the less motivated behavior. ate anxiety states. These reports have recently been
Motivational theories suggest research which might reviewed by Millenson and Leslie (1974), who point
not arise from other conceptual schemes. Empirical out the considerable advantages of the conditioned
data such as those of Millenson and de Villiers are suppression procedure in this context. First, as is the
therefore welcome and challenging. However, as with case with most operant conditioning experiments,
all such theories, there are potential disadvantages in experimental sessions may continue for long periods,
the motivational view of conditioned suppression with thereby allowing the time course of a drug’s effects to
its appeal to states which cannot be measured di¬ be measured. (See also Thompson & Boren, Chapter 18
rectly. For example, the differential effects of a pre¬ of this volume.) Second, by choosing the parameters of
shock stimulus on behavior maintained by various the conditioned suppression experiment judiciously,
schedules (discussed in the section on operant con¬ it is possible to establish partial suppression during a
ditioning parameters, above) may too easily be trans¬ preshock stimulus, thereby providing a behavioral
lated into motivational terms in a way which can be base line which is sensitive to either alleviating or en¬
difficult to refute. If a pattern of behavior proves to hancing effects of a drug on anxiety. Third, and
be susceptible to conditioned suppression, this can be perhaps most important, since the procedure includes
taken as evidence that motivation is weak. On the both signaled periods when anxiety is presumed to be
other hand, behavior which is resistant to conditioned suppressing behavior and periods of safety from aver¬
suppression can readily be described as strongly mo¬ sive stimuli, it is possible to provide a within-sessions
tivated. Since behavior which is reinforced frequently control for any side effects which a drug might have
is less disrupted by a preshock stimulus, motivation on overall motivation, on sensory function, or on loco¬
can be said to vary with reinforcement frequency in a motor activity.
way that seems acceptable. Similarly, animals can be An early experiment by Brady (1956) has been
described as weakly motivated in the postreinforce¬ widely cited as illustrating the potential of this tech¬
ment pause on a fixed-ratio schedule, thus handling nique. Brady, using rats as subjects, established partial
Lyon’s (1964) conditioned suppression data discussed conditioned suppression of intermittently reinforced
earlier. But it has also been argued (Millenson 8c de operant behavior during a preshock stimulus and
Villiers, 1972) that because high rates of responding then investigated the effects of amphetamine and of
are relatively susceptible to conditioned suppression, reserpine. Brady claimed that both these drugs had
they may be weakly motivated. This view may seem specific effects “in the affective sphere,” i.e., on the
initially less plausible. It is true that Fantino (1968) conditioned emotional response. Thus amphetamine
has shown that animals prefer situations in which strengthened the CER, for in comparison with saline
they are allowed to obtain reinforcement by respond¬ conditions the drug produced a decrease in the num¬
ing at unpaced rates to situations in which they are ber of responses emitted during the preshock stimu¬
required to respond at high rates. This might appear lus, in spite of what Brady described as a nonspecific
to be the independent evidence of the strength of side effect on the behavioral base line taking the form
356 CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

of an overall increase in control response rates. Simi¬ ating suppression in acute doses; on the other hand,
larly, it was argued that reserpine attenuated the it seems that phenothiazines and reserpine alleviate
CER: despite a nonspecific decrease in overall re¬ suppression fairly consistently in studies in which they
sponse rate, the number of responses during the pre¬ are administered chronically.
shock stimulus was greater than on saline days. The final experiment to be considered in this short
Unfortunately, subsequent work in this area has review of drug effects was reported by Miczek (1973)
not consistently produced similarly encouraging data, and suggests that an emotional substrate of condi¬
and some signs of gloom have emerged as to the gen¬ tioned suppression may indeed be specifically affected
eral usefulness of conditioned suppression as a model by some drugs. Miczek reports that chlordiazepoxide
of anxiety in this context (e.g., Davis, 1968; Kelleher 8c alleviates conditioned suppression of behavior main¬
Morse, 1964). Failures to produce clear-cut effects tained by a VI schedule. His report presents data
have led to interestingly different interpretations on (shown on the right in Figure 9) in terms both of
occasion. Thus Kinnard, Aceto, and Buckley (1962) suppression ratios and of base line response rates fol¬
were led to conclude that conditioned suppression is lowing various injections of the drug, and in this
not a model of anxiety. On the other hand, Ray (1964) case it seems that the dose-related alleviation of con¬
concluded from essentially similar results that it is a ditioned suppression is not contaminated in any gross
model of anxiety, and that therefore “tranquilizing” way by any changes in behavioral base lines. Even
drugs do not have a specific effect on anxiety. more important evidence, however, is to be found in
It seem§ likely that the conditioned suppression the effects of the drug administered to other animals
phenomenon is a simple model of anxiety only on a exposed to a slightly different situation, These rats
superficial level. We have seen some of its complex¬
were also trained on a VI schedule. In their case,
ities in preceding sections, and these must surely com¬ however, a stimulus was superimposed which ended
plicate the analysis of any drug’s effects. Thus Appel
(1963) has shown that a dosage of reserpine which reli¬
ably reduced conditioned suppression when the shock
intensity was .8 mA failed to produce consistent effects
when the shock was increased only to 1.0 mA. Simi¬
larly, the response rates during the preshock stimulus
in Brady’s (1956) experiment may haye been changed
not only by any specific effect of the drugs on the CER
but also by the “nonspecific” side effects of the drugs:
for example, the increased overall response rates pro¬
duced by amphetamine may themselves have pro¬
duced the relative decrease in response rates during
the preshock stimulus (Blackman, 1972). Or again,
since amphetamine is known to be an anorexic agent,
the relative susceptibility of behavior to suppression
after its administration may be the result of a rela¬
tively low motivational state in the subject on those
days (Millenson 8c Leslie, 1974). Finally, any attenuat¬
ing or enhancing effects of a drug on the amount of
conditioned suppression may merely be the outcome
of a drug’s rate-dependent effects on the two rates of
responding during and in the absence of the preshock
stimulus (Wuttke 8c Kelleher, 1970).
Despite the complexities of the situation, Millen¬
son and Leslie (1974) have suggested that the effects
of drugs on conditioned suppression have not been as
inconsistent as has sometimes been supposed. They CHLORDIAZEPOXIDE mg/kg

consider the reported effects of chronic and acute doses Fig. 9. The effects of chlordiazepoxide on suppression during a
of various drugs separately and conclude that minor stimulus which precedes food (left) or shock (right). Suppression
tranquilizers (benzodiazepines, barbiturates, and me¬ ratios are shown at the top and base line response rates below.
Notice that the schedules of dosages are not identical in the
probamate) have a relatively consistent effect in allevi¬ two conditions. (Redrawn from Miczek, 1973.)
Derek Blackman 3 57

not with a shock but with the delivery of free food. experiments explicitly designed to test it, will be dealt
Operant behavior was suppressed during this stimu¬ with only briefly here.
lus in much the same way as occurs during a preshock Clearly, shocks delivered “independently” of be¬
stimulus (this finding of “positive conditioned sup¬ havior may occasionally be associated with behavior
pression” is reviewed in the next section of this in this way. Gottwald (1967) has shown that the
chapter). Miczek reports, however, that this suppres¬ amount of conditioned suppression on any trial is
sion of operant behavior was not attenuated by in¬ affected by the proximity of shock to a response on the
jections of chlordiazepoxide (see the left panels in previous trial. However, there are a number of rea¬
Figure 9). These results suggest the drug has a specific sons to doubt that conditioned suppression is princi¬
effect on anxiety, rather than simply exerting differ¬ pally caused by adventitious punishment. One of the
ent effects on different rates of responding regardless most important of these is represented in Rescorla’s
of the nature of the US signaled by the CS. work reviewed earlier. In his experiments, the associa¬
Studies of drug effects on conditioned suppression tion of a stimulus with shock is accomplished “off the
usually attempt to identify specific effects on the emo¬ base line,” i.e., before operant training is begun. Sub¬
tional states which are thought to produce the sup¬ sequently only the conditioned stimulus is superim¬
pression. It may be recognized that these rather vaguely posed on operant behavior, and there is therefore
defined emotional states may be regarded as condi¬ no opportunity for any adventitious contiguity be¬
tioned responses and might therefore have been dis¬ tween shock and response. Yet, of course, conditioned
cussed in the context of the interference hypothesis suppression does occur during the CS in these experi¬
—he., emotion (anxiety) is a classically conditioned re¬ ments. Hoffman and Barrett (1971), using observa¬
sponse which disrupts ongoing operant behavior. How¬ tional techniques and initial association of a stimulus
ever, most drug studies do not attempt to identify the With shock “off the base line,” have also failed to sup¬
interfering conditioned emotional response per se, port the punishment hypothesis with pigeon subjects,
and that is why they have here been discussed in the since again conditioned suppression developed when
context of a motivational theory. possible contiguity between shock and responding was
minimized. There is also a good deal of evidence that
the development of conditioned suppression may be
The Punishment Hypothesis
accompanied at a gross level by more signs of auto¬
In the conditioned suppression procedure a stimu¬ nomic disturbance than is punished behavior (Hunt &
lus is superimposed on ongoing behavior and ends Brady, 1955). In addition, Annau and Kamin (1961)
with a shock which is delivered regardless of what the have claimed that a shock of .28 mA was sufficient in
subject does. Punishment, however, is defined as the their experiment to suppress behavior when used in a
“reduction of the future probability of a specific re¬ punishment procedure, but not when used in the
sponse as a result of the immediate delivery of a stim¬ conditioned suppression procedure. Orme-Johnson
ulus” (such as a shock) after that response (Azrin 8c and Yarczower (1974) used a yoking procedure, in
Holz, 1966, emphasis added). It has been suggested which pigeons were exposed either to a discriminated
that there are no fundamental differences between the punishment procedure or to one which delivered the
processes that lead to these two forms of suppression. same number of shocks independently of behavior.
On the one hand, it has been argued (e.g., by Estes, They reported that the latter procedure produced
1944) that responding which is explicitly followed by suppression while the former produced none. More¬
shock is suppressed by the process outlined above in over, the stimulus associated with shock in the con¬
the motivational theory of conditioned suppression. ditioned suppression procedure acquired conditioned
Thus the shock is associated with certain external punishing effects, while the discriminative stimulus
cues: these become conditioned stimuli by a Pavlovian for the punishment procedure did not.
procedure, and so behavior is suppressed as a result Lyon (1968) has argued that “punishment and con¬
of a conditioned emotional response. This account ditioned suppression do not represent a behavioral
has few advocates today as a theory of punishment dichotomy but specific points on a behavioral con¬
(Azrin 8c Holz, 1966). However, the opposite theory tinuum,” a suggestion that it is difficult to refute
has also been presented—that conditioned suppression unequivocally. Differences between the effects of the
results from an occasional chance contiguity between two procedures may always be interpreted in such a
operant behavior and the delivery of a shock. This way. However, considerable procedural differences be¬
theory has been discussed at some length by Lyon tween punishment and conditioned suppression in¬
(1968), and since there have been relatively few recent evitably present difficulties in comparing them, not
358 CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

least because suppression of responding has a conse¬ present, it is not easy to predict precisely the circum¬
quence in reducing shock frequency in the former stances in which suppression will be the rule, but it is
case but not the latter. It therefore seems unrewarding difficult to fit examples such as these into any tradi¬
to try to reduce either one to the other, and this is tional motivational theory. On the other hand, an
perhaps why there is little current research with this interference hypothesis should in principle be as ca¬
emphasis. pable of handling suppression of avoidance behavior as
of coping with suppression of positively motivated be¬
havior. The problem with this theory, however, is that
A BRIEF REVIEW OF SOME OTHER it offers little in the explanation of any acceleration of
CLASSICAL-OPERANT INTERACTIONS avoidance behavior during a conditioned stimulus.
Rescorla and Solomon (1967) also predicted on the
The problems and questions arising from the study basis of their motivational theory that, in their terms,
of interactions between classical and operant condi¬ any appetitively motivated behavioral base line will
tioning have been illustrated so far exclusively by increase in frequency during an appetitive Pavlovian
studies in which stimuli associated with shock have conditioned stimulus. In other words, food-reinforced
been superimposed on operant behavior maintained operant behavior should increase in frequency during
by food or water reinforcement. We now turn to con¬ a signal that ends with presentation of free food. We
sider briefly some other procedures. have seen already, however (Miczek, 1973), that condi¬
For some time, motivational theories of condi¬ tioned suppression may occur during such a stimulus.
tioned suppression gained considerable support from For example, using rats as subjects Azrin and Hake
their apparent ability to handle data describing the (1969), Meltzer and Brahlek (1970), and Hake and
effects of signaling an unavoidable shock when an Powell (1970) have all reported suppression of re¬
animal's operant behavior is maintained by a schedule sponding during a stimulus lasting 10 or 12 sec and
of shock avoidance. For example, Rescorla and Solo¬ ending with the presentation of free food. Similarly,
mon (1967) have argued that the laws of Pavlovian suppression has been reported with monkeys during
Conditioning are “the law§ o£ emotional conditioning prefood stimuli (Kelly, 1973a, 1973b; Miczek & Gross-
or laws of acquired drive states” and that “condi¬ man, 1971). It is difficult to see how the Rescorla and
tioned emotional states change [the subject's] motiva¬ Solomon (1967) account of classical-operant interac¬
tion level and thus can serye either as motivators or tions can handle such findings. It is intriguing, how¬
reinforcers of instrumental responses.” They there¬ ever, to see the vigor with which other theoretical
fore make the general assertion that aversively moti¬ accounts of the more traditional conditioned suppres¬
vated Operant behavior will increase in frequency dur¬ sion during a preshock stimulus have been extended
ing a stimulus which precedes an unavoidable shock, to this so-called positive conditioned suppression. In
since this conditioned emotional state will summate their study, Azrin and Hake (1969) used either food or
with the motivation maintaining avoidance behavior. water as the reinforcer for operant behavior and de¬
Studies by Sidman, Herrnstein, and Conrad (1957), livered “free” food, water, or intracranial stimulation
Kelleher, Riddle, and Cook (1963), and Waller and at the end of their stimulus. In general, they found
Waller (1963) all showed that free operant avoidance suppression during the stimulus with all the combina¬
did increase in frequency in this way during a stim¬ tions of reinforcer and US which they tested. They
ulus which preceded the delivery of an unavoidable suggested that such suppression was the result of a
shock. However, more recently there has been a num¬ general emotional state” and argued that suppression
ber of reports of conditioned suppression even of during a preshock stimulus is another example of the
avoidance behavior (e.g., Blackman, 1970b; Bryant, effects of such a state, rather than a model of a specific
1972; Hurwitz 8c Roberts, 1969; Pomerleau, 1970; anxiety state. Azrin and Hake do not specify the
Roberts 8c Hurwitz, 1970; Scobie, 1972). It seems that nature of this general emotional state, but it would
suppression of avoidance behavior may occur if the seem to be basically similar to Skinner's concept of
unavoidable shock is discriminable from avoidable emotion (1938, p. 407).
shocks (e.g., of a different intensity), or if such sup¬ Kelly (1973a) has attempted to monitor any
pression does not increase the overall frequency of changes in autonomic activity in monkeys which
shocks, either because the avoidance schedule is sus¬ might be conditioned during a prefood stimulus, with
pended during the warning signal or because the re¬ a view to evaluating the interfering respondents
sponse-shock times of the schedule are relatively long hypothesis in the context of positive conditioned sup¬
in comparison with the duration of the signal. At pression. Using the same experimental techniques to
Derek Blackman 359

monitor cardiovascular activity as Brady, Kelly, and Smith (1974), using pigeons, investigated the contribu¬
Plumlee (1969) had employed in their study of pre¬ tion of various response rates and reinforcement fre¬
shock stimuli, Kelly was again unable to detect any quencies to the effects of prefood stimuli of various
systematic covariation of changes in autonomic activ¬ lengths. He found that both high and low response
ity and operant suppression. He therefore dismisses rates were increased during a 5-sec prefood stimulus.
the idea that positive conditioned suppression is With longer stimuli, high rates were suppressed, but
caused by interfering respondents produced by the low response rates were unaffected. With two of the
Pavlovian aspects of the procedure. three subjects, high response rates were less suppressed
One difficulty in considering a theory of positive when they obtained high frequencies of reinforcement
conditioned suppression in terms of interfering re¬ rather than lower frequencies. It is clear from this
spondents is that the status of the free food as a Pav¬ study that there are considerable similarities in the
lovian unconditioned stimulus is by no means clear. variables which control the amount of suppression
In all the studies in this area, except when brain during a prefood stimulus, as here, and during a pre¬
stimulation ended a stimulus in Azrin and Hake’s shock stimulus (e.g., Blackman, 1968b).
(1969) study, the delivery of the “free” event seems to There appears to be an important species-depen¬
act more as a discriminative stimulus setting the occa¬ dent effect when relatively short prefood stimuli are
sion for an approach response to a particular part of used in experiments. Although the above review sug¬
the experimental chamber than as a stimulus which gests that the behavior of rats and monkeys is con¬
unconditionally elicits some response. This observa¬ sistently suppressed in such conditions, LoLordo
tion serves to emphasize the possibility that positive (1971) has found that pigeons’ response rates increase.
conditioned suppression might be produced by inter¬ Similarly, Smith (1974) found increases in various be¬
fering operants, a recurring theme in this research havioral base lines during a 5-sec prefood stimulus
(Farthing, 1971). Thus it may be that recorded oper¬ with his pigeons. In a recent study, LoLordo, McMil¬
ant responding decreases because the subject makes lan, and Riley (1974) have thrown considerable light
preparatory approaches to the food cup which maxi¬ on this anomaly. They found that if the operant re¬
mize the speed of taking up the free food when it sponse being studied was key pecking, response rates
is delivered, although most reports in this area claim increased if the prefood stimulus was a change in the
that such behaviors could not be detected. Also, illumination of the key. However, if the prefood stim¬
whether suppression or acceleration of responding ulus was a nonlocalized tone, there was no accelera¬
develops during a prefood stimulus, the possibility tion. Similarly, there were no consistent effects of a
must be considered that this is superstitiously rein¬ tone or light prefood stimulus if the operant was
forced by the delivery of the free food—an analogue of treadle pressing rather than key pecking. The authors
the punishment hypothesis of the effects of preshock interpret these results as suggesting that the accelera¬
stimuli. However, the evidence for superstitious rein¬ tive effect dependent on a localized prefood stimulus
forcement in this context is not strong (see Staddon, is an example of an autoshaped and automaintained
1972). response (Brown 8c Jenkins, 1968). This suggestion has
The effects of prefood stimuli are being shown in¬ the immediate effect of bringing the discussion toward
creasingly to depend on the parameters of the situa¬ the work of Gamzu and Schwartz (1973), who have
tion and on the nature of the behavior on which they developed the view that key-pecking rates of pigeons
are superimposed. Thus Meltzer and Brahlek (1970) may depend on a summation of pecking maintained
reported acceleration of rats’ variable-interval be¬ by instrumental contingencies per se and pecking
havior during a 120-sec prefood stimulus, but, as which is supported by automaintenance. Since auto¬
noted, suppression during a 12-sec stimulus. Henton shaping and automaintenance have been discussed in
and Brady (1970) trained monkeys on a DRL 30-sec the context of classical rather than operant condition¬
schedule and found no effect of a prefood stimulus of ing (Jenkins 8c Moore, 1973), the work of Gamzu and
20 or 40 sec, but they found acceleration during a pre¬ Schwartz (1973) and its extension to phenomena such
food stimulus lasting 80 sec. Kelly (1973b) also found as behavioral contrast (e.g., Keller, 1974) is clearly
acceleration of monkeys’ DRL behavior during a 60- relevant to the study of interactions between classical
sec prefood stimulus; his experiment, however, also and operant conditioning. However, since they are
made it possible to compare this effect with that of discussed elsewhere (Chapter 3), these ideas are not
the same prefood stimulus on random ratio behavior. developed here.
This revealed a schedule-dependent effect, for the It can be seen then that there has been much recent
latter behavior was suppressed during the stimulus. work on the effects of prefood stimuli on operant be-
 360 CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

havior. Some studies have even reported that such unavoidable aversive event. However, more recently
stimuli may have suppressive effects on behavior the idea of a general emotional state has been revived,
maintained by an avoidance schedule (e.g., Davis 8c of which the traditional conditioned emotional re¬
Kreuter, 1972; Henton, 1972). In general terms, sponse is said to be but one example. A further theory
studies of the effects of prefood stimuli have developed suggests that disruptive effects of a conditioned stim¬
in a similar way to those of preshock stimuli. In both ulus result from the conditioning of a motivational
cases, the parameters of the procedure and the nature state which interacts with the motivation which sup¬
of the behavioral base lines on which it is superim¬ ports the base line operant behavior. Yet a further
posed are crucial, and this makes it impossible to possibility is that disruption of operant behavior dur¬
make general assertions that a given preevent stimulus ing a preevent stimulus is the outcome of poorly con¬
will have simply suppressive or enhancing effects on trolled instrumental contingencies and hence reflects
behavior. There is at present no adequate general the strength of other interfering operants or the re¬
theory, whether this be couched in terms of a general sult of adventitious punishment or reinforcement.
emotional state, conditioned drives, competing re¬ Whether disruptions of operant behavior are
spondents or operants, or superstitious reinforcement thought to reflect underlying classical or operant con¬
of different rates by the delivery of free food, In short, ditioning effects or the development of changed moti¬
research in this area may be said to mirror almost vational states, it is quite clear that the effects of any
Exactly the problems which have been discussed in preeyent stimulus depend critically on the nature of
the context of preshock stimuli throughout this the behavioral base lines on which they are superim¬
chapter. posed. I he effects of classical conditioning procedures
on operant behavior arc therefore schedule-depen-
dent, as are the effects of so many other independent
CONCLUSION variables. The differing degrees of susceptibility to
disruption by a Pavlovian conditioned stimulus pose
The procedures we have considered in this chapter further questions: do these differences reflect different
haye an apparent simplicity that can obscure the very strengths of an underlying conditioned response, or is
real complexities both of measurement and of inter¬ this strength determined solely by CS-US parameters
pretation. In particular, the appropriate measurement so that different degrees of suppression reflect the re¬
of the disruption of operant behavior by classical con¬ sistance to disruption of different patterns of operant
ditioning procedures poses great problems. There is a behavior? Similar problems of interpretation arise
real danger that describing these effects m terms of a from the effects of drugs on disruptions of operant
relative rate during the conditioned stimulus can ob¬ behavior during a conditioned stimulus and clearly
scure important aspects of the situation. In spite of return us to the problem of appropriate measurement.
this, we have seen that the measurement of Pavlovian In spite of the many problems of measurement and
conditioning processes through what is usually re¬ interpretation which have been discussed in this chap¬
garded as their indirect effects on operant behavior ter, studies of the effects of classical conditioning pro¬
has been widely recognized as being unusually sensi¬ cedures on operant behavior have long played an
tive and thereby productive. On the other hand, at¬ honorable role in learning theory. A problem in re¬
tempts to monitor any autonomic effects which might viewing this research, however selectively, is that it
be supposed to be directly conditioned by the Pavlo¬ has been related at various times to many theoretical
vian aspects of the procedure have been generally controversies in psychology, and these general issues
disappointing: autonomic changes often do occur dur¬ have been mentioned only briefly here. The procedure
ing the conditioned stimulus, but they in no sense has proved to be successful in providing a sensitive
appear to reflect the orderliness of the indirect oper¬ dependent variable for the study of the necessary and
ant effects which one might suppose to be mediated by sufficient conditions for the development of an ac¬
the classical conditioning of autonomic processes. quired reflex. However, research using this procedure
Faced with this problem, it has been argued that it is has also provided empirical evidence which has been
a rather ill-specified conditioned emotional response related to motivational theories of behavior and the
(CER) which is the direct outcome of the Pavlovian role of classical conditioning in motivation, to the
aspects of the conditioning procedure. Some workers study of emotion, to the relations between physiolog¬
have preferred to describe this CER as conditioned ical events and overt behavior, to the study of the
anxiety, a term which has a degree of superficial effects of potential anxiolytic agents, and to many
validity in a situation in which a stimulus precedes an other important problems. Indeed, perhaps one of the
Derek Blackman 361

most important features of these studies is that they Blackman, D. E. Operant conditioning: An experimental
provide a focus for discussion between workers of analysis of behaviour. London: Methuen, 1974.
Blackman, D. E., 8c Scruton, P. M. Conditioned suppres¬
different theoretical persuasions. In this light, it seems
sion of counting behavior in rats. Journal of the Experi¬
almost symbolic that the amount of conditioned sup¬ mental Analysis of Behavior, 1973,19, 93-100. (a)
pression in a specified situation is a function of both Blackman, D. E., 8c Scruton, P. M. Conditioned suppres¬
classical and operant conditioning parameters, and sion and discriminative control of behavior. Animal
that this disruption seems at present not to be a direct Learning and Behavior, 1973, 1, 90-92. (b)
Bond, N. W., Blackman, D. E., & Scruton, P. M. Suppres¬
reflection of underlying physiological or autonomic
sion of operant behavior and schedule-induced licking
processes. This complexity emphasizes that no one ap¬ in rats. Journal of the Experimental Analysis of Be¬
proach to the problems discussed here can be thought havior, 1973, 20, 375-383.
of as dominant. Hearst and Jenkins (1975) have Brady, J. V. The effect of electroconvulsive shock on a
recently suggested that identifying the different forms conditioned emotional response: The permanence of
the effect. Journal of Comparative and Physiological
of learned behavior which develop in specified circum¬ Psychology, 1951, 447 507-511.
stances is at present preferable to the espousal of any Brady. J. V. Assessment of drug effects on emotional be¬
universal theory of learning. The results reviewed in havior. Science, 1956, 123, 1033.
this chapter support this view. Brady, J. V., Kelly, D., & Plumlee, L. Autonomic and
behavioral responses of the thes\is monkey to emotional
conditioning. Annals of the Mew York Academy of
Science, 1969, 139, 959-975.
REFERENCES Brown, p. L., & Jenkins, H. M. Auto-shaping of the pi¬
geon’s key peck. Journal of the Experimental Analysis
Annau, Z., & Kamin, L. J. The conditioned emotional of Behavior, 1968, ll> 1-8,
BryanTj R. C. Conditioned suppression of free-operarxt
response as a function of intensity of the US. Journal of
Comparative and Physiological Psychology, 19Gb 7f, avoidance. JOzirna l of the Experimental Analysis of Be
m-m. havior, 1972, 17, 257-260.
Afpel? J. B. Drugs, shock intensity, and the CER. Psycho Carlton, P U, & Didamo, P. Some notes on conditioned
pharmac&logia, 1963, 4, 148-153. suppression. Journal of the Experimental Analysis of
Azrin, N. H. Some effects of two intermittent schedules of Behaviori l$6o, 7, 233-236.
immediate and non-immediate punishment. Journal of Davi^j II, ^gndltIoned suppression: A survey of die liter¬
Psychology, 1956, 42, 3-21. ature. Psychonomic Monograph Supplements, 1968, 2
Azrin, N. fh, k Hare, D, Y, Positive conditioned suppres¬ (14, Whole No. 30), 283-291.
sion: Conditioned suppression using positive reinforcers Davis, H., & Kreuter, C. Conditioned suppression of an
as the unconditioned stimuli. Journal of the Experi¬ avoidance response by a stimulus paired with food.
mental Analysis of Behavior, 1969, 12, 167-173. Journal of the Experimental Analysis of Behavior, 1972i
Azrin, N. H., & Holz, W. C. Punishment. In W. K. Honig 17, 277-285.
(Ed.), Operant behavior: Areas of research and applied- de Toledo, L. E,, 8c Black, A, H, Heart rate- Changes
tion. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966. during conditioned suppression in rats. Science} 1966?
Black, A. H., & Prokasy, W. E. (Eds.). Classical condition¬ U2, 1404-1400,
ing, Vol. 2: Current research and theory. Englewood Dews, P. B, Behavioral effects of drugs. In S. M. Farber 8c

Cliffs, N.J.: Prentice-Hall, Inc., 1972. R, H, L, Wilson (Eds.), Conflict and creativity. New

Blackman, D. E. Response rate and conditioned suppres¬ York; McGraw-Hill, 1963.

sion. Psychological Reports, 1966, 19, 687-693. Estes, W. K. An experimental study of punishment. Psy¬
Blackman, D. E. Effects of response pacing on conditioned chological Monographs, 1944, 57 (Whole No. 263).
suppression. Quarterly Journal of Experimental Psy¬ Estes, W. K. Outline of a theory of punishment. In B. A.
chology, 1967, 19, 170-174. Campbell & R. M. Church (Eds.), Punishment and aver¬
Blackman, D. E. Conditioned suppression or acceleration sive behavior. Englewood Cliffs, N.J.: Prentice-Hall, Inc.,
as a function of the behavioral baseline. Journal of the 1969.
Experimental Analysis of Behavior, 1968, 11, 53-61. (a) Estes, W. K., 8c Skinner, B. F. Some quantitative properties
Blackman, D. E. Response rate, reinforcement frequency, of anxiety. Journal of Experimental Psychology, 1941,
and conditioned suppression. Journal of the Experi¬ 29, 390-400.
mental Analysis of Behavior, 1968, 11, 503-516. (b) Fantino, E. Effects of required rates of responding upon
Blackman, D. E. Effects of a pre-shock stimulus on tem¬ choice. Journal of the Experimental Analysis of Be¬
poral control of behavior. Journal of the Experimental havior, 1968, 11, 15-22.
Analysis of Behavior, 1970, 14, 313-319. (a) Fantino, E. Emotion. In J. A. Nevin (Ed.), The study of
Blackman, D. E. Conditioned suppression of avoidance behavior: Learning, motivation, emotion, and instinct.
behaviour in rats. Quarterly Journal of Experimental Glenview, Ill.: Scott, Foresman, 1973.
Psychology, 1970, 22, 547-553. (b) Farthing, G. W. Effect of a signal previously paired with
Blackman, D. E. Conditioned anxiety and operant be¬ free food on operant response rate in pigeons. Psycho¬
haviour. In R. M. Gilbert 8c J. D. Keehn (Eds.), Schedule nomic Science, 1971, 23, 343-344.
effects: Drugs, drinking, and aggression. Toronto: Uni¬ Ferster, C. B., 8c Skinner, B. F. Schedules of reinforcement.
versity of Toronto Press, 1972. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1957.
 362 CONDITIONED SUPPRESSION AND THE EFFECTS OF CLASSICAL CONDITIONING ON OPERANT BEHAVIOR

Flesiiler, M., 8c Hoffman, H. S. Stimulus generalization specificity of the behavioral effects of drugs. Ergehnisse
of conditioned suppression. Science, 1961, 133, 753-755. der Physiologie, 1968, 60, 1-56.
Gamzu, E., 8c Schwartz, B. The maintenance of key-pecking Kelleher, R. T., Riddle, W. C., 8c Cook, L. Persistent
by stimulus-contingent and response-independent food behavior maintained by unavoidable shocks. Journal of
presentation. Journal of the Experimental Analysis of the Experimental Analysis of Behavior, 1963, 6, 507-517.
Behavior■, 1973, 19, 65-72. Keller, K. The role of elicited responding in behavioral
Gottwald, P. The role of punishment in the development
contrast. Journal of the Experimental Analysis of Be¬
of conditioning suppression. Physiology and Behavior, havior, 1974, 21, 249-257.
1967, 2, 283-286.
Kelly, D. D. Long-term prereward suppression in monkeys
Hake, D. F., 8c Powell, J. Positive reinforcement and sup¬
unaccompanied by cardiovascular conditioning. Journal
pression from the same occurrence of the unconditioned of the Experimental Analysis of Behavior, 1973, 20, 93-
stimulus in a positive conditioned suppression proce¬ 104. (a)
dure. Journal of the Experimental Analysis of Behavior, Kelly, D. D. Suppression of random-ratio and acceleration
1970, 14, 247-257.
of temporally spaced responding by the same prereward
Hearst, E. Stress-induced breakdown of an appetitive dis¬ stimulus in monkeys. Journal of the Experimental Anal¬
crimination. Journal of the Experimental Analysis of ysis of Behavior, 1973, 20, 363-373. (b)
Behavior, 1965, 8, 135-146.
Kinnard, W. J., Aceto, M. D. G., Sc Buckley, J. P. The
Hearst, E., k Jenkins, H. m. Sign-tracking: The stimulns- effects of certain psychotropic agents on the conditioned
reinforcer relation and directed action. Austin, Texas;
emotional response behavior pattern of the albino rat.
Fsyclionomie Society, 1974.
Psychopharmacologia, 1962, 5, 227-230.
Hendry, D. P., k Van Toller, C. Alleviation of condi¬ Lea, S. E. C., 8c Morgan, M. J. The measurement of rate-
tioned suppression. Journal of Comparative and Physio¬ dependent changes in responding. In R. M. Gilbert 8c
logical Psychology, 1965, 49, 458-460.
J. R. Millenson (Eds.), Reinforcement: Behavioral analy¬
HENTON, W. W. Avoidance response rates during a pre- ses. New York; Academic Press, 1972.
fOOd stimulus in monkeys. Journal of the Experimental LoLordo, V, M, Facilitation of food-reinforced responding
Analysis of Behavior, 1972, 17, 269-275.
by a signal for response-independent food. Journal of
Henton, W. W„ k Brady, j. V. Operant acceleration dur¬ the Experimental Analysis of Behavior, 1971, 13, 49-56.
ing pre-reward stimulus. Journal of the Experimental LoLordo, V. M., McMillan, J. C., & Riley, A. L. The
Analysis of Behavior, 1970, 13, 205-209.
effects upon food-reinforced pecking and treadle-press¬
Hoffman, H. S. Stimulus generalization versus discrimina¬ ing of auditory and visual signals for response-indepen¬
tion failure in conditioned suppression. In R. M. Gilbert dent food. Learning and Motivation, 1974, 3, 24-41.
Sc N. S, Sutherland (Eds.), Animal discrimination learn¬ Lyon, D. O. Some notes on conditioned suppression and
ing. New York; Academic Press, 1969.
reinforcement schedules. Journal of the Experimental
Hoffman, H. S., & Barrett, J, Oyert activity during con¬ Analysis of Behavior, 1964, 7, 289-291.
ditioned suppression: A search for punishment artifacts.
Lyon, D. O. A note on response rate and conditioned sup¬
Journal of the Experimental Analysis of Behavior, 1971,
pression. Psychological Record, 1965, 15, 441-444.
16, 343-348.
Lyon, D. O. Conditioned suppression; Operant variables
Hunt, H. F., & Brady, J. V. Some effects of electroconvul¬
and aversive control. Psychological Record, 1968 18,
sive shock on a conditioned emotional response (“anx¬
317-338.
iety”). Journal of Comparative and Physiological Psy-
Lyon,^ D. O., 8c Felton, M. Conditioned suppression and
chology, 1931, jji 88-98.
variable ratio reinforcement. Journal of the Experi¬
Hunt, H. F., & Brady, J, V, Some effects of punishment
mental Analysis of Behavior, 1966, 9, 245-250. (a)
and inter-current “anxiety” on a simple operant. Journal
of Comparative and Physiological Psychology, 1955, 48, Lyon, D. O., & Felton, M. Conditioned suppression and
305-310. fixed ratio schedules of reinforcement. Psychological
Record, 1966, 16, 433-440. (b)
Hurwitz, H. M. B., k Roberts, A. E. Suppressing an avoid¬
ance response by a pre-aversive stimulus. Psychonomic Lyon, D. O., & Millar, R. D. Conditioned suppression on
Science, 1969, 17, 305-306. a fixed interval schedule of reinforcement. Psychonomic
Jenkins, H. M., & Moore, B. R. The form of the auto¬ Science, 1969, 17, 31-32.
shaped response with food or water reinforcers. Journal Mackintosh, N. J. The psychology of animal learning.
of the Experimental Analysis of Behavior, 1973, 20, 163- 1974 Academic Press, London, New York, San Francisco.
181. McKearney, J. W. Schedule-dependent effects: Effects of
Kamin, L. J. Temporal and intensity characteristics of the drugs, and maintenance of responding with response-
conditioned stimulus. In W. F. Prokasy (Ed.), Classical produced electric shocks. In R. M. Gilbert 8c J. D. Keehn
conditioning. Englewood Cliffs, N.J.: Prentice-Hall, (Eds.), Schedule effects: Drugs, drinking, and aggression.
Inc., 1965. Toronto: University of Toronto Press, 1972.
Kamin, L. J., 8c Schaub, R. E. Effects of conditioned stim¬ Meltzer, D., & Brahlek, J. A. Conditioned suppression
ulus intensity on the conditioned emotional response. and conditioned enhancement with the same positive
Journal of Comparative and Physiological Psychology, UCs: An effect of CS duration. Journal of the Experi¬
1963, 56, 502-507. ^ mental Analysis of Behavior, 1970, 13, 67-73.
Kelleher, R. T., 8c Morse, W. H. Escape behavior and Miczek, K. A. Effects of scopolamine, amphetamine, and
punished behavior. Federation Proceedings, 1964 23 benzodiazepines on conditioned suppression. Pharma¬
808-817. cology, Biochemistry, and Behavior, 1973, 1, 401-411.
Kelleher, R. T., 8c Morse, W. H. Determinants of the Miczek, K. A., 8c Grossman, S. Positive conditioned sup-
Derek Blackman 363

pression: Effects of CS duration. Journal of the Experi¬ W. F. Prokasy (Eds.), Classical conditioning, Vol. 2: Cur¬
mental Analysis of Behavior, 1971, 15, 243-247. rent Research and Theory. Englewood Cliffs, N.J.: Pren¬
Migler, B., 8c Brady, J. V. Timing behavior and condi¬ tice-Hall, Inc., 1972.
tioned fear. Journal of the Experimental Analysis of Roberts, A. E., 8c Hurwitz, H. M. B. The effect of a pre¬
Behavior, 1964, 7, 247-251. shock signal on a free-operant avoidance response. Jour¬
Millenson, J. R. A motivation-reinforcement theory of nal of the Experimental Analysis of Behavior, 1970, 14,
emotion. Studia Psychologica, 1971, 13, 222-238. 331-340.
Millenson, J. R., 8c de Villiers, P. A. Motivational proper¬ Scobie, S. R. Interaction of an aversive Pavlovian condi¬
ties of conditioned anxiety. In R. M. Gilbert 8c J. R. tional stimulus with aversively and appetitively moti¬
Millenson (Eds.), Reinforcement: Behavioral analyses. vated operants in rats. Journal of Comparative and
New York: Academic Press, 1972. Physiological Psychology, 1972, 79, 171-188.
Millenson, J. R., 8c Hendry, D. P. Quantification of re¬ Sidman, M., Herrnstein, R. J., 8c Conrad, D. G. Mainte¬
sponse suppression in conditioned anxiety training. nance of avoidance behavior by unavoidable shocks.
Canadian Journal of Psychology, 1967, 21, 242-252. Journal of Comparative and Physiological Psychology,
Millenson, J. R., 8c Leslie, J. The conditioned emotional 1957, 50, 553-557.
response (CER) as a baseline for the study of anti¬ Skinner, B. F. Behavior of organisms. New York: Appleton-
anxiety drugs. Neuropharmacology, 1974, 13, 1-9. Century-Crofts, 1938.
Orme-Johnson, D. W., 8c Yarczower, M. Conditioned sup¬ Smith, J. B. Effects of response rate, reinforcement fre¬
pression, punishment, and aversion. Journal of the Ex¬ quency, and the duration of a stimulus preceding
perimental Analysis of Behavior, 1974, 21, 57-74. response-independent food. Journal of the Experimental
Pavlov, I. P. Conditioned reflexes (trans. G. V. Anrep). Analysis of Behavior, 1974, 21, 215-221.
London: Oxford University Press, 1927. St addon, J. E. R. Temporal control and the theory of
Pomerleau, O. F. The effects of stimuli followed by re¬ reinforcement schedules. In R. M. Gilbert 8c J. R. Mil¬
sponse-independent shock on shock-avoidance behavior. lenson (Eds.), Reinforcement: Behavioral analyses, New
York: Academic Press, 1972.
Journal of the Experimental Analysis of Behavior, 1970,
Stein, L., Sidman, M., & Brady, J. V. Some effects of two
11, 11-21.
temporal variables on conditioned suppression. Journal
Ray, O. S. Tranquilizer effects as a function of experi¬
mental anxiety procedures. Archives Internationales de of the Experimental Analysis of Behavior, 1958, 1, 154-
Pharmacodynamie et de Therapie, 1964, 153, 49—68.
162.
Rescorla, R. A. Pavlovian conditioning and its proper
Stein, N,, Hoffman, H. S,, 8c Stitt, C. Collateral behavior
of the pigeon during conditioned suppression of
control procedures. Psychological Review, 1967, 74,
key pecking. Journal of the Experimental Analysts of
71-80.
Behavior, 1971, 15, 83-93.
Rescorla, R. A. Probability of shock in the presence and
Waller, M. B., & Waller, P. F. The effects of unavoidable
absence of CS in fear conditioning. Journal of Compara¬
shocks on a multiple schedule having an avoidance com¬
tive and Physiological Psychology, 1968, 66, 1-5.
ponent. Journal of the Experimental Analysis of Be¬
Rescorla, R. A. Conditioned inhibition of fear. In N. J.
havior, 1963, 6, 29-37.
Mackintosh 8c W. K. Honig (Eds.), Fundamental issues
Weiskrantz, L. Emotion. In L. Weiskrantz (Ed.), Analysis
in associative learning. Halifax: Dalhousie University of behavioral change. New York: Harper 8c Row, 1968.
Press, 1969. Weiss, K. M. Some effects of the conditioned suppression
Rescorla, R. A., 8c Solomon, R. Two-process learning paradigm on operant discrimination performance. Jour¬
theory: Relationships between Pavlovian conditioning nal of the Experimental Analysis of Behavior, 1968, 11,
and instrumental learning. Psychological Review, 1967, 767-775.
74, 151-182. Wuttke, W., 8c Kelleher, R. T. Effects of some benzodiaze¬
Rescorla, R. A., 8c Wagner, A. R. A theory of Pavlovian pines on punished and unpunished behavior in the
conditioning: Variations in the effectiveness of rein¬ pigeon. Journal of Pharmacology and Experimental
forcement and nonreinforcement. In A. H. Black 8c Therapeutics, 1970, 172, 397-408.
 13

Negative Reinforcement
and Avoidance*

Philip N. Hineline

INTRODUCTION the term has been identified mainly with one theory,
and with experimental procedures oriented to that
This chapter deals with behavior that is main¬ theory. The history of the interplay between avoid¬
tained when it removes, reduces, or prevents stimula¬ ance theory and experiments has been thoroughly
tion. The stimulation is called aversive on the basis documented by Herrnstein (1969) and by Bolles (1973)
of this functional relation with behavior. Through the and will not be recounted here. I will, however, sketch
same functional relation, the removal or reduction of the theory, initially noting some reasons for departing
stimulation is defined as negative reinforcement. As from it. Later I will occasionally indicate how the
the title suggests, many of the experiments to be de¬ present approach relates to it or differs from it.
scribed here were initially designed and interpreted In brief, avoidance theory has required that some
as avoidance, with this term taken both as a conve¬ stimulus, called a warning stimulus or conditioned
nient category of procedures for shaping and maintain¬ stimulus, be paired with primary aversive stimulation
ing behavior, and as a presumably valid category of such as electric shock. Through this pairing the warn¬
ing stimulus is said to become aversive. Then, an overt
behavior or behavioral processes. However, avoidance
response that is allowed to prevent the shock can also
will not be my organizing principle. The everyday
produce an immediate effect, terminating the warning
meaning of this term is too general to assist analysis.
stimulus. In some versions of the theory, the warning
Defined more precisely as the prevention, rather than
stimulus is identified as a Pavlovian conditioned stim¬
the reduction or removal of aversive stimulation,
ulus, producing a conditioned internal response; re¬
avoidance applies to only a few of the procedures and
moval of the stimulus then terminates the conditioned
data to be included here. Further, within psychology
response. In all versions, responding is said to be rein¬
forced by its immediate effects, and only incidentally
* This manuscript was written with the support of funds to prevent the primary stimulation. Paradoxically this
from a PHS research grant MH-18432 from the National In¬
stitute of Mental Health to the author, and of a Summer Re¬
is to assert that avoidance (as prevention of absent
search Fellowship from Temple University. aversive stimulation) is not a basic process; rather that

364
Philip N. Hineline 365

such prevention is always the byproduct of escape time. I will begin by describing two experiments. One
from a present conditioned stimulus. illustrates the fact that procedures conventionally
The present account departs from traditional labeled and studied as avoidance are very limited
theory, partly because warning stimuli are sometimes representatives of what this term might include. The
not aversive as predicted by the theory. Also, warning second experiment suggests that negatively reinforced
stimuli have multiple properties that the theory does behavior may be more similar to its positively rein¬
not predict. These facts will be documented below. forced counterpart than is customarily assumed. I will
There are more general reasons for taking a differ¬ then consider the forms that negative reinforcement
ent approach. Avoidance theorizing has conformed to procedures have taken, first defined in discrete-trial
a prejudgement that the reduction of aversive stimula¬ procedures, and then extended to free-operant pro¬
tion must be immediately discriminable upon occur¬ cedures in which only the aversive stimuli are manip¬
rence of a response, if that reduction is to reinforce ulated. Next, I will consider procedures in which cues
the response. Thus in the prototypical procedure are added, providing for stimulus control of negative
sketched above, the warning stimulus is included to reinforcement, and changing the way in which inter¬
provide an immediately discriminable effect. When mittent aversive stimulation affects behavior. Finally,
avoidance procedures have not provided explicit I will deal with some issues related to the shaping of
immediate consequences, their interpretations have particular responses with negative reinforcement.
focused on plausible surrogates for the warning stim¬ The following are some major points; The funda¬
ulus. The surrogates have been drawn from overt be¬ mental operations in negative reinforcement proce¬
havior, or covert behavior, or internal time-correlated dures have been shock-delay and shock-deletion. In
stimuli, but have always been said to function in a shock-delay, the timing schedules for shock are reset
manner like that described above for externally-sup¬ by responses. In shock-deletion, the timing schedules
plied warning stimuli. Explanation of avoidance, proceed independently, but responses can cancel shock
then, has been mainly an explanation of how an deliveries. In both of these, shock-frequency reduction
animal bridges gaps in time between a response and appears to be a major controlling variable. When
the consequent non-occurrence, or reduced occurrence additional stimuli are provided, their functions de¬
of the aversive stimulation. This focus came prior to pend on which procedural features they are correlated
any appreciable examination of the range of situations with. For example, added cues can control behavior
in which negative reinforcement is effective. through correlations with contingencies of reinforce¬
In defense of the strategy of beginning with ment, irrespective of the presence or absence of shock.
theoretical notions of mechanism, it can be argued Added cues can also control behavior through their
that the theory provides a means for summarizing and correlation with differing rates of shock. A distinction
organizing data. Further, when carried out with pre¬ can be made between reinforcement due to a change
cision as in mathematical models, this approach car¬ of situation, and reinforcement due to shock-frequency
ries with it a formal evaluation of explanatory as¬ reduction within a situation. Added stimuli can also
sumptions. However, when applied to avoidance be used to isolate particular variables such as shock-
theory this increased rigor has been accompanied by frequency reduction and short-term delay of shock.
restriction to small ranges of data, gathered with im¬ For the initial shaping of behavior with negative rein¬
poverished sets of procedures (e.g. Hoffman, 1966; forcement, one may encounter apparent constraints
Theios, 1971). The more common, verbal postulations on conditioning. However, this “characterization by
of avoidance mechanisms deal with more data, but deficit” is of little help. Additional principles may be
as Hoffman (1966) and Church (1973) have noted, needed to describe the maintenance of the ongoing
these accounts have at best been informal and impre¬ stream of behavior upon which negative reinforce¬
cise. In both cases, the focus on questions of mecha¬ ment must operate.
nism has tended to constrain the range of procedures
that are studied.
The present account is still concerned with orga¬ TWO ILLUSTRATIVE EXPERIMENTS
nizing principles. However, I will focus on external
controlling variables. Traditional avoidance proce¬
Broadening the Range of Avoidance Experiments
dures will be included, as part of a continuum that in¬
cludes situations where behavior is maintained by Consider first an experiment that suggests a way to
immediate consequences, as well as situations where systematically generate and examine a rich variety
its apparent controlling consequences are remote in of behavioral situations that might be called avoid-
 366 NEGATIVE REINFORCEMENT AND AVOIDANCE

30 SEC | 30 SEC .3 SEC | 27 SEC havioral output toward the one requiring fewer re¬
sponses, with a bias toward responding in the situation
BLUE ! GREEN 1 RED 1 BLACKOUT
closer to shock.

30 RESP 30 RESP
This experiment indicates a promising approach
i‘ 4
T ▼
for investigators who wish to make “avoidance” a
BLACKOUT primary concern. Krasnegor, Brady, and Findley’s
procedure could be expanded or extended, not only to
1
-- examine a wider range of response requirements, but
also to examine “reversing chains” in which respond¬
Fig. 1. Schema for a procedure using aversively maintained mul¬
ing in a proximal situation would reinstate a more
tiple operants. With no responding, the sequence progresses
as indicated at the top of the diagram, with shocks accompany¬ distal situation. For example, the procedure described
ing the three-sec red light. As indicated, ratio schedules in effect above could be modified so that responding in green
during the red and green conditions permit shock-deletion and
would put the subject back in the presence of the blue
blackout for greater portions of the fixed 90-sec cycle. (After
Krasnegor, Brady, & Findley, 1971. © 1971 by the Society for the light. I shall describe experiments with this feature
Experimental Analysis of Behavior, Inc.) later, in the context of stimulus control. One might
also study choice and preference within this type of
procedure by use of branching reverse sequences, such
ance. The experiment, by Krasnegor, Brady, and Find¬ that differing, more-or-less distal situations could be
ley (1971), used Rhesus monkeys, and was based on an made contingent upon different responses. This fol¬
abortable sequence of events diagrammed in Figure 1. lows a strategy similar to that outlined by Findley
Initial training was accomplished in phases, with the (1962) for the study of appetitively-maintained be¬
final procedure involving three stimuli in sequence on havior. The resulting procedures would more closely
a recurring 90-sec cycle. If the confined monkey did resemble what is commonly called “avoidance” than
not press a lever, a blue light was present for 30 sec, have traditional avoidance procedures.
followed by a green light for 30 sec, followed by a 3-
sec red light that was accompanied by three, brief, in¬
escapable shocks, followed by 27 sec of darkness, or Comparing Negative with Positive Reinforcement

timeout, until the beginning of the next cycle. Thirty

My second illustrative experiment is by Kelleher
responses in the presence of either the green light or and Morse (1964), who compared responding main-
the blue light, with the count starting from zero at
each stimulus change, produced timeout for the re¬
mainder of the 90-sec cycle. If fewer than 30 responses
occurred in both the blue and green 30-sec periods,
the red 3-sec light appeared with its 3 inescapable
shocks. Thus, sufficiently rapid and persistent respond¬
ing during either the blue or green light could abort
the sequence, producing timeout for the remainder of
the current 90-sec cycle. Stable performances were ob¬
tained in which the subjects received few shocks. One
monkey responded primarily in the presence of the
green light, which was the situation proximal to the
red light and shocks. The other monkey completed
the ratios about equally often in the proximal and Fixed Ratio Requirement
distal (green and blue) situations. When the size of
Fig. 2. Part A shows results of manipulating the fixed-ratio
either response requirement was varied while the
requirement in the proximal (green) stimulus while the ratio
other was held constant, the performances of both size in the distal (blue) stimulus remained constant at FR 30.
monkeys gave similar functional relationships. As Each data point represents the mean number of ratio comple¬
tions for the last five sessions at each value of the ratio in green.
shown by a comparison of the two parts of Figure 2,
Part B shows results of manipulating the FR requirement in
responding in the distal situation dropped off more the distal (blue) stimulus while the ratio requirement in the
quickly as a function of fixed ratio size in that situa¬ proximal (green) stimulus remained constant at FR 30. Each
data point represents the mean number of ratio completions
tion (Figure 2B) than it did in the proximal situation
for the last five sessions at each value of the ratio in blue. The
when that ratio was increased (Figure 2A). The main differing data points (filled squares vs. open circles) denote
effect of varying the response requirements in the different monkeys. (After Krasnegor, Brady, & Findley, 1971. ©
1971 by the Society for the Experimental Analysis of Behavior,
proximal or distal situations was to transfer the be¬ Inc.)
Philip N. Hineline 367

tained by food presentation with responding main¬ tion of the drugs’ effects is given in Chapter 7 of this
tained by termination of situations, denoted by dis¬ book. For the present purpose, these effects can be
tinctive visual cues, in which intermittent shocks easily summarized: While the two drugs had differing
occurred. The latter consequence can also be character¬ effects on behavior, the shock-maintained and food-
ized as production of a discriminable shock-free maintained responding were very similarly affected by
situation. Three squirrel monkeys were trained on a given drug. In these situations the kind of conse¬
conventional multiple schedules of positive reinforce¬ quences maintaining the behavior were less important
ment in which a 10-min fixed-interval (FI) schedule than the specific pattern of behavior. This result
alternated with a fixed-ratio (FR) schedule that re¬ recommends that appetitive /aversive distinctions not
quired 30 responses. The schedules were accompanied be taken for granted, nor too much predicated on
by white and red lights, respectively, and exposures to them. Negatively-reinforced behavior and behavior
these schedules were separated by timeout periods characterized as avoidance may have much in common
denoted by a visual pattern of horizontal bars. Three with behavior not so categorized. Nevertheless, the
similar monkeys were trained on an analogous pair of procedures for negative reinforcement differ formally
schedules of negative reinforcement: in the FI sched¬ from those of positive reinforcement, and the effects
ule, brief shocks were scheduled to occur once per sec, of negative reinforcement procedures will be the main
beginning when the white light had been present for concern here.
10 min; the first response after the 10-min period
produced the pattern of horizontal bars which de¬
noted a situation with no shocks, and no response NEGATIVE REINFORCEMENT
contingency. In the presence of the red light, brief WITHOUT ADDED CUES
shocks were scheduled to occur once every 30 sec; the
30th response m the presence of this light terminated
Ttie Escape Procedure
the light and also produced the timeout stimulus. As
shown in Figure 3, patterns of key-pressing main¬ A simple procedure for negative reinforcement is
tained by these alternating FI and FR schedules were that of escape. In this procedure an aversive stimulus
nearly identical for the appetitively-maintained and is presented and some aspect of the subject’s behavior,
the aversively-maintained procedures. With this ac¬ which the experimenter specifies as a response, can
complished Kelleher and Morse administered D- terminate that stimulus. The stimulus is identified as
amphetamine and chlorpromazine, separately and aversive if the result of its termination is an increased
with systematic variation of doses. A detailed descrip- probability or decreased latency of the response when
that stimulus is again present. In most experiments on
negative reinforcement the aversive stimulus has been
electric shock. Other stimuli, such as intense light
(Keller, 1941), loud noise (Harrison 8c Tracy, 1955),
rotation (Riccio Sc Thach, 1966), temperature change
(Weiss 8c Laties, 1961), and centrifugal force (Clark,
Lange, 8c Belleville, 1973) have been used with varying
success in aversive conditioning procedures, but the
present chapter will deal mostly with shock since it
has been almost universally used in systematic work.
In the escape procedure, behavior in the absence of
an aversive stimulus has no effect on its subsequent
recurrence. Such responding is typically ignored.
Thus, escape conditioning is aptly described as a dis¬
crete-trial procedure, where the presence of the
aversive stimulus defines a trial during which the
Fig. 3. Performance maintained by reinforcement with food subject’s behavior is under study. Yet, while a trial is
(upper records) compared to performance maintained with easily identified it is not a simple event. Three of its
electric shock (lower records) under multiple FI FR schedules.
components will be distinguished here; as shown in
The large excursions were produced when the 10-min FI
schedules were in effect; the smaller excursions were produced Part I, of Figure 4, they coincide in time on the proto¬
on the 30-response fixed-ratio schedules. At reinforcement, typical escape procedure, but can be viewed as inde¬
either through food delivery or shock prevention, the cumula¬
pendent. In the discussion that follows, these features
tive recording pen reset to the bottom of the record. (From
Kelleher & Morse, 1964.) will provide a basis for relating the escape procedure
 368 NEGATIVE REINFORCEMENT AND AVOIDANCE

I. Basic Escape Procedure

positive reinforcement—key-pecking is improbable
a Shock _ J~L when the grain hopper is accessible—typical negative
b Opportunity to Respond.... ri reinforcement and positive reinforcement situations
c Resp. Can Affect Shock_ n are similar. They differ in that positive reinforcement
durations are usually shorter than negative reinforce¬
ment durations.
I. Shock Delay with Continuous Shock in Absence of Responding

a Shock _j J”
b Opportunity to Respond..!

c Resp. Can Affect Shock.! - ~~~~ Free Operants and the Escape Paradigm

Although the escape procedure of textbooks is

BE Shock Delay with Brief Periodic Shocks in Absence of Responding usually described as requiring only a single response
a Shock I I I I I I I I I L -U to terminate the shock and to produce an intertrial
f.--- ¥_£ 1_£_
D Opportunity to Respond J interval, where neither a), b) nor c) is operative. Dins-
s Reap. Can Affect Shock.] ” ! moor and his colleagues have demonstrated schedules
of negative reinforcement superimposed on the basic
Fig. 4. Diagram illustrating three feature? they apply tO each
escape procedure. They allowed the opportunity for
of three procedures, with and without responding. Time
indicated linearly from left to right; upward displacement of a responding whenever the shock was present. Features
line indicates when a given feature is in effect. An R’ indicates a) and b) coincide as before, but c), the period during
occurrence of a response. Fart I illustrates the basic escape which responses could affect shocks, was restricted
conditioning procedure, Part II illustrates a shock-delay pro¬
cedure in which failure to respond results in continuous shock. (Dinsmoor, 1967). As in appetitive schedules, rein¬
Part III illustrates a more typical shock-delay procedure of the forcements were set up during only parts of the
kind devised hy Sidman (1953), in which the RS interval is twice periods when responding could occur. The resulting
the 55 interval.
behavior on basic ratio and on interval schedules
resembled that of comparable schedules of positive
to other negative reinforcement procedures and for reinforcement.
interrelating those procedures. The escape trial in¬ So far I have said little about intervals when
cludes; a) An aversive situation. In the basic escape neither a) nor c) is operative. Of course, these inter¬
procedure the aversive situation is defined by contin¬ trial intervals are of only peripheral interest if they
uous presence of the aversive stimulus, b) An oppor¬ include no opportunity to respond and other behavior
tunity for responses to occur and to be counted. In is not observed. However, if a completely free operant
the basic escape procedure the opportunity is often is allowed and responding can and does occur between
controlled by closing a door or removing a response trials that it cannot affect, we have a situation that
lever; typically the opportunity for responding is can be compared with other free-operant procedures.
terminated with the first response of the trial, c) An The shock can function as a discriminative stimulus,
occasion when the specified response can affect the delineating the availability of reinforcement as well as
occurrence of aversive stimulation. One can relate providing the basis for it. This resembles an appetitive
these features to those of positive reinforcement situa¬ discrimination procedure where in the presence of a
tions; a) Providing an aversive situation is a manip¬ discriminative stimulus (Sd), responses are reinforced
ulation that potentiates reinforcement. This feature according to some schedule, with extinction in the
identifies it as a “drive operation,” analogous to absence of the discriminative stimulus (SA). Keehn
deprivation procedures that are often used to poten¬ (1966) has noted patterns of intertrial responding
tiate positive reinforcement. Of course its discrimina¬ that suggest such an interpretation, with discrete-trial
tive properties differ from those of a deprivation negative reinforcement procedures seen as Sd-SA dis¬
procedure; I will discuss these properties later, b) The crimination procedures.
opportunity for responses to occur and be counted is
typically terminated with the occurrence of one re¬
The Escape/Avoidance Distinction, and a
sponse. Given that the opportunity coincides with c),
Preview of the Present Approach
the occasion during which reinforcement is “set up”
or available, this limited opportunity provides for Sometimes in appetitive discrimination procedures,
continuous reinforcement (erf), or reinforcement on a delay contingency is imposed during SA so that re¬
FR 1. To the extent that in positive reinforcement sponding during this time prevents the onset of Sd.
situations the observed response is precluded during Such a delay contingency can be added to the escape
Philip N. Hineline 369

procedure as well, and it has been done most simply ule of reinforcement, and the presence of another
in one version of the well-known shock-delay pro¬ light denotes a different schedule of reinforcement.
cedure devised by Sidman (1953)—the version with a Thus to the extent that the different negative rein¬
shock-shock interval of zero. As diagrammed in Part II forcement contingencies operate independently, de¬
of Figure 4, in the absence of responding continuous noted by distinctive cues, I will treat them like those
shock is delivered. A response can terminate this of any other multiple-contingency procedure.
shock for a period known as the response-shock (RS) Later I will describe procedures that use intermit¬
interval; such a response is negatively reinforced by tent brief shocks, producing situations that are not
removal of shock. In addition, the opportunity to re¬ easily discriminable, but that involve differing rein¬
spond is continually present, and responses in the forcement contingencies. In dealing with these I will
absence of shock delay the onset of shock. The shock distinguish between A) reinforcement by reduction of
will resume only if the response-shock interval elapses the density of aversive stimulation within a situation,
without an intervening response to restart the timing and B) reinforcement by discriminable change of
of this interval. In the appetitive case such a delay situation. Both categories include procedures that
contingency reduces the future probability of the have been called “avoidance.” The placing of a given
response to which it is applied. In the aversive case it procedure in category A) or category B) will depend
has the opposite effect; responding in the absence of upon added cues as well as upon characteristics of the
shock is maintained by delaying the onset of shock. In aversive stimulation itself. The reasons for making
the present development of procedures the added de¬ this distinction will become evident later. For the
lay feature introduces a new level of complexity. In immediate present, I shall describe procedures that
previous procedures, whenever a response occurred it cut across both escape and avoidance. These proce¬
either removed the aversive stimulus or had no effect dures require no special cues other than the shocks
on aversive stimulation. Now, different consequences themselves. They involve shock-delay or shock-dele¬
are produced by different classes of responses. tion, based on fixed or on variable time intervals.
Within the tradition of aversive conditioning the
presence vs. absence of shock has been the major dis¬
The Continuum of Shock Density or Frequency
tinction between escape and avoidance. Escape re¬
sponses remove shock; avoidance responses prevent its Electric shock continuously delivered may not be
occurrence. The validity of this distinction has been continuously received. For example, if it is grid shock
supported by experiments in which two different the animal may produce intermittency by jumping up
responses were independently maintained, one by re¬ and down. Nevertheless the escape procedure is
moval of shock, and the other by delay or prevention treated as a clear case of negative reinforcement by
of shock (e.g., Boren, 1961; Mowrer 8c Lamoreaux, removal of shock. The experimenter may even arrange
1946). The present account replaces this aspect of the an escape procedure by explicitly presenting intermit¬
escape/avoidance distinction with a more general con¬ tent pulses of shock several times per second, rather
cept of multiple contingencies of reinforcement under than presenting it continuously. But if shock is pre¬
stimulus control. Instead of the Sd — SA discrimina¬ sented several times per second, why not just twice per
tions of the escape procedure (shock vs. no shock) second, or once per second, or even less frequently? At
where responses are reinforced in the presence of one some point we tend to stop labeling it continuous
stimulus and extinguished in the presence of another, shock, and call it a stream of shocks. Responses are
there are two Sds. A response in the presence of shock reinforced by interruption of (escape from) a stream
both removes the shock and produces a shock-free of shocks. But as the pulses of shock are spaced out
interval. Responses in the absence of shock delay it, still further, to one every five, ten, or twenty seconds,
extending the shock-free interval. As with the escape/ we tend to characterize suspension of this situation
avoidance distinction, if these two consequences are not as removal of shock, or as interruption of a
made contingent upon separate responses, the sep¬ stream of shocks, but as reduction in shock frequency
arate responses will be independently maintained. or density. One aspect of the shock delivery procedure
The separate effects, however, are not seen as resulting that affects this characterization is the variability of
from distinct processes. Rather, they result from the the time between pulses of shock. Regularly spaced
fact that in discriminably different stimulus situa¬ shocks seem to be appropriately characterized as
tions, different responses are reinforced. In this re¬ streams or sequences until they become quite far
spect, it is like a positive reinforcement procedure apart. Irregularly spaced shocks are difficult to specify
where the presence of one light denotes a given sched¬ without reference to a distribution, whose measure of
 370
NEGATIVE REINFORCEMENT AND AVOIDANCE

central tendency translates into frequency or density. only marginally effective. In contrast, Clark and Hull
Frequency and density are nearly equivalent. One (1966) reported reliable acquisition with equal RS and
stresses the number of shocks per unit time; the other SS intervals of 60 sec or more. However, in unpub¬
stresses the time between events, but can also refer to lished work I have been unable to replicate this lat¬
the intensity and duration of those shocks. ter result.
Once responding is established, RS and SS intervals
Shock-Delay Procedures
affect response rates somewhat differently, as shown
by one of Sidman’s early experiments (Sidman, 1953).
The various forms of Sidman’s well-known free- Sidman measured overall response rates while he
operant shock-delay procedure (Siclman, 1953) illus¬ varied the RS interval, holding the SS interval con¬
trate the density continuum. Diagrammed in Part III stant for each series of RS values, and changing the
of Figure 4, the procedure is a more general form of SS interval between series. In this way he obtained a
the “escape-avoidance’7 procedure described above. In family of functions for each subject; data from one of
the absence of responding, instead of continuous the rats is presented in Figure 5. With SS intervals
shock, brief shocks are delivered periodically. These of 5 sec or greater, maximal response rates occurred
shocks are typically from 0.1 to 0.5 sec in duration, when the RS interval was equal to or slightly shorter
and are separated by the shock-shock (SS) interval. than the SS interval. Response rates dropped off gradu¬
The shock-shock interval ranges from zero to more ally as the RS interval was increased beyond the SS
typically 3, 5, 10, or 20 sec. A single response inter¬ interval, giving plots that were concave upward. Re¬
rupts this sequence for a time known as the response sponse rates dropped off more quickly as the RS inter¬
diock (RS) interval. With no additional intervening val was decreased, giving plots that were concave
responses, occurrence of the next shock reinstates the downward, tending sharply toward zero with small RS
SS interval as the determinant of shock delivery. How- values. With smaller SS intervals, where there was lit-
everj if additional responses occur before the RS inter¬
val has elapsed, each one “resets the clock/’ restarting
the timing of the RS interval. In the terms of the
preceding pages, a short SS interval denoting virtually
continuous shock might be considered as defining the
aversive situation, while the RS interval defines its
absence. A response during the brief SS interval pro¬
duces a relatively long RS interval, effectively remov¬
ing shock as in a standard escape procedure. However,
in procedures where the SS interval approaches the RS
interval, the term “escape” no longer seems appropri¬
ate; a response during a long SS interval can be §aid
to delay shock just as responses during the RS inter¬
val do.
While the SS and RS intervals do not cleanly dis¬
tinguish escape from avoidance, or even define the
presence versus absence of aversive situation, they do
have distinguishable effects on behavior. The effect of

each is partly determined by the value assigned to the

other. For example, Sidman (1962a) has noted that
acquisition of lever-press responding is more easily
achieved if the SS interval is substantially shorter than
the RS interval. Leaf (1965) documented this effect
with a between-group comparison in which each an¬
imal was run for only one session. Using a 20-second
RS interval combined with SS intervals of 1, 3, 5, 10,
Fig. 5. One subject’s rates of lever-press responding as response-
or 20 seconds, he found consistent acquisition with shock intervals were varied in relation to constant shock-shock
SS intervals of 5 sec or less. Acquisition was less con¬ intervals. Each column of the table indicates a series obtained,
sistently obtained with the SS interval of 10 sec; and in irregular order, with a given SS interval. Some of the series
shown in the table are plotted in the curves. (After Sidman,
an SS interval equaling the 20-sec RS interval was 1953.)
Philip N. Hineline 371

tie room to manipulate the RS interval at values be¬ This is supported by plots on the right side of Fig¬
low the SS interval, only the concave-upward portion ure 6 that tend toward zero to the right of the origin.
of the curve was obtained. The departure from a simple percentage relationship
More recently, Clark and Hull (1966) examined the was not large. It appears reliable, however, for Sid¬
maintenance of overall response rates by procedures man (1953) also reported a comparable effect when
with equal RS and SS intervals, varying these inter¬ increasing the RS interval and using the reciprocal of
vals together. While they used preselected rats that the RS interval in place of maximum shock frequency.
had produced especially low shock rates during prior
experimentation, the general features of their results Shock-Deletion Procedures
agree with other less complete, but comparable data.
As shown by the open symbols on the left side of A different type of negative reinforcement pro¬
Figure 6, increases in the RS = SS interval between cedure provides continuous opportunity to respond
10 sec and 60 sec produced decreases in response rate and also incorporates much of the continuum, from
according to a roughly hyperbolic function, trans¬ pulse streams to widely spaced shocks. Instead of pro¬
formed to roughly linear plots by using the recipro¬ viding for a response to reset the timing interval and
cals of the intervals. Clark and Hull noted that trans¬ thus delay shock, these procedures allow a response to
formations other than the reciprocal, such as semilog cancel or delete an impending shock without affecting
and log-log transformations also produced approx¬ the time cycles for shock delivery. The basic charac¬
imately linear plots of response rate as a. function of teristics are readily evident in a fixed-cycle procedure
RS = SS interval. They found no compelling basis for described by Sidman (1966). As the label implies, the
selecting one of these transformations over another. procedure is based on a timing cycle that progresses
When Sidman (1953) held the SS interval constant independently and constantly, regardless of the sub¬
at 2.5 seconds, varying the RS interval through larger ject's behavior. With no responding, each timing cycle
values he obtained response rate changes comparable ends with a brief inescapable shock and starts again.

to those obtained by Clark and Hull. This is demon¬ The first response in a cycle cancels the shock due at
strated by the plot with asterisks on the left side of
Figure 6. These points were obtained by replotting a
representative set of Sidman's data from Figure 5,
in terms of reciprocals of the RS interval. Thus, as
Sidman concluded from the experiment represented
by Figure 5, the SS interval apparently has little effect
in determining the shape of the rate versus RS func¬
tion, provided that the RS interval is not substantially
smaller than the SS interval. This is not a surprising
outcome for well-conditioned animals that eliminate
most shocks; the few shocks that they receive would
be initiated by the Response-Shock timer. The results
obtained by Clark and Hull with equal RS and SS in¬
tervals show relationships that would be expected on
Max. Shock Rate ISk/Min)
any schedules in which the RS interval exceeds the
SS interval, at least for animals that eliminate most Fig. 6. Response rates and obtained shock rates as functions of
shocks. the maximum shock rates possible under shock-delay and shock-
deletion procedures. Each open symbol represents a pair of
The right side of Figure 6 shows shock rates that
equal shock-shock and response-shock intervals on Sidman’s
correspond to the response rates just considered. The shock-delay procedure, as used by Clark and Hull (1966). Differ¬
open points indicate roughly linear increases in ently shaped data points represent performances of different
received shock rates as Clark and Hull increased the rats. The plot with asterisks shows response rate as a function
of the reciprocal of the RS interval (which, for this set of
maximum possible shock frequency. However, for intervals does not equal maximum shock rate), with the SS
most animals, received shock rate was not simply a interval held constant at 2.5 sec. These data, obtained by
percentage of maximum shock rate. Clark and Hull Sidman (1953), are taken from Figure 5. The plots with filled
symbols show response rates and received shock rates as func¬
reported that as the maximum shock rate was re¬ tions of maximum shock rates on a shock-deletion procedure
duced to low values—corresponding to long SS = RS based on variably spaced shocks (after de Villiers, 1974). Most
intervals—the percent shock reduction increased points show the mean of two determinations for a given VI
schedule: one from an ascending, and one from a descending
and more responses were emitted per shock received. series of VI values.
372 NEGATIVE REINFORCEMENT AND AVOIDANCE

the end of that cycle; additional responses during the ited to one effective response per cycle. This pro¬
cycle have no effect. Thus, as indicated in Part I of cedure has not been studied as extensively or as sys¬
Figure 7, the opportunity to respond is continuous, tematically as has the shock-delay procedure, but it is
but the occasion for a response to affect shock is lim- clearly effective for producing and maintaining lever-
press responding. The fixed-cycle shock-deletion pro¬
cedure is similar to the shock-delay procedures con¬
I. Fixed-Cycle Deletion
sidered above in that both use fixed shock-shock
No Response intervals, and both allow frequent responding to elimi¬
■ nate all shocks. The two procedures differ in that all
b. Opportunity to Respond ' responses on the delay procedure affect shock, while
c. Resp. Can Affect Sk. i on the deletion procedure not all responses are effec¬
Occasional Responses tive. They also differ with respect to which relations
^ Shnrk 1 1 between responding and shock are fixed, and which
R R R R R
b. Opportunity to Respond ■ are variable. In the delay procedures the interval be¬
c. Resp. Can Affect Sk.
tween response and shock is fixed. The amount of
shock reduction resulting from each response varies
with the subject’s spacing of responses. In deletion
I Variable-Cycle Deletion
procedures the subject’s spacing of responses affects
the interval between response and shock, but the
No Response
amount of shock reduction is tightly controlled; an
a. Shock I 1_I_IN_1_LJ_
effective response eliminates exactly one shock. Closely
b. Opportunity to Respond'! spaced responses are relatively ineffective in both: A
c. Resp. Can Affect Sk. response that closely follows another can produce only
Occasional Responses a small increment in shock delay. There is a low
a. Shock i i i : 1 1 1 1 probability of deleting shock, since it is likely to fall
, R ; R RR R ! R
1!
b. Opportunity to Respond! in the same cycle as the earlier response.
c. Resp. Can Affect Sk. Shock-delay procedures need not use constant time
intervals. Indeed, Sidman and Boren (1957a) modified
the basic delay procedure by changing the response-
shock interval after each shock. This procedure read¬
IE. Fixed-Cycle Deletion with tA Interposed
ily produced both acquisition and maintenance of
No Response
lever-pressing. Bolles and Popp (1964) used a delay
a. Shock procedure in which the response-shock interval was
b. Opportunity to Respond ■ constant but the shock-shock interval varied from one
! j
c. Resp. Can Affect Sk. rr 1 tA It® 1 tA l t° i 1 1 i i shock to the next, with a mean value slightly under
Occasional Responses seven seconds. They compared acquisition on this
a. Shock 1 i 1 1
procedure to that on a standard delay procedure
__R_R_B_JLR—RR_ where the RS interval was 15 sec and the SS interval
was constant at five sec. This was a between-subject
c. Resp. Can Affect Sk.
comparison, and their small number of rats prohib¬
Fig. 7. Diagrams showing the relations between features of ited concluding that the variable shock-shock interval
shock-deletion procedures. A given feature is in effect when its was superior to the fixed one. However, the observed
corresponding line is displaced upward. A straight line indicates
that a given feature is continuously in effect. An “R” indicates
differences were clearly in that direction. In unpub¬
the occurrence of a response. Part I illustrates a procedure lished work I have found consistently good acquisi¬
based on fixed-cycle shock delivery (after Sidman, 1966). Part tion and maintenance of lever pressing on a procedure
II illustrates a procedure that is formally identical except it is
based on variable-cycle shock delivery (after de Villiers’s 1974
where both the response-shock and shock-shock inter¬
“variable-interval” shock deletion). Part III illustrates a fixed- val varied randomly after each response and each
cycle procedure with imposed tA periods when responding shock. Clearly, fixed intervals are not critical to the
cannot affect the impending shock. On this procedure, tD indi¬
cates periods when responding is effective. In this example tA
effectiveness of the shock-delay procedures.
and tt> are of equal duration. Their relative durations can be The shock-deletion contingency, where timing cy¬
varied (after Hurwitz & Millenson, 1961). In all three parts cles are independent of behavior, is also readily
of the figure, sequences of events with and without responding
are portrayed. The dashed lines identify points at which re¬
adapted to variable shock-shock intervals, as shown by
sponding has resulted in deletion of shock. de Villiers (1974). His procedure was identical to Sid-
 Philip N. Hineline 373

man’s fixed-cycle shock-deletion procedure described

A B
above, except that the shock-shock interval varied
from cycle to cycle. In the absence of responding the
rats received brief shocks, irregularly spaced so that
the probability of a shock was roughly constant over
time. The first response in an SS interval deleted the
shock due at the end of that interval; additional re¬
sponses within the interval had no effect. When the
time for a (deleted) shock was passed, the next re¬
sponse would again be effective. A plausible sequence
of events on this procedure, showing shocks delivered
by identical random scheduling sequences with and
without responding, is shown in part II of Figure 7.
De Vdliers first established responding with a sched¬
Shock Frequency Reduction
ule whose mean shock-shock interval in the absence of
responding was 15 sec. He then varied this mean in¬ Fig. 8. Reciprocals of response rates plotted against reciprocals
terval between 15 sec and 60 sec, over blocks of ses¬ of shock-frequency reductions on shock-delay and shock-deletion
schedules. Part A shows data obtained by dc Villiers (1971)
sions. The resulting response rates are plotted in¬
with a variable-cycle shock-deletion procedure. Part B shows
dividually for the four rats, with the filled symbols in data obtained by Clark and Hull (I960) with shock-delay
Figure 6. These plots represent combined data from sehedules m which SS and RS intervals were held equal a§ they
were varied over blocks of sessions. These plots are based on
increasing and decreasing series, which de Villiers the §am? data that were presented in Figure 0.
presented separately. With maximum shock frequency
as an independent variable common to the two types Herrnstdn had derived in studies of concurrent sched¬
of procedures, de Villiers’s results are readily compared ules of positive reinforcement and then developed to
with the results obtained with fixed shock-delay pro¬ relate the rate of responding to the rate of reinforce¬
cedures by Clark and Hull (1966). Interestingly, al¬ ment on single VI schedules (Herrnstein, 1970). A
though absolute response rates are much higher for detailed treatment of this formulation can be found
two of de Villiers’s four animals, response rates versus in Chapter 9 of the present volume. For the present
maximum shock rate gave roughly similar functions purpose, suffice it to say that Herrnstein’s equation
in both cases. The “received shock frequency” plots predicts a linear relationship between the reciprocal
for the two procedures show even greater similarity of response rate and the reciprocal of the rate of rein¬
than the response rate plots. However, systematic re¬ forcement.1 Since assessment of linearity is simpler
lationships based on responses emitted per shock re¬ than other forms of curve fitting, the response rates
ceived and percent shock reduction as a function of are plotted as reciprocals in Figure 8, using the data
maximum shock frequency observed on shock-delay from Figure 6. For both de Villiers’s data and those of
procedures^ both by Sidman and by Clark and Hull Clark and Hull, shock-frequency reduction was com¬
were not consistently obtained on this procedure. puted for each animal on each schedule by subtracting
the obtained shock rate from the shock rate that
Shock-Frequency Reduction as a would occur with no responding. Part A of Figure 8
Controlling Variable shows that this formulation describes the performance
of each of de Villiers’s four animals well; each plot is
While response-shock intervals, shock-shock inter¬ linear although the slopes differ for different animals.
vals, and maximum shock rate are straightforward as De Villiers tried substituting received shock in place
independent variables, none of these by itself ade¬
quately describes the consequence of responding when
k Vi
procedures are based on brief intermittent shocks. De i The expression involved here is: Ri--—, where Ri is
Ti + r0

Villiers addressed the problem of specifying the con¬ the rate of responding; n is the frequency of reinforcement
contingent on that response; k is the response rate if no alterna¬
sequence of responding with a single expression, and
tive responses were reinforced; and r0 is the sum of reinforce¬
at the same time provided evidence for functional ment frequencies contingent on responses other than that
similarity between positive and negative reinforce¬ described by Ri (whether or not these are measured by the ex¬
perimenter). If this equation is inverted it predicts a linear
ment. He also provided an additional means for com¬
relation between performance and reinforcement:
paring the effects of shock-delay and shock-deletion
procedures. De Villiers started with a relationship that
374 NEGATIVE REINFORCEMENT AND AVOIDANCE

of shock-frequency reduction, but found this produced pose shock-frequency reduction as a critical variable.
greater departures from linearity. Interestingly, re¬ Following this lead, Herrnstein and Hineline (1966)
sponses of two of Clark and Hull’s three animals show devised a procedure that permitted direct and inde¬
a comparable degree of linearity when plotted in the pendent manipulation of both maximum shock fre¬
same way, as shown in Part B of Figure 8. De Villiers quency and the amount of shock-frequency reduction
has argued that a linear relationship would not be that responding could produce. This procedure, dia¬
expected on fixed shock-delay procedures since short¬ grammed in Figure 9, was based on two independent
term effects of spaced responding would override the random schedules for delivering brief shocks. The
overall effects of reinforcement frequency. Perhaps the schedules were generated by independently sampling
equating of relative reinforcement rate to shock fre¬ two probability distributions once every two seconds.
quency reduction can be done more generally than Averaging over longer time periods, the frequency of
de Villiers proposed. scheduled shocks was roughly constant over time, but
Both dc Villiers and Clark and Hull varied shock- the two schedules could have differing frequencies. In
frequency reduction only by manipulating the maxi¬ the absence of responding, the schedule with higher
mum shock frequency which could occur in the ab¬ probability controlled the delivery of shock. A re¬
sence of responding. Their experiments implicate sponse transferred control to the schedule with lower
shock-frequency reduction mainly on the basis of its probability, where the control remained until that
fitting a particular function when plotted in relation schedule delivered a shock, at which time the control
to response rates on these procedures. Other studies transferred back to the schedule with higher probabil¬
have more directly indicated shock-frequency reduc¬ ity. Thus, one probability or the other was operative
tion as a controlling variable, by manipulating this depending on whether a response or a shock had
variable independently of the maximum shock fre¬ occurred last. On this procedure, a response could
quency. The first instance of this was accomplished reduce shock frequency by a specified amount, but it
inadvertently by Sidman (1962a). Rats were con¬ could not impose a shock-free period, for shocks could
currently exposed to two independent shock-delay and did occur immediately after some responses. This
schedules, using separate response levers. Each sched¬ procedure permitted direct experimental manipula¬
ule was controlled by a single timer that was reset by tion of response-contingent shock-frequency reduction.
responses on its appropriate lever, recycling as it de¬ Acquisition was reliably obtained even when the two
livered shock when no such responses had occurred. shock schedules had probabilities as close as 0.2 vs. 0.3,
Hence, considered separately, each schedule was a con¬ giving mean shock frequencies of six and nine shocks
ventional shock-delay procedure with equal RS and SS per minute. However, the Herrnstein and Hineline ex¬
intervals. However, the two timers delivered indis¬ periment was mainly a study of acquisition, and in¬
tinguishable shocks, so they combined to produce cluded only a few combinations of shock frequencies.
irregular shock sequences. Even when the two timers
were given equal settings the rats tended to respond
mostly on one lever, receiving frequent shocks from b 6 Sec -I
Tape Step J_I_L
the schedule that required responses on the other
lever. The rats7 choice not to distribute responses on Post-Shock l III_L
the two levers effected a lower limit to shock-fre¬ Post - Response JJ_
quency reduction. When two timers were given un¬ Shocks Delivered I III JJ_Ll_
equal settings, responding to the one with the greater Resp. Can Affect Sk. to jfi-L
interval would produce longer delays of individual
Fig. 9. Diagram showing the relations between features of a
shocks, but would result in relatively higher overall
procedure for response-contingent shock-frequency reduction.
shock frequencies due to the continual recycling of The punched tape advances at regular 2-sec intervals. Deflections
the other timer. Responding to the lever with the on the lines marked “post-shock” and “post-response” indicate
holes in these respective channels of the tape; probability of a
shorter timer would produce shorter delays of in¬
hole is constant for a given channel, but the probability can
dividual shocks, but if this responding was suffi¬ differ between channels. Shocks are shown as deflections on the
ciently frequent it would result in relatively lower line indicated. “R” indicates a response. The delivery of shock
coincides with the occurrence of a hole in the tape channel
overall shock frequencies. With these unequal set¬
currently in control, which correlates with whether a response
tings, the rats tended to respond exclusively on the can affect shock. Control is changed from one channel to the
lever that gave a shorter shock-delay per response, but other by a shock if a response has occurred since the last shock,
and by a response if a shock has occurred since the last response.
which permitted a greater decrease in overall shock
(From Herrnstein 8c Hineline, 1966. © 1966 by the Society for
frequency. This experiment then led Sidman to pro¬ the Experimental Analysis of Behavior, Inc.)
 Philip N. Hineline 375

permitting no detailed analysis of response rate as a independently of probability, determines the mini¬
function of shock-frequency reduction. More system¬ mum response rate needed to achieve a given degree
atic data would be of interest, especially in relation of shock-frequency reduction. On Herrnstein and
to the work of de Villiers described just above. Hineline’s procedure, the minimal shock frequency
Returning to schedule classification, Herrnstein could be achieved by a minimum of one response
and Hineline’s procedure resembles the shock-deletion after each shock. On the probabilistic fixed-cycle
procedure described earlier (Part I of Figure 6), in shock-deletion procedures just outlined, a minimum
that the time-base of the shock-delivery schedule pro¬ of one response per cycle is required to achieve the
ceeds independently of behavior. No clocks are reset minimal shock frequency, no matter when specific
as would occur in shock-delay procedures. Unlike the shocks have occurred.
earlier shock-deletion procedure, an effective response I noted earlier that shock density is an alternative
deletes shock only probabilistically, as determined by specification to shock frequency. Density differs from
the difference between the two random shock-delivery frequency, partly in emphasizing the spacing between
schedules. This procedure also differs from the earlier shocks rather than shocks per unit time, but also be¬
shock-deletion procedures in that a response reduces cause it can encompass the duration and intensity of
the probability of shock, not only until the end of the shocks as well as the frequency of shock onset. For
timing cycle when a shock is deleted, but until a shock example, if shocks were to occur every five seconds, a
is delivered by the low frequency schedule. The reduction in either the duration or the intensity of
amount of exposure to the low frequency shock sched¬ shocks would count as a density reduction even
ule is determined by that schedule as well as by the though shock frequency remained constant. Powell
amount of responding. and Peck (1969) demonstrated the potency of such a
If control were to revert to the high-frequency density reduction with a procedure that illustrates an
schedule at the end of a timing cycle irrespective of interesting hybrid between shock-delay and shock-
whether a shock had been deleted, the procedure deletion. Their schedule resembled shock-deletion in
would fit a descriptive system that has recently been that the time-based shock delivery schedule proceeded
gaining some currency. This formulation has been independently of behavior. However, the effects of re¬
described by Church (1969), Catania (1971), Seligman, sponding were metered through a response-shock in¬
Maier, and Solomon (1971), and by Gibbon, Berry¬ terval identical to that of a shock-delay schedule, as
man, and Thompson (1974), as a means for defining follows: Brief shocks were delivered every five sec¬
the degree of contingent relations between responding onds. Responses started a 20-second timer with the
and rewarding stimuli or noxious stimuli. Neffinger same contingent relationship as for Sidman’s delay
and Gibbon (1975) have used the formulation to gen¬ procedure. However, instead of delaying shocks, start¬
erate negative reinforcement procedures with added ing this timer reduced the intensity of the one-per-five-
cues. It could also be used to increase the generality second shocks. So long as 20 seconds did not elapse
of uncued shock-deletion procedures that have been without a response, all shocks were delivered at the
described above (Part I of Figure 6) and enable these, lower intensity. Powell and Peck found acquisition
like Herrnstein and Hineline’s procedure, to manipu¬ with this procedure to be more rapidly and reliably
late shock frequency reduction independently of abso¬ achieved than with a standard shock-delay procedure
lute shock frequency. The formulation simply is this: with SS = 5 and RS = 20.
in any given time period, probability of shock, given
a response, can be manipulated independently of the
Schedules of Negative Reinforcement
probability of shock given no response in that time
Based On Intermittent Shocks
period. On the standard, fixed-cycle shock-deletion
procedure (Part I of Figure 6), the probability of The shock-delay procedures considered so far have
shock given no response is 1.0; the probability of provided for every occurrence of the response to be
shock given a response is zero. This need not be the effective. As will be seen below, schedules of reinforce¬
case; either probability can be manipedated between ment are easily achieved by imposing a ratio require¬
these values. So long as the probability of shock given ment or by limiting the access to reinforcement. On
a response is smaller than the probability given no the shock-deletion procedures already described, there
response, negative reinforcement can occur. The pos¬ have been intervals when responses were ineffective.
sible shock frequencies are determined jointly by However, these were periods of reinforcement, when
probabilities per cycle, and cycle length. Cycle length impending shock had already been cancelled or re¬
sets the maximum shock frequency. Also, cycle length, duced in probability. Disabling the response lever at
376 NEGATIVE REINFORCEMENT AND AVOIDANCE

these times is analogous to disabling the lever in the shocks to occur immediately after some responses—late
presence of a food reinforcer in appetitive procedures. in the cycle when no response had occurred during the
Schedules of reinforcement by shock-deletion require access period—or simply to a decrement in shock-
modifying response effectiveness at other times as well. frequency reduction.
Hurwitz and Millenson (1961) did this, examining More recently, Kadden, Schoenfeld, and Snapper
the maintenance of responding while they systemati¬ (1974) studied tA — tD schedules of shock-deletion in
cally varied the access to reinforcement in the face of which the probability of shock with a response in tA,
uncancelled shocks. Their procedure, diagrammed in and the probability of shock if no response occurred
Part III of Figure 7, was conceived within a formula¬ during tD, were manipulated independently. They
tion developed by Schoenfeld and his associates used Rhesus monkeys as subjects, and after initial
(Schoenfeld and Cole, 1972). The formulation uses shaping of the response with removal of continuous
two time periods, designated tA and tD, which alter¬ shock, the sum of tA + tD was always 60 seconds. They
nate continually to produce fixed cycles. Positive rein¬ established stable response rates with a series of tem¬
forcement on this schedule is customarily delivered poral combinations (tD = 45 sec, then 30 sec, then 6
immediately upon occurrence of the first response in sec) in which probability of shock was 1 given no
Fk Additional responses during are ineffective, as response, and zero if a response occurred. These fea¬
tho§£ during t^. In the negative reinforcement tures resemble the procedures by Hurwitz and Millen¬
case arranged by Hurwitz and Millenson, the first son, and by Sidman, described just above. Kadden et
response during to deleted a shock that was scheduled al. then independently varied the probability of shock
to occur at the end of tD. With tA equal to zero, as was
the case during initial training in Hurwitz and Mil-
lenson s £vperim£nt; this procedure is identical to the
simple fixed-cycle shock-deletion procedure dia¬
grammed in Part I of Figure 7. As tA is increased to
values greater than zero7 the procedure becomes one
in which the shock-deletion periods are alternated
with periods during which responding cannot affect
the impending shock. Hurwitz and Millenson held
the sum of tA and tD constant at 30 seconds, and
systematically increased tA, changing its value every
few sessions. The resulting response rates and shock
rates are shown in Figure 10, each giving a systematic
relation between response rate and the relative time
that reinforcement was accessible. When most of the
cycle was spent in tD, response rates were low, and
many shocks were received. The response rate func¬
tion obtained by Hurwitz and Millenson resembles a
comparable function obtained by Hearst (1960) with
analogous manipulations of a tA — tD schedule based
on positive reinforcement. The two curves differ in
the location of maximum rate, but not in their gen¬
eral shapes.
Sidman (1962b) reported virtually identical results
with the same procedure, using a 15-second cycle.
However, when he departed from the usual tA — tD
procedures and moved the access period to the middle
of the cycle, responses continued to be most probable
Fig. 10. (A) Number of shocks received per session; (B) lever-
near the end of the cycle, where shock was due. Suffi¬
press response rates for four rats as a function of the temporal
cient mid-cycle responses occurred to produce a few
schedule parameter T. T is defined as tD/(tD + tA) and repre¬
shock deletions, but response rates soon dropped to sents here the relative portion of a 80-second shock-shock
near zero. The eventual decrement in overall per¬ interval in the period during which the first response resulted
in deletion of the next shock due. (From Hurwitz and Millenson,
formance can be attributed either to the fact that
1961. © 1961 by the American Association for the Advancement
placing access to reinforcement at mid-cycle permitted of Science.)
Philip N. Hineline 377

given a response and the probability of shock given Delivery of the shock started this cycle over. On Sid-
no response, presenting the values in different orders man’s procedure responding could change the cycle
for different monkeys. Responding always ceased when length. Each response in the “effective” period (cor¬
the probability of shock was greater if a response oc¬ responding to tD on Hurwitz and Millenson’s pro¬
curred than if no response occurred. Also, responding cedure) delayed the shock and extended that period;
ceased or dropped to very low levels when the prob¬ response-shock intervals were in effect until respond¬
ability of shock given no response was reduced, pro¬ ing lapsed and shock occurred, reinstating the “tA”
vided that the probability of shock given a response period. Using a tA of 60 sec and a response-shock inter¬
was greater than zero. Responding was more persistent val of 6 sec, Sidman observed an acceleration of re¬
when the probability of shock given a response was sponse rate within the tA period. This pattern, which
kept at zero; the monkeys seldom paused long enough is characteristic of fixed-interval schedules was also
to encounter the new consequences of not responding. observed in the experiment by Kadden et al. (1974),
Kadden, Schoenfeld, and Snapper’s procedure fits described above. The pattern indicates a degree of
the contingency analysis of probabilistic fixed-cycle discrimination of nonreinforcement during the tA pe¬
shock-deletion schedules that I noted earlier (p. 375). riod. Hurwitz and Millenson did not find systematic
However, the insertion of large tA periods into the changes in response rate within the tA periods. This
cycle makes the procedure an interval schedule of difference of results is attributable to the fact that on
reinforcement as well as a probabilistic contingency Sidman’s procedure, each tA period began with a
manipulation. Indeed, the patterns of responding shock, providing a discriminable cue that was not
within cycles revealed fixed-interval scheduling effects consistently present on Hurwitz and Millenson’s pro¬
when shock deliveries permitted response patterns to cedure. The experiment by Kadden et al. confirmed
stay in synchrony with the timing cycle. Thus, when that cuing function of shock.
probability of shock given a response exceeded zero, Sidman’s fixed-interval procedure and the tA — tD
but was not high enough to eliminate responding, re¬ procedure of Kadden et al. are not entirely com¬
sponse rates increased as time for the tD period (as parable to fixed-interval positive reinforcement pro¬
well as the possibility of another shock) approached. cedures, for the periods when responses cannot affect
Shock-delay procedures have been adapted to pro¬ shock are also periods when no shocks can occur. The
duce schedules even more closely resembling the tradi¬ basis for reinforcement is missing, along with the ac¬
tional basic positive reinforcement schedules. Ver- cess to reinforcement. In positive reinforcement pro¬
have (1959) imposed ratio schedules on a shock-delay cedures the basis for reinforcement is continually
procedure. He pretrained the animals with conven¬ present. Hence, the negative reinforcement procedures
tional shock-delay procedures, starting with SS = 3 would have been more comparable to positive rein¬
and RS = 30 sec, and then moving to SS = RS = 30 forcement schedules if shocks were delivered during
sec. He then imposed ratio requirements so that more the tA periods. Similar issues arise in defining the ex¬
than one response was required during a given RS or tinction of negatively reinforced behavior.
SS interval, to delay the next shock. The response
requirement was increased from two up to eight. Re¬
Extinction After Negative Reinforcement
sponse rates increased concomitantly over those ob¬
tained when single responses could delay shock. Then, Extinction is the discontinuation of reinforcement,
holding the ratio constant at eight, Verhave varied with continued opportunity to respond. One expects
the RS interval, and found a functional relation simi¬ that responding will return to levels that occurred
lar to those obtained in experiments by Sidman (1953) prior to conditioning. By convention, a basic or “refer¬
and by Clark and Hull (1966) where every response ence” extinction procedure implies a situation in
could delay shock (see Figure 6 above). Of course the which prior conditioning occurred, unchanged ex¬
absolute response rates were substantially higher than cept for the withholding of reinforcement. Depending
in these studies with a ratio of one. on its prior scheduling, the absence of reinforcement
Sidman (1966) reported a shock-delay experiment may or may not produce a situation strikingly differ¬
that he called fixed-interval avoidance. ‘In the absence ent from that during prior conditioning. While this
of responding, the procedure resembled the tA — tD degree of difference most likely will affect how quickly
procedure of Hurwitz and Millenson already de¬ the process of extinction occurs, the degree of differ¬
scribed. A fixed interval when responses could not ence is irrelevant to the definition of an extinction
affect shock (tA) was followed by a fixed period in procedure or process. Extinction, whether quick or
which responding could affect an impending shock. slow, is still extinction.
378 NEGATIVE REINFORCEMENT AND AVOIDANCE

The experiments described below will illustrate the Some animals’ responding also declined during the
following points regarding extinction after negative conditioning periods, indicating overall extinction,
reinforcement: Two different types of extinction pro¬ but other animals continued to respond during the
cedures have been proposed. The more traditional conditioning periods. In these latter cases the periods
one simply involves the discontinuation of shock. when the shock generator was off clearly were dis¬
However, it has been recently argued that this re¬ criminated from the periods when it was on, even
moves reinforcement only indirectly, through suspen¬ though no added exteroceptive stimuli were supplied.
sion of the drive operation upon which reinforcement Boren and Sidman suggested the occurrence of un¬
was based. By this view, discontinuing all shocks is shocked nonavoidance could account for their results,
analogous to providing food continuously during ex¬ but the experiment does not permit one to distinguish
tinction of food-reinforced responding. An alternative this from a discrimination based directly on the
type of extinction procedure involves continued pres¬ shocks themselves.
entation of shocks while eliminating the effects of The elimination of all shock produces confounded
responses on shock. Proponents of this as a reference changes in two of the three basic features I have used
procedure argue that discriminability of the extinc¬ for analyzing negative reinforcement procedures. The
tion situation is an issue in defining the extinction of aversive situation is removed, and access to reinforce¬
negative reinforcement, especially if one focuses on ment is therefore modified indirectly. This is espe¬
the contingent relation between responding and rein¬ cially obvious in relation to escape procedures, where
forcement rather than merely the delivery or nonde¬ conditioning is based on removal of continuous shock.
livery of reinforcement. Finally, the fact that delivery Reinforcement of a particular response is impossible
of free or noncontingent shocks may induce respond¬ if the reinforcer (absence of shock) is continuously
ing in a way inconsistent with usual notions of nega¬ present. In principle, the confounding is no different
tive reinforcement, further complicates the definitions for procedures based on intermittent shocks, although
and interpretations of extinction of negatively rein¬ special procedures are required to determine whether
forced behavior. a reduction in responding reflects discrimination of
the absence of shock or of the absence of contingent
Alternative Procedures
relations between responses and shock.
for Extinction
Davenport, Coger, and Spector (1970), following a
related paper by Davenport and Olson (1968), stated
To extinguish responses that were conditioned with the implications of eliminating all shocks. They ar¬
shock-delay procedures in which responding could re¬ gued that removal of all shocks is either suspension of
duce the shock frequency to zero, some experimenters the drive operation, or else it constitutes reinforce¬
have simply deactivated the shock generator. Thus, for ment of all behavior since whatever the organism does
example, Shnidman (1968) eliminated all shocks after is followed by shock omission, a consequence that dur¬
rats had been trained with a few four-hour sessions of ing prior conditioning was restricted to the avoidance
shock-delay with SS = 5 sec and with RS = 20 or 40 response. To provide an alternative extinction pro¬
each rat being exposed to both values, The shock cedure Davenport, Coger, and Spector trained rats
generator was deactivated three hours after the begin¬ with 10 hours of exposure to a standard shock-delay
ning of a session that continued beyond its usual procedure (SS = 15, RS = 15). Then they delivered a
4-hour limit, if necessary until response rate declined to shock every 15 sec irrespective of behavior, start¬
zero for 15 min. Response rates dropped to zero within ing this extinction procedure an hour after a session
an hour for two of three rats, and within two hours had begun. Four of the five animals virtually ceased
for the third. Using a similar extinction procedure, responding within 90 min of the noncontingent shock
Boren and Sidman (1957) found greater persistence of procedure: the fifth had responded primarily immedi¬
responding. After first subjecting their rats to 100 or ately after shocks during conditioning (reducing its
more hours of initial training with SS and RS inter¬ shock frequency by only 6 percent), and simply con¬
vals of 20 sec, they alternated periods of conditioning tinued in this pattern of post-shock responding. Thus,
and extinction. The 6-hour sessions were divided into extinction appeared to be achieved with suspension of
two periods, with normal conditioning for the first the negative reinforcement contingency while main¬
part of each session, and with the shock generator off taining the basis for negative reinforcement. In prin¬
during the second part of each session. Responding ciple, reinforcement could have been delivered at any
during extinction declined in an orderly fashion over moment.
a few sessions, but in most cases did not reach zero. This procedure did not deal with the implications
Philip N. Hineline 379

of subjects avoiding more or less well during condi¬ various “extinction” procedures lasted five 100-min
tioning. Some people, having taken this into account, sessions each, and retraining to the same stability cri¬
focus the definition of extinction on the contingency teria was carried out after each exposure to nonrein¬
between response and reinforcement, rather than sim¬ forcement. All animals were exposed to the shock-
ply on the discontinuation of reinforcement (analo¬ omission procedures before exposure to the other two
gous considerations also apply to positive reinforce¬ procedures. Then after retraining, some rats were
ment.) To the extent that in the animal’s history given the “maximum shock frequency” of noncontin¬
shocks have set the occasion for reinforcement, the gent shock, while others were exposed to the “matched-
elimination of shocks during extinction is as much shock” procedure. After retraining, each rat was then
a change of cues as it is a discontinuation of the exposed to the other noncontingent shock procedure,
relation between responding and reinforcement. On and then retrained and reversed again. Figure 11
the other hand, to the extent that the subject’s prior shows performance of a representative animal from
performance has completely eliminated shocks during each of the two sequences of procedures.
training, the reintroduction of maximal shock fre¬ Even with prior avoidance training to a stringent
quency during extinction produces a situation in criterion, turning off the shock generator produced
extinction that little resembles recent sessions of precipitous declines in responding. In contrast, re¬
conditioning. Coulson, Coulson, and Gardner (1970) sponding persisted throughout five-day exposures to
pointed this out, suggesting that in this context a extinction procedures based on noncontingent deliv¬
proper extinction procedure would be delivery of a ery of shocks. On the matched-shock procedure all
pattern of shocks similar to that observed during the animals showed a progressive decline in responding
training sessions that just preceded extinction. They over the five-session exposure, indicating an extinc¬
carried this out after pretraining with a shock-delay tion process and suggesting that more extended ex¬
procedure (SS = 5, RS = 30), They recorded the exact posure to this procedure would have resulted in low
sequence of shocks received by each animal during its levels if not complete cessation of responding. The
final conditioning session and subsequently delivered results of noncontingent shock delivered according to
the same pattern to produce an extinction procedure the fixed shock-shock interval were more variable, usu-
based on noncontingent shocks, which they compared
to extinction with all shocks deleted. In the latter
condition, responding dropped quickly to zero; with
the “matched shock” extinction procedure responding
declined over some 6 to 15 sessions, but typically not
to zero.
Although they reported a somewhat slower decline
in response rate for extinction with noncontingent
shocks matching the shocks of avoidance conditioning,
the results of Coulson, Coulson, and Gardner (1970)
are not directly comparable to those obtained by
Davenport, Coger, and Spector (1970). The two stud¬
ies used different conditioning parameters, and differ¬
ent amounts of training prior to extinction. A more
direct comparison of these procedures has been
achieved in my laboratory by G. D. Smith (Smith,
1973). He first trained rats with a shock-delay pro¬
cedure (SS = 5, RS = 20) until they met a fairly
rigorous performance criterion: over 75% of sched¬
CONSECUTIVE SESSIONS
uled shocks avoided, and stable responding assessed
over a 2-week period. All animals were then exposed
Fig. 11. Responses and shocks per minute as a function of
to three different procedures that eliminated negative sessions for a representative rat from each of two procedural
reinforcement: shock omission; delivery of noncontin¬ sequences. Responses and shocks are indicated by separate
symbols as noted in the Figure. Each data point represents the
gent shock every five seconds, which matched the
mean rate for a single session. Transitions from one experi¬
shock-shock interval of the previous training program; mental procedure to another are indicated by dashed lines
and a pattern of shocks that matched the given an¬ drawn through individual plots. The order in which individual
rats were exposed to extinction procedures is also indicated in
imal’s final conditioning session. Exposures to the the individual graphs. (From G. D. Smith, 1973.)
380 NEGATIVE REINFORCEMENT AND AVOIDANCE

ally showing a persistent high rate of responding, but delay procedures, and found that periodic noncontin¬
occasionally showing low consistent rates. Recondi¬ gent shock produced responding in 8 of 13 rats, some
tioning, while accomplished after exposure to each persisting for as long as 180 hours of exposure to a
extinction procedure, was most disrupted after the given procedure. Aperiodic shocks produced some¬
matched-shock procedure, but virtually immediate af¬ what less persistent responding, a result consistent
ter the shock-omission procedure. Stressing these lat¬ with observations by Kadden (1973) using rhesus mon¬
ter results, Smith (1973) argued for match-shock as keys. According to Powell, the temporal relations of
the “reference” extinction procedure, and agreed with responses to shock in his study indicated that much of
Davenport et al. (1970) and Coulson et al. (1970) that the responding was shock-elicited, presumably reflect¬
the traditional procedure of omitting all shocks con¬ ing attacks on the lever. However, even though it has
stitutes a suspension of the drive operation, and is a reflex-like time relation to the noncontingent shock,
only indirectly a removal of the negative reinforce¬ this responding appears to be related to the prior
ment contingency. avoidance conditioning. When shock-delay training
The resulting treatment of extinction still has to on a second lever was interposed before the noncon¬
deal with the fact that the responding of different sub¬ tingent shock procedure was begun, the responding
jects is extinguished in the presence of different abso¬ that persisted during the subsequent noncontingent
lute shock rates corresponding to, and thus confounded shock procedure occurred almost entirely on the lever
with the effectiveness of prior conditioning. first used for shock reduction. This suggests that the
shock was functioning both as a discriminative and as
Effects of NvttCONTmOENT SHOCKS
an eliciting stimulus.
An analysis by Hake and Campbell (1972) of re¬
While the reasons for using noncontingent shock sponding produced by noncontingent shocks compli¬
as the basis for extinction are quite compelling, the cates the picture still further. Using a fixed-interval
persistence of responding on gudh. extinction pro¬ escape procedure with squirrel monkeys, they ob¬
cedures complicates the analysis of behavior in an¬ served two patterns of lever-press responding. One
imals with histories of negative reinforcement based pattern, post-shock responding, suggested elicitation;
on shock. Such persistence is well illustrated in rats the other showed characteristics of maintenance by
trained on the shock-frequency reduction procedure the fixed interval schedule. When they changed the
based on randomly spaced shocks, that I have already situation, making two operanda available—a key
described (Herrnstein 8c Hineline, 1966). While ex¬ whose pressing was reinforced on the FI schedule and
tinction was indeed achieved by delivering noncon¬ a hose conveniently available for biting—they found
tingent §hock§7 it often took many sessions. For ex¬ that the post-shock responding was confined to the
ample, one animal emitted §<?me 20,060 responses at a bite hose, while the FI responding occurred on the
dowly-deereasing rate over some 12,000 min accu¬ key. So far, this appears to be a nice test of Powell’s
mulated in 100-min sessions. In light of this, it is diffi¬ (1972) notion that the post-shock responding differed
cult to know whether experiments presented as functionally from other responding, being shock-
“maintenance of responding by noncontingent shocks” elicited rather than an unextinguished responding
are indeed what the description implies, or whether based on negative reinforcement. However, when Hake
the effects so reported are merely modulations of and Campbell made the bite hose unavailable, post¬
responding that is undergoing extinction. For exam¬ shock responding was then observed on the key.
ple, Hurwitz, Roberts, and Greenway (1972) drew Clearly, this was not a biting response; they still clas¬
conclusions regarding the response-maintaining effects sified it as shock-elicited aggressive responding, but
of response-independent shocks on the basis of only displaced to whatever manipulandum was available.
one hour on the noncontingent shock procedure, Finally, McKearney has shown that in animals with
after more than 200 hours of preliminary training. negative reinforcement histories one can readily main¬
Other studies have been carried out further, with tain responding even with response-produced shocks
a variety of interesting results. For example, Powell provided that the probability of a response producing
and Peck (1969) found that after training with shock- shock is not too high. High-probability response-pro¬
intensity reduction, noncontingent shocks maintained duced shock suppresses responding (McKearney, 1972;
subsequent responding even when it was varied over a Powell, 1972). Thus, clearly, the effects of shock de¬
fairly wide range of intensities and frequencies. Powell livered to animals with histories of negative reinforce¬
(1972) examined effects of noncontingent shocks on ment demand further analysis and will receive further
responding after more conventional Sidman shock- experimentation (cf. chapter 7 of this volume by
Philip N. Hineline 381

Morse Sc Kelleher). For present purposes, it suffices to stein and Hineline (1966), which I have described
note that long after it has been discontinued, shock previously. In the absence of responding, randomly
reduction can affect an animal’s responses to shock. spaced shocks were delivered at a mean inter-shock
An experimenter purporting to demonstrate negative time of 5.7 sec. A response resulted in another random
reinforcing effects in animals with histories of prior distribution of shocks, still giving a fairly constant
negative reinforcement must acknowledge the possible moment-to-moment probability of shock, but with an
effects of shock delivery, which result from prior as average of 20 sec until the next shock. With delivery
well as current response-contingent relationships. of this shock, the higher shock frequency was rein¬
stated. These 100-min sessions of initial training were
accomplished in a darkened chamber. Next a light was
NEGATIVE REINFORCEMENT provided as a discriminative stimulus (SD), accom¬
WITH ADDED CUES panying the same procedure as before. This light was
present for four-min periods, alternating with periods
So far, I have focused on procedures where shock of darkness during which shocks were delivered ac¬
is the only exteroceptive stimulus manipulated within cording to the high-frequency shock schedule, inde¬
conditioning sessions. In some of these procedures the pendently of any responding (Darkness — SA). The
addition of other cues would be superfluous, for the duration of the SA period was determined by a delay
shock itself denotes the opportunity to respond and contingency; a return to light and its correlated access
the availability of reinforcement, as well as producing to reinforcement could not occur until 20 sec had
the aversive situation. However, if shocks are brief elapsed without a response. Later in training this
and intermittent, if not all responses in the presence delay requirement was extended to 40 and then to 60
of shock are effective, or if responding in the absence sec for some animals. Finally, two of the rats were
of shock can affect subsequent shock delivery, then placed on an extinction procedure in which the high
added stimuli such as tones or lights become of inter¬ shock frequency occurred independently of respond¬
est. The germinal idea for the present discussion of ing, and whether or not the light was on.
added stimuli comes from Keehn (1966), from Sidman Figure 12 shows a representative performance on
(1966), and from Herrnstein (1969). They have sug¬ this procedure. Response rates in darkness came un¬
gested or argued that such added stimuli should be der control of nonreinforcement, dropping far below
considered as discriminative stimuli denoting the those in the presence of the light. These reduced
availability of negative reinforcement. In addition, rates developed systematically, and often fairly
Baum (1973a) has provided a key concept, identifying quickly, even though the shock frequency in darkness
situation transitions as reinforcements. I attempt to go was substantially higher than that in light, and dark¬
beyond these in distinguishing several interrelated ness had accompanied the initial conditioning. Dur¬
functions of added cues in negative reinforcement situa¬ ing extinction of the discriminative responding, with
tions. I shall start with multiple schedule experiments noncontingent shocks delivered on the average of
where cues correlate with whole conditioning pro¬ every 5.7 sec regardless of responding or stimulus con¬
cedures, including aversive situations, accessibility of dition, the response rates slowly moved together as
reinforcement, and periods of shock reduction pro¬ responding decreased in the presence of the light. The
duced on a given procedure. Later will come cues de¬ plots for initial conditioning and for the discrimina¬
noting more limited aspects of negative reinforcement tion are fairly representative for all animals. However,
procedures, such as particular shock-presentation the data for the final extinction procedure is not rep¬
schedules, opportunity to respond, and limited peri¬ resentative, for the other animal exposed to this pro¬
ods when responding can affect shock. cedure persisted with high response rates in the pres¬
ence of the light, and low response rates in its absence.
The persistence of the discrimination under these
Added Cues Denoting Multiple Contingencies
circumstances is somewhat surprising, given the selec¬
A simple multiple schedule can be achieved by tive extinction that produced the discrimination in
correlating a reinforcement procedure with one stim¬ the first place. It is less surprising in comparison with
ulus, correlating extinction with another, and alter¬ another experiment that used noncontingent shocks
nating the two stimuli. This was accomplished by after histories of negative reinforcement. Appel (1960)
Bersh and Lambert (1975). They initially conditioned placed rats on a discrimination procedure similar to
lever-press responding in rats, with 32 sessions of ex¬ that of Bersh and Lambert but with negative rein¬
posure to a procedure of the kind devised by Herrn¬ forcement provided by a shock-delay procedure. His
382 NEGATIVE REINFORCEMENT AND AVOIDANCE

Fig. 12. Responses per minute

for Rat 3 during initial condi¬
tioning, with response-contin¬
gent shock-reduction in dark¬
ness; discrimination training,
with response-contingent shock-
reduction in light and noncon¬
tingent shock in darkness; and
extinction, with noncontingent
shock in both light and dark¬
ness. Data points connected
with broken lines indicate re¬
sponding in darkness; data
points connected with solid lines
indicate responding in light.
(After Bersh & Lambert, 1975. ©
1975 by the Society for the Ex¬
perimental Analysis of Behavior,
SESSIONS Inc.)

animals failed t6 show discriminative control when revealed by response rate changes in the opposite
the extinction component included delivery of non- direction in the presence of the alternate stimulus
contingent shocks. Systematic comparisons would be where the schedule had remained unchanged. De Vil¬
needed tq identify critical features for establishing liers showed that these contrast effects could be sum¬
this type of discrimination. marized by an equation that Herrnstein (1970) had
The multiple schedules studied by Bersh and Lam¬ developed to deal with behavioral contrast in multi¬
bert and that studied by Appel involved reinforce¬ ple schedules of positive reinforcement.
ment in one component and nonreinforcement with Extending1 the study of stimulus control, de Villiers
delivery of noncontingent shocks in the other. De Vil- (1974) examined responding on similar multiple
liers (1972; 1974} has studied multiple schedules with schedules, but used separate levers and cue lights for
shock deletion procedures based on different shock the different component schedules. This permitted
frequencies in the different components. In both of further examination of shock-frequency reduction as
de Villiers's experiments the basic procedures were a controlling variable. He also varied the frequency
the same as in his experiment described above. In the with which the components of the multiple schedule
absence of responding, shocks were delivered at ir¬ alternated within a session. This latter manipulation
regular intervals, with a roughly constant probability permitted a detailed comparison with multiple sched¬
of shock from moment-to-moment. He specified these ules of positive reinforcement. With positive rein¬
in terms of the mean interval between shocks. The forcement it has been found (e.g., Todorov, 1972) that
first response within any scheduled shock-shock inter¬ rapid alternation between components of a multiple
val cancelled the shock due at the end of that inter¬ schedule produces relative response rates that are
val; additional responses within that interval had no nearly proportional to their relative rates of rein¬
effect. In his earlier experiment, de Villiers (1972) forcement. Also with positive reinforcement, less fre¬
demonstrated contrast effects in addition to basic mul¬ quent alternation between components produces
tiple schedule effects. He first provided identical smaller differences between response rates on the
shock-deletion schedules in the presence and absence different components. De Villiers (1974) found very
of a buzzer that alternated, on for three min and off similar relationships for multiple schedules of nega¬
for three min. Then the schedule was changed in one tive reinforcement, with amount of shock-frequency
component, providing for greater or less shock- reduction corresponding to rate of reinforcement.
frequency reduction by increasing or decreasing the This is shown in Figure 13, where different panels
maximum shock frequency. The response rate changed (bounded by vertical lines) correspond to different
in that component, in a direction directly related to pairs of component schedules in the multiple sched¬
the change in shock-frequency reduction. Thus, if ule. The horizontal line in each panel of the figure
shock frequency increased, so did responding (and the indicates the relative shock-frequency reduction pro¬
amount of shock reduction). The contrast effect was duced on the right lever, averaged over exposures to
Philip N. Hineline 383

MULTIPLE MULTIPLE MULTIPLE MULTIPLE

VI 15-SEC VI 15-SEC VI 20-SEC VI40-SEC VI 40-SEC VI 20-SEC VI45-SEC VI 15-SEC
sponding to reduce shock frequency to zero, he could
manipulate shock frequency reduction only by vary¬
ing the “background” or maximum shock frequency,
that which would occur only in the absence of re¬
sponding. A more powerful test of shock-frequency
reduction would be to hold background shock fre¬
quency constant and manipulate the minimum shock
frequency produceable by responding, including other
values as well as zero.
Wertheim (1965) also studied multiple schedules
of negative reinforcement, but used shock-delay pro¬
cedures. In addition to demonstrating potent stimulus
control of responding, and contrast effects, he found
that the relative response rate was correlated quite
strongly with relative frequencies of shock delivery.
However, the use of his procedures for a comparison
Fig. 13. Mean relative response rates of each rat for the last of shock frequency with shock-frequency reduction is
six sessions of each experimental condition. The mean relative
problematical since he used shock-shock intervals of
shock-frequency reduction for each multiple schedule is shown
by the horizontal solid lines. The data points of differing shapes zero, giving continuous shock in the absence of re¬
designate performances of different rats. (From de Villiers, sponding.
1974.)

Added Cues and the Escape Paradigm

a given pair of values. Shock-frequency reduction was
computed by subtracting the obtained shock fre¬ My discussion of negative reinforcement procedures
quency from the frequency that would have occurred began with the escape paradigm, and evolved away
in the absence of responding. For each schedule com¬ from it. A response in the presence of continuous
ponent, this measure of “r” was substituted into the shock removes that shock, producing a shock-free in¬
rr terval. Negative reinforcement is clearly the offset of
equation,-where rr and r± are reinforcement
rr 4- r± the shock; specifying the shock-free interval is no
rates on the right and left components, respectively. problem. However, there is a continuum between
When the component schedules were alternated every continuous shock and sequences of intermittent brief
40 sec, the relative response rates closely matched the shocks. The escape paradigm providing for response-
relative amounts of shock-frequency reduction. When produced shock-free intervals is effective across much
the component schedules were alternated more slowly, of this continuum, but it is not clear how to specify
response rates were more nearly equal in the two the effective consequence when the duration of the
components, giving relative values closer to 0.5. This interval between brief shocks approaches that of the
effect was similar to one seen in pigeons run on simi¬ response-produced shock-free interval. The analysis
lar schedules of positive reinforcement, as noted above. confronts a “figure-ground” problem: A response can
I noted earlier that de Villiers found, when apply¬ be considered as deleting or delaying a specific shock,
ing Herrnstein’s (1970) relative reinforcement fre¬ or the situation can be treated as one and continuous,
quency equations to responding on single variable- with responses reducing the overall aversiveness of the
interval schedules, that good fits to the matching situation. The experimental procedure may specify
relation were obtained when reinforcement rate was shock-deletion or shock-delay, but the effective vari¬
defined as degree of shock-frequency reduction. Ob¬ able may well be shock-frequency reduction.
tained shock frequency also provided a fair approx¬ When cues are used in multiple schedules such as
imation to the data (as it should, being correlated those just considered, an added stimulus is correlated
with magnitude of shock frequency reduction), but with a whole procedure. It accompanies not only a
shock-frequency reduction provided a better fit in schedule of shock presentation and the contingency or
every case. On the multiple schedules considered here, schedule of reinforcement whereby responding can
relative shock-frequency reduction was more simply affect the shock presentation, but also the periods of
related to relative response rate than was received shock reduction that responding produces. The cue
shock rate. I also noted earlier that since de Villiers’s does not simplify interpretation of the procedure and
shock-frequency reduction procedures allowed re¬ its effects; the cue merely delimits the periods during
384
NEGATIVE REINFORCEMENT AND AVOIDANCE

which the procedure operates. If, however, the cue plex being the combination of intermittent shocks
correlates only with particular features of a condi¬ and correlated stimulus. They found evidence for
tioning procedure, it does enter into the interpreta¬ direct effects of shocks. When developing fixed-inter¬
tion of the procedure’s effects. val schedules of negative reinforcement, they found
The first examples to be considered here are experi¬ that when intermittent shocks were delivered through¬
ments in which the cue correlates perfectly with the out the interreinforcement interval the positive-accel¬
shock-presentation schedule. Onset and offset of the erating rates commonly identified with this schedule
cue provide distinct boundaries to a sequence of (“fixed-interval scallop”) failed to appear. They ob¬
intermittent shocks, permitting interpretation to rely tained the characteristic fixed-interval performance
again on the escape paradigm of negative reinforce¬ only by allowing the interval to elapse before the
ment. Dinsmoor and Bonbright (reported in Dins- intermittent shock began, as described above in the
moor, 1967) directly compared such a procedure to a procedure for Figure 3. It is likely that the direct
standard escape procedure that used continuous shock. effects of intermittent shocks delivered during the
They used a multiple schedule, consisting of white interval are akin to the effects of noncontingent shocks
noise accompanied by intermittent brief shocks, alter¬ delivered during extinction after conditioning with
nating with continuous shock of lower intensity in shock-delay procedures (e.g., McKearney, 1969).
the absence of the white noise. In both situations, the Dinsmoor (1965) systematically examined some
same variable-interval schedule was in effect, with the direct effects of shocks and shock-deletion, correlated
removal of the continuous shock, or of noise plus with a cue and cue-removal, tie compared rats7 per¬
intermittent shock, contingent upon lever-pressing. By formances on two procedures, both of which could be
independently adjusting the shock intensities in the called VI 30-sec schedules of negative reinforcement.
two components of the multiple schedule, they pro¬ Both schedules of reinforcement were superimposed
duce virtually indistinguishable patterns of lever¬ on identical schedules of intermittent shock, specified
pressing in the two components. These performances by their mean shock-shock intervals. In both proce¬
were identically affected by administration of chlor- dures responding could produce fixed shock-free
promaime at various doses. Thus, in this situation, periods during which any scheduled shocks were
behavior that removed continuous shock was equiva¬ deleted. Responses during these shock-free periods had
lent to that which removed intermittent shocks and no effect.2 In one procedure an added light and tone
correlated white noise. This should come as no great denoted the presence of the schedule of intermittent
surprise after the results of Kelleher and Morse (1964) shocks and the shock-free periods, respectively. In the
who compared the effects of drug administrations on other procedure, no added cues were provided. Initial
closely similar food-maintained and shock-maintained training was accomplished with fixed shock-deletion
lever-press performances. As described at the begin¬ periods of 60 sec, and with mean shock-shock intervals
ning of this chapter, they found that drug effects were varying from day to day, between 7.5 and 120 sec. The
determined more by the specific patterns of behavior duration of shock-free periods was also systematically
than by the types of reinforcement maintaining those manipulated over sessions, between 15 and 240 sec for
patterns. one animal, and between 30 and 240 sec for the other
Procedures using cues correlated with intermittent twTo. d hus, in some sessions a reinforced response
shocks have often been characterized as “reinforce¬ could produce a shock-free period that exceeded the
ment by removal of a stimulus paired with shock/’ average shock-shock interval; in other sessions it could
(e.g,, Azrin, Holz, Hake, and Ayllon, 1963; Dinsmoor, not. In some sessions the correlated lights and tones
1967; Kelleher and Morse, 1964), which of course de¬ were used. In other sessions there were no exterocep¬
scribes the immediate response consequence. However, tive cues other than the shocks themselves. In all ses¬
this characterization implies that on these procedures, sions a variable interval schedule provided a mean
the effects of shock on responding are entirely me¬ interreinforcement time of 30 sec if the animal re¬
diated by the correlated stimulus. One must still sponded sufficiently often in the presence of the shock
consider possible direct effects that the intermittent schedule. This time was computed by subtracting the
shocks and their deletion may have on behavior, inde¬ reinforcement (shock-free) periods from the total time.
pendent of the correlated stimulus. Morse and Kel¬
leher acknowledge this in their more recent descrip¬ 2 This is similar to what I have previously called a shock-
tion of these procedures by characterizing them as deletion procedure. However, since the duration of the
reinforcement by “termination of schedule com¬ shock-deletion period is based on time, independent of the
shock-delivery schedule, I will refer to this consequence of re¬
plexes” (Morse 8c Kelleher^ 1966), the schedule com¬ sponding simply as a shock-free period.
Philip N. Hineline 38 5

Fig. 14. Families of curves show¬

ing the mean rate of bar press¬
ing by each rat as a function
of the mean interval between
successive shocks. The values
identifying each curve represent
the number of seconds for
which shocks were deleted by
an effective response. A, B, and
G denote individual rats. (From
Dinsmoor, 1962. © 1962 by the
Society for the Experimental
SECONDS BETWEEN Aiiaiy§i§ of Sehayior, Ine.)

As indicated by Figure 14, response rates were mon with behavior on comparable positive reinforce*
much higher with added cues, However, there was ment schedules.
also substantial responding when the cues were not Azrin, Holi, Hake, and Allyon (1963) provided
provided, especially when the mean shock-shock inter¬ another striking demonstration of such effects, with
val was less than SO sec. Dinsmoor found that shoek- ratio schedules. First, they exposed squirrel monkeys
free periods with durations of 30 sec or less tended to to four 6-hour preliminary sessions consisting of 5unln
be less effective than longer values, except when the periods of intermittent shock (with the mean inter¬
average shock-shock interval was extremely short. val increased over sessions, from If §eg to 2 mill) cor¬
Above 30 $ec? the duration of the shock-free period related with bright light and silence, alternated with
had little systematic effect whether or not the corre¬ 5»min shock-free periods accompanied by tone and
lated cues were supplied. Relative response rates in dimmed lights. They noted that this pretraining is
the presence and absence of the shock schedule indi¬ similar to magazine training sessions in experiments
cated discriminative control by the added cues. When using positive reinforcement. Subsequently, they made
performances were stable, fewer than 10% of the re¬ 30 -sec shock-free periods, with ee>rr^lai^d stimuli) eviir
sponses typically occurred during the shock-free pe¬ lingent on responding. Responding during shock-free
riods, while without the cues these percentages were periods was ineffective. The shock-delivery schedule
much higher. Thus considered in total, Din$moc>r?s continued to be accompanied by the bright light and
experiment indicates both discriminative control by silence. Moving from continuous reinforcement, they
the correlated cues and potent reinforcing effects of found extremely stable and persistent fixed-ratio per¬
the immediate response-contingent stimulus change formances with ratios of 25 and 50, and maintained
that these cues provide, but it also indicates the likeli¬ substantial, though less stable responding with ratios
hood of some behavioral control by the shocks, inde¬ as high as FR 300.
pendent of the cues. Azrin et al. then held the ratio requirement at 50,
Stimuli correlated with intermittent shocks en¬ and varied the shock-delivery schedule and the dura¬
hance the flexibility with which negative reinforce¬ tion of response-produced shock-free periods. They
ment can be scheduled. This was demonstrated in the found that under some conditions the duration of the
experiment by Kelleher and Morse (1964), described cued shock-free period is less critical a variable than
early in the present chapter, showing characteristic Dinsmoor’s (1962) results indicated. Over hundreds of
fixed-interval and fixed-ratio schedule effects in squir¬ hours of conditioning, changes in relative duration of
rel monkeys. In elaborating on the techniques used the shock-free periods produced by responding had
for development of such performances, Morse and very little effect on performance. For example, with a
Kelleher (1966) also summarized a number of charac¬ mean shock-shock interval of ten minutes, and the
teristics that behavior on these procedures has in com¬ duration of the cued shock-free period reduced to ten
 386
NEGATIVE REINFORCEMENT AND AVOIDANCE

seconds, only a slight reduction in overall response presence of only one color. When the scheduled inter¬
rate was observed. Responding was also maintained, val on one timer elapsed in the presence of the color
although at a lower overall rate, when the shock-free associated with the other timer, the first timer’s shock-
periods were as brief as two seconds. At the same time, free interval was not delivered until immediately after
the absolute shock rate correlated with the bright the color changed. These procedural features closely
light and silence seemed not to be critical either, pro¬ resemble those of an experiment by Baum and Rach-
vided that the shock frequencies were changed grad¬ lin (1969) using the same chamber, with concurrent
ually. For example, when every 25th response (FR 25) schedules of food delivery instead of schedules of
produced a 30-sec timeout, and the shock-shock inter¬ shock deletion. In that experiment, Baum and Rach-
val was increased gradually by giving 18 hours or lin had found that the ratio of times spent on two
more of exposure to each value, response rates re¬ sides of the chamber was proportional to the ratio of
mained virtually unchanged with the mean shock- the corresponding frequencies of positive reinforce¬
shock interval approaching one hour. To be sure, it ment. Formally, this matching relation is virtually
must be emphasized that these are maintenance data, identical to the repeatedly-verified matching relation
resulting from very gradual changes in experimental describing relative response rates (e.g. key-peeks per
conditions. IV/Fore sudden changes in shock frequency minute) as a function of relative frequencies of posi¬
timeout duration did result in performance decre¬ tive reinforcement (Herrnstein, 1970).
ments. Also, these results were obtained with monkeys. Figure 15 shows the results obtained on Baum’s
Whereas experiments on negative reinforcements procedure, including individual data points for each
have used rats. of four pigeons, showing relative time allocations as a
These experiments by Azrin et al. suggest that a function of relative frequencies with which cued
critical and potent feature of the cued escape shock-free periods were initiated in the presence of
paradigm is the stimulus change occurring at the onset
of a shock-free period. Frequencies and. patterns of
expending revealed schedule control by this event,
with only minor or limiting effects attributable to
overall shock frequency, shock-frequency reduction, <?r
duration of the stimulus-correlated shock-free periods.
This tentative conclusion gains further support from
an experiment by W. M. Baum (1973b) who studied
pigeon^ time allocation in shock-correlated stimulus
situations, as influenced by how frequently cued
shock-free periods were initiated in those situations.
The experiment is also of interest because it demon¬
strates another quantitative similarity between posi¬
tive and negative reinforcement. It can also be related
to the functions obtained by de Villim (1972, 1974)
described earlier.
Baum’s experiment used an experimental chamber
whose floor was made of two platforms. If the pigeon
stood on one platform, a red light was on; if the
pigeon stood on the other, a green light was on; stand¬
ing on both platforms simultaneously was accom¬
panied by a white light, and a move from one plat¬
form to the other produced the white light for one
second. All of these lights were accompanied by brief Fig. 15. The logarithm of the ratio of time spent on the left
to time spent on the right as a function of the logarithm of
shocks delivered once per sec. When either the red or
the ratio of number of shock-free periods obtained on the left
the green light was on, as determined by the bird’s to number of shock-free periods obtained on the right. The
position, two variable-interval programming timers data from four birds are presented; each data point represents
a value obtained from an individual bird. The solid line was
were running which could initiate two-minute shock- fitted by the method of least squares. The broken line has a
free periods, with all lights off. The two variable- slope of one, and passes through the origin; it represents the
interval timers typically had differing mean intervals, performance of perfect matching. (From W. M. Baum, 1973b. ©
1973 by the Society for the Experimental Analysis of Behavior
and each could initiate a shock-free period in the Inc.)
Philip N. Hineline 387

the red and green lights. While there was considerable Reinforcement Achieved by Shock Reduction
variability, the least-squares fit of the points on Figure Within a Situation, Contrasted with Reinforcement
15 closely matches the line indicating a matching rela¬ by Change of Situations
tion. The birds’ relative times spent on the two sides
of the chamber, excluding shock-free periods and The variable-cycle shock-deletion procedures used
changeover periods, matched the relative frequencies as components of multiple schedules by de Villiers
with which cued shock-free periods were produced on (1972, 1974) identified shock-frequency reduction as a
the two sides, again excluding the shock-free periods controlling variable, as I described earlier. The con¬
themselves. trast between de Villiers’s experiments and the one by
In introducing this experiment and in discussing Baum that I have just described, points up two dis¬
its results, Baum made the plausible assumption that tinct modes of negative reinforcement. In de Villiers’s
with a constant shock frequency in the shock situation, experiments, as in other multiple schedules, the pres¬
and a constant duration of shock-free periods, fre¬ ence of a given cue could be said to define a situation.
quency of escape from the shock situation is directly Responding within that situation reduced the shock
related to magnitude of shock-frequency reduction. frequency within the situation but did not remove the
I hus he interpreted the matching relation shown in situation itself. In Baum’s experiment, responding did
Figure 15 as showing reduction in rate of aversive not affect shock frequency in the situation itself:
stimulation to be the critical parameter of, or even rather, it occasionally produced a change of situation,
more strongly, the definition of, negative reinforce¬ to one in which shock frequency was lower. It is
ment. He also stated that it demonstrated a quantita¬ tempting to identify these two modes o£ reinforcement
tive function relating behavior to shock-frequency re¬ with the familiar labels of avoidance and escape, but
duction that was comparable to that obtained by de they do not conform to these labels as commonly
Villiers (1972, 1974) which I described earlier. The understood. The effects of both can be based on aver-
problem with this interpretation is that in Baum’s agmg events over time- Neither is reducible to the
procedure the frequency with which shock-free periods other, although there must be boundary procedures
were initiated in the presence of the red or the green that resemble both, when the response-produced stim¬
light, and from which the independent variable of ulus change is not clearly discriminable. In some ways,
Figure 15 was derived, was not proportional to shock- the relationship b^twe^n these two modes of reinforce¬
frequency reduction in the presence of those stimuli, ment resembles the relationship between magnitude
If we simply look at shock frequency in the presence and frequency of positive reinforcement, as follows.
of the red light versus that in the presence of the Hermstem (1970) pointed out that the matching
green light, disregarding shock-free periods as Baum relation for relative frequency of positive reinforce¬
did when computing the relative reinforcement rates, ment can be expected to hold only if reinforcement
we find no difference. They were both equal to one magnitudes are equal^ or if sonic mathematical adjust¬
shock per second. If instead the shock-free periods are ment is made to deal with their inequality. In addb
included when computing shock frequencies on the tion Herrnstein suggested that a matching function
two sides, relative shock-frequency reduction does obtained with equal reinforcement frequencies but
equal the relative frequency of shock-free periods, varied reinforcement magnitudes, could be used for
provided that this latter measure also includes the the scaling of reinforcement magnitude to make valid
durations of the shock-free periods. On this basis it is such a mathematical adjustment. Very tentatively, the
possible for the birds’ relative time allocations to analogous treatment of negative reinforcement can be
match the relative reductions in shock frequency. spelled out by accepting Baum’s specification of fre¬
However, a plot comparable to Figure 15, but using quency of negative reinforcement in terms of the
shock-frequency reductions computed from the data frequency with which shock-free periods (changes of
given in the appendix of Baum’s article in place of situation) are initiated. Magnitude of reinforcement
the frequency of timeout production, gave a much less could be either the duration of shock-free period or
orderly function. Thus, the measures plotted in the magnitude of change in shock schedule or in shock
Figure 15 were well chosen, but the resulting function intensity that accompanies the change of situation.
does not show the relation of time allocation to mag¬ In addition, the shock frequencies and intensities
nitude of shock-frequency reduction. Rather, the prior to change of situation could be treated as magni¬
birds’ relative time allocations matched the relative tudes of drive operation, tightly defined as operations
frequencies, in the situations where shocks occurred, that potentiate reinforcement operations. De Villiers’s
of onset of highly discriminable shock-free intervals. experiments with multiple schedules could be seen as
388 NEGATIVE REINFORCEMENT AND AVOIDANCE

potential scaling procedures to permit the use of vary¬ The use of two concurrently recordable responses per¬
ing magnitudes of reinforcement within Baum’s pro¬ mitted a dissociation of effects of reinforcement by
cedure. However, we must still contend with the dis¬ shock-removal from effects of reinforcement by shock-
tinction between magnitude of reinforcement as delay even when some of these effects (bursts) were not
achieved by varying background shock frequency and explicitly prescribed by the schedule. By itself this
as achieved by holding that frequency constant and experiment requires a distinction between the two re¬
varying response-produced lower frequencies. sponse classes only because they are controlled by
different parts of the shock-frequency continuum.
Multiple Response Patterns Produced However, it complements the preceding experiments
by Shock-Delay Procedures by de Villiers and by Baum, supporting a distinction
between change of situation and reduction of aversive¬
The experiments described above identify con¬ ness within a situation.
trolling features of negative reinforcement mainly
through long-term parametric relations between these Concurrent Schedules, with
features and overall response rates. A more direct ap¬ Reinforcement by Shock-Delay
proach has also been used, based on manipulation and by Removal of the Shock-Delay
and observation of short-term response patterns, Procedure
rather than on curve-fitting with overall rates. For
example, using a shock-delay procedure Sidman (1966) Verhave (1962) explicitly arranged for concurrent
found that when the shock-shock and response-shock but separate control of behavior by these two aspects
of negative reinforcement. He used several proce¬
intervals differ (SS=20, RS=40), the spacing of re¬
dures; in all of them, responses could delay shocks
sponses within those intervals differs accordingly.
(SS=3, RS=30). In each, some responses could pro¬
Thus differing RS and SS intervals are discriminable,
and as they alternate the animals respond appro¬ duce a “timeout” period, accompanied by a tone, in
which both the shocks and the shock-delay contin¬
priately to each. When the SS interval approaches
zero, and thus is extremely short relative to the RS gency were absent.3 In an initial experiment the onset

interval, responses during the shock-shock interval can

of the timeout period was made contingent upon the
same lever-press that delayed shock, but with a fixed-
be said to remove shock, producing a distinct change
interval schedule. Verhave found no indications of
of situation. Responses during the RS interval are
maintained on a much lower range of the shock- control by the fixed-interval schedule. However, in a

density continuum* their consequent shock-delay or

later experiment where the cue-correlated timeout
periods were made contingent upon pressing a second
shock-frequency reduction can be characterized as
lever, they had very clear effects. In this experiment,
further reduction within a less aversive situation.
initial training was accomplished by partitioning the
A distinctive pattern of responding has frequently
been noted that is related to, although not entirely conditioning chamber to provide access only to the

confined to post-shock periods. This responding has

shock-delay lever. After responding was stable on this
procedure, and after irregularly spaced tone presenta¬
been characterized as “bursting.” Ellen and Wilson
tions to ensure that the tone by itself had no effect on
(1964) described it in detail, contrasting it with the
spaced responding or “continual-responding pattern” responding, the second lever was made accessible. De¬
pressions of this lever produced 10-min timeout pe¬
characteristic of the response-shock interval. There
riods accompanied by the tone. Figure 16 portrays
have been repeated suggestions that this burst pattern
some of the results with the cumulative records indi¬
is adventitiously maintained by shock removal, an
cating responses and shocks on the shock-delay lever.
interpretation well supported by an experiment by
Downward deflections of the underlying event records
Boren (1961) which illustrates another method for
indicate timeout periods, each produced by a response
separate study of shock-removal and shock-delay. He
on the second lever. The first panels show the first
introduced a second lever into a shock-delay situation.
session in which responding could produce the time¬
One lever worked only during the response-shock
out periods. The first response-produced timeout oc-
interval; the other lever worked only during the
shock-shock interval. Using a shock-shock interval of
3 Within the present chapter the term “timeout” is reserved
zero (continuous shock), Boren found that the burst for shock-free periods that accompany suspension of a shock-
pattern, even though occurring partly in the RS inter¬ delay contingency that would permit responding to extend
val, was entirely confined to the shock-removal lever, shock-free intervals. It should be recognized that other authors
have applied the term to any clearly discriminable shock-free
the one operative only during the (zero) SS interval. period.
Philip N. Hineline 389

SESSION 7 CONTINUED

Fig. 16. Cumulative records showing performance on a shock-delay procedure, with

associated event records showing performance on a lever that produced timeout from
the shock-delay procedure. Shocks are indicated by pips on the cumulative record. Time¬
out periods are indicated by downward deflections of the event record. The top two
panels show the fourth session of shock-delay training, the first session in which the
timeout procedure was in effect. The middle pair of panels shows performance during
the fourth session in which the timeout procedure was in effect; the bottom pair shows
performance in the next session, during which the timeout procedure was discontinued,
producing extinction on the second lever while the shock-delay procedure continued.
(After Verhave, 1962. © 1962 by the Society for the Experimental Analysis of Behavior, Inc.)

curred very early in the session, perhaps facilitated by 100 minutes of exposure to the timeout situation. In
the similarity of the two levers located under identical general, responding on the delay lever decreased and
pilot lights. Responding on the delay lever persisted frequency of timeout production increased over the
undiminished throughout that timeout period. This next few hours of conditioning; the seventh six-hour
responding during timeout did not begin to diminish session indicates the typical pattern, more shock-delay
until the fourth and fifth timeouts, which followed responding and a slightly lower frequency of timeout
one another closely, suggesting a strengthening of re¬ production early rather than late in the session. When
sponding on the timeout lever. Response rates during the timeout lever was deactivated, responding quickly
timeout periods did not become reliably lower until recovered on the shock-delay lever (bottom 2 panels of
close to the 11th timeout, which was after more than Figure 16). In additional experiments, Verhave im-
390 NEGATIVE REINFORCEMENT AND AVOIDANCE

posed ratio schedules on the timeout lever. When of these that I will describe, the effects to be measured
fewer than ten responses were required to produce a were concurrent reinforcing effects superimposed on
timeout, responding persisted on the timeout lever; shock-delay procedures.
when the ratio exceeded ten, two of three animals As noted above, Verhave (1962) found that a sepa¬
reverted to responding exclusively on the delay lever. rate response, rather than the shock-delay response,
So long as responding persisted on the timeout lever, was needed for demonstrating concurrent reinforcing
enough accompanying responses were emitted on the effects of a timeout stimulus correlated with absence
delay lever that few shocks were received even though both of the shock-delay contingency and shock. Res¬
the ratio took some time to complete. corla (1969) used a two-response procedure to demon¬
Verhave concluded that for his subjects the timeout strate that at least to some degree responding main¬
was an effective but rather weak reinforcer similar in tained by shock-delay is sensitive to concurrent
its effects to food for a minimally deprived organism. reinforcing effects of a stimulus correlated only with
It seems more directly analogous to a situation with absence of shocks. Using dogs, he established panel-
effective deprivation but with two kinds of food rein¬ press responding with the shock-delay procedure
forcers available7 one slightly preferred to the other. (SS=10, RS = 30). Responding on either of two panels
Sidman (1962c) studied the effects of a similar pro¬ could eliminate all shocks, but responding exclusively
cedure with monkeys. He found that responding was on either panel for two minutes made that panel in¬
readily maintained when cued time-out periods were operative until a response had occurred on the other
contingent upon higher ratios, but that there were panel. Then in three pretest sessions with the delay
substantial interactions between responding on the procedure still in effect but without the two-min
shock-delay lever and the chain-pull response that limitation, he made brief (0.5 seconds) tones of 400
produced timeout periods. It is likely that the animals’ Hz and 1200 Hz each separately contingent on presses
prior histories of positive reinforcement were impor¬ of different panels, with the positions reversed midway
tant m maintaining the behavior with higher ratios. through the pretesting period. Panel pressing was un¬
The species difference may also have contributed to affected by these tone presentations, however, five
the differing persistence on ratio schedules. intervening sessions of conditioning with a Pavlovian
procedure changed this. With the response panels
Concurrent Reinforcement removed, a 2-sec darkening of the chamber was some¬
with Stimuli Denoting times followed by shock 8 sec later; at other times it
Shock-Free Periods was followed in 7 sec by a tone, but with shock
omitted. Negatively correlated with shock, the tone
The notion of a shock-free situation has little was either 400 Hz or 1200 Hz, depending on the
meaning except in the context of other contrasting animal. Subsequently the animals were re-exposed to
situations where shocks occur at least sometimes. This the same procedure used during pretesting, with panel
fact and its significance have been stressed by Rescorla presses producing brief tone presentations as well as
(1967), mainly in relation to Pavlovian conditioning. shock-delay. During this testing, responding clearly
He has advocated replacing traditional stimulus-pair¬ predominated on the panel that produced the tone
ing procedures with stimulus-correlation procedures. that for the given animal had been negatively corre¬
In procedures where cue presentation is determined lated with shock in the Pavlovian procedure. The
independently of shock presentation, cues have zero stimulus denoting absence of shock had a reinforcing
correlation with shock, as do cues in experiments effect revealed in response preference. However, con¬
where there are no shocks. In only a trivial sense do sistent with Verhave’s (1962) results when using a
the latter denote shock-free periods. Positively corre¬ single response, the reinforcing effect was not evident
lated cues are those for which the probability of shock in absolute rates.
is greater in their presence (or immediately following Weisman and Litner (1969) devised a more potent
their occurrence) than in their absence. Negatively procedure for demonstrating reinforcing effects of a
correlated cues include those I have already described stimulus whose negative correlation with shock was
as cues correlated with or denoting shock-free periods. established during separate, Pavlovian conditioning
Shock probability is greater in their absence than in sessions. The stimulus had clear concurrently rein¬
their presence. Frequently, stimulus correlation pro¬ forcing effects on a single response maintained by a
cedures are accomplished apart from operant rein¬ shock-delay schedule. Using rats, they established a
forcement procedures, but assessed through their wheel-turning response with a delay procedure of
effects on baselines of operant behavior. In the first SS = 5, RS=20 sec, and then divided their subjects
Philip N. Hineline 391

sessions were included to observe extinction of any

conditioned reinforcement effects generated under the
drh procedure.
During test sessions 8-11, if zero or one response
occurred in a 5-sec interval the 400-Hz tone was pre¬
sented in the succeeding 5-sec interval. Thus, during
these sessions responding at a low rate produced the
tone. As with the drh procedure, only the rate of re¬
sponding in the absence of the tone could affect the
presentation of the tone, but responses occurring at
any time during the session postponed shock. Again,
during shock-delay sessions administered on test days
12 and 13 the tone was not presented. These sessions
were included to observe extinction of any condi¬
tioned reinforcement effects generated during the drl
procedure.
As shown in Figure 17, for rats exposed to tones
Tig. 17. Mean responses per minute for baseline, differential denoting shock-free periods during intervening ses¬
reinforcement of high rate (drh), differential reinforcement Of
sions, 1 ever-pressing maintained by the shock-delay
low rate (drl), and extinction sessions. These reinforcement
contingencies refer to the presentation of a tone, contingent contingency was increased by the drh schedule and
upon changes in rates of responding that was, all the While, decreased by the drl schedule of tone presentation.
maintained by a shock-delay procedure. During intervening
Rats for which the tone had not been correlated with
sessions in a separate chamber, the tone was presented in differ¬
ent relations to shock for two of the three groups; randomly, absence of shock in the intervening days did not show
or unrelated for TRC; negatively correlated, for CS . For the any §uch reinforcing effects of contingent tone presen¬
third group (no CS) only shocks were presented during the in¬
tervening sessions. (After Wdsman & Litncr, 1909. © 1909 by the
tation.
American Psychological Association. Reprinted by permission.) Dinsmoor and Sears (1973) used dimensional con¬
trol by an added cue to reveal reinforcing effects of
the cue correlated with shock-free periods. In addi¬
into three groups, run on identical procedures during tion, the cue’s negative correlation with shock was
alternate "test” days, but differing in their treatments accomplished by presenting it in synchrony with a
on the intervening days. For one group (CS—, using shock-delay procedure by a technique §imi!ar to <?ne
Pavlovian terminology in Figure 17), 5-sec presenta¬ developed by Rescorla (1968). Dinsmoor and Sears
tions of a 400 Hz tone were correlated with shock-free used pigeons, and the shock-delay procedure (SS — 5,
periods. For another (TRC in Figure 17, for "truly RS = 20 see) contingent upon treadle-pressing. In
random after Rescorla (1967), tones and
control” addition to the 20-sec response-shock interval, each
shocks were presented independently, so that the response produced a 1000 Hz tone of 5-sec duration,
probability of shock was equal m either presence or which ensured repeated presentation of the tone dur¬
absence of tone. The third group received the same ing shock-free periods. After 58 training sessions, a
number of shocks, but the tone was not presented test procedure was introduced during the final 60 min
("No CS”) on the intervening days in the second of every third 90-min session. In this testing, tones of
chamber. various frequencies were made contingent upon the
During the test sessions, which included continued response; responding was also examined in additional
training with the shock-delay procedure, presentations sessions where no tones were presented. As indicated
of the tone were made contingent upon changes in by Figure 18, all birds responded maximally when the
response rate. First came differential reinforcement of response-produced tone had a frequency of 1000 Hz,
high rates (drh). During the first 30 min of the initial as in training. For two of the three birds, response
drh session the number of responses per 5 seconds re¬ rates dropped substantially when the frequency of
quired to produce the tone was gradually increased the tone was varied. Across birds, the degree of decre¬
from 4 to 10, where it remained. During the final 30 ment produced by changing tone frequency was cor¬
min of the second test session and test sessions 3-5, related with the degree of decrement observed when
the criterion of 10 or more responses per 5 sec re¬ the tone was deleted entirely (open data points in
mained in effect. During the shock-delay sessions on Figure 18). Noncontingent presentations of the 1000
test days 6 and 7 the tone was not presented. These Hz tone resulted in relatively little responding. The
 392 NEGATIVE REINFORCEMENT AND AVOIDANCE

• BIRD 1396 contingent specifically upon changes in response rate,

A BIRD 1929
the delay-maintained response itself is sensitive to
reinforcement by the cue correlated with shock-free
periods. In these last two experiments the correlations
between cues and shock-free periods were established
and maintained in separate training sessions, and thus
required special test sessions to demonstrate their
effects. Dinsmoor and Sears (1973) demonstrated a
method for assessing concurrent reinforcement effects
of a stimulus whose negative correlation with shock
was established within the shock-delay procedure it¬
self, although the stimulus change always coincided
with the initiation of shock-delay.

Discriminative Properties of Cues Added

to Shock-Delay Procedure

I have been emphasizing the reinforcing properties

of stimuli denoting shock-free periods. At the same
time, discriminative properties, revealed by behavior
Fig. 18. Tile mean numnei1 Of Response-contingent in the presence of the stimuli, have sometimes been
tone was produced by each of three pigeons during the final 60
observed. These appeared to be closely correlated
mm of each test session, in which tones of various frequencies
were pi-evented, contingent on responses. The open symbols with the reinforcing properties. For example, Dins-
indicate a comparable measure for periods when no tone was moor and Sears (1973), when gathering the data
presented. Training occurred with the tone of 1000 Hz; other
shown above in Figure 18, found that the frequency
frequencies used during testing are also indicated on the
abscissa. (After Dmsmoor and Sears, 10M.) of responding in the presence of a given tone fre¬
quency was inversely related to the response rate
maintained by response-contingent presentation of
effects of varying the frequency of these response-
that tone frequency. A similar, though less systemati¬
produced tones are not readily attributable to the
cally examined effect was observed in the experiment
degree of stimulus change produced by responding,
by Verhave (1962) on reinforcing effects of timeout
or to the degree of difference between stimuli cor-
from a shock-delay procedure. There, responding on
related with the presence of shock (silence) and the
the shock-delay lever during timeout from the shock-
response-produced stimulus. Rather, they are attribut¬
delay procedure decreased as responding increased on
able to the physical differences between the response-
the separate lever that produced the timeout periods.
produced stimuli and the stimulus that was previ¬
I lus relationship appears roughly similar for both
ously correlated with shock-free periods immediately
Dinsmoor and Sears’s and Verhave’s experiments
following responses.
even though the durations of the stimuli denoting
Each of the four preceding experiments has dem¬
shock-free periods differed drastically—6 sec versus 10
onstrated simultaneous control of behavior by shock-
min—and one denoted absence of an entire pro¬
delay (shock-frequency reduction without distinctive
cedure while the other denoted the momentary ab¬
change of situation) and by reinforcement based on
sence of shock within a similar procedure.
clearly discriminable changes of situation. Verhave
In the experiments that follow, discriminative
(1962) found that a response independent of the
effects of cues paired with presence and absence of
shock-delay contingency was sensitive to concurrent
shock were the main focus of study.
reinforcing effects of stimuli correlated with absence
of shocks (and of the shock-delay contingency), while
Cues Superimposed on
the shock-delay response was insensitive to these
Shock-Delay Procedures
effects. Rescorla (1969) found a choice measure to be
more sensitive than absolute response rate measures Rescorla and LoLordo (1965), with dogs as subjects,
for demonstrating concurrent reinforcement effects. found that when a cue had been correlated with
However, Weisman and Litner (1969) showed that if shock-free periods in a separate situation, superim¬
the cues correlated with shock-free periods are made posing the cue on a baseline of shock-delay respond-
Philip N. Hineline 393

ing produced a decrease in responding in its presence. consistent with the two-factor view. Clearly, Pavlo-
This finding was replicated in rats by Weisman and vian-cue relations are imbedded in cued negative rein¬
Litner (1969). With analogous procedures these two forcement procedures. We have seen that the role of
studies also demonstrated increased response rates in such cue relations can profitably be studied with the
the presence of stimuli that had separately been pre¬ aid of Pavlovian procedures applied outside the nega¬
sented in positive correlation with shock. Extrapola¬ tive reinforcement sessions, and some of their results
tions based on these results must be done with caution, can be predicted from principles of Pavlovian condi¬
for Pomerleau (1970) showed that either enhance¬ tioning.
ment or suppression could be produced by a cue However, the conditioned aversive or “fear gen¬
paired with shock, depending on the duration of the erating,” and thus response-generating role of cues
cue. Brief cues apparently produce the simplest effects, correlated with shock has come into question. For ex¬
since they are less affected by the shock frequency ample, Bolles, Stokes, and Younger (1966) and Bolles
produced by the ongoing shock-delay schedule. and Grossen (1969) found that responding was not
Of special interest for the present analysis is an very effectively maintained by removal of a stimulus
experiment by Rescorla (1968) demonstrating effects paired, with shock unless the shock was also deleted.

of cue-shock correlations established within an on¬ Kamin, Brimer, and Black (1963) used a separate
going shock-delay procedure. Using different groups procedure outside a negative reinforcement situation
Rescorla trained dogs with an SS interval of 10 sec to independently examine conditioned properties of a
and an RS interval of 30 see contingent upon a free- cue that was paired with shock in the negative rein¬
operant shuttle response. For some animals a tone was forcement situation. This separate procedure was the

presented for 5 sec after each response, for others, the well-known “conditioned anxiety*” or conditioned
5-sec tone came on 25 sec after a response. For a third suppression procedure of Estes and Skinner (1941),
group of dogs the 5=sec tones were presented at ran¬ Using subjects at various stages of training with nega¬
dom. None of these tone presentations had any ap¬ tive reinforcement, Kamin et al. took the cue that had
parent effects on acquisition, but when the tones were accompanied shock during that training, aiid pre¬

then deleted while the shock-delay procedure re¬ sented it noncontingently during food-reinforced re¬
mained in effect, response rates increased for the sponding. The cue produced least suppression of
group that had received tone immediately after re¬ appetitive responding in animals for which the cue
sponses. Finally, the shock-delay procedure was sus¬ was most effective for msinEsiniiig negatively-rein¬

pended (no shocks delivered) and the tones were pre¬ forced responding, This result runs strongly punter
sented noncontingently. Greatly decreased response to predictions of the two-factor formulation, which
rates were observed in the presence of the tone for would require that a stimulus producing greater fear
animals that had previously received tones immedi¬ would b$ more effective, both for negative reinforce¬

ately after responses. In contrast, animals that had ment and for suppression of appetitive behavior.
previously received the tone 25 sec after responses
showed greatly enhanced response rates in the presence Cues Inserted in Shock-Delay
of the tone.
Procedures: ‘Warning Stimuli”
Rescorla interpreted his results as supporting tradi¬
tional two-factor avoidance theory, in which a clas¬ The most telling evidence against the traditional
sically-conditioned response termed “fear” is said to two-factor formulation comes from procedures in
generate the operant response, and removal or inhibi¬ which preshock cues, often called “warning stimuli,”
tion of that classically-conditioned response is said to are introduced into shock-delay procedures. Some of
reinforce the operant response. Thus, the tone im¬ these experiments have not received the attention
mediately following responses would be a fear inhibi¬ they deserve, so they are presented in detail here. First
tor, and that presented just before shock (25 sec after was an experiment by Sidman (1955) who pretrained
response) would be a fear elicitor, each with appro¬ cats and rats in his noncued shock-delay procedure
priate effects on shock-delay responding. Others (e.g. (SS = 20, RS = 20) and then introduced a 5-sec pre¬
Anger, 1963) have argued that on this type of pro¬ shock cue (light) that could be either delayed or re¬
cedure the cue itself becomes aversive through its moved. That is, responses less than 15 sec apart de¬
correlation with shock. It then generates the response, layed both the onset of the light and of the shock (RL
with its removal reinforcing the response. Either way, interval = 15 sec; RS interval = 20 sec). Responses
the experiments just described (Rescorla, 1968, 1969; in the presence of the light terminated it and delayed
Weisman and Litner, 1969), demonstrated results the shock. If no response occurred the light terminated
394 NEGATIVE REINFORCEMENT AND AVOIDANCE

with the brief shock, initiating shock-light and shock- shock interval, to zero when shock was 100 sec dis¬
shock intervals identical to the response-light and tant. Figure 19 shows typical performances with audi¬
response-shock intervals. With this procedure most of tory and visual stimuli combined, with visual and
the responding came to occur in the presence of the auditory stimuli separated, and with no added warn¬
light, with only 25 to 30 percent of the responses ing stimuli. With both sets of stimuli (top panel), the
typically occurring in its absence, mainly in post¬ rat typically responded so that shock was kept 30 to 50
shock bursts. There was some tendency for response sec away. When there were no added stimuli (bottom
rates to increase as time for the light onset approached, panel), shock was typically kept 90 to 100 sec away,
but these rates were very low relative to response with occasional lapses. Most interesting are the inter¬
rates in the presence of the light. Ulrich, Holz, and mediate cases: performance with the auditory stimuli
Azrin (1964) used a similar procedure with a buzzer was very similar to that with both auditory and visual
instead of a light. The response-buzzer interval was stimuli; performance with light alone was intermedi¬
15 sec; the response-shock interval was 20 sec, both ate between that for both stimuli and that for no
comparable to Sidman’s experiment. They used a added cues. Light plus clicker can reasonably be
shorter shock-shock interval of five sec. In the absence assumed to be a more distinctive cue than light alone.
of responding the buzzer was on continuously, accom¬ Thus, with the more salient or discriminable stimu-
panied by a shock every five seconds. They found an
even greater concentration of responding in the pres¬
kat
ence of the pre-shock cue, with only a very low, con¬
stant rate of responding in its absence. When they
shortened the response-buzzer interval, responding
continued to be concentrated in the period immedi¬
ately after buzzer onset.
If the preshock cue were to be considered a con*
ditioned aversive stimulus whose removal could rein¬
force responding7 one might expect that on these pro¬
cedures the animals’ responding would be maintained
by its delay as well as by its removal, comparable to
responding maintained by shock-delay when no cue is
provided, That did not happen; the animals usually
did not delay the warning stimulus. To preserve the
notion of conditioned aversive stimuli one could say
that the warning stimulus became only moderately

ayersiyc. Its removal could maintain responding, but

its delay could not.
Whatever its plausibility this reinterpretation is
challenged by an experiment by Field and Boren
(1963) who provided “warning stimuli for warning
stimuli.” Their underlying procedure was an “adjust¬
ing avoidance schedule” (Sidman, 1962d), where in
the absence of responding brief shocks occurred every
5 sec, and each response produced 10 sec of shock-free
time. These 10-sec periods were cumulative up to 100
sec. Field and Boren used two sets of stimuli in this
procedure. One was a series of 11 pilot lights spaced
evenly in a line on the wall above the lever. Each
lamp in ordinal sequence accompanied a specific 10-
sec period within the 100-sec range. The 11th light, Ml NUTES

directly above the lever, was on during the shock-

Fig. 19. Strip chart records showing one rat’s responding with
shock interval. The light most distant from the lever
both classes of multiple warning stimuli, with each class of
was on when the next shock was not due to occur for stimuli alone, and with no warning stimuli. The pen records
100 sec. The second set of 11 stimuli was a series of the position of the stepping relay, hence the temporal proximity
of the shock is indicated in 10-sec increments. On these records
click rates ranging from 57.7 per sec during the shock- time runs from right to left. (From Field & Boren, 1963.)
Philip N. Hineline 39 5

lus changes, the animals maintained a closer prox¬ shock in the comparison procedure where responses
imity to shock. Field and Boren interpreted their could terminate the light. Taking all of these into ac¬
results in terms of degree of stimulus control, and of count, it was clear that while both procedures satisfied
stimulus generalization. They argued that with more the conditions for formation of a Pavlovian discrimi¬
highly discriminable stimuli, characterized by a rela¬ nation, only in the case of the cue denoting unavoid¬
tively sharp gradient, the animals could maintain the able shock did the cue function as a conditioned aver¬
relatively closer proximity to shock without incurring sive stimulus. However, even there we could treat it
an increase in shock frequency. An interpretation in just as a stimulus correlated with the suspension of
terms of conditioned aversive stimuli could probably the shock-delay contingency: responding in its pres¬
handle these results with certain added assumptions. ence decreased; responding in its absence could still
However, a simple and direct application of this ap¬ delay shock, and that responding persisted. Absence of
proach would seem to predict the opposite results. the light in this case, was a stimulus correlated with
When more clearly discriminable stimulus changes access to reinforcement, and responding persisted in
were provided one might expect facilitation of respond¬ the presence of that stimulus even though it was not
ing that prevented the conditioned aversive stim¬ paired with shock.
uli. Instead, the responding shifted toward stimuli A second experiment by Sidman and Boren (1957c)
more closely associated with shock. argues eyen more strongly for discriminative proper¬
Grabowski and Thompson (1972) obtained similar ties of a preshock cue within delay procedures. After
results with monkeys. They used lights correlated with pretraining on the standard noncued shock-delay pro¬
successive time segments of SS and RS intervals on cedure with SS and RS intervals of 20 see, a cue was
conventional shock-delay procedures. added. So long as 15 see did not elapse between re¬
Additional experiments by Sidman and Boren (1957 sponses, it was identical to the familiar cued shock-
b & c) and by Sidman (1957) more directly illustrate delay procedure (Sidman, 1955). Responses could
the discriminative rather than aversive role of pre¬ delay the onset of the cue light, as determined by a
shock stimuli inserted into shock-delay procedures. In response-light interval of 15 sec analogous to the
one of these, Sidman and Boren (1957b) compared usual shock-delay contingency. However, if the an¬
performance on two procedures, one of which was imal paused, allowing the cue light to come on, a
described just above (Sidman, 1955). On that pro¬ response-shock interval of 5 sec was in effect such that
cedure, responding that delayed shock could also responses delayed the shock bu t also extended the
delay or remove a tone presented 5 see before a shock duration of the cue light. With a shock, the cue light
was due. As noted above, most responding occurred was turned off, reinstating the 15-sec responsedight
in the presence of the tone instead of delaying it. The contingency. With the transition to this schedule from
comparison procedure also used shock-delay and cue- uncued shock-delay (SS = RS — 20 sec), responding
delay, with pretraining on a conventional procedure in the presence of the light increased temporarily over
whose shock-shock and response-shock intervals both that of the comparable time periods in the pretrain¬
equal to 20 sec. Similar to the above procedure, a ing schedule and then dropped well below that in
4-sec light preceded all shocks; responses in its absence the dark. For three of four rats, responding in the
delayed its onset by 16 sec while delaying shock by dark on the 15-sec light-delay schedule increased
20 sec. However, responses in the presence of the slightly over that for comparable time periods in the
light had no effect either on light or on shock; once prior schedule. For all rats the terminal response rate
the light came on the impending shock was unavoid¬ in the absence of the light substantially exceeded that
able. When this procedure was introduced to animals in the presence of the light. In short, the animals fre¬
pretrained with no warning stimuli, responding in the quently waited through the light, took the shock, and
dark, which was now the only effective responding, then responded more rapidly, delaying the renewed
initially increased but then decreased over sessions. onset of the light.
The final stable levels were not systematically related This result suggests that the low response rates
to response rates during initial conditioning on the (pauses) in the presence of the cue light in the sequen¬
uncued shock-delay procedure; some rates were higher tial procedure were reinforced by termination of the
than initially, others were lower. Any responding that stimulus and its associated contingency, even though
initially occurred in the presence of the light de¬ the shock whose reduction provided the basis for
creased systematically to very low levels. Additional maintaining the whole performance accompanied the
control procedures assessed possible effects of variable termination of that light.
light durations and varying correlations of light and The shift of responding within a sequential pro-
 396 NEGATIVE REINFORCEMENT AND AVOIDANCE

cedure, from a more demanding to a less demanding ever, when the response-light interval was increased,
component is reminiscent of the effects observed by performance changed and the animal spent most of
Krasnegor, Brady, and Findley (1971), which were de¬ its time in the dark, with responding again main¬
scribed earlier in the present chapter. It indicates that tained by associated less stringent performance require¬
the reinforcing or aversive properties of a cue-defined ment. These functions are very similar to those of
situation can be strongly affected by the scheduled Krasnegor et al. (1971), shown in Figure 2.
reinforcement contingencies in that situation, apart In a follow-up experiment, Sidman and Boren
from correlated shock delivery per se. The similarity (1957b) moved animals from a schedule of this kind,
of the two experiments is revealed by parametric with a RL interval of 15 sec and a RS interval of 10
manipulations with the Sidman and Boren procedure. sec in the presence of the light, to a procedure that
These manipulations, reported by Sidman (1957), differed in that the cue light and the associated
make the controlling features of that procedure even response-shock interval did not terminate with the
more clear. First, the response-light interval was held first shock. Once on, the light remained on for five
constant while the response-shock interval, operative min, accompanied by ten-sec response-shock and shock-
in the presence of the light, was manipulated. Re¬ shock intervals, independently of shocks taken during
sponse rates varied in both the presence and absence this period. Response rates in the absence of the light
of the light, so that more time was spent in the con¬ and maintained by the light-delay contingency, were
dition permitting lower response rates. Complemen¬ slightly higher than in the comparable procedure
tary results are shown in Figure 20; these were ob¬ where cue presentation terminated whenever a shock
tained by holding the response-shock interval constant was delivered. Response rates in the presence of the
at 10 S££ and varying the response light interval from lights were very high relative to rates on the com¬
10 to 20 sec. As the figure shows, when the two inter¬ parable procedure where the light terminated with
vals were equal at 10 sec, most responding occurred in shock. Low rates were no longer reinforced by a re¬
the presence 6£ the light, directly delaying shock, How- turn to less stringent response requirements.

The Several Properties of Added

Cues, and Their Implications

for Two-Factor Theory

Returning to the terms of my earlier analysis of

negative reinforcement procedures without added
cues, the preshock, or “warning" stimulus added to a
shock-delay or shock-deletion procedure is correlated
with several key procedural features. The added cue
accompanies the opportunity to respond, and is cor¬
related with a frequency or probability of shock. It
also denotes periods when responding is especially
effective, and thus is correlated with the availability
of reinforcement. Accordingly, behavioral effects of a
warning stimulus are confounded. The preshock cue
cannot be seen as simply providing a classically con¬
ditioned surrogate for the shock, for several effects of
the cue are independent of its relation to shock. A
change in availability of reinforcement contingent
upon behavior in the presence of a cue can markedly
change that behavior. It can also change behavior
that affects the onset of the cue. The manipulation of
one correlated feature may affect changes in behavior
directly, or indirectly by shifting control to a different
Fig. 20. Rate of responding in the light (L) and in the absence correlated feature. In short, the familiar warning
of the light (D) for rat AA-1 as a function of the interval by
stimulus, traditionally used to simplify interpretation
which responses in the dark could delay the light (the RL in¬
terval). The RS interval, by which responses during the light of negative reinforcement, is an added feature of
could delay shocks, was constant at 10 sec. (From Sidman, 1957.) great complexity.
Philip N. Hineline 397

Traditional two-factor theory of the form that (e.g. Herrnstein, 1964); it appears that short interrein¬
postulates internal mediating responses classically con¬ forcement times have disproportionately great effects
ditioned to the warning stimulus, encounters the (see also Fantino, Chapter 11 in this volume). For the
above problems and more. These conditioned re¬ aversive case, Bolles and Popp (1964) found indica¬
sponses, with their own cue properties, must them¬ tions that on a shock-delay procedure, a variable
selves be multiply correlated with the various aspects shock-shock interval produced acquisition more reli¬
of the conditioning procedure. Further, as Black ably than did a fixed shock-shock interval with
(1971) has argued in exhaustive detail, there is no reli¬ approximately the same mean value. If this is inter¬
able principle for a priori selection or identification preted as effectively shortening the shock-shock in¬
of a mediating response. The closer one comes to terval, the results are comparable to those obtained
identifying a physiological index of a classically by Leaf (1965), showing better acquisition with small
conditioned response or process that might mediate shock-shock intervals. Thus we have both an appeti¬
the operant behavior, the greater the likelihood that tive and an aversive example suggesting that arith¬
the index is an instrumentally conditioned artifact of metic averaging is inappropriate.
the operant procedure. At the same time, the less Whatever the method of averaging, the onsets and
susceptible the proposed mediator is to problems of offsets of added cues appear to enhance or even
artifact, the less adequate it is to account for the change the boundaries of time intervals over which
operant behavior, In addition, if one is concerned intermittent events are effective, With respect to inter¬
with mediation of performance, a phasic reflex re¬ mittent shocks accompanied by tones and lights* we
sponse is far too primitive to mediate the lawful but might consider the onset or offsets of the added cues
complex behavior produced by the procedures we as influencing the behavioral effects of shock in much
have been considering. In principle the Pavlovian the same way that, m visual pattern perception bound¬
model can encompass more enduring responses. In ary lines can influence the way that dots in the visual
practice a proposed diffuse but unitary process identi¬ field are grouped. If the lines fall between areas with
fied by a facile label such as “fear” is likely to provide differing but homogeneous densities of dots, the
superficial interpretation. As Myer (1971) has argued, differences between those homogeneous areas will be
present evidence does not indicate that there is a enhanced. If on the other hand* there as a continuous
single, unitary, mediating process based on pairing of gradient of density of dots, drawing a line across the
stimuli with shock. gradient will effectively separate the areas. Tbe ob¬
Thi$ is not to belittle the role of cues m negative server will report the pattern not as a gradual gradi¬
reinforcement procedures, nor to disregard all aspects ent, but rather as a group of areas with separate,
of the Pavlovian paradigm as it applies to these pro¬ more-or-less homogeneous but different densities. So,
cedures. The correlation of cues with presence or ab¬ analogously, presentation or removal of a tone or
sence of shocks, as systematized by Rescorla and his light may influence the way that a series of brief elec¬
colleagues (e.g. Rescorla & Wagner, 1972) provide the tric shocks will be averaged. This function is implicit
basis for the most comprehensive formulation of stim¬ in my distinguishing between shock-frequency reduc¬
ulus combination now available. However, instead of tion within a cue-defined situation and a change of
characterizing the positive or negative correlation of cue-defined situations with differing shock frequencies.
a cue with shock as production or inhibition of a The onset or offset of a continuous cue provides a
Pavlovian response, it is proposed here that in operant boundary for a group or groups of irregularly spaced
behavior, the role of cues correlated with shocks is bet¬ shocks.
ter characterized as affecting the averaging of shock The effect of cues on averaging of intermittent
over time. shocks is illustrated quite clearly in an experiment by
This averaging need not be construed in terms of Badia, Coker, and Harsh (1973). For initial training,
the subject’s internal mechanism, but rather in terms they exposed rats to a variable-time schedule of brief,
of what methods for summarizing the independent noncontingent, and nonreduceable shocks. The prob¬
variables lead to the simplest functional relationships. ability of shock was fairly constant over time, with a
For example, comparison of periodic with aperiodic mean shock-shock interval of four min. In addition,
events suggests that equivalencies cannot be based on each shock was preceded by a five-sec tone. After three
arithmetic averaging. In the case of concurrent chain sessions of exposure to this signaled shock procedure,
schedules of positive reinforcement, an aperiodic se¬ shocks occurred without the tone unless the animal re¬
quence of reinforcements is preferred over a periodic sponded. A response turned on a cue light for three
sequence of events with the same overall frequency minutes, during which shocks were preceded by the
398 NEGATIVE REINFORCEMENT AND AVOIDANCE

tone. Quick acquisition of responding occurred once opportunities to respond, access to reinforcement, and
an animal made contact with this contingency. The shock probability, frequency, or density.
animals consistently responded, producing the situa¬
tion defined by the cue light, in which shocks were
reliably preceded by the five-second tone. Cues Denoting Limited Response Opportunity,
After stable responding was established, Badia et Apart from the Aversive Situation
al. varied the shock frequency in the signaled condi¬
tion, using mean shock-shock intervals of 2, 1.0, or Free-operant procedures for examining negative
0.5 min. Meanwhile, they held the shock frequency in reinforcement have permitted study of schedule effects
the unsignaled condition constant at one shock per that were inconceivable within traditional escape and
four min. By not pressing the lever, the animals could avoidance procedures. They have produced diverse
always maintain this condition in which shock fre¬ but orderly behavioral phenomena that complement
quency remained at one shock per four min. All rats those produced by positive reinforcement. At the same
persisted in responding when signaled shocks averaged time, response patterns on schedules of negative rein¬
one per two min, and three of four animals persisted forcement have sometimes been attributed to contin¬
when signaled shocks averaged one per min. That is, gencies or principles that are not readily manipulable
responding was maintained by production of a situa¬ or testable within a free-operant situation itself. The
tion in which signaled shocks occurred four times as behavior often interacts with a schedule in such a
frequently as the unsignaled shocks that would occur way as to prohibit clean manipulation of the critical
m the absence of responding. A control experiment, variable. Analysis of such contingencies has required
with all shocks unsignaled, established that the vari¬ a redeparture from the free-operant paradigm to spe¬
ous shock frequencies were discriminable by the an= cial forms of limited-opportunity procedures. I shall
examine the use of limited-opportunity schedules for
imals,
The interpretation given these results by Badia et addressing three questions regarding negative rein¬
al. was that the addition o£ preshock warning signals forcement, One deals with the isolation of the con¬
effectively provided substantial shock-free intervals in tingency between response and shock, independently
the absence of those signals. Most responses produced of shock frequency and responses rate. Another deals
the light-correlated situation denoting shock-free peri¬ with the role of temporal discriminations in behavior
ods; the averaging or integrating of intermittent on shock-delay procedures. The third question con¬
shocks occurred only within the boundaries estab¬ cerns the relative contributions of short-term shock-
lished by the warning stimuli. In addition, it is clear delay and long-term shock-frequency reduction. Inter¬
that the preshock “warning signals7’ did not increase estingly, Shimp (1973) has argued for the positive
the aversiveness of the more molar situation defined reinforcement case that an analysis of the variables
by the presence of the cue light, as might be expected operative in standard schedules of reinforcement often
if they were to be interpreted simply in terms of con¬ requires a departure from those schedules. Some such
ditioned aversiveness. To the contrary, they decreased analyses have used limited-opportunity procedures
the aversiveness of that situation. Characterized as very much like those to be considered here (e.g.
affecting integration of shocks over time, they can be Jenkins, 1970).
said to have produced discriminable shock-free inter¬
vals. Recall that Dinsmoor and Sears (1973) demon¬ Isolating Contingency and Frequency
strated dimensional control by stimuli denoting shock-
free intervals, independently of stimuli denoting When discussing shock-deletion procedures without
availability of shock or setting the occasion for the added cues, I noted that a contingency between re¬
response. They, and a number of others (e.g. Bolles, sponse and shock-deletion can be efficiently specified
1970; Rescorla, 1969) have argued for the potency of in terms of two probabilities. These are the probabil¬
discriminable shock-free intervals in reinforcing be¬ ity of shock if a response occurs, and the probability
havior. An attractive feature of the stimuli denoting of shock given no response. As pointed by Church
shock-free periods for the analysis of behavior is not (1969), Catania (1971) and several others, the degree of
only their apparent potency as reinforcers, but also contingency can be described as the difference be¬
their comparatively low degree of confounding with tween the two probabilities. However, in uncued free-
various aspects of negative reinforcement procedures. operant procedures this specification tells only part
Denoting “timeout” periods, their simplicity contrasts of the story, for one must also specify the interval for
with other stimuli that are multiply correlated with which the probability operates. For example, in a
Philip N. Hineline 399

standard fixed-cycle shock-deletion procedure, the

probabilities are specified in relation to the shock-
shock interval. An example is shown in part I of
Figure 7, where the probability of shock given a
response is zero, and the probability given no response
is one. In schedules defined under the tA — tD sched¬
ules devised by Schoenfeld and his associates (e.g. part
III of Figure 7; also, see Kadden, Schoenfeld, 8c Snap¬
per, 1974), the probabilities are specified in relation to
tD, which is a portion of the shock-shock interval.
Within either of these situations, the duration of the
period for which the probabilities are specified has
strong implications. That period determines both the
maximum possible shock rate, and the minimum re¬
sponse rate required to achieve a given amount of
shock reduction. In uncued situations these are both
powerful variables. Fig. 21. The left panel shows response probability as a func¬
The confounding of rate with contingency can be
tion of the conditional probability of shock given no response,
P^Sk)— R), while the probability of sho^k given a response, P
mitigated by adding a cue that coincides with the (5k/R), was constant at zero. The right panel shows response
probabilities when the shock probabilities for responding and
period for which the probabilities are specified. Cue
for not responding were equal, at values greater than 2ero.
duration is still a variable to be considered, but to
Each plot represents a different rat. Data are taken from the
some degree it can be moved to the background along final three days of exposure to the schedule values, and the

with overall response and shock rates. If the cue is functions are plotted from left to right in the order in which
removed by a response, no more tlian one response
the probability values were studied. (After Neffinger §s Gibbon,
1975. (■c) 1975 by the Society for the Exp erimental Analysis of
can occur per cue presentation, The shock can be Behavior, Inc.)
limited to a maximum of one per cue as well so that
both can be described by probabilities instead of fre*
quencies. response, while holding the probability of shock given
Neffinger and Gibbon (1975) have reported an ex¬ a response constant at zero. With this manipulation,
response probabilities remained relatively high until
periment that separates frequency and contingency in
this way. Using rats, they superimposed a tone on a the probability of shock given no response was yery
basic fixed-cycle shock-deletion procedure that was low.
virtually identical to that diagrammed above in part The right panel of Figure 21 shows the effects of
I of Figure 7. With no responding, a 0.5 sec shock was subsequently eliminating the contingency, by making
delivered every 20 sec; a tone was on during the inter¬ the probabilities of shock equal whether or not the
vening 19.5 sec during which a response could affect rats responded. The performances separated into two
shock. A response in the presence of the tone termi¬ distinct classes: some animals quit responding; others
nated the tone, and deleted the shock due at the end responded persistently. With further analysis, Neffin¬
of that cycle. The tone came on again when the shock ger and Gibbon argued that one class was sensitive
had been deleted, starting a new 20-sec cycle. only to contingency; the other was sensitive both to
The addition of the tone converted this to a dis¬ contingency and to absolute shock density. Inter¬
crete-trial procedure. At least Neffinger and Gibbon estingly, these two classes of animals could not be dis¬
treated it as such, for they ignored responses in the tinguished in the left side of the figure, or in original
absence of the tone. They trained their rats with the training. As the authors noted, the distinctness of two
conventional probabilities: probability of a shock was classes of performance may have resulted partly from
zero if a response had occurred; probability of a shock their stringent subject selection procedures.
given no response was one. Also, in this training they
discarded approximately 70 percent of their animals, The Role of Temporal
which failed to meet a stringent performance cri¬ Discriminations in Behavior on
terion. Then the two probabilities were manipulated Shock-Delay Procedures
independently, but with interspersed retraining at the
original values. The left panel of Figure 21 shows the It is well established that subjects trained on a
effects of decreasing the probability of shock given no shock-delay schedule distribute their responses non-
400 NEGATIVE REINFORCEMENT AND AVOIDANCE

randomly in time. The momentary probability of a illustrated above in Figure 21, of dimensional control
response typically increases as time elapses during the by stimuli paired with shock-free short post-response
shock-shock or response-shock interval (Anger, 1963; times.
Sidman, 1966). Gibbon (1972) delineated some char¬ By all three accounts—Anger, Rescorla, and Dins¬
acteristics of the timing process needed to produce moor and Sears—the animal’s spacing of responses in
these response distributions, arguing that it has time reflects an underlying process fundamental to
“scalar” properties, functioning with units propor¬ the reinforcement of responding on the shock-delay
tional to the intervals being timed (such as response- procedure. However, there is another likely interpre¬
shock intervals). Libby and Church (1974) provided tation of the spaced responding. A shock-delay pro¬
additional evidence for the scalar property, showing cedure, viewed as a procedure for reinforcement by
with a free-operant shuttle response, that rats’ spacing shock-frequency reduction, provides differential rein¬
of responses is proportional to the response-shock in¬ forcement of spaced responding. Within the response-
terval. Some interpretations of the conditioning proc¬ shock interval, the more widely spaced the responses,
ess have been founded upon the observed response the greater the magnitude of reinforcement for a
distributions. For example, Anger (1963) used it to given response. This view suggests that the production
enable two-factor theory to explain conditioning on of spaced responding is a second-order process, not
uncued shock-delay procedures. He proposed that the critical to the maintenance of responding per se, but
animal’s internal stimuli correlated with times close affecting only the distribution of responses. Sidman
to the response-shock interval become aversive through (1966) supported this latter view with a variety of
pairing with shock. When responses occur these were arguments. Some of his supporting data came from
said to be replaced with other internal time-correlated concurrent schedules; some came from demonstrations
stimuli that are not closely paired with shock. Re¬ that temporal discriminations often do not appear
sponding, then, was said to be reinforced by removal until after responding is well established.
of conditioned aversive stimuli whose temporal prop¬
Hineline and Herrnstein (1970) addressed the ques¬
erties are referenced to the response that produces the tion with an experiment that eliminated the possible
shock-delay, establishing a “time-zero.”
differential reinforcement based on shock-frequency
Rescorla (1968) supported a two-factor interpreta¬
reduction. The experiment used a modified version of
tion by demonstrating the potential for Pavlovian
the fixed-cycle shock-deletion procedure described
conditioning based on features of the shock-delay pro¬
earlier (part I of Figure 7). As before, brief shocks oc¬
cedure. These features involve the pairing of re¬
curred every 20 sec, defining a 20-sec cycle. Also as in
sponses and shocks in fixed time relations. He pre¬
the free-operant fixed-cycle procedure, the first lever-
trained his subjects on a shock-delay procedure
press within a cycle cancelled the shock due at the end
(SS = 10, RS = 30) and then gave independent train¬
of that cycle. In addition, the lever was retracted from
ing with a Pavlovian trace conditioning procedure in
the chamber immediately after the response permit¬
which onset of a 5-sec tone preceded unavoidable
ting only one response per cycle, and was re-extended
shock in the same time relationship as the response-
to initiate the new cycle; a buzzer was correlated
shock interval of the shock-delay procedure. That is,
(positively or negatively for different animals) with
shock followed the offset of tone by 25 sec. Then,
when he superimposed the tone upon the ongoing presence of lever, also denoting the opportunity to re¬
shock-delay procedure, he found response acceleration spond. Thus, each response eliminated only one
in phase with the 25-sec tone-shock interval, showing shock, no matter where it occurred within the 20-sec
that the temporal features of the Pavlovian procedure cycle. The beginning of a cycle was cued either by a
affected responding maintained by shock-delay. Res¬ shock or by stimulus changes correlated with insertion
corla presented this as evidence for a mediating emo¬ of the lever into the chamber. Distributions of re¬
tional response which would normally be conditioned sponses within the cycle were compared to inter¬
to feedback stimuli from the shock-delay response. response time distributions and shock-response time
I have argued above that preshock stimuli are bet¬ distributions of a conventional shock-delay procedure
ter characterized as discriminative stimuli, rather than with shock-shock and response-shock intervals of 20
as conditioned aversive or emotion-producing stim¬ seconds.
uli. Dinsmoor and Sears (1973) provided such an ac¬ As expected, animals on the shock-delay schedule
count, proposing a “fading trace” initiated by each frequently showed temporal discriminations, indicated
response and denoting shock-free periods. They sup¬ by increasing probabilities of responding late in the
ported their interpretation with the demonstration, response-shock interval. Animals on the fixed-cycle
Philip N. Hineline 401

schedule with limited response opportunity some¬ concluded that timing is more collateral than causal
times showed greatest probability of response late in the in behavior on shock-delay procedures.
cycle, indicating a timing process. But sometimes the
probability of response decreased with time during
the cycle, and sometimes it was fairly constant, indi¬
B-6
cating randomly spaced responding. These changes
were not correlated with changes in number of shocks
deleted. The changes in distribution of responses
within a cycle were sometimes sudden, but more often
were gradual and systematic. One of the widest and
most systematic changes of response distribution is
shown in Figure 22. In this figure, the slope of each
function is the important feature. Positive slopes in¬
dicate timing, with the momentary probability of
response increasing with time since trial (cycle) onset.
A horizontal slope indicates randomly spaced re¬
sponding, with the momentary probability of response
constant over time. This rat clearly showed slowly
shifting response distributions, with continuity of per¬
formance from week to week, but with wide swings in
the distribution of responses within the 20-sec cycles.
The distribution of responding was unrelated to pro¬
ficiency of performance, for the animal took fewer
than one percent of the possible shocks in any session.
The data of this experiment clearly revealed that
processes producing spaced responding need not be
critical to the maintenance of responding. This chal¬
lenges the two-factor interpretations, since the stimu¬
lus change initiating each cycle should have provided
excellent cues for temporal conditioning. Dinsmoor

and Sears’s interpretation of timing is perhaps least

challenged by this experiment, given that a warning
stimulus (presence of lever, and correlated buzzer) was
present until the shock-deleting response had occurred,
and the response produced the same feedback when¬
ever it occurred. However, their interpretation still
would not predict the observed gradual changes in
spacing of responses within a cycle, from timing to
non-timing and back again.
On the other hand, Sidman’s notion regarding the
rig. zz. Changes in conditional probability of response as a
basis for spaced responding is not supported either,
function of time since the beginning of a cycle (or trial) for
for the experiment shows that spaced responding need one rat, The figure J5 based upon eight eonsetutiyc weeks dur¬
not be based on differential reinforcement through ing conditioning with the fixed-cycle shock-deletion procedure,
and limited response opportunity. The conditional probabilities
shock-frequency reduction. The experiment does sup¬ were computed by dividing the number of responses in a
port Sidman’s contention that spacing of responding 2-sec class interval by the number of opportunities for response
is a second-order process, and that Pavlovian pairings in that class interval. The computation compensates for the
fact that each response in a given class interval eliminated the
with temporal stimuli or stimulus traces is not neces¬ opportunity for response intervals later in that cycle. To clarify
sary to the maintenance of responding. As I described the continuity of performance changes, plots for successive
earlier, Neffinger and Gibbon (1975) used a procedure weeks, numbered at the right of the figure, are displaced up¬
ward, in consecutive order. The size of scale indicated on the
similar to that of Hineline and Herrnstein (1970), ex¬ ordinate is valid for all plots. Absolute values for the 0-2 sec
cept that their lever did not retract when responses class interval, for the successive weeks, were .32, .45, .76, .72, .70,
.55, .45, and .22, respectively. (From Hineline & Herrnstein, 1970.
occurred. They too found the distributions of respond¬
(c) 1970 by the Society for the Experimental Analysis of Be¬
ing to be unrelated to proficiency of performance, and havior, Inc.)
 402 NEGATIVE REINFORCEMENT AND AVOIDANCE

Dissociation of Shock-Delay and the chamber; a shock occurred at the eighth second,
Shock-Frequency Reduction and the lever retracted at the 10th second for the
remainder of the cycle. As before, a response removed
Two variables have been confounded in nearly all the lever and delayed the shock. However, this was
procedures for negative reinforcement. Responding accomplished by production of a 10-sec interval with
reduces overall shock-frequency or density, and at the the lever out, and a shock occurring 8 sec after the
same time it produces short-term delay of shock, a response. There was still one shock per cycle, but
shock-free interval. Even the random-shock procedures responses shortened the cycle, increasing overall shock
of Herrnstein and Hineline (1966) and of de Villiers frequency. Response rates of the two rats with previ¬
(1972, 1974) can be described in terms of average time ous training decreased systematically to negligible
between response and shock, contrasted with time be¬ levels, within five sessions and nine sessions respec¬
tween shocks in the absence of responding. Such a tively. Eleven naive rats placed directly on this pro¬
translation is mathematically tautological, in that av¬ cedure never responded on more than 30 percent of
erage delay is merely the reciprocal of shock fre¬ the cycles. Typically, responding rose to 20 or 25 per¬
quency. Still, it is also a change in emphasis from cent of the cycles during a few of the early sessions,
molar, long-term controlling relations to short-term and then systematically decreased to zero.
controlling relations. The first of these two experiments indicates that
In the experiment described just above, the molar short-term shock-delay can function as a remforcer
and molecular features of behavior were dissociable independently of overall shock frequency. The pro¬
in that the molar features—degree of shock-frequency cedure used a variable response-shock interval (10 to
reduction, or probability of response per trial—were 18 sec, depending on the placing of the response
stablewhile the mtcrastructures of performance were within a cycle), and constant shock frequency. In the
highly unstable. The experiments that follow focus second experiment, apparently shock-delay was not
on procedural dissociation of the molecular and molar sufficiently potent to override an opposing change in
aspects of negative reinforcement, shock frequency. This procedure provided a shorter,
In these experiments the opportunity to respond constant response-shock interval (8 sec), as well as in¬
was again limited, to permit precise control of the creased shock frequency contingent upon responding.
delay between response and shock with no interven¬ A complicating feature of both procedures was the
ing responses to complicate the relationship. Delivery delivery of shocks sometimes in the presence of the
of a shock at times between response opportunities lever and sometimes in its absence. While shock fre¬
permitted the shock frequency to be manipulated quencies in the presence and in the absence of the
somewhat independently of short-term delay. The first lever (each accumulated over all cycles) did not indi¬
such procedure (Hineline, 1970) provided for re¬ cate this as the source of response strength,4 another
sponses to produce shock-delay without affecting over¬ set of procedures was devised to eliminate possible
all shock frequency. As shown in part I of Figure 23, confounding effects of these relative shock frequencies.
the procedure was based on fixed 20-sec cycles. Time The revised procedure (Hineline, 1969) permitted
specifications within a cycle began with insertion of complete dissociation of the opportunity to respond
the lever into the chamber. In the absence of respond¬ from the situation in which shocks occur. It also per¬
ing the lever was accessible for alternate 10-sec peri¬ mitted wider ranges of shock-delay and shock fre¬
ods, with a shock occurring at the eighth second of the quency. As shown in Part III of Figure 23, all variants
lever’s presence. A response prior to that point pro¬ of the procedure were identical if no response oc¬
duced immediate removal of the lever and delayed the curred. That sequence of events is portrayed just
shock from the 8th to the 18th second of the cycle. above the dotted line in the figure: a cycle began
There was still one shock per 20 sec. This procedure with insertion of the lever into the chamber; it re¬
maintained responding in each of five rats exposed to tracted at the 10th sec, a shock (1.0 sec; 0.8 mA) was
it, two of which had prior training with a conven¬
tional cued shock-delay procedure, and three of which
were experimentally naive. 4 Two of the three naive animals placed on the constant
frequency procedures persistently responded so as to produce
These results are compared with a second experi¬
substantially higher shock rates in the absence of the lever than
ment, using the procedure diagrammed in part II of in its presence. Herrnstein (1969) showed that the principle of
Figure 23. In the absence of responding this pro¬ shock-frequency reduction under stimulus control can account
cedure was identical to the one just described: 20-sec for this experiment if the computation of shock frequencies is
based only on stimulus exposures (presence or absence of lever)
repeated cycles began with insertion of the lever into during which shock occurred.
Philip N. Hineline 403

|-10 Sec-1
Time
Lever In
No Response Lever Out

Response Each Cycle J~l -f sl~~f .s-T

Response On
Alternate Cycles J sj-*1_rt_ i_T

J-10 Sec-J
Time

Lever In
No Response Lever Out J |_\

Each Cycle J f s I H, , ~ j L_ j—L

■=i—f
Response On
Alternate Cycles J-L cr

I-10 Sec-1
JL———I———i- — I —JL——■!■ k

No Response Lever out J b. r Fig. 23. Diagrams for three

shock-delay procedures with re¬
sponse opportunity limited, in
order to isolate shock-delay
Resp. Deletes Shock from shock-frequency. Time is
j-1_ ----r represented linearly frem lelt te
right, as indicated on the top

Extinction line of each diagram. Upward

displacement of a line indicates
insertion of the lever at the be¬
Resp. Delays Shock j—E -s_r ginning of a cycle. Downward
displacement indicates retrac¬
—2_2_ tion of the lever. An “S” marks
—2_5 s_ the delivery of a shock; an “R”
indicates the occurrence of a
s s s s_
response. (From Hineline, 1969,
s s s s s_ 1970.)

delivered at the 11th second, and a new cycle began achieved at the expense of increases in overall shock
with reinsertion of the lever at the 60th second. frequency. These procedures were identical to the
A sequence of procedures was run; each provided a third procedure, except that one, two, three, or four
different consequence for responding. The first was a additional shocks were delivered in cycles where the
shock-deletion procedure. If a response occurred, the response occurred. For two shocks per response, the
lever retracted immediately and no shock was deliv¬ shocks occurred at the 39th and 47th secs of the cycle.
ered on that cycle. The second was an extinction pro¬ The larger numbers of shocks were added with the
cedure. If a response occurred, the lever retracted im¬ spacing indicated at the bottom of the diagram, filling
mediately, but the shock was still delivered at the 11th in the interval between the 39th and 47th secs.
sec. The third was a shock-delay procedure with con¬ Each rat was first trained until stable on the first,
stant overall shock frequency. If a response occurred, shock-deletion procedure. Next, each rat was placed
the lever retracted immediately and the shock was on the second (extinction) procedure until responding
delayed from the 11th to the 39th sec. This was fol¬ decreased to less than 5 percent of the trials. The
lowed by shock-delay procedures where the delay was third procedure, shock-delay, was instituted next; if a
404 NEGATIVE REINFORCEMENT AND AVOIDANCE

rat failed to respond on this he was reconditioned on deletion phase has been eliminated from subsequent
the shock-deletion procedure and then moved to the experiments with the same set of procedures with
shock-delay procedure. If a rat developed stable re¬ 60-second cycles. These experiments have encountered
sponding on the third procedure, the final sequence difficulties of conditioning that have also sometimes
was instituted, providing two, three, four, and then been reported with more conventional negative rein¬
five shocks per response. forcement procedures. Hence, while the following,
One of the six rats responded on over 90% of the rather autobiographical series will tell something
cycles on the initial, shock-deletion procedure. The about the contributions of shock-delay and shock-
others stabilized at approximately 80%, 75%, 70%, frequency reduction to negative reinforcement, it will
20%, and one animal seldom responded at all, even also illustrate strategies that have been used to deal
with hand-shaping. All animals’ responding ceased with difficulties of initial conditioning with negative
within a very few sessions of exposure to the second reinforcement. This will prepare the way for a discus¬
procedure where a shock occurred at the 11th second sion of special considerations regarding initial condi¬
independently of responding. Only two animals re¬ tioning with negative reinforcement.
sumed responding when transferred directly from this In an unpublished experiment, twelve naive rats
to the shock-delay procedure, the two that had re¬ were placed on the procedure diagrammed by the
sponded most on the initial, shock-deletion procedure. bottom line of Figure 23. The only response conse¬
The others failed to respond on the third procedure quences were lever-removal and shock-delay. None of
even after reconditioning with shock deletion. These the twelve acquired stable lever-pressing. Hand-shap¬
two were then run on procedures in which responses ing with some animals, by making lever-removal and
delayed shock but increased shock frequency. As the shock-delay contingent upon successively closer move¬
number of shocks per response was increased every ments toward the lever, resulted in their hovering
five sessions, from two, three* four, and then to five over the lever and spending most of their time near it,
shocks per response, they continued to respond on but did not produce reliable lever-pressing. The an¬
nearly all cycles. imals often tended to remain motionless during the
Thus, in some animals, delay of shock maintained 10-sec opportunity period, moving around at other
responding even when it produced a fivefold increase times.
in overall shock frequency. However, these animals In light of these results, supplementary positive
had prior training on a shock-deletion procedure, and reinforcement was used, in a manner similar to that
although their responding had subsequently been ex¬ used by Riess (1970) with a conventional shock-delay
tinguished with a noncontingent shock procedure, the procedure, and by Giulian and Schmaltz (1973) with
results should be interpreted writh some caution. a discrete-trial, cued shock-deletion procedure, to
Response-contingent shocks, and sometimes even non¬ bring the animals’ behavior into contact with the
contingent shocks, can maintain responding in an¬ delay contingency without a history of shock reduc¬
imals with histories of negative reinforcement (e.g. tion. In collaboration with G. D. Smith, several vari¬
McKearney, 1972; Powell, 1972). ants of this approach were used on a few animals with
In order to be sure that prior negative reinforce¬ moderate success. One is described here; it demon¬
ment based on combined shock-delay and shock- strated the clear and reliable, but weak reinforcing
frequency reduction would not contaminate the effects of delay-of-shock with constant shock frequency.
dissociation of these two variables, the initial, shock- The sequence of procedures and results is shown in

Fig. 24. Lever-presses per ses¬

sion for rat DS-6, on a sequence
of procedures: A) Responses
produce food with no shocks
delivered; B) Responses produce
food and delay of shock; C)
Same as B, but free access to food
in home cage; D) Discontinue
positive reinforcement; responses
produce shock delay only; E)
One shock delivered at the 16th
sec of each cycle; F) One shock
delivered at the 47th sec of
each cycle.
Philip N. Hineline 405

Figure 24. Prior to training the rats were food- permits complete independence of shock-delay and
deprived; then lever-pressing was established and shock-frequency changes made contingent upon re¬
maintained with delivery of sweetened condensed sponding. This flexibility was used in a recent experi¬
milk as reinforcer (part A of Figure 24). Then a ment by Lambert, Bersh, Hineline, and Smith (1973),
slightly modified version of the shock-delay procedure pitting shock-frequency reduction against shock delay,
described above (part III of Figure 23) was added, with a different part of the range of time values per¬
with shock intensities increased by increments over mitted by a 60-sec cycle. In the absence of responding
sessions (part B of Figure 24). This shock-delay pro¬ the lever was available for 10 sec, and then five brief
cedure differed from that diagrammed in Figure 23, in shocks were delivered, one per second, between the
that: with no response the shock occurred at the 16th 11th and 15th sec. A response during the first 10 sec
instead of the 11th sec; responding delayed the shock retracted the lever and deleted the five later shocks,
until the 47th sec; whenever it occurred, shock was but produced one immediate shock. Thus, responses
preceded by a three-second tone. These changes were could reduce shock frequency at the expense of reduc¬
made to minimize response-opposing effects of signaled ing shock delay to zero. Two rats placed directly on
versus unsignaled shocks (e.g. Badia et al.7 1973), since this procedure initially responded on approximately
shock immediately preceded by lever removal could 60 percent of the cycles for the first few sessions; then
be said to be signaled shock, while the delayed shock their responding dropped systematically to zero by
could not. When shock intensities were increased to session 13. Two other animals, trained with a pro¬
1.0 mA, a level that supports lever-press responding cedure where a response could terminate the stream
on conventional shock-delay procedures, food depriva¬ of five shocks, also ceased responding on the procedure
tion was discontinued by giving free access to lab permitting only shock frequency reduction with con¬
chow in the home cage, but delivery of condensed tingent zero-delay shock. Thus, shock frequency re¬
milk contingent upon lever-presses was continued duction also appeared to be a weak reinforcer when
(part Cm Figure 24). When body weights had recov¬ pitted against a reduction in shock-delay,

ered to normal free-feeding levels, the positive rein¬ Perhaps the weakness of these reinforcing effects
forcement was discontinued (condition D in Figure should be interpreted in light of a notion advanced
24), and when performance was stabilized to the point by Jenkins (1970), with reference to limited-opportu¬
of showing no consistent trends, two control or extinc¬ nity positive reinforcement procedures. It is a notion
tion procedures were used to insure that indeed it was very similar to the one I advanced earlier, regarding
shock delay that maintained the responding. First, the effects of cues on the averaging of shocks over
(condition E) all shocks were delivered at the 14th time:
sec, independently of responding. This procedure was
comparable to the extinction procedure of the pre¬ The possibility exists that distinctively differ¬
ceding experiment, and the results were similar; re¬ ent occasions for nonreinforcement and rein¬
sponding dropped quickly to negligible levels. Rein¬ forcement confine or localize the effect of rein¬
statement of the shock-delay procedure produced a forcement while more similar occasions extend
slow, systematic, but variable reacquisition over some the interval over which delayed reinforcement
40 sessions. In the next control procedure (condition supports a prior response. (Jenkins, 1970, p. 105)
F), all the shocks were delivered noncontingently at
the delayed position, the 47th sec of the cycle. On this To the extent that this is true, confining the response
procedure responding slowly but consistently declined opportunity to a clearly discriminable situation dis¬
toward zero, until reversed by a return to the shock- tinct from the effects of the response weakens the rein¬
delay procedure which again produced, slow, variable, forcing effect of either short-term delay or shock-
but systematic reacquisition. Thus, in an animal with frequency reduction.
no history of response-produced shock deletion, delay The reinforcing effect may be weak for another
of shock appeared to be an unconfounded but fairly reason, however. Acquisition of lever-pressing with
weak reinforcer. Several animals on this procedure conventional shock-delay procedures is most easily
did not persist in responding when the shock intensi¬ accomplished with short shock-shock intervals. Per¬
ties were raised; others responded at lower levels when haps the change to long cycle length, attractive be¬
the positive reinforcement was discontinued. cause it permits longer delays, is relatively ineffective
It is clear from the procedural diagrams of Figure because it imposes a low basal shock rate. If the con¬
23 that the fixed-cycle procedure separating the oppor¬ ventional shock-delay procedures, providing both
tunity to respond from the periods when shocks occur shock-delay and shock-frequency reduction within the
406 NEGATIVE REINFORCEMENT AND AVOIDANCE

situation where the response occurs, are ineffective for lar responses by reinforcement. With positive rein¬
producing initial acquisition of lever-pressing with forcement, conceptual concern over initial acquisition
similar long intervals, then the weak effects in Fig¬ has been allayed by appeal to the principle of shap¬
ure 24, and in the work by Lambert et al. (1973) are ing. This principle, of response selection through
not surprising. reinforcement of successive approximations to a spe¬
One strategy for studying weak reinforcing effects cified response, was convincingly demonstrated in
is to use a response more easily conditioned than the Skinner’s early work, and is routinely replicated in
lever-press. A likely candidate is a shuttle response, undergraduate laboratories. Over the years the as¬
which has been shown to be readily strengthened sumed generality of this principle has been occasion¬
with shock reduction (Bolles, 1970; Riess, 1971; Riess ally challenged (e.g. Breland 8c Breland, 1961), but
8c Farrar, 1972). In unpublished work, J. Harman and only recently has that challenge become a central
I have recently verified the usefulness of this approach topic supported by systematic work on autoshaping
by subjecting four rats to a shock-delay procedure (Brown 8c Jenkins, 1968) and other response-selection
with a shock-shock interval of 60 sec, and an RS in¬ effects (e.g. Staddon 8c Simmelhag, 1971).
terval of 120 sec. With a free-operant shuttle response, The present chapter has also emphasized mainte¬
all animals quickly and consistently reduced their nance rather than acquisition, partly because an in¬
shock frequencies to less than one per 10 min. Lam¬ tegrative account of negative reinforcement analogous
bert et al. (1973), using the shock-frequency reduction to that of positive reinforcement seems needed at this
procedure with nondelayed shock described above, time. Also, as argued by Morse and Kelleher (1966), it
found a shuttle response to be maintained slightly appears that once control over a response with nega¬
more easily than lever-pressing. In an additional un¬ tive reinforcement is achieved, the effects of schedul¬
published experiment, J. Harman and I used a shut¬ ing negative reinforcement are much like those of
tle response in conjunction with the shock-delay pro¬ scheduling positive reinforcement. However, initial
cedure diagrammed at the bottom of Figure 23. acquisition with negative reinforcement has consis¬
Moderate to low response rates were obtained, com¬ tently been of more concern than with positive,
parable to those of the latter parts of Figure 24. The probably because using the former, experimenters
shuttle response did not require the addition of posi¬ have often encountered difficulty in producing a chosen
tive reinforcement early in training to produce these response. The difficulties have tended to arise in rela¬
effects. Thus again, shock-delay and shock-frequency tion to particular species, and to particular responses.
had independently isolable but weak reinforcing
effects when response opportunity was discriminably
Species Differences in Acquisition
separated from the situation in which shocks oc¬
curred. According to informal laboratory lore, “monkeys
Earlier in this chapter we saw that in the case of and dogs avoid better than rats do,” and until very
weak reinforcing effects of timeout from the shock- recent years the behavior of pigeons seemed virtually
delay procedures, a two-response or choice situation immune to negative reinforcement based on shock
provided more sensitive measures than did measures reduction. However, it is impossible to deduce exactly
based on maintenance of a single response. Further, in comparable situations for different species and choose
positive reinforcement studies of reinforcement mag¬ apparatus configurations, shock intensities, and cue
nitude and frequency, choice procedures provided by intensities or modalities on the basis of the subjects’
concurrent chains have also proven especially sensi¬ differing shapes and receptors. Hence, it is difficult to
tive. It may be that choice procedure would provide devise valid cross-species comparisons. The case of
an effective strategy for the isolation of shock-fre¬ pigeons is especially instructive in this regard.
quency reduction and short-term shock delay as well, In view of several unpublished, unsuccessful at¬
permitting systematic assessment of the relative po¬ tempts to produce or maintain key-pecking with
tency of differing magnitudes of each. shock-delay or shock-reduction, Rachlin and Hineline
(1967) devised a shaping procedure based on termina¬
tion of trains of shock pulses with slowly increasing
CONSIDERATIONS REGARDING intensity. They successfully, if arduously, shaped the
INITIAL ACQUISITION key-peck response and maintained repetitive respond¬
ing within a limited range of fixed-ratio and fixed-
Operant conditioners have tended to emphasize interval schedules (Hineline 8c Rachlin, 1969a, b). More
maintenance of responding and of response patterns, recently, Ferrari, Todorov, and Graeff (1973) devel¬
rather than initial acquisition of control over particu¬ oped these shaping procedures further and maintained
 Philip N. Hineline 40 7

stable key-peck responding with a shock-delay pro¬ positive reinforcement. Earlier, I noted comparable
cedure. Using an SS interval of two seconds, manipula¬ observations by Riess and Farrar (1972) that in rats a
tions of the RS interval resulted in performance shuttling response is usually acquired with negative
changes similar to those observed by Sidman with rats reinforcement more rapidly and reliably than is a re¬
(Sidman, 1953). However, they too reported the need petitive lever-press response. Further along similar
for hours of tedious shaping during the initial train¬ lines, Keehn (1967) observed that the response of
ing. Schwartz and Coulter (1971) found that key- running in a wheel was acquired more quickly with a
pecking established with positive reinforcement did shock-delay contingency than was a lever-press re¬
not readily transfer to negative reinforcement. How¬ sponse. Bolles (1970) found differences in acquisition
ever, more recently, Lewis, Lewin, Stoyak, and Mueh- on a shock-deletion procedure, when comparing the
leisen (1974) were able to transfer control of key- responses of running in a wheel, turning, or rearing.
pecking from positive reinforcement to negative Bolles (1970) has discussed these differences in rela¬
reinforcement based on termination of a cue and tion to “species-specific defense reactions.” He noted
deletion of frequent irregularly spaced shocks, and that in the genetic histories of small animals, random
Foree and LoLordo (1974) succeeded in a similar “trial-and-error” patterns of behavior must have been
transfer, to a shock-delay procedure with added cues. maladaptive in aversive situations or situations ac¬
It is clear, however, that with the pigeon, responses companied by intense or novel stimulation. While the
other than key-pecking are more easily established first inappropriate response producing loss of food
with negative reinforcement, Hoffman and Fleshier leaves an animal still hungry, an inappropriate re¬
(1959) found a head-lifting response to be slightly sponse in an aversive situation may very well leave
more susceptible to shock-reduction, and MacPhail the animal dead. Bolles proposed that in such situa¬
(1968) obtained reliable, high-probability shuttle re¬ tions the responses of freezing, fighting, or fleeing
sponding with pigeons on a cued shock-deletion pro¬ would be most generally adaptive, so animals that
cedure. Later, Smith and Keller (1970) and Foree and tended to behave in these ways in these situations
LoLordo (1970) easily accomplished conditioning with would tend to survive and reproduce. In their off¬
free-operant shock-delay procedures contingent upon spring, these behavior patterns could be expected to
a foot-treadle response. Klein and Rilling (1972) have predominate in aversive and novel situations. This
built upon this by manipulating the parameters of line of reasoning can lead to two interpretations that
shock delay and of shock intensity, replicating in con¬ are not mutually exclusive. One proposes that par*
siderable detail the results obtained with the lever, ticular responses are especially assoeiable or condi-
press in rats (Boren, Sidman, & Herrnstein, 1959; tionable with a particular stimuli, The other analyzes
Sidman, 1953). As described above, Dinsmoor and the differences in terms of the operant levels of vari¬
Sears (1973) used the same schedule and response to ous responses in aversive situations, Bolles has clearly
demonstrate dimensional control by response-produced implied that the former is more important, arguing
stimuli, and Klein and Rilling (1974) also used the that negative reinforcement is basically selection from
treadle-press in pigeons to study generalization effects among species-specific defense reactions; other re¬
with tones accompanying the shock-delay component sponses are deemed uneonditionable, From this view¬
of a multiple schedule, point, the conditioning of a particular response with
This brief history shows that some aspects of ob¬ negative reinforcement can only be aehieved to the
served species differences m behavioral processes origi¬ extent that the response is part of a species-specific
nate as much in the method used for examining them, defense reaction. Seligman and Hager (1975), develop¬
as in any differing capacities of different species. In ing an earlier discussion by Seligman (1970), have
addition, it points up the importance of response spe¬ opted for a similar interpretation. They used a con¬
cification for ease of conditioning, a consideration cept of “preparedness” to make explicit the proposed
that has had considerable impact on recent interpreta¬ susceptibility of a response to the effects of (or in their
tions of negative reinforcement. terminology, the associability of a response to) a par¬
ticular reinforcer.
Response Differences: Apparent Susceptibility Both of these interpretations relate well to the ap¬
to Reinforcement parent differences of conditionability described above.
They also account for Bindra and AncheFs (1963)
I just reviewed evidence that in pigeons treadle- demonstration that immobility can readily be condi¬
press or shuttle responses have been more easily tioned with a shock-removal contingency, and the
brought under control of negative reinforcement than observation by Keehn (1967) that lever-holding is
has the key-peck, which is so readily controlled by more readily produced and maintained with negative
408 NEGATIVE REINFORCEMENT AND AVOIDANCE

reinforcement than is repeated lever-pressing. Both sponse is explained in terms of attributes of the
immobility and lever-holding can be components of response itself. For example, Keehn (1967) noted that
the species-specific response of freezing. The first in¬ the relatively slow acquisition of lever-press respond¬
terpretation outlined above is especially supported by ing, compared to wheel running, may occur because
an experiment by Foree and LoLordo (1973). They in the latter a continued response pattern is permit¬
found that after a combined light and tone were used ted, while the former requires repeated production
as a cue in an appetitive situation the light had and discontinuance of a response pattern. He com¬
stronger effects on behavior, whereas after the same pared acquisition of running with acquisition of lever¬
light and tone were used as a cue in a negative rein¬ holding, and found these two responses equally con-
forcement procedure the tone had stronger effects on ditionable with negative reinforcement.
behavior. Finally, as Ferrari, Todorov, and Graeff (1973) have
However, attributing species differences in condi¬ pointed out, any comparison of conditionability or
tioning to species differences in associativity or sus¬ associability of different responses must be made on
ceptibility to reinforcement does have some drawbacks. the basis of optimal procedures for each response.
First, as Schwartz (1974) has pointed out, any inde¬ This brings us inevitably to the other aspect of Bolles’s
pendent assessment of such “preparedness” of a re¬ formulation, the role of shock-produced behavior in
sponse must be made with existing conditioning determining the operant levels of different responses.
procedures. Different procedures will give differing re- Optimal procedures must accommodate or eliminate
$ultsj and any attempt to standardize the measures of this behavior.
“preparedness” will necessarily standardize a set of
“fundamental" conditioning procedures. This would
Competing, Shock-Produced Behavior
be counter to current critical examinations of com¬
mon conditioning concepts, and would impose an Any intense, novel, or aversive stimulus is likely to
arbitrariness more difficult to assess than that of the produce behavior that helps to determine the reper¬
“arbitrary response.” toire of responses available for reinforcement. This is
Second, the “species-specific77 label draws our atten¬ especially relevant in negative reinforcement pro¬
tion away from the fact that these reactions and their cedures since, by definition, such stimulation must be
conditionability, however common and stereotyped present, highly probable, or impending when the to-
among members of a species, haye ontogenetic as well be-remforced response is emitted. This contrasts with
as genetic origins. I wonder, for example, about the positive reinforcement situations, where the critical
degree to which the conditionability of a rat's run¬ stimulus is presented after the response is emitted.
ning m an aversive situation originated in its interac¬ Effects of noncontingent aversive stimulation have
tions with littermates during the early weeks of life. been reviewed by Myer (1971), who abstracted some
The label “species-typical” would be more appropri¬ useful principles. He noted, for example, that prox-
ate. This term would remind us that we have not imally received stimuli usually produce skeletal be¬
closed the question of the origins of an animal’s be¬ havior that tends to remove the stimulation. Distally
havioral characteristics, while still suggesting that we received stimuli—in themselves aversive or paired with
look to genetic factors for part of the answer. aversive stimuli—tend to produce cessation of move¬
Third, in appealing to genetic determination one ment.
appeals to a largely unspecified sequence of events. To Thus electric shock, while providing the basis for
be sure, one can breed animals that tend to respond reinforcement, produces behavior patterns that must
in this way or that, but this only establishes the interact with the responses that experimenters have
plausibility of the explanation. To the extent that chosen to reinforce. Sometimes facilitation occurs, but
species-typical reactions are specified and studied inde¬ often competition or disruption occurs instead. Specific
pendently, with adequate identification of the circum¬ competing patterns have been described and dis¬
stances under which they will occur, their use is easily cussed by experimenters who had difficulty in condi¬
justified. However the genetic explanations tend to be tioning particular responses with negative reinforce¬
used in a post hoc fashion when all else has failed. ment. For example, a number of experimenters have
The independent specification usually is not available, observed shock-produced freezing and lever-holding
and one can thus choose convenient properties for the (e.g. Dinsmoor, 1967; Feldman Sc Bremner, 1963;
genetically determined behavior. More directly test¬ Keehn, 1967). Smith, Gustavson, and Gregor (1972)
able explanations are sometimes available in which used high-speed photography to examine the pigeon’s
the relative ease of conditioning of a particular re¬ response to unsignaled shock, and found that the
Philip N. Hineline 409

shock produced head-flexions—movements in the di¬ results implicate competing behavior as disrupting
rection opposite to that required for key-pecking. negatively reinforced responding early in all sessions,
Shock-elicited aggressive patterns such as biting the competing responses are relevant to maintenance
manipulandum have also been observed and measured of the conditioned response, as well as to its initial
(Azrin, Hutchinson and Hake, 1967; Azrin, Rubin acquisition. Experimenters have often tacitly acknowl¬
and Hutchinson, 1968; Pear, Moody and Persinger, edged this by discarding data from the initial parts of
1972; Powell, 1972). The use of intermittent shocks or all sessions, even in studies of long-term maintenance.
cues correlated with these shocks may not eliminate
competing behavior, for as Hoffman (1966) and Myer
Negative Reinforcement and the
(1971) have noted, the preshock warning stimulus
Ongoing Behavioral Stream
commonly used in shock-deletion procedures fits a
paradigm for conditioned suppression of active skele¬ Whichever aspect of the shock-produced competing
tal behavior. As Estes and Skinner (1941) demon¬ responses one chooses to stress, one must come to grips
strated, appetitively maintained behavior is suppressed with their ontogenetic as well as their phylogenetic
in the presence of a stimulus that has been paired origin. Furthermore, since they affect the operant
with shock. While after initial conditioning has been level, or “behavioral stream" (Schoenfeld, 1969) upon
achieved with negative reinforcement, such a stimulus which negative reinforcement must operate, one must
may enhance active responding (e.g. Rescorla, 1968), consider specifically how they are maintained in a
its effect during initial conditioning is to produce given situation.
crouching and freezing. Of course, operant conditioning principles partly
Each of these shock-produced behavior patterns describe the shaping of the behavioral stream. These
has been seen as interfering with initial attempts to principles can be used to explicitly study the main¬
bring lever-press responding under control of nega¬ tenance of the “competing behavior." For example,
tive reinforcement. I have also found evidence that in Keehn and Walsh (1970) have examined reinforce¬
rats the disruption persists beyond initial condition¬ ment of lever-holding, and Bolles and Riley (1973)
ing (Hineline, 1966). The persisting disruption is have carefully examined reinforcement and punish¬
revealed in a commonly-observed “warm-up" effect, ment, as well as elicitation of a freezing response. In
whereby even after they have achieved proficient per¬ some cases operant principles have been used to spe¬
formance, rats continue to take many shocks early in cifically eliminate the competing behavior. For ex¬
sessions. In this experiment rats were conditioned ample, Forgione (1970) improved the acquisition of
with a shock-delay procedure (SS = RS — 20). Direct repeated lever-pressing by identifying and eliminating
observation of subjects showing pronounced warm-up reinforcement that inadvertently had been made con¬
revealed that patterns of freezing and lever-holding tingent upon short-latency post-shock responding.
were clearly evident early in the sessions, when most Feldman and Bremner (1963) eliminated freezing and
shocks were received, but not later in sessions. In a lever-holdmg by making brief shock contingent upon
subsequent set of procedures, the possible disruptive these responses, while making shock-delay contingent
effects were examined indirectly, by making positive upon repeated pressing,
reinforcement and shock-delay both contingent upon Initial training procedures for bringing a particu¬
presses of the shock-delay lever. The schedule of posi¬ lar response under control of negative reinforcement
tive reinforcement (VI 40 sec), when used alone prior are “little transfer experiments.*’ The result of one
to shock-delay training, produced performances with conditioning procedure produces the operant level for
few interresponse times exceeding 20 sec until satia¬ the next procedure to act upon. Therefore it is not
tion late in the sessions. Thus, if there were no disrup¬ surprising that the effects of negative reinforcement
tion virtually all early session shocks should have been are very different once an organism’s behavior has
eliminated by the addition of positive reinforcement. been brought under control of some negative rein¬
For rats with extensive prior training the added posi¬ forcement procedure. The situation is analogous to
tive reinforcement failed to eliminate the dispropor¬ the teaching of swimming. Once one swimming tech¬
tionately high shock frequencies early in sessions. nique is brought under control of water-filled situa¬
Animals with less extensive training showed some tions, which involves eliminating many incompatible
reduction of warm-up with the added positive rein¬ water-produced responses, it is relatively easy to train
forcement. However, four of five animals continued any of a variety of swimming patterns.
to show at least some warm-up, indicating transient However, some components of the behavioral stream
disruption of lever-pressing. To the extent that these clearly are produced in ways not subsumed under
410 NEGATIVE REINFORCEMENT AND AVOIDANCE

principles of reinforcement and punishment. Usually attack by rats in response to aversive shock. Journal
the species-typical patterns have been characterized as of the Experimental Analysis of Behavior, 1968, 11, 633-
639.
elicited even though they may include complex re¬
Badia, P., Coker, C., 8c Harsh, J. Choice of higher density
sponse patterns. Elicitation plays a role, as Bolles and
signalled shock over lower density unsignalled shock.
Riley (1973) have demonstrated, but it is somewhat Journal of the Experimental Analysis of Behavior, 1973,
inadequate in that it implies a passive organism that 20, 47-55.
does nothing except when goaded. The ethologists’ Baum, W. M. The correlation-based law of effect. Journal

concept of “releasing stimuli” for particular classes of of the Experimental Analysis of Behavior, 1973, 20, 137-
153. (a)
behavior is only slightly less limiting. It more readily
Baum, W. M. Time allocation and negative reinforcement.
encompasses complex behavior patterns, but still Journal of the Experimental Analysis of Behavior, 1973,
tends to characterize the controlling environment as 20, 313-322. (b)
made up of discrete, single stimuli. Baum, W. M., 8c Rachlin, H. C. Choice as time allocation.
Journal of the Experimental Analysis of Behavior, 1969,
To be sure, single stimuli are important, but an¬
12, 861-874.
imals are also sensitive to temporal configurations of Bersh, P. J., 8c Lambert, J, V. The discriminative control
environmental objects and events. Temporal patterns of free-operant avoidance despite exposure to shock dur¬
of environmental events interact with patterns of on¬ ing the stimulus correlated with nonreinforcement. Jour¬
going behavior. Perhaps we shall come to deal with nal of the Experimental Analysis of Behavior, 1975, 23,
111-120.
these interactions in terms of modulation (Gibbon 8c
Bindra, D„ 8c Anghel, H. Immobility as an avoidance re¬
O’Connell, 1973; Morse & Kelleher, 1970), periodicity sponse, and its disruption by drugs. Journal of the
and apenadicity (Kadden, 1973)j autocorrelation, Experimental Analysis of Behavior, 1963, 6, 213-218.
closed and open loops, and/or contingency and non- Black, A. H.^ Autonomic aversive conditioning in infra-
contingency as expressed in a statistical, rather than a fiuman subjects. In Robert F. Brush (Ed.), Aversive Con¬
ditioning and Learning. New York: Academic Press,
merely associative sense (Baum, 1973). Concepts such
1971.
as these may help m understand how some procedures Bolles, R. C. Species-specific defense reactions and avoid¬
that were designed to reinforce a specific response are ance learning. Psychological Review, 1970, 77, 32-48.
actually more effective for producing behavior that BOLLES, R. C. The avoidance learning problem. In G. H.
competes with that response. It is not yet clear whether Bower (Ed.), The Psychology of Learning and Motiva¬
tion (Vol. 6). New York; Academic Press, 1973.
these developing concepts will come to subsume what
Bolles, R. C. &! GROSSEN, N. E. Effects of an informational
wo now call reinforced as well as what we call non- stimulus on the acquisition of avoidance behavior in
reinforced behavior. Perhaps the operations we now rats. Journal of Comparative and Physiological Psychol¬
treat as discrete and unitary will be of interest mainly ogy, 1969, 68, 90-99.
as ends of contmua such as those suggested by Catania Bolles, R. G. 8c Popp, R. J. Parameters affecting the acquisi¬
tion of Sidman avoidance. Journal of the Experimental
(1971). Whatever reformulations occur, they must also Analysis of Behavior, 1964, 7, 315-321.
deal with interrelations described here, between re¬ Bolles, R. C. 8c Riley, A. L. Freezing as an avoidance
sponding and the response-contingent reduction of response; Another look at the operant-respondent dis¬
stimulation, tinction. Learning and Motivation, 1973, 4, 268-275.
Bolles, R. C., Stokes, L. W., 8c Younger, M. S. Does CS
termination reinforce avoidance behavior? Journal of
Comparative and Physiological Psychology, 1966, 62,
REFERENCES 201-207.
Boren, J. J. Isolation of post-shock responding in a free
operant avoidance procedure. Psychological Reports,
Anger, D. The role of temporal discriminations in the 1961, 9, 265-266.
reinforcement of Sidman avoidance behavior. Journal Boren, J. J., and Sidman, M. A discrimination based upon
of the Experimental Analysis of Behavior, 1963, 6, 447- repeated conditioning and extinction of avoidance be¬
506. havior. Journal of Comparative and Physiological Psy¬
Appel, J. B. The aversive control of operant discrimination. chology, 1957, 50, 18-22.
Journal of the Experimental Analysis of Behavior, 1960, Boren, J. J., Sidman, M., 8c Herrnstein, R. J. Avoidance,
3, 35-48. escape, and extinction as functions of shock intensity.
Azrin, N. H., Holz, W. C., Hake, D. F., 8c Ayllon, T. Journal of Comparative and Physiological Psychology,
Fixed-ratio escape reinforcement. Journal of the Experi¬ 1959,52, 420-425.
mental Analysis of Behavior, 1963, 6, 449-456. Breland, K., 8c Breland, M. The misbehavior of organisms.
Azrin, N. H., Hutchinson, R. R., 8c Hake, D. F. Attack, American Psychologist, 1961, 16, 661-664.
avoidance and escape reactions to aversive shock. Jour¬ Brown, P. L., 8c Jenkins, H. M. Auto-shaping of the pi¬
nal of the Experimental Analysis of Behavior, 1967, 10, geon’s key-peck. Journal of the Experimental Analysis of
131-148. Behavior, 1968, 11, 1-8.
Azrin, N. H., Rubin, H. B., 8c Hutchinson, R. R. Biting Catania, A. C. Elicitation, reinforcement, and stimulus
Philip N. Hineline 411

control. In R. Glaser (Ed). The Nature of Reinforce¬ exploring multi-operant behavior repertoires. Journal of
ment. New York: Academic Press, 1971. the Experimental Analysis of Behavior, 1962, 5, 113-166.
Church, R. M. Response Suppression. In Campbell, B. A. Foree, D., 8c LoLordo, V. Signalled and unsignalled free-
8c Church, R. M. (Eds.), Punishment and Aversive Be¬ operant avoidance in the pigeon. Journal of the Experi¬
havior. New York: Appleton-Century-Crofts, 1969. mental Analysis of Behavior, 1970, 13, 283-290.
Church, R. M. Laws o£ learning. (Review of Aversive Con¬ Foree, D., 8c LoLordo, V. M. Attention in the pigeon:
ditioning and Learning by Robert F. Brush). Contem¬ differential effects of food-getting versus shock-avoidance
porary Psychology, 1973, 18, 322-323. procedures. Journal of Comparative and Physiological
Clark, F. C. 8c Hull, L. D. Free-operant avoidance as a Psychology, 1973, 85, 551-558.
function of the response-shock-shock = shock interval. Foree, D., 8c LoLordo, V. M. Transfer of control of the
Journal of the Experimental Analysis of Behavior, pigeon's key peck from food reinforcement to avoidance
1966, 9, 641-647. of shock. Journal of the Experimental Analysis of Be¬
Clark, F. C., Lange, K. O., 8c Belleville, R. E. Behavioral havior, 1974, 22, 251-259.
regulation of gravity: schedule effects under escape- Forgione, A. G. The elimination of interfering response
avoidance procedures. Journal of the Experimental patterns in lever-press avoidance situations. Journal of
Analysis of Behavior, 1973, 20, 345-353. the Experimental Analysis of Behavior, 1970, 13, 51—56.
Coulson, G., Coulson, V., & Gardner, L. The effect of two Gibbon, J. Timing and discrimination of shock density in
extinction procedures after acquisition on a Sidman avoidance. Psychological Review, 1972, 79, 68-92.
avoidance contingency. Psychonomic Science, 1970, 18, Gibbon, J., Berryman, R., & Thompson, R. L. Contingency
309-310. spaces and measures in classical and instrumental condi¬
Davenport, D. G., Coger, R. W. and Spector, O. J. The tioning. Journal of the Experimental Analysis of Be¬
redefinition of extinction applied to Sidman free-oper¬ havior, 1974, 21, 585-605.
ant avoidance responding. Psychonomic Science, 1970, Gibbon, J., & O'Connell, M. E. Ac quisition and main¬
19, 181-182. tenance of responding by rats under non conbiigsftt
Davenport, D. G. 8c Olson, R. D. A reinterpretation of shock. Paper read at the Annual Meeting of the Eastern
extinction in discriminated avoidance. Psychonomic Psychological Association, 1973, Washington, D.C.
Science, 1968, 12, 5-6. Giuhan, D„ 8c Schmaltz* L. W. Enhanced discriminated
DE Villiers, P. A. Reinforcement and response rate inter¬ bar-press avoidance in the rat through appetitive pre¬
action in multiple random-interval avoidance schedules. conditioning. Journal of Comparative and Physiological
Journal of the Experimental Analysis of Behavior, 1972, Psychology, 1973, 83, 106-112.
18, 499-507. Grabowski, J., & Thompson, T. Response patterning an an
de Vilhers, p, A. The law of effect and avoidance: a avoidance schedule as a function of time-correlated
quantitative relationship between response rate and stimuli. Journal of the Experimental Analysis of Be*
shock-frequency reduction. Journal of the Experimental havior, 1972* 18, 525-534.
Analysis of Behavior, 1974, 21, 223 235. Hake, d. f., & Campbell, R, L, characteristics and re¬
Dinsmoor, J. A. Variable-interval escape from stimuli ac¬ sponse-displacement effects of shock-generated respond¬
companied by Journal of
shocks. the Experimental ing during negative reinforcement procedures: pre-shock
Analysis of Behavior, 1962, 5, 41-47. responding and pASt-shock aggressive responding, J ournal
Dinsmoor, j, A, Escape from shock as a conditioning tech= of the Experimental Analysis of Bs kavior, 1972, 17,
mquc, In M. R. Jones (Ed.), Miami Symposium on the 303-323.
Prediction of Behavior; 1967; Aversive Stimulation. Harrison, J. M., & Tragv, W. H. The use of auditory
Coral Gables, Fla.: University of Miami Press, 1968, stimuli to maintain lever-pressing behavior. Science,
Dinsmoor, J. A., & Sears, G, W. Control of avoidance by a 1955, 121, 373-374.
response-produced stimulus. Learning and Motivation,
Hearst, E. Multiple schedules of time-correlated reinforce¬
1973, 4, 284-293.
ment. Journal of the Experimental Analys is of Behavior,,
Ellen, P., & Wilson, A. Two patterns of avoidance re¬
1960, 3, 49-62.
sponding. Journal of the Experimental Analysis of Be¬
Herrnstein, R, J, Aperiodicity as a factor in choice,
havior, 1964, 7, 97-98.
Estes, W. K. 8c Skinner, B. F. Some quantitative properties
Journal of the Experimental Analysis of Behavior, 1964,
7, 179-182.
of anxiety. Journal of Experimental Psychology, 1941,
29, 390-400. Herrnstein, R. J. Method and theory in the study of avoid¬
Feldman, R. S., and Bremner, F. J. A method for rapid ance. Psychological Review, 1969, 76, 49-69.
conditioning of stable avoidance bar pressing behavior. Herrnstein, R. J. On the law of effect. Journal of the Ex¬
Journal of the Experimental Analysis of Behavior, 1963, perimental Analysis of Behavior, 1970, 13, 243-266.
6, 393-394. Herrnstein, R. J., 8c Hineline, P. N. Negative reinforce¬
Ferrari, E. A., Todorov, J. C., 8c Graeff, F. G. Nondis- ment as shock-frequency reduction. Journal of the Ex¬
criminated avoidance of shock by pigeons pecking a key. perimental Analysis of Behavior, 1966, 9, 421-430.
Journal of the Experimental Analysis of Behavior, 1973, Hineline, P. N. The warm-up effect in avoidance condi¬
19, 211-218. tioning. Unpublished Doctoral Dissertation, Harvard
Field, G. E., 8c Boren, J. J. An adjusting avoidance pro¬ University, 1966.
cedure with multiple auditory and visual warning Hineline, P. N. One shock now or five shocks later. Paper
stimuli. Journal of the Experimental Analysis of Be¬ read at the Annual Meeting of the Eastern Psychological
havior, 1963, 6, 537-543. Association, Philadelphia, 1969.
Findley, J. D. An experimental outline for building and Hineline, P. N. Negative reinforcement without shock re-
412 NEGATIVE REINFORCEMENT AND AVOIDANCE

duction. Journal of the Experimental Analysis of Be¬ Krasnegor, N. A., Brady, J. V., 8c Findley, J. D. Second-
havior•, 1970, 14, 259-268. order optional avoidance as a function of fixed-ratio
Hineline, P. N., 8c Herrnstein, R. J. Timing in free-oper¬ requirements. Journal of the Experimental Analysis of
ant and discrete-trial avoidance. Journal of the Experi¬ Behavior, 1971, 15, 181-187. © 1971 by the Society for
mental Analysis of Behavior, 1970, 13, 113-126. the Experimental Analysis of Behavior, Inc.
Hineline, P. N., 8c Rachlin, H. Notes on fixed-ratio and Lambert, J. V., Bersh, P. J., Hineline, P. N., 8c Smith,
fixed-interval escape responding in the pigeon. Journal G. D. Avoidance conditioning with shock contingent
of the Experimental Analysis of Behavior, 1969, 12, 397- upon the avoidance response. Journal of the Experi¬
401. (a) mental Analysis of Behavior, 1973, 19, 361-367.
Hineline, P. N., 8c Rachlin, H. Escape and avoidance of Leaf, R. C. Acquisition of Sidman avoidance responding
shock by pigeons pecking a key. Journal of the Experi¬ as a function of S-S interval. Journal of Comparative
mental Analysis of Behavior, 1969, 12, 533-538. (b) and Physiological Psychology, 1965, 59, 298-300.
Hoffman, H. S. The analysis of discriminated avoidance. Lewis, P., Lewin, L., Stoyak, M., 8c Muehleisen, P. Nega¬
In W. K. Honig (Ed.), Operant Behavior: Areas of Re¬ tively reinforced key pecking. Journal of the Experi¬
search and Application. Englewood Cliffs, N.J.: Prentice- mental Analysis of Behavior, 1974, 22, 83-90.
Hall, Inc., 1966. Libby, M. E., 8c Church, R. M. Timing of avoidance re¬
Hoffman, H., 8c Fleshler, M. Aversive control with the sponses by rats. Journal of the Experimental Analysis of
pigeon. Journal of the Experimental Analysis of Be¬ Behavior, 1974, 22, 513-517.
havior, 1959, 2, 213-218. Macphail, E. M. Avoidance responding in pigeons. Journal
Hurwitz, H. M. B., 8c Millenson, J. R. Maintenance of of the Experimental Analysis of Behavior, 1968, 11,
avoidance behavior under temporally defined contingen¬ 629-632.
cies. Science, 1961, 133, 284-285. McKearney, J. W. Fixed-interval schedules of electric
Hurwitz, H. M. B., Roberts, A. E., 8c Greenway, L. Ex¬ shock presentation: extinction and recovery of perfor¬
tinction and maintenance of avoidance behavior using mance under different shock intensities and fixed-interval
response-independent shocks. Psychonomic Science, 1972, durations. Journal of the Experimental Analysis of Be¬
28, 176-178. havior, 1969, 12, 301-313.
Jenkins, H. M. Sequential organization in schedules of McKearney, J. W. Maintenance and suppression of re¬
reinforcement. In W. N. Schoenfeld (Ed.), The Theory sponding under schedules of electric shock presentation.
of Reinforcement Schedules. Englewood Cliffs, N.J.: Journal of the Experimental Analysis of Behavior, 1972,
Prentice-Hall, Inc., 1970. 17, 425-432.
Kadden, R. M. Facilitation and suppression of responding Morse, W. H., 8c Kelleher, R. T. Schedules using noxious
under temporally defined schedules of negative rein¬ stimuli. I. Multiple fixed-ratio and fixed-interval termi¬
forcement. Journal of the Experimental Analysis of Be¬ nation of schedule complexes. Journal of the Experi¬
havior, 1973, 19, 469-480. mental Analysis of Behavior, 1966, 9, 267-290.
Kadden, R. M„ Schoenfeld, W, N., 8c Snapper, A. G, Morse, W. H., 8c Kelleher, R. T. Schedules as fundamen¬
Aversive schedules with independent probabilities of tal determinants of behavior. In W. N. Schoenfeld
reinforcement for responding and not responding by (Ed.), The Theory of Reinforcement Schedules. Engle¬
Rhesus monkeys: II. Without Signal. Journal of Com¬ wood Cliffs, N.J.: Prentice-Hall, Inc., 1970.
parative and Physiological Psychology, 1974, 87, 1189- Mowrer, O. H., 8c Lamoreaux, R. R. Fear as an interven¬
1197. ing variable in avoidance conditioning. Journal of Com¬
Kamin, L. J., Brimer, C. J., 8c Black, A. H. Conditioned parative and Physiological Psychology, 1946, 39, 29-50.
suppression as a monitor of fear of the CS in the course Myer, J. S. Some effects of noncontingent aversive stimula¬
of avoidance training. Journal of Comparative and Phys¬ tion. In Robert F. Brush (Ed.), Aversive Conditioning
iological Psychology, 1963, 96, 497-501. and Learning. New York: Academic Press, 1971.
Keehn, J. D. Avoidance responses as discriminated oper¬ Neffinger, G. G., 8c Gibbon, J. Partial avoidance con¬
ants. British Journal of Psychology, 1966, 57, 375-380. tingencies. Journal of the Experimental Analysis of Be¬
Keehn, J. D. Running and bar pressing as avoidance re¬ havior, 1975, 23, 437-450.
sponses. Psychological Reports, 1967, 20, 591-602. Pear, J. J., Moody, J. E., 8c Persinger, M. A. Lever at¬
Keehn, J. D., 8c Walsh, M. Barholding with negative rein¬ tacking by rats during free operant avoidance. Journal of
forcement as a function of press- and release-shock inter¬ the Experimental Analysis of Behavior, 1972, 18, 517-
vals. Learning and Motivation, 1970, 1, 36-43. 523.
Kelleher, R. T., 8c Morse, W. H. Escape behavior and Pomerleau, O. F. The effects of stimuli followed by re¬
punishment behavior. Federation Proceedings, 1964, 23, sponse-independent shock on shock-avoidance behavior.
808-817. Journal of the Experimental Analysis of Behavior,
Keller, F. S. Light aversion in the white rat. Psychological 1970, 14, 11-21.
Record, 1941, 4, 235-250. Powell, R. W. Some effects of response-independent shocks
Klein, M., 8c Rilling, M. Effects of response-shock interval after unsignalled avoidance conditioning in rats. Learn¬
and shock intensity on free-operant avoidance respond¬ ing and Motivation, 1972, 3, 420-441.
ing in the pigeon. Journal of Experimental Analysis of Powell, R. W., 8c Peck, S. Persistent shock-elicited respond¬
Behavior, 1972, 18, 295-303. ing engendered by a negative-reinforcement procedure.
Klein, M., 8c Rilling, M. Generalization of free-operant Journal of the Experimental Analysis of Behavior, 1969,
avoidance behavior in pigeons. Journal of the Experi¬ 12, 1049-1062.
mental Analysis of Behavior, 1974, 21, 75-88. Rachlin, H., 8c Hineline, P. N. Training and maintenance
Philip N. Hineline 413

of keypecking in the pigeon by negative reinforcement. parative and Physiological Psychology, 1953, 46, 253-261.
Science, 1967, 157, 954-955. Sidman, M. Some properties of the warning stimulus in
Rescorla, R. A. Pavlovian conditioning and its proper avoidance behavior. Journal of Comparative and Phys¬
control procedures. Psychological Review, 1967, 74, 71- iological Psychology, 1955, 48, 444-450.
80. Sidman, M. Conditioned reinforcing and aversive stimuli
Rescorla, R. A. Pavlovian conditioned fear in Sidman in an avoidance situation. Proceedings of New York
avoidance learning. Journal of Comparative and Phys¬ Academy of Sciences, 1957, 534-544.
iological Psychology, 1968, 65, 55-60. Sidman, M. Reduction of shock frequency as reinforcement
Rescorla, R. A. Establishment of a positive rein forcer for avoidance behavior. Journal of the Experimental
through contrast with shock. Journal of Comparative Analysis of Behavior, 1962, 5, 247-257 (a).
and Physiological Psychology, 1969, 67, 260-263. Sidman, M. Classical avoidance without a warning stim¬
Rescorla, R. A., k LoLordo, V. M. Pavlovian inhibition of ulus. Journal of the Experimental Analysis of Behavior,
avoidance behavior. Journal of Comparative and Phys¬ 1962, 5, 97-104 (b).
iological Psychology, 1965, 59, 406-412. Sidman, M. Timeout from avoidance as a reinforcer: a
Rescorla, R. A., & Wagner, A. R. A theory of Pavlovian study of response interaction. Journal of the Experi¬
conditioning: variations in the effectiveness of reinforce¬ mental Analysis of Behavior, 1962, 5, 423—434 (c).
ment and nonreinforcement. In A. H. Black & W. F. Sidman, M. An adjusting avoidance schedule. Journal of
Prokasy (Eds.), Classical Conditioning II: Current Re» the Experimental Analysis of Behavior, 1962, 5, 271—
search and Theory, Englewood Cliffs, N.J,: Prentice- 277 (d).
Hall, Inc., 1972. Sidman, M, Avoidance behavior- In W. K, lining (Ed,),
RiCCio, D. C., 8c TriACH, J. S. Rotation as an aversive stim¬ Operant Behavior: Areas of Research and Application.
ulus for rats. Psychonomic Science, 1966, 5, 267-268. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966.
Riess, D. A shaping technique for producing rapid and Sidman, M., k Boren, J. J. The use of shock-contingent
reliable Sidman barpress avoidance. Journal of the Ex¬ variations in response-shock intervals for the mainte¬
perimental Analysis of Behavior, 1970, 15, 279-280. nance of avoidance behavior. Journal of Comparative
Riess, D. Shuttle boxes, Skinner boxes, and Sidman avoid¬ and Physiological Psychology, 1957, 50, 558-562 (a),
ance in rats: acquisition and terminal performance as a Sidman, M., 8c Boren, J. J. A comparise>n q£ two types of
function of response topography. Psychonomic Science, Warning stimulus m an avoidance situation. Journal of
1971, 25, 283-286. Comparative and Physiological Psychology, 1957, 50,
Riess, D., 8c Farrar, C. H. Unsignalled avoidance in a 282-287. (b)
shuttle box: a rapid acquisition, high-efficiency para¬ Sidman, M., & Boren, J. J. The relative aversiveness of
digm. Journal of the Experimental Analysis of Behavior, warning signal and shock in an avoidance situation.
1972, IS, 169-178. Journal of Abnormal and Social Psychology, 1957, 55,
Sciioenfeld,W, N. “Avoidance” in behavior theory, Jour¬ 339-344. (c)
nal of the Experimental Analysis of Behavior, 1969, 12, Smith, G. D. Extinction of free-operant avoidance in rats.
669-674, Unpublished Doctoral Dissertation, Temple University.
Schoenfeld, W, N., 8c Cole, B. K. Stimulus schedules; the 1973.
t—t systems, New York: Harper & Row, 1972. Smith, R. E.,Gustavson, G. R., 8c Gregor, G. L. Incom-
Schwartz, B. On going back to nature: a review of Selig- patability between the pigeons' unconditioned response
man and Hager’s Biological Boundaries of Learning. to shock and the conditioned key-peck response. Journal
Journal of the Experimental Analysis of Behavior, 19?4j of the Experimental Analysis of Behavior, 1972, IS,
21, 183-198, 147-153.
Schwartz, B., & Coulter, G. A failure to transfer control Smith, R., & Keller, F. Free-operant avoidance in the
of key pecking from food reinforcement to escape from pigeon using a treadle response. Journal of the Em peri-
and avoidance of shock. Bulletin of the Psychonomic
mental Analysis of Behavior, 1970, II, 211-214.
Society, 1971, 1, 307-309.
Staddon, J. E. R., & Simmelhag, V. L. The ‘‘gup^rstitiW'
SELIGMAN, M. E. P. On the generality of laws of learning.
experiment: a reexamination of its implications for the
Psychological Review, 1970, 77, 406—418.
principles of adaptive behavior. Psychological Review,
Seligman, M. E. P,, 8c Hager, J. L. Biological Boundaries
1971, 78, 3-43.
of Learning, Englewood Cliffs, N.J.: Prentice-Hall, Inc.,
Theios, J. Mathematical models for aversive conditioning.
1972.
In R. F. Brush (Ed.), Aversive Conditioning and Learn¬
Seligman, M. E. P., Maier, S. F., k Solomon, R. L. Unpre¬
dictable and Uncontrollable Aversive Events. In F. R. ing. New York: Academic Press, 1971.
Brush (Ed.), Aversive Conditioning and Learning, New Todorov, J. C. Component duration and relative response

York: Academic Press, 1971. rates in multiple schedules. Journal of the Experimental
Shimp, C. P. Synthetic variable-interval schedules of rein¬ Analysis of Behavior, 1972, 17, 45-59.
forcement. Journal of the Experimental Analysis of Be¬ Ulrich, R. E., Holz, W. C., 8c Azrin, N. H. Stimulus con¬
havior, 1973,19, 311-330. trol of avoidance behavior. Journal of the Experimental
Shnidman, S. R. Extinction of Sidman avoidance behavior. Analysis of Behavior, 1964, 7, 129-133.
Journal of the Experimental Analysis of Behavior, 1968, Verhave, T. Technique for differential reinforcement of
11, 153-156. rate of avoidance responding. Science, 1959, 129, 959-
Sidman, M. Two temporal parameters in the maintenance 960.
of avoidance behavior by the white rat. Journal of Com¬ Verhave, T. The functional properties of a time out from
414 NEGATIVE REINFORCEMENT AND AVOIDANCE

an avoidance schedule. Journal of the Experimental Weiss, B., and Laties, V. G. Behavioral thermoregulation.
Analysis of Behavior, 1962, 5, 391-422. Science, 1961, 133, 1338-1344.
Weisman, R. C., & Litner, J. S. Positive conditioned rein¬ Wertheim, G. A. Behavioral contrast during multiple
forcement of Sidman avoidance behavior in rats. Journal avoidance schedules. Journal of the Experimental
of Comparative and Physiological Psychology, 1969, 68, Analysis of Behavior, 1965, 8, 269-278.
597-603.
 14

By-Products of Aversive
Control*

R, Ri Hutchinson

INTRODUCTION maintained by other stimuli. Thus, aversive stimuli

arc often referred to as negative reinforcers or punish¬
The concept of aversive control is familiar to all ers. It is in such a context that the concept of by¬
psychologists who work with operant methods, and products of aversive control has developed. When a
commonly refers to the conditioning procedures of stimulus is thought of as having properties defined by
escape, avoidance, and punishment. Under such condi¬ its capacity to alter behavior through a contingent
tions, it has often been found that secondary or *‘by¬ relation with that behavior, simultaneous influence
product" performances are generated. This chapter upon behavioral processes other than by contingent
discusses some of the secondary effects of aversive con¬ influence have typically been discussed as by-product,
trol and describes how these may be functionally secondary, or adjunctive influences of the stimulus.
related to aversive control procedures. Yet attention to the contingent control of certain en¬
The concept of aversive control connotes the appli¬ vironmental stimuli, is not a sufficient reason to con¬
cation of aversive stimuli in a manner such that its sider other performances or effects which are produced
consequences affect performance. Aversiveness is as¬ simultaneously and non-contingently by such stimuli,
sessed by the capacity of a stimuli to support responses to be secondary in nature. Only recently has it become
which eliminate or reduce such stimulation, or alter¬ understood that aversive stimuli may produce com¬
natively by its capacity to suppress performances plex chains of reactions directly, and that though
these performances may be seen in environments
* The assistance of G. Emley, E. Hallin, N. Murray, V. where the contingent control of some behavior is be¬
Pufpaff, B. Snowden, S. Crawford, K. Dinzik, G. Pierce, I. Wing, ing studied, such direct effects are also present in
S. Carpenter, T. Sammons, N. Hunter, D. Mann, T. Proni, V.
environments where no contingency arrangements are
Lane, P. Reynolds, L. Peebles, D. Marine, and R. Sewell was
essential to this work. Financial support came from the Depart¬ present.
ment of Mental Health of the State of Michigan, National Recent work shows that aversive stimulation pro¬
Science Foundation grant GB-33620X, Office of Naval Research
duces complex, highly coordinated performance se¬
grant N00014-70-A-0183-0001, and National Aeronautics and
Space Administration grant NGR-23-014-002. quences in a wide range of species. These performances

415
416 BY-PRODUCTS OF AVERSIVE CONTROL

are reliably produced and constant during extended tance, electrode pressures, etc., must be routinely
periods of observation. Though such behaviors may available on a moment-to-moment basis.
be modified in learning paradigms, their expression Response-sensor mechanisms must be located ap¬
is fundamentally dependent neither upon historical propriately within the experimental space so as to
nor contemporary response-contingent environmental make optimal contact with performances as they oc¬
events. Recent findings in our laboratory suggest that cur. No control via reinforcement of successive re¬
such reactions produced by aversive stimuli may par¬ sponse approximations (“shaping”) or contingency
tially form the basis for the complex performances management is possible in such experiments. The ex¬
which often result from the application of response- perimenter records the behavior where he finds it.
contingent procedures. Constant attempts at improving the suitability of con¬
tact surfaces and feedback stimuli are required, as are
sensors which are indestructible and operate reliably
METHODS over hundreds of thousands of occurrences* It is also
necessary to tailor chamber spaces to suit an individ¬
The methods employed for the work described in ual subject’s physical and behavioral variations,
the present chapter, are m the main, described in Figure 1 illustrates an apparatus which we have
detail in earlier publications. It may be useful, how- designed for testing biting attack responses in mice.
&VQV} t6 describe several experimental guidelines The mouse here shown while biting a small protrud¬
which have emerged over a number of different stud¬ ing nylon object is restrained in a small plexiglas
ies, designed to measure behavioral sequences not usu¬ tube. The tube is removable from the apparatus so
ally undergoing response-contingent control. that most subject preparation can be accomplished in
By the very nature of aversive stimulation, vigorous the home colony. Subsequent to placement in the test¬
attempts by subject^ to escape or avoid lull or direct ing apparatus, the subject’s tail is cleansed and placed
stimulus contact is likely. To prevent unwarranted under two contact electrodes at the rear of the ap¬
stimulus variability, apparatus must be designed to paratus. In front of the subject i§ an object (in most
insure stable long term contact a§ experimentally §pe cases a small bit of flexible nylon) attached to a vari¬
cified. Stimuli should be brief to prevent momentary able force and displacement sensor. A standard tele¬
variations in posturing and orientation which can graph key has proved useful for this purpose. The
produce marked alterations in stimulus intensity. method has been suitable for the study of effects of
Additionally, continuous feedback to the experimen¬ genetic variations, drug influences, and social living
ter regarding contact dimensions such as skin resis- conditions on attack behavior. Figure 2 is an illustra-

Fig. 1. Photograph of apparatus

used for testing the effects of
aversive stimulation on biting
attack by mice. The subject
shown here biting a protruding
nylon object is partially re¬
strained in a cylindrical cap¬
sule mounted at the center of
the apparatus. The subject’s tail
is lightly taped to a plexiglas
rod. Two brass electrodes rest
upon cleansed and prepared
portions of the tail. The nylon
bite object is attached directly
to a- telegraph key which allows
force and displacement adjust¬
ments.
R. R. Hutchinson 417

mounted on the panel either in front of, or to the left

of the subject. Response levers, chains, etc., can be
mounted suitably for manual response contact.
The human testing paradigms have been used to
study the phyletic continuity process of aggression and
anger. These paradigms in earlier tests with other
species revealed the effects of social contingencies,
symbolic communication upon operant and respon¬
dent processes, and the effect of several drugs upon
aggressive reactions.
Studies with humans have required preliminary de¬
velopment of force transducers inserted directly into
the mouth. These devices have allowed perfection of
the methods of recording biting contractions externally
and without awareness by the subjects. Electrodes are
placed over the temporalis and masseter muscles at
positions illustrated in Figure 3. The muscles control¬
ling the eccentric and concentric occlusal patterns
contract, i.c., biting occurs, during both the presenta¬
tion of aversive stimuli and subsequent to the with¬
drawal of positive reinforcers (Hutchinson & Pierce,
1971; Pierce, 1971; Proni, 1973). Small needle elec¬
Fig. 2- Apparatus used for testing effects of aversive stimulation
upon manual manipulative, biting attack, and drinking reac¬ trodes are used rather than surface electrodes, as these
tions in the squirrel monkey. The subject is restrained by a allow far more precise, noise-free recordings of elec¬
waist lock assembly and is seated on two plexiglas rods. The
tromyographic (EMG) potentials which covary with
tail is placed in a stockade device. Two brass electrodes rest
upon cleansed and prepared portions of the distal section of bite contraction force (Hutchinson 8c Pierce, 1971).
the tail. A rubber hose may be connected between two pipe
stanchions located external to the right and left walls of the
test space. Compressive forces possible only by biting attack
are recorded via an air flow switch mounted at the rear of the HUMAN BITE
intelligence panel.A response lever may be seen mounted on
the lower left hand quadrant of the removable front intelligence
panel. In studies where chain pulling was measured, the chain ELECTRODE PLACEMENTS
is suspended from the chamber ceiling.

tion of the apparatus used for the study of squirrel

monkey subjects. The chamber was originally devel¬
oped by Drs. D. F. Hake and N. H. A2rin at Anna
State Hospital for work on shock^avoidance behavior
(Hake & Azrin, 1963; Hake, 1968) and incorporates
a number of helpful features for the study of elicited
behavior. Gross physical movement of the subject is
controlled such that an aversive stimulus can be ap¬
plied precisely through electrodes upon a shaved por¬
tion of the tail which is restrained under a stockade.
The upper torso and limbs are left unrestrained so
that a considerable range of behavior is possible. In
studies of biting attack, a rubber hose is suspended
several inches in front of the subject approximately at
Fig. 3. Schematic illustration of electrode placement positions
head level. Bites on the rubber hose produce sufficient
for recording of temporalis and masseter electromyographic
air displacement to trigger an air flow switch mounted activity. The indifferent electrodes are paralleled electrically to
on the rear of the panel, but grasping, tugging, and provide a balanced reference. The electrode on the nose is a
pulling have too little effect to trip the switch. A silver disc. Standard clip electrodes are used on each ear. All
other electrodes are standard 14" subdermal EEG needle elec¬
drink tube and drinkometer to record drinking can be trodes.
 418 BY-PRODUCTS OF AVERSIVE CONTROL

Hutchinson, 8c Sallery, 1964; Hutchinson, Azrin, &

Hake, 1966; Ulrich, 8c Azrin, 1962; Ulrich, Hutchin¬
son, 8c Azrin, 1965).
As experience has been gained in the sensing of
complex reactions in several species, it has become
apparent that the aggression-attack reaction to aver¬
sive stimulation is only one of several identifiable
behavioral sequences and patterns which relate to one
another and to aversive stimulation in a reliable
fashion.
Numerous studies have shown that the delivery of
an intense aversive, noxious, or unpleasant stimulus
will produce, in a variety of species, movement to¬
ward, contact with, and possibly destruction of, an¬
imate or inanimate objects in the environment (Ulrich
Fig. 4. Photograph of the actual test space for assessing biting & Azrin, 1962; Ulrich, Hutchinson, 8c Azrin, 1965).
and other motor responses by human subjects subsequent to Figure 5 illustrates some temporal and intensive rela¬
aversive stimulation. Electrode leads from head recording areas tions which have been recorded from several species
are provided strain relief under a standard athletic bandage
wrapped loosely around the forehead and returned to an input
following delivery of a noxious or aversive stimulus.
box at the rear of the chair. Other recording electrodes are Shortly after the stimulus event, attack or biting be¬
evident on the forearm and wrist. On the intelligence panel gins at a high intensity. Repetitions of this response
vf the console are signal lights and response buttons. At center
left i$ an intercom for continuous communications with the are likely with the frequency and intensity gradually
experimenter or other subjects during social procedures. A falling over a period of seconds or minutes (Azrin,
magazine cup for delivery of coins is also provided. The test Hutchinson, & Sallery, 1964; Hutchinson, Azrin, &
chamber is linked visually via a window arrangement with an
identical tc§t space, The two spaces may be linked or separated.
Renfrew, 1968).
If aversive stimuli are repeatedly delivered in a
discrimmable temporal pattern a display of aggression-
EMG recordings from the forearm, neck, and other
attack reactions assumes additional features. During
areas allow confirmation of differential activity of the
the period prior to an ensuing aversive stimulus (and
temporalis and master muscles relative to these lat¬
at a discriminable temporal period after aversive stim¬
ter muscle groups at different times. An actual test
uli) additional biting attack reactions will occur. Fig¬
setting of a human subject is shown in Figure 4. After
ure 6 illustrates this temporal pattern for one species
preparation for an experimental session a subject is
and subject. Biting attack reactions occur for a period
seated in electrostatically and acoustically isolated
before the occurrence of an aversive stimulus, but
test space where various stimuli and responses may be usually cease just before shock (Hutchinson 8c Emley,
recorded. Test spaces are arranged in pairs to allow,
1972; Hutchinson, Renfrew, & Young, 1971).
when experimentally desirable, visual and auditory
Although aggression-attack sequences often occur
contact with partners in social experiments.
in reaction to conditional stimuli, more recent studies
have shown that other reaction sequences are also
likely. These reactions include sensory scanning, man¬
BEHAVIOR CAUSED BY AVERSIVE
ual manipulation, and locomotion sequences (Hutch¬
STIMULATION inson 8c Emley, 1972; Hutchinson, Renfrew, 8c Young,
1971). Figure 7 illustrates the automatic recording of
This section will discuss the present state of our shock-induced lever pressing in a squirrel monkey and
understanding of several major classes of behavioral noise-induced movement by a human. During the
sequences as they relate to the occurrence of aversive presentation of a conditional stimulus, activities oc¬
stimulation and to one another. Historically, our lab¬ cur at a progressively increasing rate until just before
oratory has worked to develop methods and tech¬ the aversive stimulus, when all reactions cease.
niques for the objective long term study of aggression- Sensory scanning, and locomotion and manual
attack sequences in animals and man. The departure manipulative behaviors are prepotent to attack reac¬
point for this work is the series of studies by Ulrich, tions during the period prior to unconditional stimu¬
Azrin, and their colleagues (Azrin, Hake, 8c Hutchin¬ lus occurrence (Hutchinson 8c Emley, 1972; Hutchin¬
son, 1965; Azrin, Hutchinson, & Hake, 1963; Azrin, son 8c Emley, 1973). Figure 8 shows the records of a
R. R. Hutchinson 419

MOUSE MC-40
SHOCK
M-2
- SHOCK

BITES
_L~ ^. BITING tttti-mm-
4 SEC.'
I- 4 MINUTES -1

Fig. 6. Reconstruction of actual event records obtained from

one squirrel monkey subject of biting attack responses both
prior and subsequent to aversive stimulation. Shock was 400 v,
delivered for 200 millisec each four min.
MONKEY
monkey and a human male when the separate reac¬
SHOCK tion classes could, each occur. Leygir presses the
monkey, and movements of the torso and upper limbs
of the male human, progressively increase in fre¬
quency during a period of conditional gtimulus de-

BITING
MQNKE.Y
4 SEC.
MQ‘22
SHOCK

LEVER PRESSES
MAN I II II IIII llll Ii-
N-4
NOISE
-4 MINUTES -1

MAN
K-4
NOISE

Fig. 5. Automatic recordings of biting responses by a mouse, a

squirrel monkey, and an adult male human subsequent to PHYSICAL MOVEMENTS
aversive stimulation. The mouse subject was of the Swiss-
Webster strain. Shock was via tail electrodes, at 600 v for 200
msec. Biting attack upon a nylon object began shortly there¬
after. The nylon bite object was connected to a Statham force I- 3 MINUTES -1
transducer, the output of which was amplified and recorded by
a Grass Model 5 polygraph. Static peak force required to pro¬ Fig. 7. Reconstruction of actual event records obtained from a
duce excursions as seen here is approximately 35 g. The squirrel squirrel monkey subject and an adult male human during
monkey was shocked for 200 msec via tail electrodes. The pres¬ periods of aversive stimulation. For the squirrel monkey, electric
sure tracing of biting was obtained from a Statham P23 pres¬ shock was 400 v for 200 msec delivered every 4 min. The re¬
sure transducer, the output of which was amplified and dis¬ sponse lever was mounted on the intelligence panel immediately
played by a Grass Model 5 polygraph. The human subject was ahead of and to the left of the subject. For the human, aversive
stimulated with 110 db of 2000-Hz tone. Speakers were mounted stimulation was 2 seconds of 2000-Hz pure tone delivered each 3
on either side, and several feet to the front of the subject. minutes at an intensity of 110 db (measured at the amplifier
Biting, in the form of nonfunctional, concentric occlusion, output transformer). Aversive stimulation is preceded by pro¬
began shortly thereafter. Responses were recorded with sub- gressively increasing and then decreasing frequencies of manual
dermal EEG electrodes. Signals were amplified, integrated, and responding and/or the physical movements of upper torso or
recorded by a Grass Model 5 polygraph. arms.
 420 BY-PRODUCTS OF AVERSIVE CONTROL

MONKEY portunity to bite (Hutchinson 8c Emley, 1972; Hutch¬

inson, 1970). The pair of cumulative records obtained
MC-64 on Day 5, for a squirrel monkey subject illustrates
SHOCK that chain-pulling responses occurred predominately
before the deliveries of electric shocks. Immediately
before shock delivery, chain pulling was absent. Sub¬
sequent to shock delivery, biting attack occurred and
LEVER PRESSES
then progressively diminished. Later, for a series of
tfflttlf
tests, the rubber hose was removed from the chamber
for the entire session. Both post-shock and pre-shock
BITES chain pulling increased. These effects can be noted on
Day 22 and Day 34. After replacement of the hose,
performance returned to the earlier pattern. An exag¬
4 MINUTES
gerated illustration of this dual effect of interaction
between the pre-event and post-event behaviors is
illustrated in the lower graph of Figure 9. Here for a
MAN squirrel monkey subject, tests were conducted of the
interactions between hose biting and lever pressing.
Studies of such interactions are important in under¬
standing the separate processes for two reasons. On
the one hand, the elimination of the opportunity to
engage in biting attack reactions subsequent to the
physical'Movements delivery of the aversive stimulus generated greater
rim -—-
numbers of anticipatory or pre-shock manual manipu¬
lative reactions—an effect similar to only one other
condition known to us—that produced by an increase
BITES
in shock intensity or duration. Thus, preventing at¬
tack subsequent to aversive stimulation is function¬
I- 3 MINUTES -1 ally similar to increasing the intensity of an aversive
stimulus. This suggests the possibility that the influ¬
Fig. 8. Reconstruction of actual event records obtained for a ences of a biting attack are similar to those of shock
squirrel monkey subject and an adult male human. For each
subject simultaneous records of pre*aversive stimulus manual reduction (Hutchinson, Renfrew, 8c Young, 1971). At
manipulative or movement reactions and post-aversive Stimulus a more speculative level, this effect may account for
biting was obtained. Parameters of aversive stimulation were at least a portion of the reinforcement inherent in bit¬
as described previously.
ing attack sequences which has been reported in
previous studies (Azrin, Hutchinson, & McLaughlin,
livery, but immediately prior to unconditional stimu¬ 1965).
lus delivery these reactions cease. Following delivery Of greater relevance to the present discussion is
of the unconditional stimulus biting reactions occur. the second effect noted; removal of the opportunity to
The addition of apparatus suitable for sensing a attack caused an increase in post-shock manual manip¬
second reaction of different topography is not only a ulative and sensory scanning responses of chain pull¬
technical exercise. Such a change alters the environ¬ ing and lever pressing. We have repeatedly found that
ment and may influence the relative and combined post-shock manual responses have increased when the
behavioral expressions which occur. An example of opportunity to attack was absent, to a level almost
the influence of a change in features of the environ¬ identical in absolute number with the frequency of
ment (established at the convenience of the investiga¬ post-shock attack responses which occurred during
tor to allow simultaneous sensing and recording of periods when a hose was present (Hutchinson, 1970;
several responses) upon the several reactions which Hutchinson 8c Emley, 1972). Conversely, the provision
may occur is illustrated by the upper graph in Figure of opportunity to attack causes a shift from post-shock
9. In Figure 9, the aggression-attack reactions of bit¬ manual manipulative and sensory scanning responses
ing a rubber hose, and the manual-manipulative and to biting attack. When allowed, attack reactions are
sensory-scanning reactions involved in pulling a chain, prepotent over other locomotor and manipulative re¬
are influenced by the presence or absence of the op¬ actions subsequent to aversive stimulation: this is the
R. R. Hutchinson 421

MD~3 that any movements or other efforts by the subject

CO might be followed by a reduction or termination of
UJ BITES CHAIN PULLS
GO the stimulus and thus be reinforced. With the human
z
o testing techniques, actual reliable contact between an
CL
CO aversive stimulus and the subject is provided by the
LU
tr recruitment of volunteers who are fully informed of
the noxiousness which may occur. In addition the
UJ
> volunteers are assured that no unreasonable or poten¬
tially hazardous stimulation will be used, that finan¬
<
_i cial remuneration will be received, that only mild
3
noxious stimuli will be used, and that escape is always
O immediately available. In summary, for both animals
MINUTES and humans, the testing paradigms have been de¬
signed to eliminate a class of reactions which is known
to be highly probable during or immediately subse¬
quent to the occurrence of an aversive stimulus, e.g.,
LEVER PRESSES physical movement and escape from the noxious stim¬
ulus. Innate reactions of flinching, jumping, and run¬
ning immediately after application of an aversive
stimulus are thoroughly documented m the litera¬
ture (Brogden, Lipman, 8c Culler, 1938; Campbell k
Teghtsoonian, 1958; Liddell, 1934). Also, in more
recent studies, it has been shown that an experimen¬
tally learned escape reaction will become prepotent
over aggression-attack sequences (Azrin, Hutchinson,
& Hake, 1907; Ulrich, 1907).
Are there reactions which are equally or more
potent than attack sequences in £nyironm£nt§ where
no reinforcement for such non-attack behavior oc¬
curs? In fact the temporal primacy of locomotor and
MINUTES manual manipulative reactions over aggression attack
behaviors has been observed, A minority of subjects in
Fig. 9 (upper). Cumulative response records for biting attack our laboratory continue to make a small number of
and chain pulling by 6ile squirrel monkey subject before, dur¬
ing. and after removal of the opportunity to engage in biting
manual responses for long periods subsequent to sho^k
attack responses, During the period illustrated on Days 22 and delivery and before attack, even when attack oppor¬
34 while the bite hose was removed from the test chamber} the
tunity is present. Figure 10 is an illustration of the
subject demonstrated increased pre shock chain-pulling response
hursts equivalent to response bursting of biting attacks during
high speed event records obtained for both lever
previous periods of hose availability. Subsequent to return of presses and biting attacks, Whereas lever presses domi¬
the opportunity to attack, performance returned to original nate before shock delivery, the typical pattern of bit¬
levels, (lower). Cumulative response record segment showing
magnified illustration of the increase of pre-shock lever presses ing attacks predominates after shock delivery, The
and the instatement of post-shock lever-press burst reactions important feature of this illustration, however, is the
subsequent to the removal of the opportunity to attack. The
inter-shock interval was 4 min.

MC-40

suggestion presented, but it does not take into account SHOCK _

several features of the testing apparatus and methods

LEVER PRESSES
employed in the studies. First, the testing environ¬
ments all include features of physical or social re¬
straint. For infrahuman subjects, cage arrangements
- 4 MINUTES -1 |- 4 MINUTES -
have stockades or other restraint devices which guaran¬
tee contact between an applied aversive stimulus and Fig. 10. Reconstruction of actual event records obtained for
two squirrel monkey subjects of lever-press responding and
the organism. Additionally, the stimulus is delivered
biting attacks. Note that lever presses which occur subsequent to
for only a brief instant to minimize the possibility shock delivery, occur prior to ensuing episodes of biting attack.
 BY-PRODUCTS OF AVERSIVE CONTROL
422

differences in our handling methods or recording ap¬

LEVER PRESSES
paratus. Nevertheless, the general pattern is clear.
Manual manipulative, locomotor, and sensory scan¬
ning responses occur prior to noxious stimulation, but
HOSE BITES cease immediately before aversive stimulation. Fol¬
lowing the aversive stimulus, tendencies towards loco¬
motion, sensory scanning, and manual manipulation
o
are later followed by aggression-attack sequences.
si
<
_1 LEVER PRESSES
The behavioral sequences which occur both before
ZD and after aversive stimulation may vary even when
stimulus parameters are held constant. Two separate
HOSE BITES
and opposite processes, habituation or facilitation, can
occur even with invariant features of aversive stimuli.
If aversive stimuli are relatively mild and/or frequent,
MINUTES the reaction sequences show progressive reductions in
amplitude and frequency over successive occurrences of
Fig. 11. Cumulative response record segments for simultaneously
recorded manual manipulative and aggressive attack sequences
the stimulus (Hutchinson, Renfrew, &r Young, 1971;
for four different squirrel monkey subjects. The general pattern Ulrich 8c Azrin, 1962). Figure 12 (upper panel) illus¬
for each subject is a relatively greater probability of pre-shock trates the cumulative response records of post-shock
manual manipulative responding and a relatively greater proba¬
bility of post-shock biting attack responding. Individual differ¬ attack behaviors over a series of relatively mild and
ences from these general patterns are circled and are discussed frequent shock occurrences. Successive shocks result in
in the text. progressively fewer attacks.
The repetitive, but infrequent delivery of intense
temporal position of the manual behaviors which do aversive stimuli can result in increased responses or
occur following shock. When such manual responses facilitation, rather than habituation. Figure 12 (lower
occur, they predominately occur immediately after panel) illustrates for one squirrel monkey subject the
shock and before the biting attack responses. Thus, effect of infrequent, intense shock deliveries upon
even in environments where there has been no rein¬ pre-shock lever pressing and post-shock biting attack.
forcement for long periods of time, a number of sub¬ The rate of both responses increases progressively un¬
jects continue to engage in locomotion, manual til responding is almost continuous. Some time is
manipulation, and sensory scanning reactions immedi¬ always necessary for the development of this response
ately after shock and before the aggressive-attack reac¬ facilitation effect; somewhere between 20 and 40 min¬
tion sequences. utes seems to be the necessary amount of time for
Considerable individual variation in the temporal several species tested. Once the process is begun it is
and intensive patterns of the reaction sequences is possible to terminate all aversive stimulation, yet
typically observed (Hutchinson & Emley, 1972; Hutch¬ continue to observe the occurrence of these reactions
inson, Renfrew, & Young, 1971). Figure 11 is an ex¬ for hours and even days (Hutchinson 8c Emley, 1972;
ample of four subjects. For each subject, the relative Hutchinson, Renfrew, 8c Young, 1971).
probability of manual manipulations is greatest before Due to the processes of habituation and facilita¬
shock and the relative probability of biting attack tion, a momentary measurement of the amplitude or
greatest after shock. For several subjects, there is also frequency of display of reactions is not a reliable esti¬
a tendency toward the brief absence of all reactions mate of the current level of “aversiveness” to which
immediately prior to shock. Each subject, however, the organism is exposed, since, depending upon the
provides a slightly different variant on these general history of aversive stimulus encounter, the subject’s
statements. Subject MC-30 illustrates a relatively clear current reactions may be excessive or diminished by
temporal differentiation of these reaction patterns. considerable degree relative to what another subject
MC-1 shows some post-shock as well as pre-shock lever or the same subject would have shown at an earlier
pressing. MC-12 shows some pre-shock biting attack time.
in addition to greater pre-shock lever pressing. MC-13 Further experiments have found that additional
shows both post-shock lever pressing and pre-shock reaction sequences are influenced by the occurrence of
biting attack. Currently our understanding of these aversive stimulation. After the delivery of an aversive
processes does not allow us to be certain whether these stimulus, a series of manual manipulative and loco¬
differences depend on constitutional or on technical motive responses, or after a series of biting attack
R. R. Hutchinson 423

stimulus period. This behavior is absent for a brief

period immediately before the aversive stimulus oc¬
curs. After shock delivery there is a rapid flurry of
biting attacks. Following this, both subjects show a
series of lick responses which first increase and then
decrease over ensuing seconds. Figure 13 (lower panel)
shows in cumulative record form, the response pat¬
terns for lever presses, bites, and licks for one subject.
Pre-shock lever pressing increases up until almost time
for shock delivery, but shows a tendency for reduction
immediately before shock. Subsequent to shock, a
rapid series of biting attack reactions occurs. Follow¬
ing this, a negatively accelerated burst of water lick¬
ing responses takes place.
Figure 14 shows the temporal distribution of man¬
ual manipulative and locomotor responses, biting at-

MGi2 MC-30
SMOCK

) Hitt LEYER PRESSES ttrarr

BITES

LICKS
—IIHII -H—K-
MtNMTC? -

MINUTES

Fig. 12 (upper). Cumulative response record, for one scpdrrel

monkey subject during a session where frequent and relatively
mild aversive stimulation was delivered. Habituation of re¬
sponding over successive shock episodes is evident. Shocks were
100 msec duration, 100 V intensity, delivered once each minute,
(lower). Cumulative response record for one squirrel monkey
subject where biting attack reactions and manual lever pressing
responses were simultaneously recorded during response facili¬
tation over successive shock occurrences. Shock was 400 v deliv¬
ered for 200 msec every 4 min.

MINUTES
reactions, subjects begin drinking (Hutchinson 8c Em-
ley, unpublished research). Figure 13 (upper graph) Fig. 13 (upper). Reconstruction of actual event records obtained
presents event recordings for two subjects during a for two squirrel monkey subjects when manual lever pressing,
biting attacks, and water licking were each simultaneously
period before and after electric shock delivery. For measured during periods of aversive stimulation. Following
each subject the typical pattern of manual manipula¬ biting attack, subjects begin drinking for several seconds,
tive and locomotor reactions (in this case lever presses) (lower). Cumulative response record segments for one squirrel
monkey subject of manual lever pressing, biting attacks, and
is apparent for both subjects during the pre-aversive water licking during periods of aversive stimulation.
424 BY-PRODUCTS OF AVERSIVE CONTROL

SHOCK

LICKS

MC-30

HOSE PRESENT

DAY 42

HOSE ABSENT

DAY 56

SHOCK

HOSE PRESENT

SECONDS DAY 67

Fig. 14. Temporal distribution of manual lever pressing, biting

attack reaction and water intake responses prior and sub¬
sequent to aversive stimulation. Responses are totalled over 12- I—12 MINUTES—I
f?ec intervals. Shock was delivered at “0” seconds.
Fig. 15. Cumulative response record segments of fluid intake
responses for one squirrel monkey subject before, during and
subsequent to removal of the opportunity to attack. Intake
tuck responses, and licking responses averaged across responses and the amount of fluid consumed are increased in
the absence of attack opportunity.
an entire session for one subject. Here, in the in¬
dividual intershock intervals, lever pressing shows a
progressive increase in the period before shock. As the
manner similar to that already illustrated for pre-
minimum class intervals are 12 seconds, no abrupt
shock manual responses and post-shock biting attack
termination of responding in the few seconds prior to
reactions. If intense aversive stimuli are delivered
shock is evident, although this is frequently seen in
infrequently, licking responses will begin to occur at
individual records. Subsequent to shock delivery, a
a far greater rate and in an almost continuous episode
brief series of lever presses occurs, followed immedi¬
as compared with earlier occasions. Figure 16 dis¬
ately by biting attacks upon the rubber hose. This
plays for one subject, this facilitation of drinking re-
attack reaction subsides progressively and then water
licking responses occur first at a higher and then at a
lower frequency falling over some seconds to zero. LICKS
Limiting or expanding other response opportuni¬
ties has effects on the magnitude or frequency of
MC-12
drinking responses which are similar to those previ¬
ously described for lever pressing and biting, Remov¬
ing the opportunity to attack, by removing the rubber
hose from the chamber, increases the number of licks
and the amount of water consumed. These reaction
shifts remain for as long as the environment is altered.
Figure 15 portrays the effect for one subject in succes¬
sive experimental sessions where the hose first was pres¬
ent then removed for several days and then returned
to the chamber. Removal of the hose, and thus the
opportunity to attack caused biting attacks to again
quency of licking subsequent to shock. Return of the
opportunity to attack caused biting attacks to again
occur as before. Licking responses were reduced to i-12 MINUTES -1
their original level. Fig. 16. Cumulative response record segments of fluid intake
Drinking reactions are also influenced by the tem¬ responses for one squirrel monkey subject in early and later
portions of a period of aversive stimulation showing develop¬
poral pattern and intensity of shock delivery in a ment of response facilitation.
R. R. Hutchinson 425

actions when continuing, infrequent, intense, and ing. Sometimes, only partial sequences will occur since
aversive stimuli are presented. alternate response reaction classes may, through rein¬
From the foregoing discussion and illustrations, it forcement, completely or substantially displace in
is evident that occurrence of aversive stimuli can in¬ time and strength the basic reaction patterns noted.
fluence a series of complex motor reactions for a Each of these reaction types involves a complex se¬
period after aversive stimulation and, through condi¬ quence of coordinated movements, frequently involv¬
tioning, to stimulus periods prior to the delivery of ing many muscle groups in concert with sensory
a particular ensuing aversive stimulus. Further, there scanning processes acting upon particular objects or
is a predictable ordering of the frequencies or tenden¬ places in the environment. So far, our findings sug¬
cies for various reaction sequences to occur, both prior gest that these reactions, though flexible and capable
to, and after aversive stimulus delivery. During condi¬ of considerable modification, nevertheless tend to oc¬
tional stimulation, locomotor, manual manipulative, cur in an essentially identical fashion in many mem¬
and sensory scanning reactions occur at a progres¬ bers of the same species. Additionally, individual
sively increasing frequency until some period immedi¬ subjects show great consistency between successive
ately before the aversive unconditional stimulus de¬ behavioral episodes oyer long periods. Such species
livery. During that same period, occurrence of aggres¬ specificity, coupled with uniformity across successive
sion-attack sequences and drinking reactions can instances makes it tempting to refer to these reactions
occur, though these are of lesser strength than the as reflexes or innate reactions. We have refrained
locomotor and manual manipulative reactions. Im¬ from this for several reasons. We haye little evidence
mediately prior to unconditional stimulus delivery, all regarding the portions or features of these perfor¬
reaction tendencies cease, a period of behavioral arrest mances which may be independent of learning. Also,
or suppression occurs. After the occurrence of an un¬ the modification of these reaction sequences which
conditional aversive stimulus, locomotor, manual ma¬ comes about from alterations in post-response environ¬
nipulative, and sensory scanning responses are maximal ment (reinforcement) implies a reliance on feedback
for a brief period. Subsequent to this time, aggression that is perhaps not a common feature of other proc¬
attack reaction sequences develop to a high frequency esses referred to as reflexive or innate, and has been
and then gradually return to zero. Later yet, and sub» little emphasized in connection with them.
sequent to attack reactions, drinking responses occur,
first, at a progressively increasing and then a gradu¬
ally decreasing frequency over ensuing seconds and BEHAVIOR CAUSED BY AVERSIVE
minutes. These sequential relationships are sketched STIMULI IN ESCAPE PARADIGMS

symbolically in Figure 17.

Manual manipulation, arrest, attack, and drink Operant conditioning procedures involving aver¬
reactions may or may not occur in the particular stim sive control may be divided into the response-produced
ulus context present at the time a subject is respond- stimulus-offset (“escape”) and response-produced-
stimulus^onset (“punishment”) paradigms. In escape
conditioning or performance paradigms some aspect
AVERSIUE STIMULATION
of behavior results in the termination of either the
J LNCOromONAL STIMULUS PERIOD conditional aversive stimulus or the unconditional
aversive stimulus. Often where the conditional stimu¬
lus is terminated the paradigm is defined as avoid¬
ance conditioning—a reference to the fact that such
responding also avoids occurrence of the unconditional
stimulus. For both unconditional-stimulus and condi¬
tional-stimulus escape, the response occurs in the pres¬
ence of an identifiable unique aspect of the environ¬
Fig. 17. Schematic illustration of the sequential features of be¬
havioral processes produced by aversive stimulation. During ment. As such, the reaction sequences which will
the conditional stimulus period, locomotor, manual manipula¬ occur can be predicted, at least in part, by knowledge
tive and visual scanning responses are progressively elevated. In
of how behavior is influenced directly by aversive
a more imminent temporal position to unconditional stimula¬
tion, behavioral reactions are arrested or suppressed. Subsequent stimuli as described in the previous section.
to unconditional stimulus delivery, manual manipulative, loco¬ Several general behavioral relationships are ob¬
motor and visual scanning responses are increased. Subsequent
served during escape/avoidance-learning and perfor¬
to this, aggression-attack sequences occur. At yet a later point
in time, fluid intake responses begin. mance. During presentation of the conditional stimu-
426 BY-PRODUCTS OF AVERSIVE CONTROL

lus, locomotor, manual manipulative, and visual Thus the temporal and intensive patterns of man¬
scanning reactions will be likely (Brogden, Lipman, 8c ual and locomotor behaviors observed on schedules
Culler, 1938; Miller, 1948; Mowrer Sc Lamoreaux, involving escape-avoidance procedures are similar or
1942). As the conditional stimulus continues toward identical to those observed routinely when such con¬
the time ordinarily corresponding to introduction of tingencies are not in effect. Therefore, to what extent
the unconditional stimulus, behavior will initially in¬ are the performance elements, normally observed un¬
crease in probability and then decrease (Anger, 1963; der escape and/or avoidance training and learning
Hoffman, 1966; Sidman, 1955). After the occurrence procedures, attributable to the contingencies rather
of the unconditional stimulus, a brief period of than as a direct result of contact with conditional and
heightened probability of manipulative, locomotor, unconditional aversive stimuli? This question can
and visual scanning responses will be evident (Sidman, only be answered conclusively, it seems to me, by a
1958; Hoffman, 1966). Later yet, or in situations
where escape reactions are not reinforced, aggression-
attack sequences will occur (Azrin, Hutchinson, 8c
Hake, 1967). Each of these patterns is similar to that co
LlJ
described in the previous section where such per¬ CO
formances were shown to result from conditional and o
unconditional stimulation directly. Figure 18 (upper CL
CO
UJ
panel) presents a segment from a cumulative record of
tr
one subject on a Sidman avoidance schedule, during a
LlJ
portion of the session when the unconditional aversive >
stimulus actually did occur twice. The rapid post¬ I—

stimulus burst of lever pressing behavior frequently

<
reported in this circumstance, is evident. The lower
graph of Figure 18 presents a segment from two simul¬ o
taneous cumulative records obtained from another
subject on similar experimental parameters, except
that a bite hose was placed in the test chamber. For
this animal, the opportunity to engage in attack se¬
quences, caused a total displacement of the post¬
stimulus lever-pressing behavior by a flurry of biting
CS ESCAPE
attacks on the rubber hose (Azrin, Hutchinson, 8c MCi (AVOIDANCE)
Hake, 1967). In numerous avoidance sessions where
the attack opportunity is present we have observed
that subjects typically show a nearly or totally com¬
plete shift in the post-shock reaction sequences to that
pattern shown in the lower graph of Figure 18. As
suggested on the basis of data presented in earlier
sections, biting may somehow serve to reduce the
effect of shock. Coupled with an essentially zero rein¬
forcement probability for bursting responses on the
lever, this may produce alteration of the two per¬
formances seen in the lower portion of Figure 18.
The literature also shows that escape or avoidance
behavior is often absent or infrequent early in a test¬ b- II MINUTES —-1
ing session even after learning seems stable in earlier
Fig. 18 (upper). Cumulative response record segments during
tests. This daily or weekly recurrent absence of initial Sidman avoidance conditioning for one squirrel monkey subject.
strength coupled with the progressive increase in re¬ Subsequent to shock delivery, a brief flurry of lever presses
occurs. Avoidance parameters were response-shock interval, 30
sponding over subsequent shocks during testing has
sec, shock-shock interval, 30 sec, shock intensity 400 v 200 milli-
been referred to as “warm up” (Hoffman, 1966; Hoff¬ sec duration, (lower) Cumulative response record segments
man, Fleshier, Sc Chorny, 1961). In the previous sec¬ during a Sidman avoidance program. Simultaneously recorded
manual lever pressing and biting attacks for one squirrel monkey
tion the similar facilitation of shock-produced per¬
subject are shown. Note that subsequent to shock delivery, the
formances by repetitive shock deliveries was illustrated. flurry of responding occurs upon the bite hose.
R. R. Hutchinson 427

careful three-part experimental sequence. Subjects LEVER PRESSES

must first be exposed for an extended period to the MC-I
conditional and unconditional stimuli in the absence
of any contingencies. Subsequent to this, contingencies DAY 767

may be introduced and shifts in performance noted. DAY 22

To establish the certainty of the contingency’s long

term influences, at a later time a return to original DAY I

noncontingent stimulation must be arranged. Un¬

fortunately, these conditions are almost never met
experimentally. Typically, a shift in some feature of
the contingencies or in their temporal or interlocking
character, has been the primary experimental manipu¬
lation and a shift from one level of performance to DAY 31

another has been the observed result. Such tests are

helpful, but they leave unanswered the question of
what topographic and intensive features of behavior
are independent of, or dependent on, contingencies
per se. In cur studies we typically do not find that
DAY 50
subjects respond at high continuous response rates
during response-independent stimulus presentation.
A shift to avoidance conditions from response-inde¬
DAY 100
pendent shock usually produces large increments in
response rate. An example of such an effect may be
seen in Figure 19. Here selected segments of cumula¬ DAY 130
tive records under three successive experimental con¬
ditions are displayed. This particular subject had a
two-year history of response-independent shock every
four minutes for one hour sessions. During this period MINUTES
a variety of tests were performed using this behavioral
Fig. 19. Cumulative response record segment for one squirrel
baseline. The baseline behavioral pattern is shown as monkey subject during response-independent shock, Sidman
the uppermost segment of Figure 19. At this point avoidance conditioning, and subsequent response-independent

avoidance conditioning was begun. By Day 22 of the shock sessions. Avoidance conditioning results in a large in¬
crease in manual responding compared to initial response-
avoidance program a high steady rate of nearly shock- independent shock baseline performance. Subsequent to avoid¬
free responding was occurring. In subsequent tests the ance, responding gradually falls until it stabilizes but at a level
subject was returned to response-independent shock of responding many times greater than prior to avoidance
learning.
conditions and behavior was observed over an ex¬
tended period. The rate of responding gradually fell
to the level seen in the bottom segment of Figure 19. the occurrence of some response. Since in this para¬
Two features of these results are important. First, it digm the unconditional stimulus and the conditional
took a long time for performance to stabilize at a stimulus are not present during a response, but only
lower level after exposure to avoidance conditioning; afterward, the ongoing response rate will be deter¬
second, the level at which performance stabilized was mined by other conditions which are present, such
10-15 times greater than it had been before the as food deprivation, food reinforcement schedule, etc.
avoidance training. Each of these results has also Generally, since the establishment of a contingency
been noted in earlier studies (Kelleher, Riddle, & between a response and an unconditional aversive
Cook, 1963; Sidman, 1960). stimulus simultaneously places a negative reinforce¬
ment contingency on behaviors other than the to-be-
punished response (Dinsmoor, 1954), and since, as has
BEHAVIOR CAUSED BY AVERSIVE been shown in the prior section, the terminal effect of
STIMULI IN PUNISHMENT PARADIGMS conditional stimuli is to produce depression or an ab¬
sence of behavior, any responding which results in the
In punishment paradigms, presentation of the con¬ occurrence of unconditional stimulation will be re¬
ditional or unconditional stimulus is contingent upon duced. After the actual occurrence of an uncondi-
428 BY-PRODUCTS OF AVERSIVE CONTROL

PUNISHMENT
first at a high rate and then at a decreasing rate. The
subject who experienced shock delivery contingent
upon lever pressing (labeled Punishment), produced
the pattern illustrated, one seen in numerous other
subjects in the laboratory. Generally, no lever press¬
ing occurred during the first few minutes of the
response-independent shock delivery sessions, nor on
the first day when shocks were response produced. At
about seven minutes into the first punishment session
the initial lever press occurred and produced a shock.
Subsequent to this shock a flurry of biting attacks
I- I MINUTE —I occurred. For this subject; no further lever presses oc¬
curred during any session for the next 20 punishment
Fig, gO, Cumulative response record segments for one squirrel
sessions. A quite different performance is illustrated
monkey subject during variably-interval food reinforcement and
simultaneous continuous punishment of manual lev^r pressing. in the right hand column, for subject MC-5. This sub¬
Biting attack episodes oeeiiy subsequent to shock deliveries. Data ject had a much higher rate of pre-shock manual re¬
are reconstructions of event records obtained from R. E. Ulrich,
sponding during the response-independent shock ses¬
Western Michigan University, Kalamazoo, Michigan.
sions. d he consistent pattern was a progressively
increasing lever press rate up to almost the instant of
tional stimulus, however* other reaction sequences shock delivery, with at most only a very brief pause
ordinarily produced by unconditional aversive stimu¬ prior to shock occurrence. Further, little or no biting
lation will be observed, Figure 20 presents the simul¬ occurred subsequent to shock. Introduction of punish¬
taneous cumulative records of a subject on a variable- ment contingencies produced an actual increase in
interval food-reinforcement schedule and simultaneous pre-shock lever pressing. By Day 20 on this procedure,
continuous shock punishment for lever pressing. Also, m fact, an increase in responding under contingent
the opportunity to attack a rubber hose was present. shock conditions was evident. Results similar to those
Note that bursts of biting attack tend to occur after for MC-5 have also been reported in recent expert
the delivery of electric shocks (Ulrich, personal com¬
ments (Morse, Mead, & Kelleher, 1967; Stretch, Orloff,
munication). Whether fluid intake responses are in¬ k Dalrymple, 1968) in other laboratories and have
creased subsequent to the occurrence of punishment served as the evidential base for a theoretical position
by an unconditional stimulus is unknown to me.
When responding is maintained and produced by
arrangements other than positive reinforcement sched¬ NO PUNISHMENT
ules, additional interactions with punishment pro¬
grams can occur. When unconditional or conditional
aversive stimulation is made contingent on perfor¬
mances which have been generated by aversive stimu¬
lation (either by escape-avoidance routines, or by re¬
sponse-independent shock application), the results can
appear confusing. Sometimes in these situations, the
contingent application of conditional aversive stim¬
ulation can actually elevate behavior. Figure 21 pre¬
sents selected cumulative record segments for two
squirrel monkey subjects during response-independent
shock delivery programs and subsequent response-
contingent shock delivery schedules. The two records
show the simultaneous results for pre-shock manual
manipulative responses and post-shock biting attacks.
In the case of subject MC-30, during response-inde¬
pendent shock delivery (labeled No Punishment) lever MINUTES
pressing occurred at a progressively increasing rate Fig. 21. Cumulative response record segments obtained simul¬
until just before shock when behavior was absent. taneously for manual lever pressing and biting attack reactions
for two squirrel monkey subjects prior to and during punish¬
Subsequent to shock delivery, biting attacks occurred. ment of lever pressing.
 R. R. Hutchinson 429

that electric shock and other aversive stimuli, can ever a part of these experiments. In the left hand
serve as positive reinforcer-like events, depending column, the upper tracing illustrates final perfor¬
upon the schedule of presentation of such events. mance under response-independent shock conditions.
However, other interpretations are possible. Figure Over a series of months, the fixed-interval value of the
21 shows prompt and total cessation of responding for punishment schedule was gradually reduced from four
one subject (MC-30) when shifted from a response- minutes to two minutes, one minute, 15 seconds, and
independent shock schedule to a response-contingent finally to 5 seconds. During these tests responding was
shock schedule. Figure 22, on the other hand, shows not reduced, rather, overall rate of response increased.
that reinstatement of response-independent shock pro¬ Throughout these tests, it was noted that not on a
cedures for subject MC-30 produced a gradual, but single instance was responding low enough to cause
progressive return of responding to a pattern and shock to be delayed more than a few seconds, as com¬
level identical to that observed prior to punishment pared to when shock would have occurred on a
conditions. In the case of subject MC-5 and others dis¬ response-independent shock baseline. The high fre¬
playing similar high response rate patterns, additional quency of responding exhibited by MC-5 caused the
experiments have been conducted. Figure 23 presents shock delivery to occur without change, just as though
full cumulative records for MC-5 through a series of scheduled by a clock without regard to behavior. For
experimental tests involving successive alterations in reasons unknown to us, this situation was altered for
the fixed-interval shock punishment schedule, and the first time at point A on Day 18 of the 5-sec FI
finally followed by a return to response-independent punishment condition, shown in the center column
shock procedures. No other response maintenance pro¬ of Figure 23. On the next day (Day 19) the pattern of
cedures (such as food, time out from shock, etc.) are performance was immediately different from the be¬
ginning of the session. No responding occurred for
almost 4 minutes. The first response produced a
LEVER PRESSES shock. This happened three more times during session
19. Subsequently, no responding was observed during
MC-30 any of the 50 additional punishment test sessions.
A return to response-independent shock conditions
CO LAST DAY PUNISHMENT also produced for this subject, a gradual increase in
LU responding until it assumed the same temporal and
CO
intensive pattern initially displayed prior to punish¬
~ZL
O ment testing. This behavior which might have been
CL RESPONSE INDEPENDENT SHOCK interpreted as “maintained” by the response-produced
(O shock contingency, was in fact, a shock-produced-
W
DC TT response performance, Pr^yiyu? r^p^ts o£ similar
o DAY I
effects have in all cases inadvertently employed one or
O more of several different collateral response producing
CM
or re&ponse-reinfwcmg procedures Frequently these
studies have employed an avoidance history (Byrd,
>1 —^
h 1969; Kelleher & Morse, 1968; McKearney, 1968;
DAY 5
< Stretch, Orloff, & Dalrymple, 1968). In the previous
_i section we saw that an avoidance history, even long
13
past, could markedly increase the response generating
Z) effects of subsequent response-independent shock de¬
DAY 7
o livery schedules. Other experiments have employed
free-shock baselines (McKearney, 1969; Morse, Mead,
8c Kelleher, 1967). In this and earlier sections we have
shown that free shock can generate responding. In
■- 12—- still other experiments, simultaneous reinforcement
MINUTES (Azrin, Holz, Hake, 8c Ayllon, 1963) procedures such
Fig. 22. Cumulative response record segments for one squirrel as time out from shock (McKearney, 1970; Morse 8c
monkey subject during final testing of punishment for manual Kelleher, 1970), or previous food reinforcement his¬
responding and subsequent response-independent shock condi¬
tories have been used (Morse 8c Kelleher, 1970). Such
tions. Responding gradually recovers upon removal of the pun¬
ishment contingency. procedures produced high levels of responding neces-
 430
BY-PRODUCTS OF AVERSIVE CONTROL

MC-5
FREE SHOCK FI PUNISHMENT PI PUNISHMENT

Fig. 23. Cumulative response

records for one squirrel monkey
subject during stable perfor¬
mance on response-independent
shock and during a series of ex¬
periments where manual lever
pressing was punished. Later
yet, the subject was returned to
response-independent shock con¬
ditions. Whereas punishment,
particularly at long intervals,
appears to increase responding
temporarily, behavior is eventu¬
ally totally suppressed. Return
to response-independent shock
produces a gradual reinstate¬
ment of performance.

sary to both compete with the response-suppressing Brocben, W. J., Lipman, E. A., 8c Culler, E. The role of
effects of the punishment schedule and jo? to generate incentive m conditioning and extinction. American
behavioral patterns which obscure or eliminate dis- Journal of Psychology, 1938, SI, 109-11?.
Byrd, L. D. Responding in the cat maintained under re¬
criminable episodes of shock reduction following ab¬
sponse-independent electric shock. Journal of the Ex¬
sence of responding. Each of these practices can
perimental Analysis of Behavior, 1969, 12, 1-10.
confuse the outcome of a particular study and cloud Campbell, B. A., 8c Teghtsoonian, R. Electrical and be¬
understanding of the basic behavior-generating effects havioral effects of different types of shock stimuli on the
of aversive stimulation. rat, Journal of Comparative and Physiological Psychol¬
ogy, 1958, Sly 185-192.
Dinsmoor, J. A. Punishment; I. The avoidance hypothesis.
REFERENCES Psychological Review, 1954, 61, 34-46.
HAKE, D. F. Actual versus potential shock in making shock
situations function as negative reinforcers. Journal of
Anger, D. The role of temporal discriminations in the rein¬ the Experimental Analysis of Behavior, 1968, 11 385-
forcement of Sidman avoidance behavior. Journal of the 403.
Experimental Analysis of Behavior, 1963, 6, 477-506. Hake, D. F., 8c Azrin, N. H. An apparatus for delivering
Azrin, N. H., Hake, D. F., & Hutchinson, R. R. Elicitation
pam-shock to monkeys. Journal of the Experimental
of aggression by a physical blow. Journal of the Experi¬ Analysts of Behavior, 1963, 4, 297-298.
mental Analysis of Behavior, 1965, 8, 55-57.
Hoffman, H. S. The analysis of discriminated avoidance.
Azrin, N. H., Holz; W. C., Hake, D. F., 8c Ayllon, T.
In W. K. Honig (Ed.), Operant behavior: areas of re¬
Fixed-ratio escape reinforcement. Journal of the Experi¬
search and application. Englewood Cliffs, N.T.: Prentice-
mental Analysis of Behavior, 1963, 6, 449-456. Hall, Inc., 1966, 499-530.
Azrin, N. H., Hutchinson, R. R., 8c Hake, D. F. Pain-in¬
Hoffman, H. S., Fleshler, M., 8c Chorny, H. Discrim¬
duced fighting in the squirrel monkey. Journal of the inated bar-press avoidance. Journal of the Experimental
Experimental Analysis of Behavior, 1963, 6, 620. Analysis of Behavior, 1961, 4, 309-316.
Azrin, N. H., Hutchinson, R. R., & Hake, D. F. Attack
Hutchinson, R. R. Effects of post-shock attack opportunity
avoidance and escape reactions to aversive shock. upon pre-shock manipulative responding. Paper pre¬
Journal of the Experimental Analysis of Behavior, 1967 sented at Psychonomic Society, 1970.
10, 131-148.
Hutchinson, R. R., Azrin, N. H., 8c Hake, D. F. An auto¬
Azrin, N. H., Hutchinson, R. R., 8c McLaughlin, R. The matic method for the study of aggression in squirrel
opportunity for aggression as an operant reinforcer dur¬ monkeys. Journal of the Experimental Analysis of Be¬
ing aversive stimulation. Journal of the Experimental havior, 1966, 9, 233-237.
Analysis of Behavior, 1965, 8, 171-180. Hutchinson, R. R., Azrin, N. H., 8c Renfrew, J. W. Effects
Azrin, N. H., Hutchinson, R. R., 8c Sallery, R. D. Pain-
of shock intensity and duration on frequency of biting
aggression toward inanimate objects. Journal of the attack by squirrel monkeys. Journal of the Experimental
Experimental Analysis of Behavior, 1964, 7, 223-228. Analysis of Behavior, 1968, 11, 83-88.
R. R. Hutchinson 431

Hutchinson, R. R., 8c Emley, G. S. Schedule-independent Fear as motivation and fear-reduction as reinforcement

factors contributing to schedule-induced phenomena. In in the learning of new responses. Journal of Experi¬
R. M. Gilbert, 8c J. D. Keehn (Eds.), Schedule Effects: mental Psychology, 1948, 38, 89-101.
Drugs, Drinking, and Aggression. Toronto: University Morse, W. H., 8c Kelleher, R. T. Schedules as fundamental
of Toronto Press, 1972, 174-202. determinants of behavior. In W. N. Schoenfeld (Ed.),
Hutchinson, R. R., 8c Emley, G. S. Effects of nicotine on The theory of reinforcement schedules. Englewood Cliffs,
avoidance, conditioned suppression and aggression re¬ N.J.: Prentice-Hall, Inc., 1970, 139-185.
sponse measures in animals and man. In W. L. Dunn Morse, W. H., Mead, R. N., 8c Kelleher, R. T. Modulation
(Ed.), Smoking Behavior: Motives and Incentives. Wash¬ of elicited behavior by a fixed-interval schedule of elec¬
ington, D.C.: V. H. Winston, 1973, 171-196. tric shock presentation. Science, 1967, 157, 215-217.
Hutchinson, R. R., 8c Pierce, G. E. Jaw clenching in Mowrer, O. H., 8c Lamoreaux, R. R, Avoidance condition¬
humans: Its measurement and effects produced by con¬ ing and signal duration—a study of secondary motivation
ditions of reinforcement and extinction. Presented at and reward. Psychological Monographs, 1942, 54, 5, 247.
American Psychological Association, 1971. Pierce, G. E. Effects of several fixed-ratio schedules of rein¬
Hutchinson, R. R., Renfrew, J. W., 8c Young, G. A. forcement and of extinction upon temporalis and mas-
Effects of long-term shock and associated stimuli on seter muscle contractions in humans. Masters thesis.
aggressive and manual responses. Journal of the Experi¬ Western Michigan University, 197L
mental Analysis of Behavior, 1971, 15, 141-166. PRONI, T. J. The effect of changes in response-independent
Kelleher, R. T., 8c Morse, W. H. Schedules using noxious pay upon human masseter EMG. Masters thesis, Western
stimuli. III. Responding maintained with response-pro¬ Michigan University, 1973.
duced electric shocks. Journal of the Experimental Sidman, M. Some properties of the warning stimulus in
Analysts of Behavior, 1966, 77, 819-836. avoidance behavior. Journal of Comparative and Physio¬
Kelleher, R. Tv Riddle, W. C., 8c Cook, L. Persistent be¬ logical Psychology, 1955, 48, 444—450-
havior maintained by unavoidable shocks. Journal of SlDMAN, M. Normal sources of pathological behavior. Sci¬
the Experimental Analysis of Behavior, 1963, 6, 507-517. ence, 1960, 132, 61-68.
Liddell, H. S. The conditioned reflex. In F. A. Moss (Ed.), Sidman, M. Some notes on “bursts” in free-operant avoid¬
Comparative Psychology. Englewood Gliffs, N.J.: Pren¬ ance experiments. Journal of the Experimental Analysis
tice-Hall, Inc., 1934. of Behavior, 1958, 13 167-172.
McKearney, J. W. Maintenance of responding under a Stretch, R., Orloff, E. R,, & Balrymvle, §, B, Mainte¬
fixed-interval schedule of electric shock presentation. nance of responding by fixed-interval schedule of electric
Science, 1968, 160, 1249-1251. shock presentatian in squirrel monkeys. Seieme, 1968,
McKearney, J. W. Fixed-interval schedules of electric 162, 583-586.
shock presentation. Extinction and recovery of perfor¬ Ulrich. K. E. Interaction between reflexive fighting and
mance under different shock intensities and fixed-interval cooperative escape. Journal of the Experimental Analysis
durations. Journal of the Experimental Analysis of Be¬ of Behaviori 1967, 10> 311-317.
havior, 1969,12, 301-313. Ulrich, R, R„ & Azams, N. H. Reflexive fighting in re¬
McKearney, J. W. Responding under fixed-ratio and sponse to ayerslys stimulation. Journal of the Experi¬
multiple fixed-interval fixed-ratio schedules of electric mental Analysis of Behavior, 196£*; 5, 511—5^0.
shock presentation. Journal of the Experimental Anal¬ Ulrich, R. E., Hutchinson, R. R., 8c Azrin, N. H. Pain-
ysis of Behavior, 1970, 14, 1-6. elicited aggregsisn. Psyehologieal Record, 1965, 15, 111—
Miller, N. E. Studies of fear as an acquirable drive: I. 126.
 15

Stimulus Control
and Inhibitory Processes*

Mark Rilling

OVERVIEW
Discrimination training is one of the most powerful
determinants of stimulus control. The fourth section
A reinforcer never occurs in a vacuum, isolated defines the various types of discrimination training
from outside influences. Environmental events, the and gives particular attention to the measurement
stimuli of stimulus control, are present before, during, and interpretation of inhibitory phenomena. Spence’s
and after the occurrence of the reinforcer. The rapid theory of discrimination learning is reexamined in the
solution of major substantive problems in the area of context of recent evidence on the impact of discrimi¬
stimulus control is due in large measure to the ele¬ nation training on stimuli varying within a single
gance of techniques for reliably assessing the control dimension on the stimulus-generalization gradient. In
of behavior by these environmental events. These addition, the newer dynamic models of stimulus con¬
techniques are considered in the following section. trol are considered as they apply to inhibitory phe¬
The third section considers the influence of a num¬ nomena.
ber of specific factors on the characteristics of the The basic assumption in the fifth section is that
empirically obtained stimulus-generalization gradient. the determinants of inhibitory phenomena are inde¬
Discussed are the effects of the schedule of reinforce¬ pendent of whether the discriminative stimuli are
ment during training and a microanalysis of the selected from a single or two independent dimensions.
generalization gradient in terms of interresponse times. This section examines the effect on inhibitory phe¬
nomena of amount of training, sequence of the dis¬
* Preparation of this chapter and the author’s research was criminative stimuli, and schedule of reinforcement.
supported in part by NIMH grant No. 5 R01 MH 18342. I thank The joint effect of the response rate and incentive
the many graduate and undergraduate students at Michigan
State University for their comments on an earlier version of
differences between the two components of the multi¬
this chapter. I am especially indebted to Vern Honig, Tom ple schedule on the shape of the generalization gradi¬
Kodera, John Staddon, Herb Terrace, Dave Thomas, Stan Weiss, ent is also considered.
and many other colleagues for their constructive comments and
guidance in our pursuit of a clearer understanding of stimulus
The phenomenon of errorless learning has played
control. a major role in theories of stimulus control. It refers

432
Mark Rilling 433

to discriminations in which the rate of responding ment is a class of training procedures in which
during (S-) is negligible from the first session of dis¬ responses are unequally reinforced. When differential
crimination training. Errorless learning was consid¬ reinforcement depends upon the stimuli in the en¬
ered an exception to the basic laws of discrimination vironment, as in most experiments in stimulus con¬
learning in that S~ was apparently neutral, rather trol, the procedure is called discrimination training.
than acquiring inhibitory properties during errorless In discrimination training, certain stimuli predict
discrimination training. In the sixth section data are occasions when a class of responses is reinforced and
presented which indicate that basic inhibitory phe¬ other stimuli predict occasions when those responses
nomena are in fact obtained following errorless learn¬ are not reinforced or when they are reinforced ac¬
ing. cording to a different schedule. Stimuli that are cor¬
related with periods of reinforcement are often desig¬
nated positive stimuli (S+), while those stimuli which
THE DEFINITION AND MEASUREMENT are correlated with periods of extinction are desig¬
OF STIMULUS CONTROL nated negative stimuli (£—),

The Vocabulary of Stimulus Control The Stimulus-generalization

Gradient

Stimulus Control
At the completion of discrimination training, dif¬
Stimulus control is observed when a change in a ferent rates of responding are typically associated with
particular property of a stimulus produces a change in each stimulus. However, it i§ not apparent how the
some response characteristic, as in the rate or prob¬ organism will respond to test stimuli that have not
ability with which a response occurs. For example, the been previously presented. It is not clear to what
onset of a light is said to control behavior if respond¬ properties of the stimulus the organism is responding,
ing occurs at a higher (or lower) rate in the presence To answer these questions, the stimulus may be
of the light than in its absence. varied along its various dimensions. For example, in
The rationale for introducing the new term stim¬ a discrimination between red and green, dimensions
ulus control stems from the semantic confusion which such as the size of the stimulus, its luminance, and its
Brown (1965) noted as existing between the terms wavelength may have acquired control over respond¬
discrimination and generalization. To illustrate this, ing. The dimension of generalization is the con¬
suppose different rates of responding to a red stimulus tinuum along which a particular property of a stim
and to a green stimulus are established. Then the ulus is varied during a test for stimulus control.
data can be described as indicating a discrimination Physical dimensions such as the wavelength of a light,
between red and green—or, with the logically equiva¬ the frequency or intensity of a tone, or line orienta¬
lent statement, as indicating a failure to generalize tion are usually selected. A stimulus-generalization
between red and green. Therefore, theoretical at¬ gradient is the function obtained when the total num¬
tempts to explain generalization as a failure to dis¬ ber of responses to each of the stimulus values pre¬
criminate, or discrimination as a failure to generalize, sented during the generalization test are plotted
may involve the fallacy of using different words to against the dimension of generalization.
describe the same behavioral process. This problem is
avoided when discrimination and generalization are
Technlcjues for Obto Ining «
defined as opposite ends of the single continuum of
Stimulus-generalization Gradient
stimulus control.
Nondifferential and differential reinforcement are A stimulus-generalization gradient is employed to
two training procedures which are frequently em¬ determine the properties of the stimuli that have
ployed in experiments on stimulus control. In non¬ acquired control over responding. Responding may
differential reinforcement, a response is equally rein¬ decrease, increase, or remain unchanged during a
forced in the presence of all the stimuli in the stimulus-generalization test. A variety of procedures
environment, so that the consequences of responding have been developed for obtaining stimulus-general¬
remain identical independent of stimulus change. ization gradients. No single procedure is appropriate
This procedure is typically employed to obtain base for all experimental problems, and the shape of the
line levels of responding to stimuli that subsequently gradient depends upon the procedure employed. The
are differentially reinforced. Differential reinforce¬ advantages and disadvantages of each procedure are
434 STIMULUS CONTROL AND INHIBITORY PROCESSES

detailed in the sections that follow. Note that in each than the gradient obtained from the first test. This
of these procedures, specific training is usually given illustrates a major disadvantage of the extinction
with respect to only one or two stimuli on a dimen¬ methods: the measurement of generalization is con¬
sion prior to the stimulus-generalization test. Conse¬ taminated by the effects of extinction. Optimal assess¬
quently, most of the stimuli presented during the ment of stimulus generalization requires comparison
generalization test are novel. with a behavioral base line which is both stable and
recoverable. Unfortunately, responding during extinc¬
Transient or Extinction Methods tion lacks both of these characteristics. The rate of
responding decreases, eventually to zero, and repeated
In the single-stimulus method, a response is rein¬ exposures to extinction reduce the number of re¬
forced in the presence of one stimulus. In a subse¬ sponses obtained.
quent test, extinction to a single test stimulus occurs. The advantage of the multiple-stimulus method is
A separate, independent group is required for each that a stimulus-generalization gradient is obtained
data point on the stimulus-generalization gradient, from a single organism and averaging of data from
which is obtained by averaging the total number of different organisms is not required.
responses during extinction for each animal in the
group. While single-stimulus tests can be used with
Maintained Generalization Methods
operant methods, and have indeed been studied by
Hiss and Thomas (1963), the advantages of presenting Procedures in which a constant rate of responding
all test values to each individual subject are so great is maintained by reinforcement are generally pre¬
that this method has been used almost exclusively. ferred to transient or transition procedures in which
Skinner developed the multiple-stimulus method the rate of responding is changing, as described above.
for assessing generalization which was first reported Several investigators (D. Plough, 1969, 1975; P. Blough,
in 1950. In a report prepared in 1944 but not pub¬ 1972; Malott, Malott 8c Glenn, 1973; Pierrel, 1958)
lished until 1965, Skinner described a precursor of have employed maintained generalization procedures.
the most common method for obtaining generaliza¬ The session is divided into trials of generally short
tion gradients with operant methods. The classic duration—e.g., 20 sec. On training trials responding is
study reported by Guttman and Kalish (1956) in¬ reinforced intermittently in order to maintain a base
corporated many aspects of Skinner's procedure and line rate of responding. On test trials, responding is
determined the direction of subsequent research in never reinforced, and a generalization gradient is ob¬
stimulus generalization. tained during each session by presenting the test
Guttman and Kalish selected the visual spectrum stimuli in random order. As long as the animal fails
as the stimulus dimension to exploit the excellent to discriminate test from training trials, responding
color vision of pigeons. During trammg7 the response occurs during the test trials even though reinforce¬
key was illuminated with a monochromatic light ment never occurs. As D. Blough (1969) has shown,
source. Responses on the key were reinforced with the technique is extremely powerful, since hundreds
food on a variable-interval (VI) 1-min schedule of of generalization gradients can be obtained from the
reinforcement. After a substantial rate of responding same animal over a period of many months. A typical
was established, generalization testing was carried out finding with the maintained procedure (P. Blough,
during extinction in a session which began with 1972) is that the stimulus-generalization gradient
several reinforcements to the training stimulus. Dur¬ around the training stimulus gradually becomes
ing the generalization test, 11 different-colored sharper within the sensory limits of the organism.
stimuli, including the training stimulus, were ran¬ This method is very useful when closely spaced
domly presented on the key each for 60 sec. Each stimuli are used in the test.
stimulus was repeated 12 times within the test in an A disadvantage of maintained procedures is that
attempt to average out the differences due to the slow several weeks of pretraining are required to establish
decrease in the response rate produced by extinction. a constant rate of responding on the base line sched¬
After the first generalization test, the birds were re¬ ule of reinforcement before a stimulus-generalization
trained with reinforcement and a second generaliza¬ gradient can be obtained. Another disadvantage is
tion test identical to the first was administered. that when the test stimuli are spaced far apart, a dis¬
When Guttman and Kalish administered this crimination between training and test stimuli is ac¬
second generalization test, the generalization gradient quired and responding to the test stimuli rapidly falls
from the second test showed fewer total responses to zero. In this latter case, the extinction technique
Mark Rilling 435

may be more appropriate. Whether the maintained tion data. However, some important experimental
procedures will eventually replace the transient pro¬ questions require a comparison of different conditions
cedures of obtaining the gradient during extinction and cannot be answered with absolute gradients. Sup¬
remains to be seen. pose that an experimenter wants to compare the
amount of generalization produced by two conditions
Simultaneous or Concurrent which produce an extreme difference in the total
Methods number of responses during extinction. For instance,
two different schedules of reinforcement may produce
The methods just described are appropriate for
drastic differences in the rate of responding during
assessing stimulus control in successive discrimina¬
extinction for two different groups of subjects. A rela¬
tions. Stimulus generalization is also measured in
tive stimulus-generalization gradient can be used to
simultaneous discriminations. In a simultaneous dis¬
compare conditions in such cases, and also when indi¬
crimination two stimuli are presented to the organism
vidual absolute gradients differ substantially in their
at the same time; S+, correlated with reinforcement, mean rates.
and S , correlated with extinction. When a discrete-
In a relative gradient, the number of responses to
trial procedure is employed, each response to S+ is
each test stimulus is expressed as a percentage of the
reinforced and each response to S~ (or error) produces
total responses to all stimuli. Relative gradients are
a time-out during which the onset of the next trial is
also sometimes plotted As a proportion of responses
delayed. When responses at two different locations are
made to the training nmudus, When relative grad¬
intermittently reinforced in the presence of two differ-
ients are averaged, equal weight is given to each
ent stimuli, the procedure is called a concurrent
gradient. In constructing a relative gradient, the
schedule of reinforcement. During a concurrent gen-
experimenter assumes that a given absolute decrement
erdUzatioH test, a variety of test stimuli are presented
is psychologically greater against a base line of low
at two locations during extinction, and the number
responding to the training value than against a high
of responses to each stimulus which occur at each
base line. Comparisons can be made between the
location are recorded. As employed by D. Blough
slopes of relative gradients since the gradients have
(1978) and Catania, Silverman, and Stubbs (1974), this
been equated the differences in the number of re¬
procedure produces two stimulus-generalization grad¬
sponses obtained during the generalization test. How»
ients, one for each stimulus location.
ever, as Morgan (1969) points out, conclusions about
A second procedure, described by Honig, Beale,
slopes and differences are safest when absolute as well
Seraganian, Lander, and Muir (1972), employs a con¬
as relative gradients intersect.
current schedule with an explicit changeover or ad¬
vance response. Two discriminative stimuli, S+ and
S~, are presented alternately at one location so that Control for Stimulus Preferences
only one stimulus is present at a time. A second re¬
Suppose an experimentally naive pigeon is placed
sponse, at a different location, terminates the current
in an experimental chamber which contains a key
stimulus and produces the next stimulus in a predict¬
which can be illuminated with various stimuli. The
able series. The advantage of this procedure is that it
bird's responses on the key are not reinforced, but
expands the range of dependent variables to include
various stimuli are projected on the key and the
the time as well as the number of responses in the
number of responses to each stimulus are recorded.
presence of each stimulus (or class of stimuli). This
Most investigators assume that such a procedure will
procedure has been employed by Honig et al. and
produce a flat gradient with few or no responses to
Beale and Winton (1970) to measure generalization of
each stimulus and conclude that no stimulus prefer¬
the pigeon’s response of terminating a stimulus as¬
ences are present. This conclusion is probably incor¬
sociated with extinction.
rect. Some species exhibit marked preferences for
certain stimuli which are determined by hereditary
The Analysis of Data from a Generalization Test: and developmental rather than by reinforcement vari¬
Absolute vs. Relative Gradients ables. If these effects are not considered, the results of
a stimulus-generalization test may be misinterpreted.
A generalization gradient based on the total num¬ For example, by measuring the unconditioned peck¬
ber of responses obtained during extinction is called ing behavior of newly hatched gull chicks which were
an absolute generalization gradient. An absolute grad¬ presented with various monochromatic stimuli. Hail-
ient is the simplest method of presenting generaliza¬ man (1969) obtained a preference function resembling
436 STIMULUS CONTROL AND INHIBITORY PROCESSES

a stimulus-generalization gradient. Several reviewers—

e.g., Hinde & Hinde (1973); Seligman & Hager (1972);
Shettleworth (1972)—have stressed that the experi¬
menter must consider the constraints which the
organism’s heredity, anatomy, and development im¬
pose upon the behavior of an organism in a learning
experiment.

SOME DETERMINANTS OF
GENERALIZATION GRADIENTS

Nondifferential Reinforcement

A VI schedule of positive reinforcement is often

employed to sustain a moderate rate of responding
prior to the generalization test in each of the tech¬
niques for measuring stimulus generalization. In these
techniques the schedule of reinforcement is usually
held constant at VI 1-min, at least when pigeons serve
as subjects. However, by manipulating the schedule
of reinforcement in effect prior to the generalization
test, Hearst and his colleagues discovered that the
schedule of reinforcement is one of the most potent Fig. 1. Gradients of relative generalization for five groups of
determinants of the slope of a stimulus-generalization pigeons in which each group received training on a different
value of a VI schedule prior to the generalization test. The S+
gradient. Different schedules of reinforcement pro¬
was a vertical line (0°) for all subjects. In general, the gradient
duce widely divergent absolute gradients because they becomes flatter as the value of the VI schedule increases. (From
differ in their resistance to extinction. Therefore, rela¬ Hearst, Koresko, & Poppen, 1964. (c) 1964 by the Society for the
Experimental Analysis of Behavior, Inc.)
tive gradients are employed to compare the effects of
schedule of reinforcement on the slope of the general¬
ization gradient. Thomas and Switalski (1966) compared stimulus
Hearst, Roresko, and Poppen (1964) found that generalization following variable-ratio (VR) and VI
relative gradients obtained after differential reinforce¬ training. In order to equate the two schedules for the
ment of low rate (DRL) training were much flatter frequency and pattern of reinforcement, pairs of
than gradients obtained after VI training. On a DRL pigeons were matched through a yoking procedure.
schedule an interresponse time greater than t sec pro¬ The time required by a pigeon on VR training to
duces reinforcement, while an interresponse time less complete each ratio determined the interval at which
than t sec is extinguished. In a second experiment, its yoked pigeon on VI training was reinforced. Thus
Hearst, Koresko, and Poppen (1964) trained each when one pigeon’s response was reinforced on the VR
group of animals on a different value of a VI sched¬ schedule, the next response of the VI bird was also
ule. The longer the mean value of a VI schedule, the reinforced. The VR schedule generated a higher rate
lower the response rate and frequency of reinforce¬ of responding than the VI schedule, but the gradient
ment. Generalization was measured during extinction for the VR group was slightly flatter than the VI
with the multiple-stimulus method. Figure 1 shows gradient.
that a VI 4-min schedule produces a rather flat relative What is the explanation for these results? A simple
gradient, indicating that more generalization is ob¬ explanation is that each response is determined to
served with long VI schedules than with the short VI some extent by previous responses (factor A) and to
1-min schedule which is usually employed in general¬ some extent by external stimuli (factor B), where
ization experiments. These data demonstrate that the A + B = l (i.e., complete determination). When fac¬
slope of a stimulus-generalization gradient can be tor A is important, as on DRL and (perhaps) ratio
drastically altered by manipulating the temporal dis¬ schedules, then factor B is correspondingly less so;
tribution of food deliveries while holding stimulus hence the flatter gradients. While this hypothesis is an
variables constant. attractive device for integrating data on the effects of
Mark Rilling 437

schedules of reinforcement on stimulus generalization, responses to a 582-nm stimulus, and the data for gen¬
the explanation remains post hoc until a method is eralization were obtained with a maintained pro¬
developed for measuring the extent to which a re¬ cedure by randomly presenting a series of adjacent
sponse is determined by previous responses. wavelengths during extinction.
Three stimulus-generalization gradients were ob¬
tained by dividing the number of responses into four
Microstructure of the Stimulus-generalization IRT categories or class intervals: .6 to 1.0 sec, 1.0 to
Gradient 2.0 sec, 2.0 to 4.0 sec, and greater than 4.0 sec. Few
responses occurred between 0 and .6 sec because the
This subsection provides a review of the summa¬
key was darkened during this period to provide stim¬
tion or averaging procedures that are employed on
ulus feedback for each response. The dependent vari¬
the data obtained during a stimulus-generalization
able, IRTs/OP was the conditional probability that a
test. The basic question is whether all responses are
response fell within one of the four IRT categories.
equivalent. The question has been divided into three
Figure 2 dearly indicates that the stimulus on the
subsidiary questions which will be considered in turn:
key acquires control over the pecking response only
(1) Does the shape of the gradient depend upon the
within an IRT range of 2.0-4.0 sec. The Rgure shows
amount of time which has preceded the response? (2)
a fairly flat gradient and complete generalisation for
Does the shape of the gradient change during the test
IRTs within the .G-hQ-sec and 1.0-2.0-sec categories.
session? (3) Is the gradient an artifact of inappro¬
In other words* the stimulus on the key does not con¬
priate averaging of responses of different topog¬
trol the rate of responding when an animal responds
raphies?
with IRTs less than 2.0 sec.
Similar data have been obtained with the IcYer-
IRT Analysis of the
pressing response of rats by Crites. Harris, Rosenquist,
Stimulus-generalization Gradient
and Thomas (1967). However, an experiment by
An interresponse time (IRT) analysis of the rate of White (1973) has restricted the generality of this
responding is a useful technique for determining the phenomenon. Stimulus control over the responding
essential characteristics of stimulus control. One of of pigeons was acquired by responses in all IRT class
D. Blough's (1969) experiments neatly illustrates the intervals, including those of less than 2»0 sec. White’s
contribution of responses within various IRT cate¬ procedure differed from Blough s in a number of sig¬
gories to the shape of the stimulus-generalization nificant respects. The generalization test in White’s
gradient. Pigeons were intermittently reinforced for experiment was preceded by differential reinforce*

Fig. 2. Maintained gradients

obtained following reinforce¬
ment of r^pynding at D62 nm*
allowing the probability of a

response as a function of wave¬

length and inteirefip6ftg£ tim^

(IRT). The numbers next to
each function indicate the IRT
class intervals in sec. The over¬
all level of the curves varies
with the number of responses
in the class interval. The sig¬
nificant aspect is that the gradi¬
ents with the class interval be¬
tween 2 and 4 sec is steeper
than the flat gradients obtained
between .6 and 1 sec. (From D.
Blough, 1969. © 1969 by the
Society for the Experimental
WAVELENGTH (nm)
Analysis of Behavior, Inc.)
438 STIMULUS CONTROL AND INHIBITORY PROCESSES

ment, while Blough employed nondifferential rein¬ tion is based on an IRT analysis. When the rate of
forcement. Additional research is necessary to specify responding is high, short IRTs predominate, while
the conditions under which stimulus control is ac¬ long IRTs emerge when the rate of responding is low.
quired or is not acquired over responses following Since the rate of responding slows during extinction,
various IRTS. The data base of this research should IRTs become longer. Blough’s data demonstrated
be broadened by employing responses other than that responses preceded by long IRTs may acquire
pecking. For example, Hemmes (1973) has shown that more stimulus control than do responses preceded by
pigeons acquire a discrimination between a red or a short IRTs. Since long IRTs predominate toward the
white houselight located above the ceiling of the end of a stimulus-generalization test carried out
experimental chamber when responses on a foot during extinction, the gradient should become
treadle located on the floor of the chamber are rein¬ steeper.
forced with food. It would be interesting to know if
The Stimulus-generalization
stimulus control over the treadle response depends
Gradient: Fact or Artifact?
upon the duration of the preceding IRT.
Blough’s experiments, like those discussed in the Several investigators (Migler, 1964; Migler 8c Mil-
preceding section, imply that the sensitivity of a re¬ lenson, 1969; Ray 8c Sidman, 1970; Stoddard 8c Sid-
sponse to stimulus variation depends upon the man, 1967, 1971) view the stimulus-generalization
amount of time which has elapsed since the preceding gradient as a continuous function, consisting of vary¬
response. ing proportions of discrete elements. The proportions
can vary continuously, but the elements (IRTs or
Steepening of the
whatever) are discrete or “quantal.” Consider the re¬
Stimulus-generalization
duced number of responses to an intermediate test
Gradient During Extinction
stimulus. The same number of responses could be
In the multiple-stimulus method, each stimulus is produced by a constant, intermediate rate of respond¬
presented several times in extinction during the gen¬ ing or by averaging brief periods of responding at
eralization test. In general, fewer responses are ob¬ the previously reinforced rate with long periods con¬
tained with each successive presentation of the same taining few or no responses. These investigators em¬
stimulus. Several experiments (Friedman 8c Guttman, ployed simultaneous methods for assessing stimulus
1965; Thomas k Barker, 1964) have demonstrated a control following discrimination training in which
steepening of the relative stimulus-generalization responses at more than one location were reinforced.
gradients during extinction. Friedman and Guttman In general, the test stimuli controlled the relative fre¬
analyzed the changes in the generalization gradient quency of the two responses that were reinforced dur¬
which occurred during testing by dividing the total ing training so that a mixture of the two responses
number of responses during extinction into successive was obtained at intermediate test stimuli. Mixing of
quarters. A relative generalization gradient was con¬ different responses is likely to occur during stimulus
structed for each of the four quarters. The gradients generalization when two incompatible responses have
became steeper as extinction progressed because the been reinforced during simultaneous discrimination
rate of responding dropped to zero more rapidly for training prior to the generalization test, so simultane¬
the stimuli which were remote from the training stim¬ ous methods are well suited to the measurement of
ulus while responding to the training stimulus de¬ competing responses during generalization.
creased less rapidly. Thus a long generalization test Collins (1974) obtained IRT distributions of the
in which each stimulus is presented many times is pecking response of pigeons while generalization was
biased toward a steep gradient, while a short general¬ measured with the multiple-stimulus method during
ization test with few stimulus presentations is biased extinction. Following single-stimulus training in
toward a flat gradient. While the magnitude of the which responding to a 554-nm stimulus was reinforced
bias is small, caution is required in interpreting differ¬ on a VI schedule, the number of responses in the
ences in the slopes of the gradients from different longer IRT class intervals (>6 sec) increased system¬
experiments because the differences could be due to atically with divergence from S+, while the frequency
comparing generalization test sessions of different of responses with short IRTs decreased. Following
lengths. successive discrimination training between two stim¬
Why does the generalization gradient become uli, the IRT distribution for an intermediate test
steeper as extinction progresses? A plausible explana- stimulus was a mixture, in varying proportions, of the
Mark Rilling 439

response patterns conditioned in the presence of S+ Two Types of Discrimination Training

and S~.
Using rats as subjects, Weiss (1972b) measured IRT The term discrimination training is very broad,
distributions in the presence of a light associated with since there are many different procedures which share
a DRL schedule and a tone associated with a VR the defining characteristic of an S+ which is corre¬
schedule. The absence of the light or tone was asso¬ lated with reinforcement and an S~ which is cor¬
ciated with extinction. After a low rate of responding related with extinction. Switalsky, Lyons, and Thomas
was established on the DRL schedule and a high rate (1966) have developed useful terminology for classify¬
of responding was established on the VR schedule, the ing the types of discrimination training on the basis
tone and light were presented simultaneously as a of the relationship between the discriminative stimuli
compound stimulus during extinction. An IRT anal¬ (S+ and S—) and the stimulus dimension on which the
ysis of responding during the compound stimulus re¬ generalization gradient is obtained.
vealed few patterns of responding during the com¬ The first type is intradimcnsional training* which
pound stimulus that were not present during the occurs when S+ and S~ are selected from the same
individual presentations of the light and the tone. stimulus dimension and a generalization test is carried
All of these experiments, which employed a wide out within that dimension. A common example of
variety of methods for assessing stimulus control, are intradimemioiial training is a successive discrimina¬
in agreement that the presentation of an intermediate tion in which responding is reinforced in the presence
test stimulus following discrimination training be¬ of one wavelength, S+, and extinguished in the
tween two stimuli does not produce a constant* inter¬ presence of another wavelength, 5-. Other dimen¬
mediate rate of responding. A significant component sions frequently used are the frequency of an acoustic
of the original behaviors which were conditioned dur¬ Stimulus and orientation of a line.
ing training remains during generalization testing. An important characteristic of intradimensional
Therefore, as Weiss (1972b) points out* the stimulus- training is that it is impossible to vary the psycholog¬
generalization gradient is probably a product of the ical distance of a test stimulus from S+ without also
mixing of a small number of response classes. The varying its psychological distance from S~. Therefore,
task of the microanalysis of stimulus control is to intradimensional training is employed when the ex¬
determine and isolate the variables responsible for the perimenter wants to study the interaction between
mixture of responses which result in the stimulus- reinforcement at S+ and extinction at S on respond¬
generalization gradient, ing to each stimulus.
A microanalysis of the generalization gradient in d he second type of discrimination training is inter-
terms of interresponse times or competing responses is dimemional training which occurs when S+ is equally
compatible with research whose goal is to determine distant psychologically from each of the stimuli on the
the effects of various variables upon the shape of the S- dimension or when S~ is equally distant from each
stimulus-generalization gradient. It is to this body of of the stimuli on the Si- dimension. When two dimen¬
research that attention is now directed. sions are psychologically independent) each stimulus
from the S+ dimension is equally distant psycholog*
ically from each of the stimulus on the dimension.
INFLUENCE OF DISCRIMINATION
Interdimensional training is employed when the ex¬
TRAINING ON THE GENERALIZATION perimenter wants to compare responding to stimuli
GRADIENT similar to S“ with responding to stimuli similar to
S+, under conditions where the two kinds of respond¬
One of the main problems in the discrimination ing are assumed to be independent. Two separate
learning of animals is to specify the conditions under generalization gradients for the S+ and S~ dimensions
which a change in a stimulus produces a change in may be obtained with interdimensional training,
the probability with which a response occurs. A gen¬ thereby avoiding the interaction obtained with intra¬
eral finding (see the reviews by Thomas, 1969, 1970) dimensional training. This rationale for interdimen¬
is that nondifferential reinforcement produces a flat¬ sional training was stated by Jenkins (1965) and is
ter stimulus-generalization gradient than does differ¬ described in greater detail in monographs by Hearst,
ential reinforcement. Discrimination training is one Besley, and Farthing (1970) and Hearst (1972).
of the most effective procedures for increasing the Silence, the absence of S+ or S-, is a stimulus at
slope of the stimulus-generalization gradient. the end point of the intensity (loudness) dimension
440 STIMULUS CONTROL AND INHIBITORY PROCESSES

which is often employed in interdimensional training. ence for certain stimuli, the experimenter may con¬
For example, suppose that silence is the stimulus that sider this response bias in selecting the most appro¬
is employed as S~ and a tone of 1,000 Hz is employed priate stimuli for interdimensional training. Thus, for
as S+. If intensity and frequency are independent example, the experimenter may deliberately design
dimensions, then the frequency of the tone may be the experiment to counteract the anticipated bias in
varied during a stimulus generalization test without order to make such an outcome more convincing.
changing the psychological distance between the test
values from the S+ dimension and S~.
Comparing Nondifferential with
The stimuli for the early experiments in inter¬
Differential Reinforcement
dimensional discrimination training were a priori as¬
sumed to be psychologically independent dimensions. The effects of nondifferential and differential train¬
However, psychological independence is an empirical ing procedures on stimulus generalization are illus¬
concept, and a preliminary experiment is required to trated by a widely cited experiment of Jenkins and
demonstrate the stimuli that appear independent to Harrison (1960). The data from a subsequent experi¬
the experimenter are also functionally independent, ment of Jenkins and Harrison (1962) extend these
as indicated by the behavior of the organism. A test results comparing the effects of interdimensional and
for psychological independence is conducted as fol¬ intradimensional training on the slope of the stim¬
lows. First a response is reinforced in the presence of ulus-generalization gradient. In their first experiment,
a stimulus from the T-dimension without the presen¬ one group of pigeons was given nondifferential train¬
tation of a stimulus from the B-dimension. Then a ing in which a 1,000-Hz tone signaled that a VI sched¬
stimulus-generalization gradient is obtained by pre¬ ule of reinforcement wag in effect. No training stim¬
senting test stimuli from the 13-dimension. If a hori¬ ulus explicitly correlated with extinction was intro¬
zontal gradient is obtained, the stimulus from the A- duced until the generalization test. A second group
dimension is independent of the ^-dimension. The was given interdimensional training in which S+ was
converse experiment could be carried out by first a 1,000-Hz tone and $- was silence. The tone and
training with stimulus B and then testing on the A~ silence were randomly presented go that the animals
dimension to determine if stimulus B is independent learned to respond in the presence of the tone and not
of the ^-dimension* to respond in the absence of the tone.
Giurintano, Schadler, and Thomas (1972) condi¬ The individual gradients for a representative bird
tioned the pecking response of different groups of from each group which received each type of training
pigeons in the presence of stimuli that are a prion are shown in Figure 3, During generalization, the test
independent of the dimension of line orientation; a stimuli were tones widely separated in frequency from
white light, a green light, and a white dot. Then each the training stimulus, as well as silence. The relative
group was given a stimulus-generalization test in gradient following nondifferential training demon¬
which the angle of a white line on a dark background strated weak stimulus control, since the gradient was
was varied, A preexperimental preference for a par¬ relatively flat with a maximum at S + .
ticular orientation was not obtained. Training with In contrast with nondifferential training, inter¬
the white or dim light produced a preference for dimensional training produced a steeper gradient
vertical, while training with the dot produced a pref¬ with a dear maximum at 1,000 Hz and over 20% of
erence for 30°. Only the green stimulus resulted in no the total responses occurring to the training stimulus
preferred orientation and therefore was functionally for each animal. Following interdimensional training,
orthogonal to the angle of the line. Using a similar the frequency of the tone during the test system¬
procedure, Selekman (1973) conditioned pecking in atically controlled the rate of pecking. Why does the
the presence of a white key and obtained a nonhori¬ frequency of the tone acquire control over responding
zontal gradient on the wavelength dimension. The when the only source of discrimination training is be¬
pigeons demonstrated a preference for short wave¬ tween the presence and absence of the tone? Clearly,
lengths between 510 and 560 nm. These data demon¬ differential reinforcement is more effective than non¬
strate that an experiment is necessary to determine if differential reinforcement in activating stimulus con¬
the two dimensions employed in interdimensional trol, but the question has no adequate answer.
training are functionally independent. Complete func¬ In a second experiment, Jenkins and Harrison
tional independence is probably an ideal state, and it (1962) obtained much steeper generalization gradients
is unlikely that any two dimensions are completely when several of the birds were given additional intra¬
independent. By determining an organism’s prefer¬ dimensional training between two closely spaced tones
Mark Rilling 441

sities of the subjects in these experiments. Therefore,

the effect of nondifferential reinforcement on stimulus
control depends upon the stimulus dimension, rein¬
forcer, and species of the subject. (See Mackintosh,
chapter 16 in this volume.)
Using pigeons and visual stimuli, Switalski, Lyons,
and Thomas (1966) and Lyons and Thomas (1967)
compared the effects of interdimensional nondifferen¬
tial reinforcement and interdimensional differential
reinforcement on the slope of the stimulus-generaliza¬
tion gradient. In the Lyons and Thomas study, the
stimuli were a white vertical line on a black back¬
ground and a 555-nm light. When responding to both
stimuli was equally reinforced, the gradients for wave¬
length were flattened. Following interdimensional
training between 555 nm (S+) and the vertical line
(S~)7 the gradients for wavelength were steepened.
Fig. 3. Representative individual generalization gradients of
total frequency obtained from pigeons following three training Interdimensional training does not guarantee stim¬
conditions. The open circles show a relatively flat gradient ulus control over the behavior of the organism by the
obtained after nondifferential reinforcement in the presence dimension selected by the experimenter, but it is more
of a 1,000-Hz tone. The closed circles show that a steeper
gradient was obtained following interdimensional training in effective than nondifferential reinforcement. When
which 5H was a 1,000-Hz tone and was no tone. The open compared with interdimensional training, intradimen¬
triangles show that the steepest gradient was obtained following
sional training between stimuli which are closely
intradimensional training in which $+ was a 1,000'Hz tone and
S" a 950-Hz tone. (Adapted from Jenkins & Harrison, 1960. (c) spaced on the stimulus dimension further sharpens
1900 by the American Psychological Association, and reprinted the stimulus-generalization gradient.
by permission. And Jenkins & Harrison, 1902. © 1902 by the
Society for the Experimental Analysis of Behavior, Inc.)
Effects of Intradimensional Training

rather than between a tone and the absence of the

The Positive Pear Shift
tone as in their original experiment. Notice in Figure
3 that following intradimensional training between an The classic peak shift experiment was performed
S+ of 1,000 Hz and an S~ of 950 Hz, the maximum by Hanson (1959). Four groups of pigeons were given
did not occur at S+, but occurred instead at 1,050 Hz. intradimensional training on the wavelength dimen¬
This phenomenon is called the peak shift and is the sion with the same S4-, 550 nm. The groups differed
general result of intradimensional training. In the only with respect to the employed: 555, 560, 570,
peak shift, the peak or mode of the generalization and 590 nm. The experimental groups were trained
gradient occurs at a test stimulus which is displaced on the discrimination until the response rate during
from S+ in a direction away from S~. The significance S~ reached zero. The number of sessions required to
of this important phenomenon is discussed later. reach this discrimination criterion increased as the
In an earlier review of the Jenkins and Harrison difference between S+ and S~ decreased. Following
experiment, Terrace (1966c) concluded that “differ¬ discrimination training, each animal was given a gen¬
ential reinforcement was necessary to establish stim¬ eralization test with stimuli which ranged from 480 to
ulus control” (p. 281). However, this conclusion was 620 nm.
not correct since Jenkins and Harrison obtained weak The results for these groups plus a control group
stimulus control following nondifferential reinforce¬ that received nondifferential reinforcement with S +
ment. In addition, Thomas and Setzer (1972) showed only are presented in Figure 4. The peak of the
reliable auditory frequency generalization gradients generalization gradient for the four experimental
in both rats and guinea pigs following nondifferential groups did not occur at S+, 550 nm. Rather, the peak
reinforcement. The finding that auditory stimuli ac¬ occurred at 540 nm, a stimulus that the birds had
quire good stimulus control during nondifferential never seen until the generalization test. This is the
reinforcement with food for rats and guinea pigs and positive peak shift, which is usually called simply the
poor control for pigeons may reflect a difference be¬ peak shift. Extrapolation of the gradient obtained
tween either the ontogeny or the inherited propen¬ with S~ at 555 nm suggests that the peak would have
442 STIMULUS CONTROL AND INHIBITORY PROCESSES

tion test and produces a higher rate of responding to

the stimuli in the vicinity of S+. When behavioral
contrast is assessed with a control group, the control
group should receive the same amount of exposure to
S+ as the experimental group.
Discrimination training reduced the number of re¬
sponses in the vicinity of S~~ as compared with the
control group. This produced asymmetrical gradients
with more area on the side of S+ that was away from
S-. The area shift is an index of a gradient in which
more than 50% of the area of the gradient lies on the
side of S+ away from S~. The area shift is based on
the assumption that the theoretical excitatory gradi¬
ent around S+ is symmetrical. The area shift is more
sensitive to the effects of intradimensional training
than is the peak shift, since some animals which do
not show a peak shift may still show an area shift
(Terrace, 1966c, p. 328). However, the peak shift is
Fig. 4. Effects of intradimensional discrimination training on
the stimulus-generalization gradient for different groups of superior to the area shift as a dependent variable.
pigeons. For all groups, S+ was 550 nm. Four groups received Area shifts must be interpreted with caution, since an
discrimination training with S~ at 555, 560, 570, or 590 nm,
respectively, as indicated by the vertical arrows. A control group
asymmetric gradient may reflect chance variability,
(solid curve) received nondifferential reinforcement at 550 nm stimulus preferences, or lack of excitation in the
and showed a maximum at S+. The groups given discrimina¬ vicinity of S~. In Hanson’s experiment the area shift,
tion training showed a positive peak shift with the maximum
displaced from S+ to 540 nm. (From Hanson, 1959. (c) 1959 by
the percentage of responses below 550 nm increased as
the American Psychological Association. Reprinted by permis¬ the difference between S+ and S~ decreased.
sion.) One of the most convincing demonstrations of the
peak shift was a study by Thomas and Williams
occurred at 535 nm if that test stimulus had been (1963) which employed an S~ located between two S+
presented, and extrapolation of the gradient with S~ at stimuli. The S~ was 560 nm which alternated suc¬
590 nm suggests a peak shift at 545 nm. Hanson’s data cessively with an S+ of 540 and 580 nm. The postdis¬
suggest that the magnitude of the peak shift depends crimination gradient showed a double peak shift with
upon the difference between S+ and S~. Most subse¬ the two maximums located at 530 and 590 nm. This
quent experiments (see the review by Pur tie, 1973, p. study clearly demonstrates that the peak of the gradi¬
410) agree that the closer the spacing between S+ and ent shifts away from S-.
S~, the greater the probability of obtaining a peak A comprehensive review of the literature on the
shift. Successful research on the determinants of the peak shift by Purtle (1973) indicates that the phe¬
peak shift requires the inclusion of several test stim¬ nomenon has been obtained following intradimen¬
uli which are spaced closely to S+. sional discrimination training on a variety of dimen¬
Figure 4 also shows that the experimental groups sions with several species of organisms, including
emitted more responses in the vicinity of S+ than did humans. Notwithstanding the generality of the phe¬
the nondifferential control group. This difference nomenon, research on the determinants of the peak
may simply reflect the fact that the control group shift is confronted with the problem of individual
received only five days of nondifferential reinforce¬ differences. Published articles are filled with restric¬
ment followed by the generalization test, while the tive statements such as “only one of the four subjects
experimental groups received up to 25 additional days displayed the peak shift” or “three of the five birds
of discrimination training prior to the generalization produced the peak shift,” etc. When the stimuli are
test. Alternatively, the increased output in the vicinity spaced closely along the stimulus dimension, more
of S+ may reflect the occurrence of behavioral con¬ responses may by chance occur to a test stimulus
trast. When behavioral contrast occurs during dis¬ other than S+ producing an artifactual peak shift.
crimination training, the rate of responding to S+ is The optimal parameters for producing the peak shift
elevated relative to a base line of nondifferential rein¬ should be determined. One problem is that the mea¬
forcement prior to the stimulus-generalization test. surement of the peak shift during extinction with the
The higher rate to S+ carries over to the generaliza¬ Guttman and Kalish technique may increase the vari-
Mark Rilling 443

ability of the data. The use of a maintained pro¬ Similarly, the maximum did not occur at S+, test
cedure may increase the reliability of the phenomenon. stimulus 9, but was displaced beyond S+ away from
S- to test stimulus 8.
The Negative Peak Shift Stevenson (1966) has also obtained a negative peak
shift with a method which involves testing over only
A negative peak shift occurs when the minimum of
the half of the stimulus dimension centered around
the generalization gradient occurs at a test stimulus
S —. When stimuli in the vicinity of S+ were not pre¬
which is displaced from S~ in a direction away from
sented, the rate of responding in the vicinity of S-
S+. Careful examination of Figure 4 shows that sev¬
was much higher than in Hanson’s experiment. The
eral of the experimental groups showed a negative
advantage of Stevenson’s procedure is that a negative
peak shift. However, reliable measurement of the
peak shift can be obtained after the birds meet a
negative peak shift was difficult in Hanson’s experi¬
strict criterion of no responding to S~ without previ¬
ment due to the low response output for test stimuli
ously reinforcing responses to any of the test stimuli
in the vicinity of S-.
except S + . Stevenson’s experiment demonstrates that
Guttman (1965) solved the problem of the zero rate
the presentation of S+ during the generalization test
of responding in the vicinity of S- by nondifferen-
is not necessary for the negative peak shift.
tially reinforcing responses to a range of spectral
Since most of the research has concentrated on the
stimuli that were subsequently employed in the gen¬
positive peak shift, the negative peak shift has been a
eralization test. This was followed by intradimensional
neglected experimental phenomenon. More research
discrimination training between S+ and S~ to a
is needed to determine the correlation between these
criterion less strict than Hanson’s. Then a generaliza¬
two phenomena of intradimensional discrimination
tion test was administered during extinction, The
training. The simplest assumption is that the determi¬
results of this experiment, presented in Figure 5, show
nants of the positive and negative peak shifts are
both a positive and a negative peak shift. The data in
identical.
Figure 5 represent an average for six animals given
intradimensional discrimination training on the wave¬
Additive and Suppressive Summation
length dimension. The minimum did not occur at
S~, test stimulus 11, but was displaced beyond S+ In the experiments on the positive and negative
away from S~ to test stimulus 14. This a negative peak shifts, the continuum was defined by a dimen¬
peak shift of 15 nm on the wavelength dimension. sion of a stimulus. Weiss (1972a) defines a stimulus
set in terms of the on and olf states of twq discrimina¬
tive stimuli; a tone (T) and a light (L). The set ex¬
tends from the all-off extreme (T 4- L) through the
one-stimulus on conditions (T + L) or (T + L) to the
all-on extreme (T + L). The preceding peak shift ex¬
periments employed two-component multiple sched¬
ules in which each component was associated with the
on state of a stimulus. Weiss argues that consideration
of the rate of responding and conditions of reinforce¬
ment in the off as well as the on states is essential for
a complete understanding of a stimulus control.
Therefore, Weiss employs three-component multiple
schedules in which the third component is associated
with the all-off extreme of the ordered stimulus set.
Typically rats are intermittently reinforced for
pressing a lever in these experiments. Consider a
Fig. 5. A stimulus-generalization gradient illustrating both a
positive and a negative peak shift. Responding to the test three-component multiple schedule in which respond¬
stimuli was nondifferentially reinforced. Then the rate of re¬ ing is reinforced in the presence of a tone (T + L)
sponding to S" was reduced by intradimensional discrimination
and a light (T + L), while responses are extinguished
training. Finally generalization was measured by presenting the
test stimuli during extinction. The maximum did not occur in the absence of the tone and light (T + L). Re¬
at S+, test stimulus 9, but was displaced to test stimulus 8, a sponding is maintained at approximately equal rates
positive peak shift. The minimum did not occur at S_, test
stimulus 11, but was displaced to test stimulus 14, a negative in the tone and the light. After discrimination train¬
peak shift. (From Guttman, 1965.) ing, stimulus control is assessed in a compounding
444
STIMULUS CONTROL AND INHIBITORY PROCESSES

test in which the tone and the light are presented stimuli might lead to additive summation under
separately and, in addition, the tone-plus-light com¬ one set of circumstances, suppressive summation
pound stimulus is presented for the first time. The under another or even averaging in a third
typical outcome of this type of experiment is additive complex schedule context. . . . [This means]
summation, in which more responses are obtained to that absolute properties cannot be attributed to
the compound stimulus than to either of the single the schedule-associated behaviors without due re¬
training stimuli alone. gard to the total schedule context of which they
are a part. (p. 206, italics in original)
Weiss (1972a, p. 194) explained additive summa¬
tion as follows. When T + L is associated with extinc¬
tion, the response which is conditioned is response Effects of Interdimensional Training
cessation (R^. When responding is reinforced in the
presence of T + L, the behavior which is conditioned
Excitation, Inhibition, and
consists of a mixture of the response controlled by the Dimensional Control
tone (R2) and response cessation (Rx) which is con¬
trolled by the absence of the light. Similarly, the Hearst, Besley, and Farthing (1970) define exci¬
(T -f L) stimulus controls a mixture of behavior con¬ tatory and inhibitory stimuli as follows:
trolled by the light (R3) and response cessation (Rx)
controlled by the absence of the tone. During presen¬ An “excitatory stimulus” is a stimulus . . . that
tation of T + L, additive summation occurs be¬ develops during conditioning . . . the capacity
cause responding consists of a mixture of responses to increase response strength above the level
controlled by the light (Rs) and responses controlled occurring when that stimulus is absent. An “in¬
hibitory stimulus” is a stimulus that develops
by the tone (R3), The mixture of R2 and R2 produces
during conditioning the capacity to decrease re¬
a higher rate of responding than the mixture of either
sponse strength below the level occurring when
Rs or R3 with Rr According to this formulation the
that stimulus is absent, (p. 376)
habits conditioned in T + L are influencing the be¬
havioral control in I 4- L and L -{- T through L and
Consider a hypothetical experiment in which in¬
T, respectively. Therefore, according to Weiss, the terdimensional training is employed with S+ cor¬
response rate and reinforcement frequency deter¬ related with reinforcement and S~ correlated with
mined by the T + L contingency sets the condition¬ extinction. After discrimination training, a test is re¬
ing context within which control is acquired by quired to determine if S+ has acquired the capacity
T + L and L -{- T. to increase responding, and a separate test is required
Now consider a second experiment in which re¬ to determine if S~ has acquired the capacity to de¬
sponding is reinforced in the absence of the light and crease responding. As the following quotation illus¬
tone and responding is reinforced at a lower rate or trates, Hearst et al. distinguish these tests from
punished in the presence of a tone and light. The procedures for obtaining a stimulus-generalization
typical outcome of this experiment is called suppres¬ gradient.
sive summation, in which fewer responses are ob¬
tained to the compound stimulus than to either train¬ The term “excitatory dimensional control,” in
ing stimulus alone. Weiss also explained suppressive our view, would be applied when new stimulus
summation in terms of a mixture of responses con¬ values that lie at progressively greater distances
trolled by each stimulus element. along a specific dimension from an excitatory
Additive summation is analogous to the positive stimulus show a graded decremental effect. The
peak shift, since in each case the maximum rate of term “inhibitory dimensional control” would be
applied when new stimulus values at progres¬
responding is controlled by a stimulus removed from
sively greater distances from an inhibitory stimu¬
S+ in a direction away from S~. Similarly, suppressive
lus show a graded incremental effect on the
summation is analogous to the negative peak shift.
strength of an operant response. It is important
Weiss (1971) noted that these phenomena are func¬ to point out that an incremental gradient
tionally similar in the sense that each has the same around some stimulus value is necessary but not
determinants. Weiss (1972a) summarizes the effects of sufficient for defining inhibitory dimensional
conditioning context as follows: control. The specific stimulus at which respond¬
ing is minimal must also be shown to be inhibi¬
tory by some independent test, since it is logi¬
Compounding the same schedule-associated cally possible that such a stimulus is relatively
Mark Rilling 4 45

“neutral” and the other values progressively sured by presenting the line at various angles with the
more excitatory, (p. 377) Guttman and Kalish technique.
If the effects of extinction generalize, S~ should
Stimulus-generalization gradients are classified as show the fewest responses and the number of re¬
excitatory or inhibitory on the basis of whether re¬ sponses to a test stimulus should increase with increas¬
sponding decreases or increases with increasing dis¬ ing distance from S~. For the groups with the line
tance from the training stimulus. An excitatory or correlated with S~, Figure 6 shows that an inhibitory
decremental stimulus-generalization gradient has a gradient was obtained around S~. With S+ correlated
maximum at or near S+. The number of responses to with reinforcement, an excitatory or decremental
a test stimulus decreases with increasing distance gradient was obtained. This experiment is one of the
from S + . An inhibitory or incremental stimulus- clearest demonstrations of parallel generalization for
generalization gradient has a minimum at or near extinction and reinforcement.
Sf% The number of responses to a test stimulus in¬ The major advantage of interdimensional training
creases with the distance from S-. Excitatory gradi¬ for the measurement of inhibitory dimensional con¬
ents measure generalization of reinforcement while trol is that the stimuli can he varied along the S~
inhibitory gradients measure generalization of extinc¬ dimension without interacting with S+. Excitatory
tion. gradients around S+ and inhibitory gradients around
S“ can be compared with this procedure. Notice in
Figure 6 that more responses were obtained during
The Measurement of Inhibitory Stimulus Control
the generalization test when the orientation of the
This section describes four techniques that have line was employed as the 5+ dimension. The major
been developed for the measurement of inhibitory weakness of the extinction procedure is that when
stimulus control following interdimensional discrimi¬ very few responses are obtained during the generaliza¬
nation training. As Rescorla (1969a, 1969b) and Hearst tion test along the dimension, a “floor effect’1
(1972) have pointed out, a variety of different experi¬ makes detection of inhibition extremely difficult.
mental operations are used to define a conditioned
inhibitor. Furthermore, the measurement of condi¬
tioned inhibition is more difficult to measure than is
conditioned excitation, thus necessitating special con¬
trol procedures to avoid confounding conditioned
inhibition with other behavioral processes. Parallel¬
ing the research on conditioned inhibition, a variety
of methods have emerged for the measurement of in¬
hibitory stimulus control. These include (1) resistance
to extinction, (2) resistance to reinforcement, (3) com¬
bined cue or summation tests, and (4) stimulus re¬
duction.

Resistance to Extinction

Three groups of investigators (Honig, Boneau, Bur-

stein, Sc Pennypacker, 1963; Jenkins 8c Harrison, 1962;
Schwartzbaum 8c Kellicut, 1962) independently devel¬
oped interdimensional procedures to measure gen¬
eralization of extinction independently of generaliza¬
tion of reinforcement. For example, Honig et al. ran
Degrees of tilt
two similar studies in which interdimensional dis¬
crimination training was given to one group of Fig. 6. The functions with triangles demonstrate excitatory or
pigeons with S+ as a homogeneous white key and S~ decremental gradients of stimulus control and were obtained
following interdimensional training between a vertical line (S+)
as a black vertical line bisecting the white key. For and no line (S-). The functions with circles demonstrate in¬
the other group, S+ was a vertical line on the key and hibitory or incremental gradients of stimulus control and were
obtained following interdimensional training between a vertical
S- was the white key. After differential responding
line (S~) and no line (S+). (From Honig, Boneau, Burstein, &
was well established, stimulus generalization was mea¬ Pennypacker, 1963.)
446 STIMULUS CONTROL AND INHIBITORY PROCESSES

Resistance to Reinforcement
RESISTANCE TO REINFORCEMENT
WITHOUT COMPOUNDING TEST
Hearst, Besley, and Farthing (1970) adapted the
1 O _ 2 _ 3 4
method of retardation of the development of a condi¬ o o o E 5 °
tioned response from Rescorla’s (1969a) analysis of wL-
1 t : 1 1 11 1 11 1 1
inhibition in classical conditioning. This procedure
6 7 _ 8 9
substantially increased the reliability of the inhibitory -v-i/'* §
I ° o ’T
gradient. Rather than measuring the resistance to ‘ Q
extinction on the S- dimension, as in the extinction
UJ r- II r '3 „ _ 14 « r 15 “
f- 1 ° : 12 °
methods, the resistance to reinforcement on the S~ D -\ir
Z
dimension is obtained. First, behavior is extinguished
1 1 111111 1 11111 i 11 11111111
to S~ during interdimensional discrimination train¬
ing. Then, during the generalization test, responses to
1 2 3 4 5
all of the test stimuli are reinforced on a VI schedule.
The procedure is repeated for several successive daily rN'-o
o
iiii iiii 1 I t 1 i 1 1 o1 i iiiiiii 111 111 11 11 1111l1
sessions. The procedure assumes that the resistance of _ 7 9
r 8. r 10 S
a response to the effects of reinforcement, in the 55
: ° o
presence of a former S“, provides an index of the in¬ cc
CD
hibitory properties of that stimulus. For example, an II 12 _ 14 15
13
animal will acquire a response to a novel stimulus 0
more rapidly than to a stimulus to which responding 1 1 111111 LJLLULUJ I111i11I
-90 O S+ -90 0 S+ -90 0 SV ~9Q 0 St “SO 0 St
has been previously extinguished. The advantage of
DEGREES FROM S-
the re^i^tance-tQ-reinforcvment method is that it elim¬
inates the problem of the zero base line which occurs Fig. 7. Rate of responding to each test stimulus during each of
when no responses are obtained during the generali¬ the 15 days of generalization testing with the resistance to rein¬
forcement procedure for birds 4589 and 5199. (From Rilling,
zation test. A limitation the procedure that equal Caplan, Howard, & Brown, 1975. © 1975 by the Society for the
rates of reinforcement are required in the presence of Experimental Analysis of Behavior, Inc.)
each test stimulus. Otherwise, differences in response
rate during testing could be attributed to differential
of measuring inhibition, especially when a classical
reinforcement,
conditioning paradigm is employed. In this method,
Rilling, Caplan, Howard, and Brown (1975) dem¬
S+ is presented simultaneously with test stimuli from
onstrated the effectiveness of the resistance-to-rem-
the S~ dimension. The assumption is that if the test
forcement procedure in elevating responding to the
stimulus has inhibitory properties, its presentation
S~ dimension following errorless learning in which
should produce a decrement in the presence of a
the rate of responding to S- was essentially zero
Stimulus associated with reinforced responding. In a
throughout discrimination training. The results of this
study by Lyons (1969), using a combined-cues tech¬
procedure for 15 successive days of generalization test¬
nique, S+ was a monochromatic light of 550 nm and
ing are illustrated in Figure 7. For both birds, the
S- was a white vertical line on a black background.
slope of the inhibitory gradient remained essentially
During a stimulus-generalization test, the angle of the
unchanged during many sessions. While individual
line was varied, but each test stimulus from the S~~
variability was observed in the shape of the gradients
dimension was superimposed upon S+. Surprisingly,
during the early sessions, the inhibitory gradients
excitatory generalization gradients were obtained
showed no tendency to invert and became excitatory
with the peaks at S_. In a similar experiment using a
with extended training as reported by Hearst, Besley,
combined-cues test in which lines at various angles
and Farthing (1970). Therefore, the resistance-to-
were superimposed upon S+, Davis (1971) obtained
reinforcement procedure reliably measures inhibitory
evidence for the inhibitory property of the line, since
stimulus control.
all such compound stimuli produced lower rates of
responding than S+ alone. However, for some birds
Combined Cues or Summation
maximum responding occurred when S— was com¬
Combined-cue tests are also designed to elevate bined with S + . These data are anomalous, since S-
response output to the S~ dimension during the gen¬ was an inhibitory stimulus, while a decremental
eralization test. Many investigators—e.g., Rescorla gradient was obtained around S~.
(1969a)—regard summation as the most direct method However, Drexler and Terrace’s unpublished data
Mark Rilling 447

suggest that Lyons’s and Davis’s failure to obtain in¬ The problem here is that no criterion is employed for
hibitory stimulus control may have been due to a ranking procedures on a scale of “directness.” The
high rate of responding to S- prior to the stimulus- combined-cues test is probably employed more fre¬
generalization test. These studies indicate that ade¬ quently because of its historical precedent in Pavlov’s
quate interpretation of stimulus-generalization gradi¬ work. An empirical criterion should be employed in
ents requires data showing the rates of responding to selecting a procedure for measuring inhibitory stim¬
S+ and S~ during the acquisition of the discrimina¬ ulus control. The most sensitive procedure is usu¬
tion prior to the generalization test. ally the best choice. Hearst’s (1972) point that ex¬
At present, the combined-cues method is the least perimenters should employ a variety of methods and
satisfactory procedure for measuring inhibitory di¬ compare the results of different procedures is worthy
mensional control. Superimposing S+ upon S~ may of further emphasis. It seems likely that the results of
not produce the desired increase in responding in the such research will demonstrate that the different mea¬
presence of S- during the generalization test, as sures of inhibitory stimulus control are not always
Yarczower (1970) noted. The presence of S+ may highly correlated.
overshadow stimuli from the S~ dimension and pro¬

duce a flat gradient. Furthermore, the measurement

Spencers Theory of Discrimination Learning
of inhibition is dependent upon the strength of rein¬
forced responding, so that the detection of inhibition
is more likely when the reinforced behavior is weak Assumptions and
and susceptible to disruption than when the rein¬ Qualitative Predictions
forced behavior is strong and resistant to disruption.
Spence’s (1937) analysis of what is now called in¬
Consistent with this view, Yarczower and Evans (1974)
tradimensional learning has become a classic. Al¬
found that an increase in the amount of training was
though this theory evolved before the development of
accompanied by a reduction in the amount of ex¬
operant techniques, it provides the best explanation
ternal inhibition to a novel stimulus.
for the effects of intradimCttsioilal training en the
postdiscrimination gradient. The theory includes the
Stimulus-reduction or
following five assumptions:
Advance Procedure

The fourth technique for measuring inhibitory 1. Reinforcement of responding to a stimulus (S+)
stimulus control is stimulus reduction. Several in¬ produces an excitatory tendency to respond to St.
vestigators (Rilling, Askew, Ahlskog, 8c Kramer, 1969; 2. Excitation generalizes around S-K
Rilling, Kramer, 8c Richards, 1973; Terrace, 1971) 3. Extinction of responding to a stimulus (S~) pro¬
have demonstrated that pigeons acquire a response duces an inhibitory tendency opposite to the ten¬
that terminates and thereby reduces the duration of dency associated with S + .
the stimulus associated with extinction. These data 4. Inhibition generalizes around S~.
indicate that a stimulus associated with the absence 5. The predicted response to any tost stimulus is
of reinforcement may become a conditioned aversive obtained by subtracting the amount of inhibition
stimulus. As Honig, Beale, Soraganian, Lauda*, and to the stimulus from the amount o£ saltation to
the stimulus.
Muir (1972) point out, none of the current definitions
of inhibition deals with the possibility that it may be
Spence developed his theory to account for trans¬
defined by a reduction in the duration of S_. Rather,
position, which is observed in a simultaneous discrim¬
the definitions concentrate upon a reduced rate of re¬
ination when the subject prefers to S+ a novel stim¬
sponding in the presence of a stimulus as the criterion
ulus which is displaced from St in a direction away
for inhibition. Unfortunately, this had led most in¬
from S~. Riley (1968) has provided a thorough review
vestigators to neglect duration as a parameter of an
of the theories and research on the transposition prob¬
inhibitory stimulus.
lem. However, the transposition experiments did not
provide a crucial test of Spence’s theory, since the
Selecting a Method for Assessing
hypothetical gradients of excitation and inhibition
Inhibitory Stimulus Control
were never directly measured.
Rescorla (1969a) and Hearst (1972) argue that the Spence’s theory is easily extended to successive dis¬
“most direct” method is the best, and they lean to¬ crimination experiments. Following intradimensional
ward the combined-cues test as the method of choice. training, the postdiscrimination gradient is the re-
 448
STIMULUS CONTROL AND INHIBITORY PROCESSES

sultant of the interaction between excitation and in¬

hibition. Therefore, the number of responses emitted
to each test stimulus during a generalization test is
obtained by subtracting the amount of generalized

RESPONSES
inhibition to S” from the amount of generalized ex¬
citation to S + .
Spence’s algebraic summation or gradient-interac¬
tion theory produces the following major predictions
about the shape of the postdiscrimination gradient
following intradimensional discrimination training:

1. The maximum or peak of the generalization gradi¬

ent occurs at a test stimulus which is displaced from
S4- m a direction away from S . This is the posh
the peak shift.
T The minimum the generalization gradient occurs
at a test stimulus which is displaced from 5^ in a
direction away from ST This is the negative peak
shift.
%. Th^ magnitude of the peak shift increases as the
difference between S+ and S^ is reduced. The
peak shift is not obtained with a large difference
between Si" and S~, since there is no overlap or WAVELENGTH (nm)
interaction between the excitatory and inhibitory
Fig. 8. Post hoc prediction of the postdiscrimmation gradient
gradients.
using Spence s algebraic summation theory. The empirical post-
4. The peak shift does not occur if the inhibitory discrimination gradient was obtained by Hanson (1959) with S+
gradient is flat or horizontal. Subtracting a constant at 550 and S- at 579 nm The empirical gradient is obtained
by subtracting the hypothetical inhibitory gradient from the
from the excitatory gradient yields a predicted
hypothetical excitatory gradient. The hypothetical functions
gradient with the peak at ST were selected to yield the positive peak shift, in which the peak
5, Rate of responding to 5+ is reduced by discrimina¬ of the empirical gradient was displaced from 550 nm to 540
nm and the negative peak shift in which the minimum was
tion training relative to tbe single-stimulus base
displaced from 570 to 580 nm. (Original figure prepared bv
line. Therefore, the number of responses to each Marty Klein,)
stimulus in the postdiscrimination gradient should
be less than the number of responses in the excita¬
tory gradient obtained following single-stimulus quantitative prediction of the positive and negative
training. peak shifts? T o answer this consider Figure 8, which
presents one of Hanson’s (1959) empirical postdis¬
Mathematical Constraints on crimination gradients from Figure 4 of this chapter,
Spence’s Theory which shows both positive and negative peak shifts.
Hypothetical gradients of excitation and inhibition
When Spence (1987) first proposed his theory, he
which sum algebraically to yield Hanson’s data were
was forced to speculate about the theoretical shape of
devised post hoc. These are also presented in Figure 8.
the excitatory and inhibitory gradients. This was a
The hypothetical gradient of excitation has a maxi¬
weakness which he readily acknowledged. His ap¬
mum at ST and the hypothetical gradient of inhibi¬
proach was essentially intuitive, illustrated with
tion has a minimum at S~.
graphs of convex hypothetical generalization curves
On the empirical postdiscrimination gradient, max¬
which have been reproduced in most learning texts.
imal responding occurred to SI, a stimulus displaced
Critics have pointed out that not all excitatory and
from S+ in a direction away from S~. Minimal re¬
inhibitory functions generate the predictions de¬
sponding occurred to S2, a stimulus displaced from S~
scribed above. For example, Hull (1948) noted that
in a direction away from ST It can be proven mathe¬
exponential or concave gradients fail to predict the
matically1 that in order to obtain a positive peak
peak shift. In addition, Hebert and Krantz (1965) and
shift from S+ to SI, the slope of the inhibitory gra¬
D. Blough (1969) have shown that linear or tent¬
dient between S+ and SI must be steeper than the
shaped gradients also fail to predict peak shifts.
slope of the excitatory gradient between S+ and SI.
What relationship between the excitatory and in¬
hibitory gradients is necessary and sufficient for a
11 thank Marty Klein for developing this proof.
Mark Rilling 449

Similarly, a necessary and sufficient condition for the interdimensional training techniques has removed this
negative peak shift from S~ to S2 is that the slope of ambiguity, since the shape of the excitatory and in¬
the excitatory gradient between S~ and S2 be steeper hibitory gradients can be empirically determined. The
than the slope of the inhibitory gradient between S~ methodology for the prediction of the peak shift from
and S2. gradients of excitation and inhibition has been devel¬
The next problem is the selection of a family of oped by Hearst (1968, 1969b).
curves which best fit the excitatory and inhibitory The basic design involves three groups which are
gradients obtained after interdimensional training. given successive discrimination training. The pre-
Since theorists (e.g., Rescorla, 1969a; Spence, 1936, excitation group is given interdimensional training
1937) define excitation and inhibition as opposite be¬ with an S+ selected from the dimension of generaliza¬
havioral processes, excitatory and inhibitory gradients tion and an orthogonal S ~ . Discrimination training is
from the same family are most appropriate. Gaussian followed by a generalization test which produces an
discriminal distributions, of which the bell-shaped empirical excitatory gradient. The preinhibition
normal distribution curve is the most familiar group is given interdimensional training with an S“
example, are the best mathematical model for describ¬ selected from the dimension of generalization and an
ing the shape of the excitatory and inhibitory gra¬ orthogonal S"K Tbe generalization test produces an
dients obtained after interdimensional training. (See empirical inhibitory gradient, The third group is
Blough, 1969, and Nunaily, 1967, for further discus¬ given intradimensional training with the Si~ of the
sion of these distributions.) Blough (1967, 1969) ob¬ preexcitation group and the S~ of the preinhibition
tained bell-shaped gradients by presenting several test group. A generalization gradient is algo obtained after
Stimuli which were spaced closely to the training intradimensional training,
stimulus. Sharp peaks in published gradients may just Hearst’s analysis does not assume that the gradient
reflect a lack of data points in the vicinity of S+. obtained after intradimensional training has the same
In order to predict the positive and negative peak form ag those obtained after interdimensional train¬
shifts simultaneously, and still have some positive ing. In fact, intradimensional gradient arc oitrcn
values in the postdiscrimmation gradient, either the asymmetric, while interdimensional gradients are
entire excitatory gradient must be further from the often symmetrical. The analysis simply predicts the
abcissa than the inhibitory gradient or the inhibitory postdigcrimmation gradient for the intradimensional
gradient must be a flatter bell than the excitatory group by subtracting the inhibitory gradient from the
gradient. Empirical evidence (Hearn, 1968, 1969b; excitatory gradient.
Honig et al., 1963; Jenkins & Harrison, 1962) suggests One problem is that the absolute gradients vary
that inhibitory gradients are indeed flatter than greatly m the mean total number of responses to the
excitatory gradients. Jenkins (1963, pp. 58-39) implies test stimuli due to the occurrence of behavioral contrast
that flatter inhibitory gradients must be the case when during the acquisition of the discrimination. When
stimulus control along the S- dimension is measured the groups were equated in Hearsds study (1968) by
by the number of responses (such as key pecking) converting the absolute gradients to relative gradients,
which are reinforced during S+. There are two sub¬ a reasonably good ht was obtained between a pre¬
classes of responses, other than key pecking, which dicted and the obtained postdiscrimination gradient
must be considered: (1) incompatible responses, such by subtracting the relative inhibitory gradient from
as turning away from the key, which are presumably the relative excitatory gradient. Unfortunately, the
conditioned during S , and (2) all other responses. In analysis was weakened by Hearst’g failure to obtain
plotting the inhibitory gradient, only a decrease in the a peak shift for the group that received intradimen¬
subclass of incompatible responses which appears as sional training.
an increase in the response that is reinforced in S+ Hearst (1968, 1969b) employed the dimension of
contributes to the slope of the inhibitory gradient. An the angle of the line in his experiments. Using the
increase in all other responses lowers the overall level design developed by Hearst, Marsh (1972) employed
of responding and flattens the inhibitory gradient. the wavelength dimension to see if the subtraction of
an empirical inhibitory gradient from an excitatory
Quantitative Tests of gradient produced a predicted gradient including the
Spence’s Theory peak shift which corresponded to the gradient ob¬
tained after intradimensional training. Marsh’s pre¬
Appetitive base lines. The hypothetical nature of dicted postdiscrimination gradient displayed a peak
the excitatory and inhibitory gradients is a major shift and showed a rough correspondence to the actual
weakness of Spence’s theory. The development of postdiscrimination gradient. Thus the data derived
450 STIMULUS CONTROL AND INHIBITORY PROCESSES

from an appetitive base line provide some support for ulus control in a conditioned suppression paradigm.
the gradient-interaction theory proposed by Spence. The relative gradients for the three groups are pre¬
sented in Figure 9. The left panel shows the gradients
Aversive base lines. In contrast with the extensive obtained with shock, and the right panel shows the
literature on the influence of environmental stimuli gradients obtained without shock. For Group I, the
on behavior maintained by schedules of positive rein¬ excitatory group, in which S+ was 1,000 Hz and S~
forcement, very little is known about the influence of was noise, the peak of the gradients occurred at S+.
environmental stimuli on behavior maintained by For Group II, the inhibitory group, in which S+ was
schedules of negative reinforcement, although Sidman noise and S~ was 1,500 Hz, an inhibitory gradient
(1966, p. 494) demonstrated stimulus control of free with a minimum at S~ was obtained. For Group III,
operant avoidance behavior. The earlier experiments
in the stimulus control of behavior maintained by
aversive stimuli were analyzed by Hearst (1969a) in a
pioneering paper which concluded that similar laws
of generalization applied to positive and negative
reinforcement. Hearst found that the parameters of
the base line rather than the type of behavioral base
line determined the shape of the stimulus-generaliza¬
tion gradient.
Basing their experiment upon Hearst’s design,
Klein and Rilling (1972) investigated the prediction
of gradients following intradimensional training with
an aversive base line, Pigeons were trained to press a
treadle on a shock-postponement schedule in which
brief 4-mA shocks followed one another at 5-sec inter¬
vals (S-S interval) unless the treadle was pressed.
After a treadle press, the next shock occurred after 20
sec (R-S interval). After avoidance responding stabil¬
ized, auditory discriminative stimuli were introduced.
The positive stimulus (S+) was associated with the
avoidance schedule, while the negative stimulus (S~)
was associated with extinction of avoidance without
shocks. For the excitatory group, S+ was a 1,000-Hz
tone and S~ was noise; for the inhibitory group, S+
was noise and S™ was a 1,500-Hz tone; and for the
intradimensional group, S+ was a 1,000-Hz tone and
S“ was a 1,500-Hz tone. After reaching a criterion for
the acquisition of the discrimination, each group was
given an identical stimulus-generalization test along
the frequency dimension.
Two types of generalization tests were employed:
resistance to extinction, in which there were no sched¬
uled shocks; and resistance to (negative) reinforce¬
Fig. 9. The upper panel presents the relative excitatory gradi¬
ment, in which one avoidable shock occurred 5 sec ents obtained for Group I. The center panel presents the relative
after the beginning of each test tone presentation if inhibitory gradients obtained for Group II. The lower panel
the animal failed to respond during the first 5 sec. presents the relative postdiscrimination gradients. The gradients
on the left are the results of tests conducted with avoidable
The resistance-to-negative-reinforcement procedure shocks. In these gradients, the solid-lined gradients (WSE)
was developed as an analog to the procedure devel¬ include shock-elicited responses, while the broken-lined gradi¬
oped by Hearst et al., for positive reinforcement, in ents (W/OSE) exclude shock-elicited responses. The gradients
on the right are from tests conducted without scheduled shocks.
order to elevate the rate of responding along the S~ Tones are spaced at approximately equal intervals along a log
dimension. This technique is also similar to Hoff¬ scale. Note that the ordinate scale for the Group I no-shock
man’s (1966, pp. 516-517) use of noncontingent shocks gradient (upper right) differs from the other ordinate scales.
(From Klein and Rilling, 1974. @ 1974 by the Society for the
between stimulus presentations which increased stim¬ Experimental Analysis of Behavior, Inc.)
Mark Rilling 451

the intradimensional condition, in which S+ was shock probably provide a more accurate picture of the
1,000 Hz and S” was 1,500 Hz, an asymmetric gradient generalization process.
with a maximum at S+ and a minimum at S~ was In general, the results of the experiments of Hearst,
obtained. Only one of the four birds in the intra¬ Marsh, and Klein and Rilling support Spence’s gra¬
dimensional group showed a peak shift. When pecking dient-interaction theory and suggest that the deter¬
was reinforced with food, Jenkins and Harrison (1962) minants of generalization on appetitive and aversive
obtained a peak shift with an S+ of 1,000 Hz and an base lines are similar.
S~ of 950 Hz. Possibly the difference between S+ and
S~ in the Klein and Rilling experiment may have
Dynamic Models of Stimulus Control
been too large to obtain a peak shift.
The left panel of Figure 10 illustrates how the While Spence’s theory integrates much of the data
inhibitory gradient was subtracted from the excitatory on stimulus control, it is inadequate in accounting for
gradient to yield a derived postdiscrimination gra¬ a growing number of phenomena, In some cases, re¬
dient (PDG). The right panel of Figure 10 shows that sponding is found to be enhanced in the presence of
reasonable agreement was obtained between the pre¬ a stimulus near an 5“, and responding is diminished
dicted PDG 1 and the empirical relative gradient ob¬ in the presence of a stimulus near an S+. Several
tained from Group III, since four of the seven points dynamic models have been developed (D. Blough,
matched almost exactly. The prediction is improved 1975; Rescorla 8c Wagner; 1972; Wagner 8c Rescorla,
further by converting to proportions in PDG 2. 1972) which incorporate such phenomena that are not
Figure 9 shows that the relative excitatory gradient easily handled by Spence’s theory. The basic assump¬
was much steeper for the tests without shock than for tions of their model are stated by Rescorla and Wag*
the tests with shock. Actually, very few responses to ner (1972) as follows!
each test stimulus were obtained for the excitatory
gradients without shock. The predictions in Figure 10 The effect of a reinforcement or nonreinforce¬
would have been much worse had the data for the ment in changing the associative strength of a
tests without shock been employed, but the tests with stimulus depends upon the existing associative
strength, not only of that stimulus, but also of
other stimuli concurrently present. It appears
that the changes in associative strength of a
stimulus as a result of a trial can be welhpre-
dicted from the composite strength resulting
from all stimuli present on that trial. If this
composite strength is low, the ability of a rein¬
forcement to produce increments in the strength
of component stimuli will be high; jf the com¬
posite strength is high, reinforcement will be
relatively less effective. Similar generalizations
appear to govern the effectiveness of a nonrein-
forced stimulus presentation. If the composite
300 I 670 I 1500 I 3400 300 I 670 I 1500 I 3400 associative strength of a stimulus compound is
460 1000 2550 450 1000 2250
FREQUENCY (Hz) high, then the degree to which a nonreinforced
presentation will produce decrements in the
Fig. 10. (Left) Relative gradients of excitation (closed triangles) associative strength of the components will be
and inhibition (squares). The excitatory gradient is the WSE
large; if the composite strength is low, the effect
gradient of Figure 9 (upper left). The inhibitory gradient is
calculated from the WSE gradient of Figure 9 (center left). The of a nonreinforcement will be reduced, (p. 73)
numbers in the square brackets alongside the vertical lines
between the gradients represent the algebraic sum of the two Two experiments by D. Blough (1975) illustrate
points that the particular line connects. The numbers in paren¬
theses along the same vertical lines were obtained by trans¬
the utility of a dynamic model of stimulus control. In
forming the numbers in square brackets to a scale of 100. these experiments the rate of responding to stimuli
(Right) Empirical (closed-circles) postintradimensional discrimi¬ along a continuum is observed. Associative strength is
nation gradient (PDG) of relative generalization from Figure 9
(WSE), compared with PDGs derived from the calculations on
manipulated by associating a moderate rate of rein¬
the left. The values on derived PDG 1 are from the square forcement with each stimulus on the continuum
brackets on the left, while the values on derived PDG 2 are which produces an above-zero base line. A decre-
from the parentheses on the left. (From Klein & Rilling, 1974. ©
1974 by the Society for the Experimental Analysis of Behavior,
mental gradient is obtained by increasing the fre¬
Inc.) quency of reinforcement in the presence of one stim-
 452
STIMULUS CONTROL AND INHIBITORY PROCESSES

Fig. 11. (Left) A family of excitatory gradients for one bird classified by breaking down
the 20-sec interval into four 5-sec periods. The gradients become flatter as the end of
the interval approaches. (Right) A family of inhibitory gradients for one bird classified
within the FI 20-sec base line. Note the “shoulders” to the left and right of 597 for the
0-5-sec functions and the apparent ceiling effect in curves collected during the last two
quarters of the trial interval. The excitatory gradients were obtained by increasing the
probability of reinforcement in the presence of 597 relative to the other test stimuli, and
the inhibitory gradients were obtained by decreasing the probability of reinforcement at
597 nm. (From Blough, 1975. © 1975 by the American Psychological Association. Re¬
printed by permission.)

ulus, while an incremental gradient is obtained by procedure is that an excitatory gradient was obtained
decreasing the frequency of reinforcement, by increasing the probability of reinforcement during
The base line was an FI 20-sec schedule and re¬ 597 nm, S + ; and following a return to the base line,
sponses were recorded within 5-sec intervals: 0-5, 5- an inhibitory gradient was obtained by decreasing the
10, and 15-^0 sec from the onset of the fixed Interval. probability of reinforcement to 597 nm, now an S-.
The probability of food at the end of the interval was These results are presented in Figure 11. Responses
.1. The moderate rate of reinforcement for all of the are shown separately for each of the four 5-sec periods
stimuli assured an above-zero base line prior to gen¬ within the 20-sec trial interval. The FI schedule was
eralization testing, The most important aspect of this selected to avoid a “ceiling effect” in which the rate
of responding is so high that it is not influenced by
the value of the test stimulus. Generalization was
measured each day with a maintained procedure in
which several test trials were included within each
session.
In Figure 12 the probability of reinforcement was
four times higher for the long-wavelength trials than
it was for the reinforcer that ended short-wavelength
trials. The rate of responding immediately to the
right of the boundary between reinforcement prob¬
abilities is accentuated relative to the other stimuli
receiving the same probability of reinforcement, while
the rate of responding to the left of the boundary is
570 587 597 606 617 attenuated relative to the other stimuli receiving the
STIMULUS WAVELENGTH - nm
same probability of reinforcement. Thus the boun¬
Fig. 12. Mean relative rate of responding for three birds. Data dary between reinforcement conditions for multiple
were obtained during discrimination training in which rein¬ stimuli on a continuum is an important determinant
forcement was four times as frequent in the conditioned rein¬
forcer that ended long-wavelength trials as it was in the con¬ of inhibitory stimulus control. These data confirm
ditioned reinforcer that ended short-wavelength trials. Note the experiments of Catania and Gill (1964) and Farthing
trough and peak to the left and right of the dividing line
(1974). Blough has developed a dynamic mathematical
between low and high reinforcement. (From Blough, 1975. ©
1975 by the American Psychological Association. Reprinted by model, similar to that of Rescorla and Wagner (1972),
permission.) that provides an excellent fit to these data. This
Mark Rilling 453

phenomenon is also analogous to the Mach bands ob¬ that the determinants of the peak shift and the inhib¬
tained in visual perception which are produced by itory gradient are identical.
differences in luminance between adjacent regions of Since Klein and Rilling obtained inhibitory gra¬
the stimulus. dients on an avoidance base line, it appears that the
In Blough’s experiments, each stimulus presenta¬ peak shift and inhibitory control do not depend on
tion was followed by an interval of 5-12 sec during whether responding is maintained by an appetitive or
which any responses on the dark key were ex¬ an aversive base line.
tinguished. Weiss’s research on stimulus compounding In the sections which follow, a unified theory of the
suggests that what is learned when the discriminative peak shift and inhibitory dimensional control will be
stimuli are “off” is an important determinant of stim¬ presented. Consideration will be given to those vari¬
ulus control. Therefore, perhaps Blough’s results ables which fail to produce the peak shift and inhib¬
would have been different without the dark key. itory dimensional control. This analysis sheds some
One of the controversial issues in stimulus control, light on the necessary and sufficient conditions for
as Hearst et al. (1970) point out, has been the search these effects.
for a neutral zone between excitation and inhibition.
They point out that an incremental gradient around
AtVldUnt &f Training
S- does not indicate whether that stimulus is neutral
or inhibitory. It could be argued that a concept of
Most theories of discrimination learning, including
inhibition is not necessary to explain Blough’s data,
those of Spence (1936) and Hull (1943), predict that
since the behavior could also be explained in terms of
the excitatory and inhibitory gradients become steeper
reduced excitation. It would be useful to extend
as a function of the amount of discrimination train¬
Blough’s procedure to interdimensional training to
ing. If the determinants of the peak shift and inhib¬
determine if the combination of S~ with S+ produces
itory stimulus control are the same, the probability of
a decrement in responding. Such a procedure would
obtaining the peak shift should also increase as a func¬
establish that S_ is an inhibitory stimulus*
tion of the amount of training. Empirical investiga¬
Blough’s data suggest that inhibition occurs rela¬
tion of the influence of the amount of training, de¬
tive to a base line probability of reinforcement in the
fined by the number of sessions, indieateg that this
presence of the background stimuli. There is, there¬
variable i§ mdeed one of the most important deter¬
fore, no point of absolute neutrality, only increments
minants of the peak shift and inhibitory stimulus
or decrements from the base line of nondifferential
control. It is not dear at present, though, which
reinforcement. Stimuli associated with reduced rates
aspects of this variable—e.g., number of reinforce¬
of reinforcement demonstrate some of the same in¬
ments, duration of exposure to the discriminative
hibitory phenomena as do stimuli associated with
stimuli, number of alternations between S+ and S~,
extinction. An important empirical question is the
etc.—are critical in determining the slope of the gen¬
extent to which similar functional relationships are
eralization gradient.
obtained between these two conditions.

iNTRADIMEN^IQNAjy TttAimiNO

DETERMINANTS Of THE PEAK SHIFT Thomas (1962) observed the acquisition of the area
AND INHIBITORY STIMULUS CONTROL peak shift by administering a short generalization test
to each subject following every even-numbered session
Historically, the peak shift was investigated before of discrimination training, After only 30 min of ex¬
interdimensional training led to the discovery of the posure to S~, a reliable shift in the area of the gra¬
inhibitory stimulus-generalization gradient. Conse¬ dient was observed, sometimes even before the rates of
quently, much more is known about the determinants responding to S+ and S~ began to separate. The mag¬
of the former than of the latter. Hearst’s three-group nitude of the area shift increased as a negatively
design provides the most promising framework for a accelerated function of the amount of training.
quantitative analysis of the relationship between inter¬ Once the effects of short amounts of training were
dimensional and intradimensional training. Unfor¬ determined, investigators turned to the effects on the
tunately, experiments which compare the two types peak shift of training extended over many sessions.
of discrimination training under equivalent condi¬ Terrace (1966a) administered generalization tests fol¬
tions are rare. However, the available data (Hearst, lowing the 15th, 30th, 45th, and 60th sessions of dis¬
1969b; Klein 8c Rilling, 1974; Marsh, 1972) suggest crimination training between two wavelength stimuli
454 STIMULUS CONTROL AND INHIBITORY PROCESSES

with: 580 nm (S + ) and 506 nm (S~). The peak of the lowing different amounts of training in which re¬
first gradient occurred at 600 nm, a peak shift of 20 sponding in the presence of S+ was nondifferentially
nm. During the next two generalization tests, the reinforced. The shallow excitatory gradient which was
magnitude of the peak shift decreased to 10 nm, and obtained on the second clay of training became pro¬
by the last generalization test, the peak shift had dis¬ gressively steeper through 14 days of training.
appeared and the peak reverted back to S+. Positive In the Hearst and Koresko study, the acquisition of
behavioral contrast also decreased with extended the response was confounded with the acquisition of
training. In positive behavioral contrast, the response dimensional stimulus control, since both processes oc¬
rate in the constant component increases when the curred simultaneously. These processes were isolated
response rate in the variable component is decreased by Schadler and Thomas (1972). In order to firmly
by extinction. These data led Terrace to the conclu¬ establish the response, pecking the key was first non¬
sion that determinants of behavioral contrast and of differentially reinforced during 10 daily 30-min ses¬
the peak shift are identical. sions in the presence of a white key that was ortho¬
Data obtained in subsequent experiments disprove gonal to angularity, the dimension of stimulus control.
Terrace’s interpretation. Dukliayyil and Lyons (1973) Then generalization gradients were obtained after 0,
determined the effects of 105 days of intradimensional 5, 10, or 20 min of nondifferential training in the
training on behavioral contrast and the peak shift. As presence of a single white vertical line on a black
in Terrace’s experiment, generalization tests were ad¬ background. Dimensional control by angularity was
ministered at regular intervals. While no bird showed acquired rapidly and approached asymptote after 20
a peak shift on each test, a majority of the birds min of nondifferential training. The Schadler and
showed a peak shift after 105 days of training. Sub¬ Thomas study demonstrated that the acquisition of
stantial fluctuations in the rate of responding to S+ response strength and the acquisition of stimulus con¬
were obtained. Therefore on a day-by-day basis, the trol over responding are distinct, separable behavioral
occurrence of the peak shift is not correlated with the processes. Once the response is acquired, control by a
existence of contrast (i.e.7 with response rate in S4-). stimulus associated with reinforcement may develop
Roth behavioral contrast and the peak shift are ob¬ in a matter of minutes.
tained after extended discrimination training. Farthing and Hearst (1968) determined how the
amount of interdimensional training affected general¬
Interdimensional Training
ization of extinction. After seven sessions of nondiffer¬
In an independent groups design, Hearst and ential reinforcement which established the pecking
Koresko (1968) administered a generalisation test fol¬ response, five groups of pigeons received from 1 to f6

Fig. 13. Relative gradients of

inhibition for individual birds
in five groups that received
from 1 to 16 days of discrimina¬
tion training prior to the stim¬
ulus-generalization test. The
probability of obtaining an in¬
hibitory gradient within each
group increases with the amount
of training. The numbers in
parentheses for each bird in¬
dicate, on the left, the total
responses to the six-line test
stimuli, and, on the right, the
number of test responses to S +
(no line). (From Farthing &
Hearst, 1968. © 1968 by the So¬
ciety for the Experimental
DEGREES FROM S- Analysis of Behavior, Inc.)
Mark Rilling 455

days of discrimination training. The S+ was a plain GROUP I S+=1000 Hz S- = NOISE

NO PROBE SHOCK I PROBE SHOCK
white field and S~ was a white field bisected by a DAYS 1-3

270f
black vertical line. Following discrimination training,
each animal was given a generalization test in extinc¬
tion, during which the angle of the line was varied.
LA,
GROUP 2
lLLL S+ = 1000 Hz S-= NOISE
Figure 13 shows the relative gradient obtained from
PROBE SHOCK
each bird. At least one steep inhibitory gradient was
z
obtained at each condition of days of training. The o
CL
CO
basic effect of the independent variable was to in¬
crease over days the probability of obtaining an in¬
o GROUP 3 S+ = NOISE S- = 1000 Hz
hibitory gradient from each session. By Day 8, the an
LU NO PROBE SHOCK | PROBE SHOCK
probability was 1.0. CD
ZD
Hearst (1971) found that an inhibitory gradient
obtained after 64 sessions of prolonged discrimination
training had essentially the same shape as those ob¬ GROUP 4 + = NOISE S- = 1000 Hz
tained after 8 or 16 hours. Selekman (1973) replicated PROBE SHOCK

Farthing and Hearst’s experiment, adding a correc¬

tion procedure m which each response to S~~ extended
its duration. He found no relationship between the
number of sessions and the slope of the inhibitory
FREQUENCY(HZ)
gradient which formed during the first session of dis¬
crimination training. Using stimulus compounding as Fig. 14. Acquisition of gradients of excitation and inhibition.
Groups 1 and 2 show the development of the excitatory gradient
an index of inhibition, Yarczower and Curio (197?) around. S4-, while Groups 3 and 4 show the development of the
found that S suppressed positively reinforced re¬ inhibitory gradient around S~. Each gradient was formed by
sponding during the first 10 min of discrimination adding the total number of responses to each stimulus across
three days of training. Each data point is a sum for the three
training.
birds in each group. The closed circles show the gradients ob¬
Thus in both the excitatory and inhibitory case, it tained without a probe shock, while the open circles show the
appears that stimulus control is acquired in minutes gradients obtained With the addition pf &n nnaypidahl? probe
sh6£k during each test stimulus. For Groups 1 and 3, tlie prefer
rather than hours. The failure to find a relation be¬
shock was present during sessions 16-21. For Groups 2 and 4.
tween the amount of interdimensional training and the probe shock was presented from the beginning of discrimi¬
nation training. The arrow indicates the 1,000-Hz stimulus for
the amount of inhibition appears related to the use
each group. Note the different ordinate scales for Groups 1 and
of conditions that measure inhibition hours or sessions
2 and for Groups 3 and 4. The frequencies employed were from
after inhibitory stimulus control has reached asymp¬ left to right; 300, 450, 070, 1,000, 1,500, 2,500, and 3,400 Hz.
tote. (From Rilling Sc Budnik, 19*?$. (6) 197# by the Society for the
Experimental Analysis of Behavior, Ine.^

Aversive Baselines
The gradients are presented in Figure 14. For
Rilling and Budnik (1975) determined the in¬ Group 1, an increasing number of responses to the
fluence of the amount of discrimination training on 1,000-Hz tone (designated by an arrow) was observed
the acquisition of excitatory and inhibitory stimulus as a function of the number of days of training. This
control in a treadle-press avoidance paradigm. During increase indicates the birds’ discrimination of the test
21 days of interdimensional discrimination training, stimuli, which were always presented during extinc¬
generalization was measured daily with a maintained tion, from the 1,000-Hz training stimulus, which was
procedure by randomly substituting each of six test usually associated with the avoidance schedule. As
frequencies for the 1,000-Hz training stimulus. Groups discrimination training progressed, the gradients be¬
1 and 2 were designed to measure the acquisition of came steeper. The addition of the unavoidable shocks
excitatory stimulus control by associating the 1,000-Hz in Session 16 flattened the generalization gradient by
tone (S + ) with a schedule of free operant avoidance elevating responding to the test stimuli more than re¬
and noise (S~) with extinction. Groups 3 and 4 were sponding to S + . For Group 2, unavoidable probe
designed to measure the acquisition of inhibitory shock elevated responding to the test stimuli in com¬
stimulus control by associating the noise (S+) with parison with Group 1. The steepening of the excita¬
free operant avoidance and the 1,000-Hz tone (S-) tory gradient with training was much less pronounced
with extinction. than in Group 1.
 456
STIMULUS CONTROL AND INHIBITORY PROCESSES

For Group 3, the acquisition of the discrimination tion of S~ which contrasts with the distributed extinc¬
is reflected in a rapid decrease in the number of tion obtained in a successive discrimination when S+
responses to S~ and flat gradients for Sessions 7-15. and S~ are briefly presented in alternate or random
Flat gradients with low responding to the test stimuli sequences. Spence’s theory of discrimination learning
are difficult to interpret because both neutral and predicts a peak shift following massed extinction.
inhibitory stimuli produce the same outcome in such As it turns out, however, the sequential presenta¬
situations. When the unavoidable probe shocks in¬ tion of S+ and S~ is essential for the peak shift and
creased responding, inhibitory gradients emerged in inhibitory stimulus control, a finding that Spence's
Sessions 16-21. The emergence of the inhibitory gra¬ theory of discrimination learning fails to predict.
dients following introduction of the probe shocks re¬ When S+ and S — were selected from the wavelength
emphasized the equivocal nature of a flat stimulus- dimension, Honig, Thomas, and Guttman (1959)
generalization gradient with a low response output. found that after 20 or 40 min of massed extinction to
For Group 4, in which the probe shocks were present S~, the peak of the generalization gradient occurred
from Day 1, inhibitory gradients were obtained at S+. A peak shift was obtained following an equiv¬
throughout discrimination training. alent amount of exposure to each stimulus when the
These results show striking parallels between posi¬ stimuli were presented alternately. Yarczower and
tive and negative reinforcement for the acquisition of Switalski (1969) also obtained no peak shift following
stimulus control and support Hearst’s views that massed extinction using goldfish as the experimental
similar laws of generalization apply to positive and organism. Weisman and Palmer (1969) compared
negative reinforcement. Both the excitatory and in¬ massed extinction and an equivalent amount of dis¬
hibitory gradients become steeper during the acquisi¬ crimination training with interdimensional presenta¬
tion of the discrimination.
tions of the discriminative stimuli. The birds receiv¬
ing massed extinction showed a flat gradient along the
Summary S~ dimension, whde an inhibitory gradient with the
minimum at S~ was obtained following successive dis¬
The amount of training has a similar influence on
crimination training.
the acquisition of stimulus control following inter-
4 he typical successive discrimination is a mixture
dimensional and intradimetisional discrimination
of four possible transitions: S+S+, S“S“, S+S~, and
training and supports a unified treatment of the two
S“S+. The results of the above experiments indicate
types of discrimination training. The acquisition of
that the identical transitions, S+S+ and S-S-, are
the response has often been confounded with the
not effective in producing the peak shift and inhib¬
acquisition of stimulus control. When these processes
itory stimulus control. In a well-designed and care¬
are separated by providing nondifferential reinforce¬
fully controlled experiment, Ellis (1970) compared the
ment of the response prior to discrimination training,
effectiveness of the two opposite transitions: S+S-
excitatory and inhibitory stimulus control is acquired
and S“S+. One group experienced a single S+S“
within minutes rather than hours. While more data
alternation within each of several daily sessions, while
on the minimum amount of training necessary for
the other group experienced a single S~S+ alterna¬
stimulus control would be desirable, the data suggest
tion. Ellis obtained a peak shift for the birds that
that the probability of obtaining the peak shift and
received a transition between massed extinction and
inhibitory gradient increases in the course of discrim¬
massed reinforcement, but the peak shift was absent
ination training. Extended training has little in¬
for the group that received massed reinforcement fol¬
fluence once the necessary and sufficient conditions for
lowed by massed extinction. These data suggest that
the inhibitory gradient or peak shift have been
established. S~S+ transition is the necessary and sufficient condi¬
tion for the peak shift. It would be interesting to
replicate this experiment with interdimensional train¬
Stimulus Sequences and Massed Extinction ing to determine if the S“S + transition is also neces¬
sary and sufficient for inhibitory stimulus control.
Spence’s theory of discrimination learning does not An experiment by Rosen and Terrace (1975) con¬
explicitly consider sequential variables, such as fre¬ firmed the absence of the peak shift and inhibitory
quency of alternation of S+ and S~. For Spence, stimulus control following massed extinction in the
the relevant variable determining inhibition is the presence of intradimensional and interdimensional
amount of exposure to S~. In massed extinction, the stimuli. These investigators found that the peak shift
subject is exposed to a single relatively long presenta¬ and inhibitory stimulus control were reinstated by the
Mark Rilling 457

interpolation for 3 min between massed extinction present throughout the three phases. For Group
and generalization testing of each of the following CiC2C2, the stimulus context changed from C± to C2
procedures: (1) the presentation of S+ with rein¬ during Phase 2 and remained constant during the
forced responding; (2) the presentation of S+ without stimulus-generalization test. For Group CJC2C!, the
reinforced responding; and (3) the presentation of stimulus context changed from Ci to C2 during Phase
food independently of responding in the presence of 2 and reverted back to Ci during generalization test¬
the dark key. ing. Miller changed the stimuli gradually rather than
Yarczower (1974) found that, as compared with a abruptly between phases in order to avoid the loss of
group that received a generalization test without any responding with changes in the contextual stimuli.
extinction sessions, massed extinction sharpened con¬ The question of primary interest is whether the peak
trol by the S+ stimulus. Although Yarczower’s results shift was appropriate to the discriminative stimuli of
are at variance with some of the earlier experiments, Phase 1 or Phase 2.
the data are in agreement with the finding that, al¬ The main results of Miller's experiment are pre¬
though massed extinction may sharpen an excitatory sented in Figure 15. For Group CiCiCi, a bimodal
gradient, it reduces the probability of obtaining in¬ gradient with maximums at 549 and 559 was obtained.
hibitory phenomena such as the peak shift and inhib¬ The peak shift at 549 is appropriate for the final
itory stimulus control. phase of discrimination training, while the peak at
These experiments demonstrate that extinction of 559 reflects generalization from the S4- employed cj.ur_
responding in the presence of S~ is not inevitably ing the first phase. For Group GiCqCq, the peak shift
followed by a peak shift or inhibitory stimulus con¬ at 549 nm is appropriate to the contextual stimuli
trol. The relevant variable seems to be the number of present during Phase 2 and generalization testing.
transitions from S~ to S + .

Stimulus Context and Memory

Contextual stimuli are stimuli within the experi¬

mental apparatus that remain uncorrelated with the
contingencies of reinforcement in the experimental
chamber. Potential contextual stimuli are the lamps
that illuminate the chamber (houselights), the level of
the background noise, and the walls and floor of the
chamber. Since these stimuli are not usually varied
systematically during the course of an experiment*
most investigators have assumed that they acquire no
control over responding. However, contextual stimuli
play a significant role in models of the inhibitory
process (e.g., Wagner & Rescorla, 1972) and a theory
of forgetting retrieval failure (Spear, 1971). There¬
fore, it was inevitable that investigators would re¬
assess the role of these background stimuli in the stim¬
ulus control of behavior.
Miller (1972) employed an ABA sequence of dis¬
crimination reversals to assess the role of stimulus WAVELENGTH (nm)
context on stimulus control. Each of the three major
Fig. 15. Dependence of the peak shift upon the discriminative
experimental groups received identical conditions of and background stimuli present during discrimination training
discrimination training and generalization testing. In and the generalization test. When the background stimuli re¬
mained constant, Group C1C1C1, a bimodal gradient with
Phase 1, the discriminative stimuli were 576 nm (S+)
maximums at 549 and 559 was obtained. With the presence of
and 555 nm (S~), while during Phase 2 the discrim¬ C2 during reversal training and generalization testing, Group
inative stimuli were reversed. In the third phase, each CiC2C2, a peak shift occurred at 549 nm away from S2. With
pigeon received a stimulus-generalization test on the the presence of during initial discrimination training and
generalization testing, Group a peak shift occurred at
wavelength dimension.
587 nm away from S3, rather than the negative stimulus
The groups differed with respect to the stimulus present during reversal training immediately prior to the gen¬
context. For Group the stimulus Cx remained eralization test, S2. (Redrawn from Miller, 1972.)
 458 STIMULUS CONTROL AND INHIBITORY PROCESSES

The most dramatic influence of the contextual stimuli forced response is zero immediately after reinforce¬
was obtained for Group C1C2C1, in which the con¬ ment and accelerates rapidly through the interval
textual stimuli present during generalization testing until the response is reinforced. This pattern is pro¬
were associated with Phase 1 of discrimination train¬ duced by the alternation of a period of extinction
ing. This group yielded a peak shift at 587 nm appro¬ with a period of reinforcement availability. Staddon
priate to Phase 1. For this group, interference from (1969) suggested that a stimulus-generalization test
Phase 2 was eliminated by manipulating the con¬ conducted by varying the stimulus present throughout
textual stimuli. The contextual stimuli enabled the the interval should produce inhibitory stimulus con¬
subjects to store the memory for the discriminative trol at the beginning of the interval and excitatory
stimuli of Phase 1 independently of the memory for stimulus control at the end of the interval. This is
the discriminative stimuli of Phase 2. Although the exactly the result obtained in an elegant experiment
contextual stimuli are presumably neutral since they by Wilkie (1974).
are present during both S+ and S~, they exert power¬ Responses in the presence of a vertical line (0°)
ful control over responding. were reinforced on FI 3-min for two birds and FI 6-
Miller’s research demonstrates that paradigms orig¬ min for the third bird. In order to increase stimulus
inally developed for research on information process¬ control by the line, each reinforcement was followed
ing in humans are also relevant to the stimulus con¬ by a blackout of 10 min. Generalization was measured
trol of behavior in infrahuman organisms and that by randomly varying the angle of the line during
the location of the peak of the stimulus-generaliza¬ different intervals. Each stimulus was present through¬
tion gradient is a sensitive index of interference. out the interval. An essential component of Wilkie’s
experiment was a maintained procedure in which re¬
Schedules of Reinforcement sponses were reinforced as usual throughout the gen¬
eralization test.
Variables associated with the schedule of reinforce¬
The generalization gradients in the left-hand panel
ment during discrimination training influence inhib¬
of Figure 16 show that an inhibitory gradient with a
itory stimulus control and the peak shift to a signif¬
minimum at the vertical line was obtained from each
icant degree. For example, most experiments on stim¬
bird during the first third of the interval, while an
ulus control have employed a multiple schedule in
excitatory gradient was obtained during the last third.
which S+ was associated with reinforcement and S~
During the middle of the interval, the rate of re¬
with extinction. The alternation of these stimuli in a
sponding was not influenced by the angle of the line.
successive discrimination produces successive periods
These results are dependent upon maintaining re¬
of responding at relatively high or low rates.
sponding during the generalization test with reinforce¬
Whenever multiple schedules of reinforcement are
ment at the end of the interval. The right-hand panel
utilized, two potential sources of influence on inhib¬
of Figure 16 demonstrates that a family of excitatory
itory phenomena are observed. One relates to the
gradients were obtained when the test was carried out
interaction between the relative rates of responding
during extinction with the omission of reinforcement.
emitted to each stimulus. The other derives from the
Wilkie’s results support the theory of temporal
incentive contrast produced by the difference between
control developed by Staddon (1974). When food is
the density of reinforcement in the two components.
delivered on a schedule with a constant period, Stad¬
The following sections will deal with such questions
don predicts that food acquires inhibitory after¬
as: (1) How much of a difference between the two
effects that decay with the passage of time. There¬
rates of responding is necessary before inhibitory
fore, a stimulus associated with the termination of
phenomena are produced? (2) Is inhibitory stimulus
food acquires inhibitory stimulus control.
control present when a discrimination is established
by different rates of reinforcement during the two
stimuli? (3) If this is so, how much of an incentive Multiple Schedules: Response Rate
and Incentive Interactions
difference between the reinforcers in the two com¬
ponents is necessary?
Under the typical conditions of discrimination
training, a mult VI extinction schedule produces a
Fixed-interval Schedules
peak shift following intradimensional training and an
Every student of operant behavior is familiar with inhibitory gradient following interdimensional train¬
the scalloped pattern of responding produced by the ing. The data reviewed in this section demonstrate
fixed-interval (FI) schedule. The rate of the rein¬ that the peak shift and inhibitory stimulus control
Mark Rilling 459

(1971, 1974) and Weiss and Van Ost (1974) in the area
of stimulus compounding. Weiss (1974) formulated a
model of stimulus control in which response and in¬
centive properties conditioned to tone and light com¬
bine to produce the behavior resulting in T + L.
Maximum additive summation occurs when T + L is
composed of stimulus elements associated with an in¬
crease in both response and reinforcement rates. For
maximum suppressive summation, T + L is composed
of stimulus elements associated with a decrease in both.
When only one factor is operating, response or rein¬
forcement, while the other is constant over multiple-
schedule components, T T L should control only
moderate summation. When response and reinforce¬
ment properties are conflicting, one increasing and
the other decreasing, minimal or no summation
94
should occur to T -f- L. Since summation and peak
shift are sensitive to many of the same variables, it
could prove useful to apply Weiss’s formulations to
predictions of peak shift and inhibitory stimulus con¬
trol.
Some new terminology must be introduced before
translating from the stimuliis-compounding to the
0 -45° ' 6° ' r45° -45* ‘ 0s ‘ 745®
stimulus-gen£ralizati9n paradigm, Foi the many ea§e§
LINE TILT
AFTER in which a successive discrimination is established by
HEINE NON-HEINE reinforcing responding at different rates during each
of the two components of a multiple schedule, the
Tig. 16. Response rate in the presence of the lines at different
angles during successive thirds of the fixed intervals that notation of S-!- and S~ 15 inadequate? In the inven¬
followed 0* intervals ending in reinforcement (left-hand panels) tion followed here, the stimulus associated with con-
or nonreinforcement (right-hand panels). When responding is
stant conditions of reinforcement during prediscrimi¬
reinforced, an inhibitory gradient is obtained at the beginning
of the interval and an excitatory gradient is obtained at the nation and discrimination training is SI, and the
end of the interval. Rate is averaged for each of the three birds stimulus associated with a change in the conditions of
over five test sessions. (From Willtie, 1974. © 1974 by the So¬
reinforcement is S2, whether the change is an increase
ciety for the Experimental Analysis of Behavior, Inc.)
or decrease in reinforcement density. Thus the rate
of responding during S2 may either increase or de¬
are also obtained under a wide range of conditions in crease during the acquisition of the discrimination
which responding is reinforced during each of the depending upon the nature of the change in rein¬
two components. Incentive variables are the proper¬ forcement, In the special multiple schedule in which
ties of the reinforcer that determine its effectiveness extinction is introduced after a period of nondiffer¬
in maintaining responding. The delay, frequency of ential reinforcement, S+ corresponds to SI and S-
occurrence, magnitude, or intensity of a positive or to S2.
negative reinforcer are incentive variables that com¬ Translating from the stimulus-compounding to the
bine with the rate of responding to determine the stimulus-generalization paradigm, the theory predicts
probability of the peak shift and the amount of in¬ that the probability of the peak shift and an inhibi¬
hibitory stimulus control. The relative incentive tory gradient is maximized when SI controls a higher
value of two schedules of reinforcement is operation¬ response rate and reinforcement frequency than S2.
ally defined by the organism’s degree of preference for When SI is associated with a change in only one of
the stimuli associated with each schedule when they these variables, the probability of the peak shift and
are presented simultaneously. Measuring this prefer¬ an inhibitory gradient is less. Finally, when SI is dis¬
ence requires an independent condition which un¬ criminative for an increase in the response rate, but
fortunately is rarely employed in experiments on signals a decrease in the frequency of reinforcement,
stimulus control. the probability of the peak shift and inhibitory gradi¬
The analysis was suggested by some work of Weiss ent is minimized or eliminated.
 460
STIMULUS CONTROL AND INHIBITORY PROCESSES

The frequency of reinforcement during SI and S2 intradimensional discrimination training on a mult

has been specified and carefully controlled in many VI 1-min VI 5-min schedule instead of a mult VI
experiments. Therefore, the frequency of reinforce¬ 1-min extinction schedule. A peak shift away from
ment rather than a direct measure of the incentive the stimulus associated with the VI 5-min schedule
value or response strength is employed in the analy¬ was obtained during a stimulus-generalization test.
sis of these experiments. The assumption made in this The shape of the postdiscrimination gradient in Gutt-
analysis is that the frequency of reinforcement is di¬ man’s experiment was virtually identical to that ob¬
rectly related to incentive value and may therefore tained in Hanson’s experiment, so the VI 5-min
serve as an estimate of the incentive value, although schedule was as effective as extinction in producing
the correlation between the two remains to be veri¬ the peak shift. This result was confirmed by Terrace
fied. It is reasonable to assume that other variations (1968) and by Wheatley and Thomas (1974). Since the
in reinforcement affecting preference-e.g., delay, mag¬ peak shift was obtained when each discriminative
nitude, and intensity—would act similarly in deter¬ stimulus was associated with a schedule of reinforce¬
mining incentive differences. ment, extinction during S2 is not necessary for the
Table 1, which is an adaptation of one Weiss (1974) production of the peak shift.
presented in his analysis of response and incentive Using intradimensional training Dysart, Marx, Mc¬
variables in the results of stimulus compounding, Lean, and Nelson (1974) systematically varied the
presents the results of stimulus-generalization tests frequency of reinforcement during S2 from VI 1-min
following discrimination training in a number of through VI 5-min to extinction using a separate
studies. These experiments are classified with respect group of pigeons for each condition. Within each
to the reinforcement and response rates in S2 as com¬ group, the probability of obtaining a peak shift away
pared to SI. SI is the stimulus associated with con¬ from S2 increased as a function of the decrease in the
stant conditions of reinforcement during prediscrimi¬ relative frequency of reinforcement associated with
nation and discrimination training. S2 is the stimulus S2. Since the correlation between the rate of re¬
associated with a change in the conditions of rein¬ sponding and the rate of reinforcement during S2 was
forcement and/or response requirements, whether -4--97, it was not possible to specify which of these fac¬
these changes are increases or decreases. tors was responsible for the peak shift.
1 he response rate and reinforcement frequency Among the most impressive evidence for a rela¬
m S2 may each decrease, increase, or remain unchanged tive interpretation of the peak shift was a study by
during the acquisition of the discrimination. This Wheatley and Thomas (1974) in which four of six
yields a 3 X 3 matrix with nine cells. Each cell is sub¬ pigeons showed a peak shift away from the VI 24-sec
divided according to whether intra- or interdimen- component of a mult VI 12-sec VI 24-sec schedule.
sional training was employed. For each experiment, The best predictor of the peak shift in their experi¬
the schedule associated with SI is given first, followed ment was a discrimination index, responses to SI
by S2 in parentheses. The outcomes predicted by ex¬ divided by total responses. A poor discrimination
trapolation of Weiss’s theory are indicated within index in the study by Yarczower, Dickson, and Gollub
each cell as the probability of the peak shift follow¬ (1966) may explain why this study was the only fail¬
ing intradimensional training. The probability of an ure to obtain a peak shift for the experiments in the
inhibitory gradient following interdimensional train¬ upper left cell of Table 1. Wheatley and Thomas con¬
ing is also presented. Each cell contains investiga¬ clude that, given a good index of discrimination be¬
tions meeting the response and reinforcement rela¬ tween SI and S2, a peak shift is obtained away from
tions indicated for SI and S2. In general, the fit is S2 when S2 is associated with a high frequency of
excellent, and, with the exceptions noted in the foot¬ reinforcement, provided SI is associated with a still
notes, the experiments support the predictions from higher frequency.
Weiss’s model.
The Wheatley and Thomas study was carried out
The upper left cell, in which both the response with a two-component multiple schedule. Suppose,
rate and frequency of reinforcement during S2 de¬ following Weiss, that a three-component schedule is
creased, was the starting point of this research effort. employed-e.g., mult VI 12-sec VI 24-sec EXT, with
Hanson’s (1959) original experiment, where respond¬ extinction associated with the absence of the discrim¬
ing in S2 was reduced through extinction while a VI inative stimuli. By this temporal manipulation of the
schedule remained operative in SI, is the extreme ar¬ stimulus context it seems possible that the peak shift
rangement for this cell. Guttman (1959) extended would not be obtained.
Hanson’s original experiment on the peak shift with Weisman (1969) ran a parallel interdimensional
 Table 1 Results of Stimulus-generalization Experiments Classified According to Response Rate and Reinforcement Rate Differ¬
ences Between S2 and SI

RESPONSE RATE IN S2 COMPARED WITH Si

Decrease No Change Increase

PREDICTION MAXIMUM PEAK SHIFT MODERATE PEAK SHIFT MINIMAL PEAK SHIFT
INTRADIMEN- VI V (VI 5') VI 2.5' (VI 50 No data
SIONAL TRAINING Guttman (1959) Wheatley 8c Thomas
Terrace (1968) (1974)*
Wheatley 8c Thomas (1974) VI 30"-DRL 4" (VI 3'-
VI 30" (VI T)a, DRL 2"
VI T (VI 20 or (VI 50 Yarczower et al. (1966)a
Dysart et al. (1974)
VI 30"-DRL 4" (VI 4'-
DRL 8")
Yarczower et al. (1966)a
VI 12" (VI 24") or (VI 60")
Wheatley & Thomas (1974)

PREDICTION MAXIMUM INHIBITORY MODERATE INHIBITORY MINIMAL INHIBITORY

GRADIENT GRADIENT GRADIENT
INTERDIMEN- VI V (VI 50 No data No data
Si

SIONAL TRAINING Weisman (1969)

COMPARED WITH

PREDICTION MODERATE PEAK SHIFT MINIMAL PEAK SHIFT MODERATE PEAK SHIFT
INTRADIMEN- VI T (DRL 6") VI V (DRL 6'0 No data
SIONAL TRAINING VI T (VI T + punishment) Terrace (1968)
Terrace (1968)
VI V (DRO 50")
Yarczower et al. (1968)
S2

VI l'-6" Reinf. (VI l'-2"

REINFORCEMENT? RATE IN

Reinf.)
Mariner & Thomas (1969)
VI T (VI T 4- delay)
Wilkie (1972)
VI V (YT T)
Huff et al. (1975)
£
PREDICTION MODERATE INHIBITORY MINIMAL INHIBITORY MODERATE INHIBITORY
GRADIENT GRADIENT GRADIENT
INTERDIMEN- VI V (DRL) or (DRO) VI V (VT V) No data
SIONAL TRAINING Weisman (1969, 1970) Weisman 8c Ramsden
VI V (VT V) (1973)
Weisman 8c Ramsden
(1973)
VI V (VI T + delay)
Richards (1973)
PREDICTION MINIMAL PEAK SHIFT MODERATE PEAK SHIFT MAXIMAL PEAK SHIFT
INTRADIMEN- VI 30"-DRL 4" (DRO 10") No data VI 5' (VI T)
SIONAL TRAINING Yarczower et al. (1968) Terrace (1968)b
VI T-2" Reinf.
(VI l'-6" Reinf.)
Mariner 8c Thomas (1969)
* PREDICTION MINIMAL INHIBITORY MODERATE INHIBITORY MAXIMAL INHIBITORY
GRADIENT GRADIENT GRADIENT
INTERDIMEN- No data No data No data
SIONAL TRAINING

a A peak shift was not obtained.

b Only one of the three birds showed a peak shift.
The Schedule Associated with SI is Given First Followed by S2 in Parentheses.
Source:Predictions adapted from Weiss's (1974) two-factor combinational model of stimulus control.

461
462 STIMULUS CONTROL AND INHIBITORY PROCESSES

experiment on a multiple VI 1-min and VI 5-min response to S2 is punished (Terrace, 1968), when the
schedule. A shallow inhibitory gradient on the dimen¬ magnitude of reinforcement during S2 is smaller
sion of line orientation was obtained for each of the than SI (Mariner 8c Thomas, 1969), when reinforce¬
animals where the rate of responding to S2 was re¬ ment during S2 is delayed (Richards, 1973; Wilkie,
duced by a shift from VI 1-min to VI 5-min. If the 1972), or when reinforcement during S2 is indepen¬
determinants of the peak shift and the inhibitory dent of responding (Huff, Sherman, & Cohn, 1975).
gradient are identical, then, extending Wheatley and Additional data are needed to determine if VI sched¬
Thomas (1974), some pigeons should demonstrate an ules are preferred to the DRL, (DRO), and VT
inhibitory gradient around a stimulus associated with schedules employed by Terrace (1968), Yarczower,
a VI 24-sec schedule following interdimensional train¬ Gollub, and Dickson (1968), Weisman (1969, 1970),
ing on a mult VI 12-sec VI 24-sec schedule. Additional and Weisman and Ramsden (1973). No data are avail¬
data are needed to determine the limits of inhibitory able to determine if the peak shift and inhibitory
stimulus control following interdimensional training stimulus control are obtained when the relative in¬
with various schedules associated with SI and S2. centive values of SI and S2 are identical.
Spence viewed inhibition in absolute rather than A study by Yarczower et al. (1968), in the lower left
relative terms and always identified inhibition with cell of Table 1, suggests that the peak shift does not
extinction. One of the major conclusions from the occur when the frequency of reinforcement during S2
experiments in the upper left cell in Table 1 is that is increased, but the rate of responding is decreased.
the peak shift and inhibitory stimulus control are The paucity of data in this cell indicates the lack of
determined by the relative rates of reinforcement research under these conditions.
during SI and S2. Richards’s unpublished experiment The “No Change” column of Table 1 presents data
demonstrated that S2, a stimulus associated with a VI for experiments in which the rate of responding dur¬
1-min schedule in which reinforcement was delayed ing SI was identical to the rate during S2 so that
for 10 sec, was inhibitory and produced a decrement differential responding between these stimuli was not
in responding when presented simultaneously with SI present prior to the stimulus-generalization test. The
in a combined-cues test. The amount of inhibition data are remarkably consistent. Independently of the
was less than was obtained when S2 was associated reinforcement rate during S2, the peak shift and in¬
with extinction. These data reduce the appeal of hibitory stimulus control are not obtained.
Spence’s original formulation and require a relative Weiss predicts that incentive differences alone
theory which assumes that S2 becomes an inhibitory should produce a moderate probability of the peak
stimulus when it is associated with a less favorable shift and inhibitory stimulus control when the rates
schedule of reinforcement. of responding to SI and S2 are identical. The Wheat-
In the experiments in the upper left cell of Table ley and Thomas (1974) and Yarczower et al. (1966)
1, the reduction in the frequency of reinforcement experiments fail to support this prediction, but we do
during S2 was confounded with a reduction in the not know if these subjects were discriminating the
rate of responding during S2. Therefore, the next reinforcement differences between components, a nec¬
logical step was to hold the frequency of reinforce¬ essary precondition for establishing different incen¬
ment constant during SI and S2 while reducing the tive values.
rate of responding in the presence of S2. The experi¬ The “Increase” column of Table 1 indicates that
ments in which this procedure was employed are in¬ in only two experiments has the discrimination been
dicated in the middle cell of the “Decrease” column acquired by increasing the response rate during S2
in Table 1. The evidence from these experiments is while a lower rate is maintained during SI. The
quite consistent: both the peak shift and inhibitory lower right cell indicates that the peak shift has been
gradient are obtained. Therefore, reduction in rein¬ obtained by increasing both the frequency of rein¬
forcement frequency during S2 is not necessary for forcement and response rate during S2 in the experi¬
these phenomena. Reduction of the response rate ments of Terrace (1968) and Mariner and Thomas
alone might be adequate. However, an equal fre¬ (1969). However, the data from these experiments are
quency of reinforcement during SI and S2 does not not as consistent as those in the upper left cell of
imply that the incentive values of SI and S2 were Table 1. Perhaps this is because it is easier to estab¬
identical. Although the relative incentive values of lish a discrimination between SI and S2 by decreasing
SI and S2 were not measured in any of these experi¬ rather than increasing the rate of responding during
ments, it is likely that SI is preferred to S2 when each S2.
Mark Rilling 463

In general, these data support the conceptual left key. Similarly, reinforcing a response on the right
framework developed by Weiss (1971, 1972, 1974) key inhibits responding on the left key. As Catania
through his work in stimulus compounding, suggest¬ puts it, “the rate of a reinforced response is inhibited
ing that incentive and rate differences between SI and by the reinforcement of other responses” (p. 741). The
S2 combine to produce the peak shift and inhibitory advantage of Catania’s formulation is that it treats
stimulus control. Thus when the incentive parameters reinforcement (e.g., of a response on the left key) as the
of SI and S2 are held constant, the greater the index causal variable which produces inhibition (of re¬
of discrimination between SI and S2, the higher the sponding on the right key). However, the “inhibition”
probability of obtaining the peak shift or inhibitory of one response by the reinforcement of another does
stimulus control. When SI and S2 control comparable not necessarily imply that the stimulus controlling
response rates, this formulation predicts that the that response, or the lack of it, is an inhibitory stim¬
greater the incentive difference between SI and S2, ulus. This requires an independent test for an in¬
the higher the probability of obtaining the peak shift hibitory stimulus.
or inhibitory stimulus control. However, since it ap¬ D. Blough (1973) obtained a peak shilt on both the
pears that these two variables might combine, the right and left keys following successive intradimen-
greatest probability of peak shift and the largest in¬ sional training by reinforcing a response on the oppo¬
hibitory effects are predicted for the upper left and site key during S . On the right key, stimuli of 550
lower right cells on Table 1 where large incentive and and 559 nm were presented alternately as in the usual
rate differences between SI and S2 are established multiple schedule. The left key was always illumi¬
during discrimination training. Although this analy¬ nated With a white diamond, Responses on the right
sis was limited to positive reinforcement, similar re¬ key were reinforced in the presence of 550 nm and
sults are predicted for responding maintained by extinguished in the presence of 559 nm. On the left
negative reinforcement. The extent to which these key the contingencies were reversed. Responses on the
variables determine the peak shift and inhibitory left key were reinforced in the presence of 559 nm
stimulus control awaits the results of further para¬ (when it appeared on the right key) and extinguished
metric research, but a glance at Table 1 indicates a in the presence of 550 nm. After discrimination train¬
remarkable consistency in the data from many differ¬ ing, two stimulus-generalization gradients were ob¬
ent laboratories. tained by recording the number of responses to the
left and right keys while the wavelength of the stim=
ulus on the right key was varied.
Concurrent Schedules
The left panel of Figure 17 ?how§ the gradients
In a concurrent schedule, two discriminative stim¬ obtained from responses on the left key, and the
uli are presented at separate locations and responding right panel shows the gradients from the right key.
to each stimulus is reinforced independently. Con¬ The left panel of Figure 17 §how§ a peak shift for
current schedules are ideally suited for experiments
designed to measure the influence of incompatible
responses on the peak shift and inhibitory stimulus
control. Although Spence’s theory of discrimination
learning was designed for the simultaneous paradigm,
one of the ironies of research based on his theory is
that some of the data that best fit Spence’s theory
were obtained from a successive paradigm.
Catania (1969) has developed a theory of inhibition
which removes much of the mystery from this phe¬
nomenon by specifying the experimental procedures
which produce the inhibited and inhibiting events. Fig. 17. (Left) Left-key generalization gradients from individual
Consider the case in which a stimulus is presented on birds trained with VI reinforcement for pecks at the left key
during 559 (S+) and extinguished during 550 (S-). On the left
each of the left and right keys and responding on each
key, a positive peak shift with more responding to the longer
key is reinforced according to a concurrent schedule. wavelengths occurred. (Right) Right-key generalization gradients
According to Catania, reinforcement of a response on from individual birds reinforced for pecks on the right key
during 550 nm (S+) and extinguished during 559 nm (S~). On
the left key inhibits responding on the right key by the right key, a positive peak shift with more responding to the
increasing the probability of a further response on the shorter wavelengths occurred. (From D. Blough, 1973.)
464 STIMULUS CONTROL AND INHIBITORY PROCESSES

each of the birds with the maximum displaced from What is Errorless Learning?

S+ (559 nm) toward a longer wavelength. The right

panel shows a peak shift for each of the birds with The Definition of
the maximum displaced from S+ (550 nm) toward the Errorless Learning
short wavelengths.
In a similar experiment with interdimensional The “error” of errorless learning is the occurrence
training, Catania, Silverman, and Stubbs (1974) ob¬ during S" of the response that is reinforced during
tained an inhibitory gradient on the left key by rein¬ S + . The definition of an error depends upon the cri¬
forcing responses on the right key when S~ was pres¬ teria employed for defining response. Consider the
ent on the left key. Inhibitory processes have also pecking response of the pigeon, which is the typical
been successfully investigated by Beale and Winton response in most experiments in errorless learning.
(1970), Winton and Beale (1971), and Honig et al. This response consists of an orientation toward the
(1972), who measured generalization with a concur¬ key, an approach to the key, and a peck on the key.
rent procedure in which an “advance” or changeover A response is usually defined automatically by a force
response was employed. of about 20 g on the key, and other components of
The experiments of D. Blough (1973) and Catania, the response are typically not recorded. The number
Silverman, and Stubbs (1974) provide excellent sup¬ of errors may be increased by relaxing the criterion for
port for Catania’s theory of inhibition of one response a response to include approach responses which do
by reinforcement of another response. However, it is not contact the key, and the number of errors may be
important to emphasize that in Blough’s experiment decreased by increasing the force of the peck required
the peak shift was obtained when S+ alternated with to activate the key or by restraining the animal from
S~ in a successive discrimination on the same key. It contacting the key. For example, a pigeon which
is interesting to note that the peak shift was not ob¬ made zero pecks on the key could be converted into a
tained in experiments—e.g., Honig (1962, 1967)—in subject which learned with errors by counting near-
which S+ and S~ were presented on different keys. misses, pecks around the key, etc.
When S+ was located on one key and S~ on a differ¬ Ethological analyses using procedures developed
ent key, the probability of the peak shift might have by Staddon and Simmelhag (1971) reveal a number of
decreased as a function of the distance between the potentially interesting responses and response se¬
keys, since this made the comparison of S+ with S~ quences which are modified during errorless discrim¬
more difficult. The advantage of the concurrent pro¬ ination training. These analyses indicate that error¬
cedures, which reinforce a response on the left key less discrimination training procedures are effective
when extinction is in effect on the right key, is that in eliminating the terminal peck response from the
the competing response is brought under experimen¬ sequence of observing and approach responses to S~.
tal control. For example, Wessells (1974, Experiment III) ob¬
tained an errorless discrimination without fading us¬
ing an autoshaping procedure. On half of the trials,
ERRORLESS LEARNING RECONSIDERED a white light, (CS+), on the left key was immediately
followed by food. On the other trials, a green key
The previous sections described the influence of light, (CS~), was not followed by food. Wessells re¬
discrimination training on the shape of the generali¬ corded three key-directed behaviors during each trial:
zation gradient and demonstrated the variables that (1) an orienting response described as “looking at the
have been established as the determinants of the peak key,” (2) an approach response defined as any move¬
shift and inhibitory stimulus control. This section ment toward the key, and (3) the peck at the key.
considers the influence of a special type of discrimina¬ Wessells found that the emergence of the pecking re¬
tion training, “errorless” discrimination training, on sponse was always preceded by an orientation-approach
stimulus control and inhibitory processes. Errorless sequence. During S~~ isolated approach responses in¬
learning refers to the class of discriminations in which creased rapidly during the first few trials of dis¬
the rate of responding to S- approaches zero from the crimination training and then decreased, while during
first session of discrimination training. The major S+ the approach toward the key consistently increased.
reason for the extensive interest in errorless learning The orientation response behaves exactly as the tra¬
is that it is considered an exception to the basic laws ditional theory of discrimination predicts. During S+,
of discrimination learning developed in the previous the orientation response is maintained by reinforce¬
sections. ment, while the response extinguishes during S~.
Mark Rilling 465

In order to obtain more data on the behaviors

which are observed during the acquisition of an error¬
less discrimination. Rilling, Caplan, and Brown's un¬
published study recorded the three responses specified
by Wessells during errorless autoshaping and subse¬
quent sessions of discrimination training in which the
duration of each component was gradually increased
to 1 min. After autoshaping, responses during S+ were
reinforced on a VI 30-sec schedule. For 10 of the 12
birds in the experiment, the following sequence of >
o 10 r
responses emerged during successive S+ trials: an ob¬
serving response appeared earliest, followed by an
$ 5
approach to the key, which was finally followed by a
LlS
pecks on the key. A hierarchy of responses was ob¬ CtL 1 0
served during S~. For each of the birds, the probabil¬ 8 io
tu o s-
ity of an observing response was greater than the sum CO o. g-
z Sl 5
of the approach and key-peck responses during S~~. o
£L
The difference in behavior between S+ and S~ is that </) o
UJ
reliable pecking does not emerge from the approach CL
response during S~. Although many of the birds were 10
errorless with respect to pecks on the key, they were s+
not “errorless” with respect to the response of ap¬ »- 5
ID
X
proaching the key. The responses for one of the error¬
less pigeons are presented in Figure 18.
110 c
Since the definition of an error response during S_
* I s-

A
is arbitrary, it follows that the definition of errorless O 5 (■
learning is also arbitrary. Researchers of errorless
learning tend to differentiate cases of learning with¬ 20 30 40 50
out errors from cases of learning with errors based
upon the absolute number of errors produced through¬ BLOCKS OF 10 TRIALS
out the experiment. Terrace (1972) described subjects Fig. 18. Number of rappiucs pecking the key, approaching the
exhibiting few errors (on the order of 25 or fewer for key, and 6i*ientmg toward the key during S4- and S~ for bird
5189 during the acquisition of a “errorless diserimmatisn.”
the parameters of his experiment) as performing
During 8^ orienting and approaching the key preceded the
fundamentally the same as subjects which made no emergence of pecking, wbil^ during g- grkming and approach¬
errors at all. Both groups were labeled “errorless," as ing the key were nt>l followed by pecking. (From Rilling, Gap-
lan, & Broivn, unpublished data.)
they appeared distinct from subjects which made sub¬
stantially more nonreinforced responses during S~.
This classification is theoretically useful if the per¬ Errorless Learning as the

formance of those subjects which make zero or a few Transfer or Stimulus Control

errors, however defined, during S- is fundamentally

One of the most potent determinants of the num¬
different from those subjects which make many errors.
ber of errors during the acquisition of a discrimina¬
The data reviewed in this section demonstrate that
tion is the physical similarity between S+ and S~. For
this classification is not useful, because few phenomena
example, when S+ was a green and S~ a dark key,
of discrimination learning depend reliably upon the
Kodera and Rilling (1975) obtained fewer errors than
rate of responding during S~. Therefore, the distinc¬ when S+ was a green and S~ a red key. As Terrace
tion between learning without errors and learning (1973) points out, the failure to respond to the dark
with errors is arbitrary. A more fruitful strategy may key is presumably a transfer from the discrimination
be to isolate those variables that determine the rate training that took place in the animal's environment
of responding during S~ and to develop theories that prior to the experiment. Pigeons are reinforced for
predict these rates. This strategy avoids the embar¬ pecking at brightly colored bits of grain, but are not
rassing question of how many errors are necessary be¬ reinforced for pecking at dark holes. These experi¬
fore errorless learning becomes errorful learning. ments suggest that interdimensional training often
 466 STIMULUS CONTROL AND INHIBITORY PROCESSES

produces fewer errors than intradimensional training alone. In the superimposition and fading procedure,
because of the greater similarity of the stimuli in the the vertical and horizontal lines were superimposed
intradimensional case. upon the chromatic stimuli for five sessions, but the
Therefore, special techniques are necessary to es¬ intensities of the red and green lights were gradually
tablish an errorless discrimination when S+ and S~ reduced within one session until they were no longer
are similar as in intradimensional training. In fading, visible to the human observer. A fourth group re¬
a property of a stimulus is gradually changed on suc¬ ceived discrimination training between the vertical
cessive trials to transfer control of responding from and horizontal lines without pretraining on the dis¬
one property of a stimulus to another. For example, crimination between red and green.
in the experiment which initiated the current interest The results of this experiment are presented in
in errorless learning, Terrace (1963a) first established Figure 19, which shows that the total number of
an errorless discrimination between a red S+ and a errors during S~ depended upon the method of
dark S~ by beginning with a dark key with a dura¬ transfer from the red-green to the vertical-horizontal
tion of 1 sec. In Phase 1, the duration of the dark key discrimination. The superimposition and fading pro¬
was lengthened from 1 to 30 sec. In Phase 2, the dis¬ cedure produced errorless transfer, while the abrupt
crimination was transferred from a dark key, S~, to procedure produced the most errors. This experiment
a green S~ by reducing the duration of the dark key demonstrates that both fading and superimposition
to 1 sec and gradually increasing the intensity of the independently facilitate transfer of stimulus control
chromatic S~ to match that of S+. In Phase 3, the and supports an interpretation of errorless learning
duration of the green S~ was gradually increased as a transfer of stimulus control from one dimension
from 1 to 30 sec to match the duration of S+. to another.
Regrettably, fading remains a part of the art rather
than the science of operant conditioning. Many in¬
vestigators—e.g., Rilling, Kramer, and Richards (1973),
Karpicke and Hearst (1975)—have obtained more er¬
rors than Terrace reported in attempts to obtain er¬
rorless discriminations. The parameters of fading
which are necessary for errorless learning remain un¬
</>
investigated. How rapidly should the intensity and
duration of S~ be increased in order to obtain opti¬
mal errorless discrimination learning? Furthermore,
tf)
the effectiveness of fading is rarely compared with z
o
appropriate control conditions in which fading is not CL
WO

employed. The procedural variables which are re¬

sponsible for errorless discrimination learning have O
been neglected in the rush to compare errorless learn¬ QL
LlJ
ing with learning with errors. 0D
2
3
In a second experiment Terrace (1963b) compared Z
three procedures for transferring stimulus control -J
<
l-
from the wavelength dimension to the line-orientation O
dimension. In the first phase of the experiment, the
pigeons acquired an errorless discrimination between
red (S+) and green (S~), and in the second phase the
discrimination was between a Vertical line (S+) and a
horizontal line (S~). Transfer was accomplished by
several procedures. In the abrupt procedure, the red
stimulus on the S+ trials was replaced with a vertical VERTICAL - HORIZONTAL DISCRIMINATION
line and the green stimulus on the S~ trials was re¬
Fig. 19. The number of errors made by each bird during the
placed with a horizontal line. In the superimposition- acquisition of a vertical-horizontal discrimination following
only procedure, the vertical line was superimposed on errorless red-green discrimination training. Errors during a
red and the horizontal line was superimposed on second series of red-green discrimination sessions are also indi¬
cated. (From Terrace, 1963b. © 1963 by the Society for the
green for five sessions before the lines were presented Experimental Analysis of Behavior, Inc.)
Mark Rilling 467

Terrace’s Theory of acquisition of a higher rate of responding to S+ than

Errorless Learning to S~. For convenience, Terrace (1972) has classified
other behavioral phenomena which occur during the
Errorless learning has been observed occasionally acquisition of a discrimination as by-products of dis¬
for many years, but has remained little more than a crimination learning. The by-products of discrimina¬
laboratory curiosity and incidental observation. For tion learning include behavioral contrast, emotional
example, Skinner (1938, pp. 203-206) established a and aggressive behavior during S~, aversive proper¬
brightness discrimination in rats without errors by ties of S~, inhibitory properties of S~, and the posi¬
introducing S— during the first session of training. tive and negative peak shift. Terrace (1972) set out to
Errorless learning was also observed by Schlosberg assess whether each of the by-products had similar
and Solomon (1943). underlying mechanisms which would be the case if
The popularization of programmed instruction by they covaried as a function of the same variables.
Skinner in the late fifties and early sixties led to in¬ Terrace's theoretical position i$ simply stated; “None
tense interest in errorless learning. Skinner argued of [the by-products of discrimination learning] occurs
that one of the major advantages of programmed in¬ following discrimination learning without errors” (p.
struction was that the student is seldom wrong. In 251). This is reiterated in each of his papers on error¬
1961, he eloquently advocated training procedures less learning (Terrace, 1963a, 1963b, 1964, 1966a,
which produce as few errors as possible. This was 1966b, 1966c, 1968, 1972, 1973). As the review of the
accomplished through stimulus generalization by literature m tins section will show, each of the by¬
changing each successive question only slightly from products of discrimination learning, except the peak
one frame to the next. Although Skinner offered no shift, has been obtained following errorless discrimi¬
data, errors presumably had aversive consequences nation learning. Therefore, a revision of Terrace’s
which might lead to escape behavior. Skinner’s theory theory is required.
that optimal learning is accomplished by maximizing Rilling and his students (Kodera k Rilling, 1973;
positive reinforcement and minimizing aversive con¬ Rilling k Gaplan, 1973, 1975; Rilling, Gaplan, How¬
sequences forms a background for the theory of error¬ ard, & Brown, 1975; Rilling, Kramer, & Richards,
less learning developed by Terrace. 1973) have carried out a series of experiments which
Most of the research in errorless learning is not demonstrate that the by-products of discrimination
concerned with the transfer of stimulus control. learning bear little relationship to the occurrence or
Rather, it is concerned with testing Terrace’s theory nonoccurrence of errors during S~. These experi¬
that the phenomena of errorless learning are funda¬ ments suggest that the parameters which most read¬
mentally different from the phenomena obtained ily produce one by-product—e,g„ extinction-induced
when the discrimination is acquired with errors. aggression—may not produce another—e.g., behavioral
Terrace has consistently argued that errorless learn¬ contrast. Only the peak shift and inhibitory stimulus
ing is fundamentally different from learning with control appear to have identical underlying mech¬
errors. In discussing the results of his original experi¬ anisms. Therefore, the theory which explains behav¬
ment, Terrace (1963a) wrote, “Discriminations ac¬ ioral contrast may differ from the theory that explains
quired with zero, or a near zero, number of responses extinction-induced aggression, and so on. It follows
to S, can be clearly distinguished from discrimina¬ that the best strategy for research is to identify those
tions acquired with large amounts of responding to variables which produce each behavioral phenomenon
S- by criteria other than the number of acquisition and to develop separate theories for each. Whether a
responses to S-” (p. 23). The theory was subsequently behavioral process is classified as a by-product or not
refined by Terrace (1966b) to specify that S~ functions is irrelevant for the task of isolating its determinants.
as a neutral stimulus following errorless discrimina¬
tion learning, while S~ functions as an aversive and
What Is Learned in Errorless Learning?
inhibitory stimulus following learning with errors. A
similar distinction between the two different types of This section demonstrates how errorless learning
discrimination learning is maintained in the most modifies the behavior of the organism. The experi¬
recent version of this theory (Terrace, 1972). ments were designed to determine if it is necessary to
Terrace distinguishes the product of discrimination modify traditional theories of discrimination learn¬
learning from the by-product of discrimination learn¬ ing for the errorless case. The procedures employed
ing. The product of differential reinforcement is the are: (1) observations of aggressive behavior, (2) escape
468 STIMULUS CONTROL AND INHIBITORY PROCESSES

from S-, and (3) assessment of inhibitory stimulus tects the final component of the attack sequence.
control. Rilling and Caplan (1973) photographed some of the
species-specific precursors of the attack response, flight
Aggressive Behavior During behaviors, and the actual attack response for two birds
Errorless Learning that were errorless throughout the experiment. For
pigeons, the first response in the sequence is bowing,
Azrin, Hutchinson, and Hake (1966) developed the which frequently precedes attack. In bowing, follow¬
standard procedure for measuring extinction-induced ing erection of the head and body, the bird ruffles the
aggression between pigeons in the experimental lab¬ feathers of the neck and bows the head coward the
oratory. They alternated periods in which each re¬ ground while walking in circles and emitting cooing
sponse was reinforced with periods of extinction. In calls. The second response, illustrated in part A of
addition to the experimental pigeon, a second, par¬ Figure 20, is attack intention in which, while stand¬
tially restrained pigeon was also present at the rear ing upright, the bird raises the feathers of its neck
of the experimental chamber* Azrin et al. found that and pecks in an open space in front of its opponent
the probability of attack against the target pigeon while vibrating its wings. The final response is attack
wae low when the opportunity to eat was available. itself. Part B shows an attack response for bird 5. Part
However, when the opportunity for reinforcement C shows an attack response by bird 1. Part D shows
was withdrawn, the probability of aggression was bird 5 immediately after an attack terminated. While
high. The duration of attack was a direct function of these preattack behaviors are difficult to measure au¬
the number of food reinforcements and decreased as tomatically, they are reliably obtained during an er¬
a function of the time since the withdrawal of the rorless S_ in the presence of a target pigeon.
opportunity to eat. With a high degree of consis¬
tency, they found that attack occurs at the moment of
transition from reinforcement to extinction, which
led them to suggest that the interruption of eating
produced the aggression.
In the typical procedure for measuring aggres¬
sion during extinction, a substantial number of re¬
sponses on the key are observed. This raises the ques¬
tion of whether the aggression is produced from the
frustration which occurs as a by-product of the non-
reinforced responses, as Terrace argued (1966c, 1971,
1972), or by the withdrawal of the opportunity for
reinforcement. If aggression occurred during an error¬
less S-, then nonreinforced responses are not neces¬
sary for this species-specific behavior. This would
argue instead that withdrawal of reinforcement is the
crucial determinant of aggression. In order to obtain
quantitative data on the phenomenon of extinction-
induced aggression during errorless learning, Rilling
and Caplan (1973) trained pigeons to discriminate
without errors between a green light as S+ and a dark
key as S_. The opportunity to attack a restrained tar¬
get bird was also present. During discrimination train¬
ing, the rate of attack in the presence of the dark key
was higher for each animal than the operant level,
even though most of the animals acquired the dis¬
crimination without errors. Furthermore, the rate of
attack did not decrease during 45 sessions of discrimi¬
Fig. 20. Photographs taken during S- of two pigeons that ac¬
nation training. These data demonstrate that attack quired a discrimination without errors. Section A illustrates
during S_ also occurs during errorless discrimination attack intention and B illustrates attack for bird 5. Section C
training and fails to confirm Terrace’s theory. shows attack for bird 1 and D shows bird 1 shortly after an
attack response. (From Rilling & Caplan, 1973. © 1973 by the So¬
The procedure developed by Azrin et al. only de¬ ciety for the Experimental Analysis of Behavior, Inc.)
Mark Rilling 469

The data of Azrin et al. (1966) and Rilling and race’s (1963) original experiment, the groups differed
Caplan (1973) suggest that the withdrawal of the with respect to the procedure used to introduce S~:
opportunity for reinforcement is one of the primary early-progressive, early-constant, late-progressive, and
determinants of extinction-induced aggression. The late-constant. The aversive properties of S~ were mea¬
probability of attack is highest immediately following sured using the escape procedure of Rilling et al.
the termination of S+ and decreases thereafter. Fur¬ (1969) in which a single peck at a second key termi¬
thermore, high rates of attack during extinction oc¬ nated S~ for 10 sec and darkened the chamber.
curred only during those sessions in which S+ and S“ Figure 21 shows the number of errors and time¬
alternated, but did not occur in sessions in which S~ outs for each animal in each of the four conditions.
was presented alone. Extinction per se did not induce The data in the upper panel are ordered from left to
aggression. This suggests that the less the incentive right by ranking the animals from least to most er¬
contrast between S+ and S~, the less the amount of rors. The lower panel shows the number of time-outs
aggression obtained. In a subsequent experiment. for the corresponding bird in the upper panel. The
Rilling and Caplan (1975) found that the frequency procedures for introducing S~ had a significant effect
of reinforcement during S+ was a determinant of on the number of responses to 5~ during the acquisi¬
extinction-induced aggression during errorless dis¬ tion of the discrimination. The constant procedure
crimination learning. A VI 30-sec schedule induced a
higher rate of attack during extinction than a VI
5-min schedule.
The results of these experiments demonstrate that
the aggression-inducing properties of S“ are not pri
marily due to the contingencies of reinforcement pre¬
vailing during S~, but are a contrast effect determined
by the contingencies prevailing during S+.

Escape from S-

Rilling, Askew, Ahlskog, and Kramer (1969) con

ducted a series of experiments which demonstrated
that an escape paradigm can be used to detect the
aversive properties of 5“ in a successive discrimina¬
tion. In their procedure, a successive discrimination
was programmed on one key. A peck on a second key
produced a time-out which terminated 5“ or S+ and
darkened the chamber. During the acquisition of the
discrimination, time-outs occurred during S~. The
probability of a time-out was highest early in S~, a
finding which paralleled the occurrence of attack
behavior in a similar situation (Azrin, Hutchinson, 8c
Hake, 1966; Rilling & Caplan, 1973). These data sup¬
port the assertion that the time-out response is an
escape response and an index of the aversive proper¬
ties of the stimuli present when the response occurs.
One of Skinner’s arguments in favor of learning
without errors is that errors make the situation so
aversive that the learner tends to escape from the
environment in which learning is supposed to occur.
Rilling, Kramer, and Richards (1973) designed an Fig. 21. (Upper panel) Total number of responses to S“ for
experiment to test directly Skinner’s interpretation each animal during all sessions of discrimination training.
(Lower panel) Total number of time-outs from S~ during the
of errorless learning using the escape procedure of 10 final sessions of discrimination training. The arrows between
Rilling et al. (1969). In a successive discrimination panels indicate the mean for each group. Note the lack of
four groups of pigeons were trained to discriminate correlation between the number of responses to S~ and the
number of time-outs from S~. (Rilling, Kramer, & Richards,
between red and green. Following the design of Ter¬ 1973.)
470 STIMULUS CONTROL AND INHIBITORY PROCESSES

produced more errors than the progressive procedure. tests, Wessells (1973) demonstrated that an errorless
The procedures for introducing S- also had a signifi¬ S~ was inhibitory.
cant effect on the number of time-outs from S~~. How¬ Wessells established errorless learning with an au¬
ever, here late introduction of S~ produced more toshaping paradigm. On a CS+ trial, the key was
time-outs from S~ than early introduction of S“. illuminated with a green light for 6 sec and always
In interpreting the results of an experiment in followed immediately by food independently of the
which pigeons escaped from S_, Terrace (1971) con¬ pigeon’s behavior. On a CS~ trial, the key was illumi¬
cluded that “the occurrence of nonreinforced respond¬ nated with a white vertical line on a black back¬
ing to S~ is the crucial factor in rendering S~ aver¬ ground which was never followed by food. Half of the
sive.” Therefore, Terrace predicts that a positive trials were CS+ trials and half were CS~ trials. The
correlation is obtained between the number of errors key was dark between trials. During Phase 1, pecking
and an index of the aversiveness of S~. Figure 21 on the key emerged during CS+ for each bird, while
demonstrates that, within each condition, the number the birds were errorless during CS~ using a criterion
of responses to S~ is a poor predictor of the number of 25 or fewer errors. The tests for inhibition were
of time-outs from S~. To the extent that time-outs carried out during the second phase of the experi¬
from S~~ are an index of the aversive properties of ment.
S~, these data do not support the view that the aver¬ Four groups were employed in Wessells’s experi¬
siveness of a stimulus is directly proportional to the ment. Group 1 received 80 presentations each of CS +
number of unreinforced responses emitted in its pres¬ and CS- in Phase 1, while Group 2 received 200
ence. presentations each of CS+ and CS~. In Phase 2,
Groups 1 and 2 received a test in which the resistance
to autoshaping of the former CS~ was measured. On
Inhibitory Stimulus Control half of the trials, the former CS- was presented on
the left key while a novel stimulus was presented on
Independent tests for inhibition. One of the first the right key on the other trials. For each bird, a peck
experiments to suggest that an errorless S— is inhibi¬ at the key with the novel stimulus emerged earlier
tory was an experiment of Marsh and Johnson (1968). than pecks at the key with the former CS~. The
In the first phase of the experiment, an errorless succes¬ amount of inhibition was a function of the number
sive discrimination between red and green was estab¬ of conditioning trials: the birds with 400 differential
lished with a fading procedure. In the second phase conditioning trials showed more inhibition than the
of the experiment, the reinforcement contingencies groups receiving only 160 trials.
for S+ and S~ were reversed, so that the previously Group 3 also received 400 differential conditioning
positive stimulus was extinguished and the previously trials followed by a combined-cues test in which CS~,
negative stimulus was reinforced. Four of the five sub¬ the white line, was superimposed upon CS+, the green
jects did not respond to S+ (the former S_) more background. Group 4 was a control group for the
than once during five days of reversal training. A con¬ possible unconditioned suppressive effects of the white
trol group that did not receive errorless training line. Combining the CS— with the CS+ completely
acquired the reversal rapidly. Therefore, a history of suppressed responding for the birds that received
errorless learning appears to retard the detection of errorless differential conditioning. In contrast, com¬
changes in the reinforcement contingencies. As com¬ bining the novel white line with CS+ produced only
pared to a subject that has acquired a discrimination a slight decrement in responding. The results of Wes¬
with errors, the errorless bird is at a relative disad¬ sells’s experiment are a convincing demonstration
vantage in coping with an environment in which the that an errorless S~ acquires inhibitory properties. A
conditions of reinforcement change. negative contingency between the CS~ and the un¬
As mentioned earlier, Hearst, Besley, and Farthing conditioned stimulus (food), so that the CS~ predicts
(1970) argue that an independent test is necessary to the absence of food, appears sufficient to account for
demonstrate that a particular stimulus is inhibitory. the development of conditioned inhibition (see
Two procedures have been developed for such tests. Rescorla, 1969a).
In the combined-cues test, S~ is an inhibitory stimu¬ Using an errorless autoshaping procedure similar
lus if its combination with S+ produces a decrement to that employed by Wessells (1973), Wilkie and
in responding. In the resistance-to-reinforcement test, Ramer (1974) found that an errorless S- is more resis¬
S“ is an inhibitory stimulus if the acquisition of a tant to autoshaping than an S~ with errors. Therefore,
conditioned response is retarded. Using both of these as measured by the resistance-to-reinforcement pro-
Mark Rilling
471

cedure, an errorless S~ is more inhibitory than an S~ with the highest rates of responding to S~. These
with errors. Clearly, an errorless S~ is not a neutral data led Terrace to conclude that inhibitory stimulus
stimulus, as was first proposed by Terrace (1966). control was absent following errorless learning.
These experiments have been extensively criticized
Generalization following errorless intradimensional by subsequent investigators. Bernheim (1968) pointed
training. The evidence that S“~ remains a neutral out that the post hoc method of dividing the subjects
stimulus following errorless learning is based on two into groups with and without errors biased the error¬
experiments of Terrace (1964, 1966) in which a less group toward pigeons that did not respond to
stimulus-generalization test was administered follow¬ wavelength and therefore virtually guaranteed a zero
ing errorless discrimination learning. In one of the base line. Terrace attempted to meet this criticism in
most widely cited experiments. Terrace (1964) estab¬ a second experiment (Terrace, 1972) in which a fad¬
lished an errorless intradimensional discrimination ing procedure was used to train the discrimination
with an early-progressive procedure. S+ was 540 nm without errors. As was the case in the first expert-
and S- was 580 nm. During a generalization test ment, the gradients obtained from the errorless group
which was administered following 14 days of discrimi¬ were flat. Birds that acquired the discrimination with¬
nation training, the peak of the gradient for all three out the fading procedure responded to S- and demon¬
errorless birds occurred at S + . When a late-constant strated inhibitory gradients during the stimulus-gen¬
procedure was employed, a peak shift was obtained eralization test.
for two of the three subjects. These data led Terrace A second criticism that applies equally to both
to conclude that S- is a neutral stimulus when the experiments is that virtually no responses were emit¬
discrimination is acquired without errors. While Ter ted to each test stimulus by the errorless birds. Deutsch
race attributed the difference between the groups to (1967)and Hearst, Beslcy* and Farthing (1970, p. 388)
the number of errors during S~, an equally plausible noted that the assessment of stimulus control by the
interpretation is that the probability of obtaining the S~ dimension is ambiguous when stimulus values far
peak shift depends upon the procedure for introduc¬ from S- produce roro responding, because a "floor
ing S“. effect" prevents the detection of a minimum in re¬
The peak shift is the only phenomenon of stimulus sponding at S~. Hearst et al. pointed out that the
control which has not yet been obtained following assessment of stimuli? central by the S dimension
errorless discrimination training. In view of the evi¬ requires an elevation in the overall level of respond¬
dence cited above that S functions as an inhibitory ing to each of the stimuli during the stimulus-gener¬
stimulus following errorless learning, it seems likely alization test. If Terrace’s interpretation is correct*
that the peak shift might occur following intradimen- the gradient should remain flat when elevated.
sional errorless learning. The problem is to select In an experiment designed to verify the Hearst ei
values of S+ and S~ which are spaced closely enough ah interpretation of Terrace’s experiment, Rilling,
to permit the observation of the peak shift yet which Caplan, Howard, and Brown (1975) compared the
do not preclude errorless learning. resistance-to-reinforcement and combined-cues tech¬
Generalization Following Errorless Interdimen- niques for elevating responding to the test stimuli
sional Training. When a stimulus=generalization test from the dimension. Discrimination training be¬
is administered following mterdimemional di§crimi- gan with autoshaping to increase the probability of
nation learning, an inhibitory gradient is usually errorless learning. The positive stimulus was a green
obtained around S~. Two experiments of Terrace key light that was followed immediately by food after
(1966b, 1972) are apparently exceptions to this gen¬ 8 sec, while the negative stimulus was a black vertical
eralization. In the first experiment, a single group of or horizontal line that was never immediately fol¬
birds were trained to discriminate between a white lowed by food. Autoshaping was followed by eight
vertical line (S+) and a wavelength of 580 nm (S~) days of successive discrimination training during
using traditional discrimination-training procedures. which responding to S+ was reinforced on a VI sched¬
Substantial individual differences in the number of ule. After identical conditions of discrimination train¬
responses to S~ were obtained. Flat gradients with ing, three different types of generalization tests were
virtually zero responses to each test stimulus were ob¬ employed: resistance to extinction with compound¬
tained for those subjects that acquired the discrimina¬ ing, resistance to reinforcement without compounding,
tion with the lowest rates of responding to S—. Inhibi¬ and resistance to reinforcement with compounding.
tory gradients with a minimum at S- were obtained In the compounding tests, various line angles were
for those subjects that acquired the discrimination presented on a green background. In the tests without
 472 STIMULUS CONTROL AND INHIBITORY PROCESSES

compounding, these same line angles were presented test, because compounding produced responses early
without the green background. Terrace’s experiments in generalization that were reinforced on the VI
employed resistance to extinction without compound¬ schedule. A flat gradient above the zero base line was
ing. obtained for the experimental group that was tested
Figure 22 presents the average number of responses during extinction with the compound stimuli, in¬
to each of the test stimuli during days 1-5 of gen¬ dicating that inhibitory stimulus control by the angle
eralization testing for each of the groups. An inhibi¬ of the line was not obtained for this condition of
tory gradient with a minimum at S- was obtained testing.
for the two groups tested with the resistance-to-rein- These results emphasize that flat gradients with
forcement procedure. Therefore, stimulus control was zero responses to each test stimulus, such as those
acquired by the S~ dimension even though the dis¬ obtained by Terrace (1966, 1972), are an equivocal
crimination was acquired with a very low rate of outcome. Deutsch (1967) and Hearst et al. (1970)
responding to S~. More responses were obtained dur¬ were correct in their initial criticism of Terrace’s ex¬
ing the compound test than during the noncompound periment that a “floor effect” prevented the detection
of inhibitory stimulus, control. A rate of responding
to the test stimuli greater than zero is necessary for
the assessment of inhibitory stimulus control. In the
Rilling et al. experiment, the resistance-to-reinforce-
ment procedure was the more sensitive index of in¬
hibitory stimulus control. Similar results have been
obtained by Karpicke and Hearst (1975).

Behavioral Contrast

The phenomenon of behavioral contrast receives

particular emphasis here, as its investigation (Terrace,
1963a) formed the basis for Terrace’s theory of error¬
less discrimination learning.
Fhe four training groups—early-progressive, early-
constant, late-progressive, and late-constant—differed
considerably in terms of the number of responses
emitted to S~, with the subjects of the early-progres¬
sive group displaying errorless or nearly errorless
performance. Substantially more errors to S- were
observed for the other three groups. Terrace subse¬
quently attributed special significance to the distinc¬
tion between discrimination learning with and with¬
out errors based on other differences between early-
progressive birds and the others. The most basic of
these differences was the absence of behavioral con¬
trast in the early-progressive group. From this observa¬
tion, Terrace (1972) concluded that behavioral con¬
trast does not occur if the discrimination is acquired
without errors.
In contrast with Terrace’s interpretation, Reynolds
(1961), Friedman and Guttman (1965), Taus and
Hearst (1970), Vieth and Rilling (1972), and Sadowsky
(1973) obtained behavioral contrast from pigeons
when a blackout, during which the chamber was
Fig. 22. Average rate of responding for the experimental and completely dark, was employed as S~. While the num¬
control groups to each of the test stimuli during days 1-5 of
ber of errors during S~ was not always recorded in
generalization testing. (From Rilling, Caplan, Howard, 8c Brown,
1975. © 1975 by the Society for the Experimental Analysis of these experiments, the blackout presumably func¬
Behavior, Inc.) tioned as an errorless S~, since pigeons readily dis-
Mark Rilling 473

criminate a blackout from an illuminated key as¬

S+ s+ls- 1 S+ ONLY S+IS- 2 S+0NLY s+ls^TI
sociated with reinforcement. These data suggest that
1 EARLY PROGRESSIVE GROUP
nonreinforced responding to S~ plays a minor role in 80 1
. *****
the production of behavioral contrast.
w 70 A**
Since the early-progressive group failed to display j| 60
behavioral contrast in Terrace’s experiment, an alter¬ s 50 -*.-*•*
CE ' /.Mill
native interpretation, which logically must be enter¬ LxJ 1
till l/.l 1 1 1 1 1111 i/d 1111 III! L1 1 1 1 nni^miY

tained, is that some aspect of this procedure pre¬

O- L EARLY CONSTANT GROUP
a so
cluded the observation of contrast. In Terrace’s CD
-W
g ™
(1963a) design, it is impossible to partition the effects Q_
co $0
LU ***** *V»s*»*
on behavioral contrast of learning the discrimination
£ 50;
with or without errors from the immediate effects of -Ad 1 II 1 ii 11 i/d 111 i LLLLL/iLLLLi 1 11 1 UA111 1 \\\\\.A\\\\f(
co I LATE PROGRESSIVE GROUP
the specific training procedure employed. Conse¬ o
quently, Kodera and Rilling (1975) systematically 90
replicated Terrace’s original experiment in errorless * 80
Q
learning. However, a dark key was used as S~ with a § 70
•v
green key as S+, rather than the red and green stim¬ £3 60 *«**•

uli used by Terrace. This substitution of S~ stimuli E 50

produced errorless acquisition of the discrimination
o ft 1111 \,f\ I I I I n
e r LATE CONSTANT GROUP
g 100 " -
in many pigeons from groups in addition to those re¬
=» 90 _
ceiving early-progressive discrimination training.
w 90 - y _
At the completion of the experiment, all eight
birds of the early-progressive group were still errorless, 7Q - -

using a criterion of 25 or fewer total responses to S~ 60 - -

sn r~r*
for classifying a bird as errorless. Six of the eight UJJLLLL Ml 1 hi Mill IIII hi 11 11 Mill,Jllll 1 1 1 1 I.J 1 1 1 1 i
pigeons in the early-constant group were errorless, SESSIONS
three of the late-progressive group, and four of the rig. g$, Mean rate of responding to Sf for the early-progressive,
late-constant group. These data clearly confirm Ter¬ £§.¥ly-£<Snstant, late-progressive, and late-constant training groups.

Because the early introduction of S~ precluded an initial phase

race’s finding that the early introduction of S was
°f bass line training, the data for the early groups are displaced
very effective in reducing the number of errors during to the ri ght to place them in phase with the data for the late

discrimination training. The progressive groups dif¬ groups. In the top column, S+ refers to the baseline; S+|S“ 1,
S+|S“ 2, and S+|S~ 3 refer respectively to the first, second.
fered from the constant groups in errors only during and third phases of discrimination training. All compar|suj]g yf

the first five days of discrimination training when S~ the magnitude of behavioral contrast are based upon this con¬
was faded in for the progressive groups. The progres¬ figuration of the data. (From Kodera & Rilling, 1975. © 1975 by
the Society for the Experimental Analysis of Behavior, Inc.)
sive groups made significantly fewer total errors dur¬
ing the first five days of discrimination training than
did the constant groups. During the first five days of more contrast than did the progressive group’s S~
discrimination training, the early-progressive group introduction.
emitted fewer responses to S~ than did the early- In his original experiment, Terrace (1963a) ob¬
constant group. served a close relationship between the occurrence of
Figure 23 shows the mean daily rates of responding errors during S~ and behavioral contrast during S +
to S+ throughout the various phases for each of the and went on to argue that behavioral contrast was
four groups. Of primary importance was the observa¬ produced by nonreinforced responding during S~.
tion that behavioral contrast occurred in all four This interpretation of behavioral contrast predicts a
groups. The increase in rate was, in general, slightly positive correlation between the number of errors dur¬
greater following a transition from base line to dis¬ ing S~ and the magnitude of behavioral contrast dur¬
crimination than was the subsequent decrease follow¬ ing S+.
ing the opposite transition. Figure 23 also reveals a Figure 24 compares the number of responses dur¬
trend for the late groups to exceed the early groups ing S- (upper panel) with the magnitude of behav¬
in the mean amount of behavioral contrast produced. ioral contrast during S+ (lower panel). The left
While time of S— introduction was a more powerful panels compare the number of errors during the first
variable than the method of introducing S~, the con¬ phase of discrimination training with the amount of
stant group’s introduction of S~ tended to produce behavioral contrast exhibited during the first discrim-
474 STIMULUS CONTROL AND INHIBITORY PROCESSES

of behavioral contrast are indicated in Figure 24 for

each group. None of these correlations was significant.
In other aspects the Kodera and Rilling experi¬
ment confirms the results obtained in Terrace's ex¬
periment. The major discrepancy between the two
studies is the occurrence of behavioral contrast fol¬
lowing early-progressive training in the Kodera and
Rilling study. Another important difference is that
many of the birds in the early-constant, late-progres¬
sive, and late-constant groups were also errorless, yet
demonstrated behavioral contrast. Which differences
in procedure are responsible for Terrace’s failure to
observe behavioral contrast in the early-progressive
group? His original experiment included a red key
as S+ and a green key as S~. Behavioral contrast is
widely assumed to be independent of whether the
stimuli used as S+ and S~ lie on the same or different
dimensions, since contrast is obtained after both inter-
and intradimensional training. Using a separate group
of pigeons trained on the early-progressive procedure
of the main experiment, Kodera and Rilling substi¬
tuted a red S~ for the dark key. The mean magnitude
of behavioral contrast was 23.2 responses per min for
Fig. 24. Relationships between responding during S~ (errors)
and the magnitude of behavioral contrast produced. The bars
the red group as compared with 15.3 responses per
within each panel are ordered with respect to the total number min for the group with a dark key as S~. These re¬
of errors produced during S+|S~ 1 (the first discrimination
sults suggest that the magnitude of behavioral contrast
phase). Arrows within each panel mark the group mean for
that measure. The left half of this figure depicts the relation¬
is greater following intradimensional training than
ship between the number of errors made during S+|S“ 1 and following interdimensional training. In any event,
the amount of behavioral contrast observed for each pigeon. The
Terrace’s use of a red S_ did not preclude the occur¬
right half of the figure presents the relationship between the
total number of errors occurring during the first two phases of
rence of contrast.
discrimination training and the mean behavioral contrast pro¬ A more significant difference between the two stud¬
duced overall. Spearman rank-order correlation coefficients for ies was Terrace’s use of 3-min components during S +
each group are indicated by the numbers appearing just below
the error data. (From Kodera & Rilling. 1975. © 1975 by the So¬ and S~ while Rilling and Kodera used 1-min com¬
ciety for the Experimental Analysis of Behavior, Inc.) ponents. Using a group of pigeons for whom the dura¬
tion of S+ was 3 min, which also decreased the
ination-training phase. The right panels relate the number of alternations between S+ and S~ from 25
total number of errors made throughout the experi¬ to 8, the magnitude of behavioral contrast following
ment with the mean behavioral contrast produced by early-progressive training decreased to only 7.8 re¬
each subject. The birds are ordered according to the sponses per min and contrast was not obtained from
number of errors made during the first discrimination each bird. The total duration of S+ remained con¬
phase. The group means are indicated by horizontal stant in these experiments. These data suggest that
arrows. Terrace’s failure to obtain behavioral contrast was
A direct relationship between errors and behav¬ due in part to the use of stimulus components with a
ioral contrast within groups would be represented by duration of 3 min. This is coupled with the fact that
a rank-ordering of the amount of behavioral contrast Terrace’s data were subject to more random daily
from least to most within each of the four conditions, fluctuation than were those of Kodera and Rilling
reflecting the symmetry of the error data. Clearly, no (1975), who obtained more stable response rates
such relationship is visible in the data of Figure 24. through the imposition of response stability require¬
Pigeons that produced the fewest errors in each con¬ ments before instituting changes in training condi¬
dition were as likely to show the greatest amount of tions. What is not clear is whether the attenuation in
contrast as were those that produced the greatest num¬ the behavioral contrast was due to increased duration
ber of errors. Spearman rank-order correlation coeffi¬ of S+ or to the decrease in the number of alternations
cients between the number of errors and the amount between S+ and S~.
Mark Rilling 475

Errorless Learning in Perspective ulus control are also greater when S~ is introduced
abruptly.
Errorless learning is the transfer of stimulus con¬ Frequency of reinforcement during S+ affects be¬
trol from one dimension associated with S~ to an¬ havior during S~. Aggressive behavior during extinc¬
other, provided the transfer is obtained with a zero tion is induced by the withdrawal of the stimulus
rate of responding to the new S— from the first ses¬ associated with reinforcement, rather than by the
sion of transfer. When the pecking response of the change in the consequences of the pecking response
pigeon is reinforced during S+, errorless discrimina¬ during extinction. The higher the frequency of rein¬
tion-training procedures are effective in eliminating forcement during S+, the higher the rate of attack
the terminal peck response from the sequence of ob¬ during S~. Since conditioned inhibition is a contrast
serving and approaching S~. Therefore, the behavior phenomenon produced by the absence of reinforce¬
of the organism during S~ is modified or conditioned ment, this variable may have a similar influence on
by errorless discrimination training. conditioned inhibition and inhibitory stimulus con¬
Errorless learning includes the tail with few errors trol.
of the distribution of total errors for the acquisition A pigeon that has acquired a discrimination with a
of a discrimination. Since the definition of an error procedure in which S is introduced gradually and
response is arbitrary, it follows that the definition of early in training may show “errorless learning” and
errorless learning is also arbitrary. Extensive research less aggression and more escape and behavioral con¬
has demonstrated that the behavior of subjects with trast than a pigeon that has acquired the same dis¬
few errors is not fundamentally different from the be¬ crimination but with a S introduced abruptly and
havior of subjects with many errors, except for the late in training. However, the organism that has ac¬
difference in errors. quired the discrimination without errors is retarded
Escape from S~, aggression during S~, conditioned in detecting changes in the response-reinforcer rela¬
inhibition, inhibitory stimulus control, and behav¬ tionship in the presence of S~ compared t<?
ioral contrast have all been obtained independently organism that mastered the discrimination with er¬
of whether the discrimination was acquired with or rors. When inhibition is measured with a resistance-
without errors. Therefore, a theoretical classification to-reinforcement procedure, more conditioned inhibi¬
based on the distinction between learning with er¬ tion is obtained for the birds that acquired the
rors and learning without errors is not useful, discrimination without errors than for the birds that
By concentrating upon the number of errors, acquired the discrimination with errors. Therefore,
investigators have overlooked key determinants of errorless learning is clearly not the best learning for
the many phenomena produced by differential rein¬ an organism exposed to a changing environment.
forcement. The research of Rilling and his students
demonstrates that the manner of presentation of
conditions during discrimination learning, not the SUMMARY
production of errors, determine the so-called by-prod¬
ucts of discrimination learning. The acquisition of stimulus control in a successive
Two procedural variables that have been identi¬ discrimination occurs as follows. In discrimination
fied are: (1) the time in the subject’s experimental training, certain stimuli predict occasions when a
history at which S~ is introduced (e.g., early or late), class of responses is reinforced, and other stimuli pre¬
and (2) the rapidity with which S~ is introduced (e.g., dict occasions when those responses are not reinforced
progressively or abruptly). The data indicate that S~ or when they are reinforced according to a different
is more aversive, as measured by escape behavior (and program. By definition, stimulus control requires
more behavioral contrast is observed during S+), different rates or patterns of responding in the pres¬
when S- is introduced late in training. While data ence of each stimulus. Following the acquisition of a
are not yet available, the previous findings suggest discrimination, a test is necessary to determine which
that conditioned inhibition and inhibitory stimulus aspects of the training stimuli have acquired control
control are also greater when S~ is introduced late in over responding. The stimulus-generalization gradient
training. The time of S~ introduction is a more pow¬ is the primary index of stimulus control. Gradients
erful variable than the method of introducing S~. are obtained by extinction or steady-state methods in
Abrupt introduction of S~ produces more behavioral which the stimuli are presented singly or simultane¬
contrast than gradual introduction. This analysis sug¬ ously.
gests that conditioned inhibition and inhibitory stim¬ In intradimensional training, the interaction be-
 476 STIMULUS CONTROL AND INHIBITORY PROCESSES

tween reinforcement at S+ and extinction at S~ is extinction. Such theories are an inadequate oversimpli¬
reflected in the stimulus-generalization gradient as the fication, since an experimental analysis demonstrates
positive and negative peak shifts. These processes are multiple determinants of inhibitory phenomena.
separated in interdimensional training in which an Spence’s theory of discrimination learning provides a
excitatory gradient is obtained around S+ and an in¬ simple integrated account of the peak shift and in¬
hibitory gradient is obtained around S~. Following hibitory stimulus control. However, Spence’s theory
interdimensional training, the normal distribution fails to account for the effects of conditioning con¬
curve provides the best fit to empirically obtained text, the relativity of inhibitory stimulus control, the
gradients of excitation and inhibition. For the vari¬ effects of transitions between S“ and S+, and en¬
ables that have been investigated to date, the deter¬ hanced responding within a spatially defined dimen¬
minants of the peak shift and the inhibitory gradient sion.
are identical. The time is ripe for the development of new quan¬
A functional analysis reveals a wide range of vari¬ titative theories that integrate the extensive data re¬
ables that determine inhibitory stimulus control. viewed in this chapter. The extension to operant
Once the reinforced response is acquired, inhibitory behavior of theories of Pavlovian differential condi¬
control by stimuli associated with the absence of rein¬ tioning may lead to a single integrated formulation
forcement may develop rapidly in a matter of min¬ of stimulus control. Many of the questions raised by
utes. Extended training has relatively little influence investigators of human cognition and information
once the necessary and sufficient conditions for in¬ processing are the same questions asked by investi¬
hibitory stimulus control have been established. In¬ gators in stimulus control. The extent to which these
hibitory stimulus control requires relatively brief suc¬ developments are relevant to research on stimulus
cessive alternations of the discriminative stimuli, as control remains to be determined.
opposed to massed presentations, and the presence
during the generalization test of the background stim¬
uli present during training. The determinants of the
peak shift and inhibitory stimulus control are inde¬ REFERENCES
pendent of whether responding is positively or nega¬
tively reinforced.
Azrin, N. H., Hutchinson, R. R., 8c Hake, D. F. Extinc¬
Extinction is not necessary for the acquisition of
tion-induced aggression. Journal of the Experimental
inhibitory stimulus control. Stimuli associated with Analysis of Behavior, 1966, 9, 191-204.
less favorable schedules of reinforcement demonstrate Beale, I. L., 8c Winton, A. S. W. Inhibitory stimulus con¬
some of the same inhibitory phenomena as do stimuli trol in concurrent schedules. Journal of the Experi¬
mental Analysis of Behavior, 1970, 14, 133-137.
associated with extinction. Inhibitory control is a
Bernheim, J. W. Comment. Psychonomic Science, 1968, 11,
relative rather than an absolute property of a dis¬ 327.
criminative stimulus. Blough, D. S. Stimulus generalization as signal detection
In the many cases in which responding is rein¬ in pigeons. Science, 1967, 158, 940-941.
forced differently in the presence of SI and S2, stim¬ Blough, D. S. Generalization gradient shape and summa¬
tion in steady-state tests. Journal of the Experimental
ulus control is dependent upon the combination of
Analysis of Behavior, 1969, 12, 91-104.
the discriminative and incentive properties of the Blough, D. S. Two-way generalization peak shift after two-

stimuli. When the response rates to SI and S2 are key training in the pigeon. Animal Learning and Be¬
identical, the peak shift and inhibitory stimulus con¬ havior, 1973, 1, 171-174.
trol are not obtained. The greater the index of dis¬ Blough, D. S. Steady-state data and a quantitative model of
operant generalization. Journal of Experimental Psy¬
crimination between SI and S2, the higher the
chology: Animal Behavior Process, 1975, 104, 3-21.
probability of obtaining a peak shift. Provided a dis¬ Blough, P. M. Wavelength generalization and discrimina¬
crimination between SI and S2 is established, the tion in the pigeon. Perception and Psychophysics, 1972,
peak shift and inhibitory stimulus control are ob¬ 12, 342-348.
Brown, J. S. Generalization and discrimination. In D. I.
tained even though the frequencies of reinforcement
Mostofsky (Ed.), Stimulus generalization. Stanford: Stan¬
in SI and S2 are identical. Inhibitory phenomena are
ford University Press, 1965.
maximized by large incentive and rate differences Catania, A. C. Concurrent performances: Inhibition of one
between SI and S2. response by reinforcement of another. Journal of the
Some theories of the peak shift and inhibitory stim¬ Experimental Analysis of Behavior, 1969, 12, 731-744.
Catania, A. C., 8c Gill, C. A. Inhibition and behavioral
ulus control have focused upon a single primary
contrast. Psychonomic Science, 1964, 1, 257-258.
determinant such as nonreinforced responding during Catania, A. C., Silverman, P. J., 8c Stubbs, D. A. Concur-
 Mark Rilling
477

rent performances: Stimulus-control gradients during mental issues in associative learning. Halifax: Dalhousie
schedules of signalled and unsignalled concurrent rein¬ University Press, 1969. (b)
forcement. Journal of the Experimental Analysis of Be¬ Hearst, E. Contrast and stimulus generalization following
havior, 1974, 21, 99-108. prolonged discrimination training. Journal of the Ex¬
Collins, J. P. Generalization and decision theory. Doctoral perimental Analysis of Behavior, 1971, 15, 355-363.
dissertation, University of Massachusetts, 1974. Hearst, E. Some persistent problems in the analysis of
Crites, R. J., Harris, R. T., Rosenquist, H., 8c Thomas, conditioned inhibition. In R. A. Boakes & M. S. Halli-
D. R. Response patterning during stimulus generaliza¬ day (Eds.), Inhibition and learning. New York: Acade¬
tion in the rat. Journal of the Experimental Analysis of mic Press, 1972.
Behavior, 1967, 10, 165-168. Hearst, E., Besley, S., 8c Farthing, G. W. Inhibition and
Davis, J. M. Testing for inhibitory stimulus control with the stimulus control of operant behavior. Journal of the
S— superimposed on S+. Journal of the Experimental Experimental Analysis of Behavior, 1970, 14, 373-409.
Analysis of Behavior, 1971, 15, 365-369. Hearst, E., 8c Koresko, M. B. Stimulus generalization and
Deutscli, J. A. Discrimination learning and inhibition. the amount of prior training on variable-interval rein¬
Science, 1967, 156, 988. (Technical comment.) forcement. Journal of Comparative and Physiological
Dukhayyil, A., 8c Lyons, J. E. The effect of over-training Psychology, 1968, 66, 133-138.
on behavioral contrast and the peak shift. Journal of Hearst, E., Koresko, M., & Poppen, R. Stimul us generaliza¬
the Experimental Analysis of Behavior, 1973, 20, 253- tion and the response-reinforcement contingency. Jour¬
263.
nal of the Experimental Analysis of Behavior, 1964, 7,
Dysart, J., Marx, M. H., McLean, J., 8c Nelson, J. A. Peak 369-380.
shift as a function of multiple schedules of reinforce¬ Hebert, D. L., 8c Frantz, J, A. Transposition; A re-evalua-
ment. Journal of the Experimental Analysis of Behavior, tion. Psychological Bulletin, 1965, 6?, 244-257,
1974, 22, 463-470. HEMMES, N. S. Behavioral contrast in pigeons d^p^nds
Ellis,W. R. Role of stimulus sequences in stimulus dis¬ upon the operant. Journal of Comparative and Physio¬
crimination and stimulus generalization. Journal of logical Psychology, 1973, 85, 171-178.
Experimental Psychology, 1970, 85, 155—163. Hinde, R. A., 8c Stevenson-Hinde, J. (Eds.). Constraints on
Farthing, G. W. Behavioral contrast with multiple positive learning. New York: Academic Press, 1973.
and negative stimuli on a continuum. Journal of the R. H„ 8c Thomas, D. R. Stimulus generalization as a
Experimental Analysis of Behavior, 1974, 22, 419-425. function testing procedure and response measure.
Farthing, G. W., 8c Hearst, E. Generalization gradients of Journal of Experimental Psychology, 1963, 61, 587-592.
inhibition after different amounts of training. Journal Hoffman, H. S. The analysis of discriminated avoidance.
of the Experimental Analysis of Behavior, 1968, IE In \Y. K. Honig (Ed.), Operant behavior: Areas of re¬
743-752. search and application. New York: Appleton-Century-
Friedman, H,, 8c Guttman, N. Further analysis of the Crofts, 1966.
various effects of discrimination training on stimulus Honig, W. K. Prediction of preference, transposition, and
generalization gradients. In D. I. Mostofsky (Ed.), Stim¬ transposition-reversal from the generalization gradient.
ulus generalization, Stanford; Stanford University Press, Journal of Experimental Psychology, 1962, 64, 239-248.
1965.
Honig, W. K. Prediction of preference, transposition, and
Giurintano, L. P., Schadler, M., 8c Thomas, D. R. Angu¬
transposition-reversal from the generalization gradients.
larity preferences and stimulus orthogonality in pigeons. In G. A. Kimble (Ed.), Foundations of conditioning and
Psychonomic Science, 1972, 26, 273~275. learning. New York: Appleton-Century-Crofts, 1967.
Guttman, N. Generalization gradients around stimuli asso¬ Honig, W. K , Beale, I., Seracanian, P., Lanber, D., &
ciated with different reinforcement schedules. Journal Mum, D. Stimulus and response reduction: Two aspects
of Experimental Psychology, 1959, 58, 335-340. of inhibitory control in learning. In R. A. Boakes &
Guttman, N. Effects of discrimination formation on gener¬ M. 5. Halliday (Eds.), Inhibition and learning. New
alization measured from the positiye-rate baseline. In York: Academic Press, 1972.
D, Mostofsky (Ed.), Stimulus generalisation, Stanford; Honig, W. K., Boneau, C. A., Burstein, K. R., & Pennv
Stanford University Press, 1965. packer, H. S. Positive and negative generalization gradi¬
Guttman, N., & Kalish, H. I. Discriminability and stimulus ents obtained after equivalent training conditions. Jour¬
generalization. Journal of Experimental Psychology, nal of Comparative and Physiological Psychology, 1963,
1956, 51, 79-88.
56, 111-116.
Mailman, J. P. Spectral pecking preference in gull chicks. Honig, W. K.; Thomas, D. R., 8c Guttman, N. Differential
Journal of Comparative Psychology, 1969, 67, 465-467.
effects of continuous extinction and discrimination train¬
Hanson, H. M. Effects of discrimination training on stim¬
ing on the generalization gradient. Journal of Experi¬
ulus generalization. Journal of Experimental Psychology, mental Psychology, 1959, 58, 145-152.
1959, 58, 321-334.
Huff, R. G., Sherman, J. E., 8c Cohn, M. Some effects of
Hearst, E. Discrimination learning as the summation of
response-independent reinforcement on auditory gener¬
excitation and inhibition. Science, 1968, 162, 1303-1306. alization gradients. Journal of the Experimental Analysis
Hearst, E. Aversive conditioning and external stimulus of Behavior, 1975, 23, 81-86.
control. In B. A. Campbell 8c R. M. Church (Eds.), Hull, C. L. Principles of behavior. New York: Appleton-
Punishment and aversive behavior. New York: Appleton- Century-Crofts, 1943.
Century-Crofts, 1969. (a) Jenkins, H. M. Generalization gradients and the concept of
Hearst, E. Excitation, inhibition, and discrimination learn¬ inhibition. In D. I. Mostofsky (Ed.), Stimulus generaliza¬
ing. In N. J. Mackintosh 8c W. K. Honig (Eds.), Funda¬ tion. Stanford: Stanford University Press, 1965.
 478 STIMULUS CONTROL AND INHIBITORY PROCESSES

Jenkins, H. M., 8c Harrison, R. H. Effect of discrimination Rescorla, R. A. Pavlovian conditioned inhibition. Psycho¬
training on auditory generalization. Journal of Experi¬ logical Bulletin, 1969, 72, 77-94. (a)
mental Psychology, 1960, 59, 246-253. Rescorla, R. A. Conditioned inhibition of fear. In N. J.
Jenkins, H. M., 8c Harrison, R. H. Generalization gradients Mackintosh 8c W. K. Honig (Eds.), Fundamental issues
of inhibition following auditory discrimination learning. in associate learning. Halifax: Dalhousie University
Journal of the Experimental Analysis of Behavior, 1962, Press, 1969. (b)
5, 435-441. Rescorla, R. A., 8c Wagner, A. R. A theory of Pavlovian
Karpicke, J., 8c Hearst, E. Inhibitory control and errorless conditioning: Variations in the effectiveness of reinforce¬
discrimination learning. Journal of the Experimental ment and non-reinforcement. In A. Black 8c W. F. Pro-
Analysis of Behavior, 1975, 23, 76-83. kosy (Eds.), Classical conditioning (Vol. 2). Englewood
Klein, M., 8c Rilling, M. Generalization of free-operant Cliffs, N. J.: Prentice-Hall, Inc., 1972.
avoidance behavior in pigeons. Journal of the Experi¬ Reynolds, G. S. Behavioral contrast. Journal of the Experi¬
mental Analysis of Behavior, 1974, 21, 75-88. mental Analysis of Behavior, 1961, 4, 57-71.
Kodera, T., 8c Rilling, M. Procedural antecedents of be¬ Richards, R. W. Stimulus generalization and delay of
havioral contrast: A reexamination of errorless learning. reinforcement during one component of a multiple
Journal of the Experimental Analysis of Behavior, in schedule. Journal of the Experimental Analysis of Be¬
press. havior, 1973, 19, 303-309.
Lyons, J. Stimulus generalization as a function of discrimi¬
Riley, D. A. Discrimination learning. Boston: Allyn 8c
nation learning with and without errors. Science, 1969, Bacon, 1968.
163, 490-491.
Rilling, M. Effects of timeout on a discrimination between
Lyons, J., & Tliomas, P. R. Effects of interdimensional fixed-ratio schedules. Journal of the Experimental Analy¬
training on stimulus generalization, II: Within-subjects sis of Behavior, 1968, 11, 129-132.
design. Journal of Experimental Psychology, 1967, 75, Rilling, M., Askew, H. R.; Ahlskog, J, E„ 8c Kramer, T.
572-574. J. Aversive properties of the negative stimulus in a suc¬
Malott, K„ Malott, R. W., & Glenn, M. E. Maintaining cessive discrimination. Journal of the Experimental
responding during stimulus generalization testing in Analysis of Behavior, 1969, 12, 917-932.
extinction.Journal of the Experimental Analysis of Rilling, M., &? Budnik, J. E. Generalization of excitation
Behavior, 1973, 19, 199-209. and inhibition after different amounts of training on
Mariner, R, W., 8c Tliomas, D. R. Reinforcement duration an avoidance baseline. Journal of the Experimental
and the peak shift in post-discrimination gradients. Jour¬ Analysis of Behavior, 1975, 23, 207-215.
nal of the Experimental Analysis of Behavior, 1969, 12, RlLLING, M., 8c Caplan, H. J. Extinction-induced aggression
759-766.
during errorless discrimination learning. Journal of the
Marsh, G. Prediction of the peak shift in pigeons from Experimental Analysis of Behavior, 1973, 20, 85-91.
gradients of excitation and inhibition. Journal of Com¬ Rilling, M., 8c Caplan, H. Frequency of reinforcement as
parative and Physiological Psychology, 1972, SI, 262-266.
a determinant of extinction-induced aggression. Journal
MARSH, G., 8c Johnson, R. Discrimination reversal following
of the Experimental Analysis of Behavior, 1975, 23, 121—
learning without “errors.” Psychonomic Science, 1968, 129.
10, 261-262. Rilling, M., Caplan, H., Howard, R., 8c Brown, C. H. In¬
Migler, B. Effects of averaging data during stimulus gen¬ hibitory stimulus control following errorless discrimina¬
eralization. Journal of the Experimental Analysis of tion learning. Journal of the Experimental Analysis of
Behavior, 1964, 7, 303-307. Behavior, 1975, 24, 121-133.
Migler, B., 8c Millenson, J. R. Analysis of response rates Rilling, M., Kramer, T. J., 8c Richards, R. W. Aversive
during stimulus generalization. Journal of the Experi¬
properties of the negative stimulus during learning with
mental Analysis of Behavior, 1969, 12, 81-87.
and without errors. Learning and Motivation, (New
Miller, J. T. The effect of contextual cue change during
York: Academic Press, Inc., 1973) 4, 1-10.
reversal training on post-discrimination generalization Rosen, A. P., 8c Terrace, H. S. On the minimal conditions
gradients. Doctoral dissertation, University of Colorado, for the development of a peak-shift and inhibitory
1972. (University Microfilms No. 73-18, 582)
stimulus control. Journal of the Experimental Analysis
MORGAN, M. J. Book review of animal discrimination learn¬ of Behavior, 1975, 23, 385-414.
ing. In R. M. Gilbert 8c N. S. Sutherland (Eds.), Quar¬ Sadowsky, S. Behavioral contrast with timeout, blackout,
terly journal of experimental psychology, 1969, 21, 291— or extinction as the negative condition. Journal of the
292.
Experimental Analysis of Behavior, 1973, 19, 499-507.
Nunnally, J. C. Psychometric theory. New York: McGraw- Schadler, M., 8c Thomas, D. R. On the acquisition of
Hill, 1967.
dimensional stimulus control by the pigeon. Journal of
Pierrel, R. A. Generalization gradient for auditory inten¬ Comparative and Physiological Psychology, 1972 79
sity in the rat. Journal of the Experimental Analysis of 82-89.
Behavior, 1958, 1, 303-313.
Schlosberg, H., 8c Solomon, R. L. Latency of response in a
Purtle, R. B. Peak shift: A review. Psychological Bulletin,
choice discrimination. Journal of Experimental Psychol¬
1973, 80, 408-421. ogy, 1943, 33, 22-39.
Ray, B. A., 8c Sidman, M. Reinforcement and stimulus Schwartzbaum, J. S., 8c Kellicut, M. H. Inverted gen¬
control. In W. N. Schoenfeld (Ed.), The theory of rein¬
eralization gradients about a nonreinforced stimulus.
forcement schedules. Englewood Cliffs, N.J.: Prentice- Psychological Reports, 1962, 11, 791-792.
Hall, Inc., 1970.
Selekman, W. Behavioral contrast and inhibitory stimulus
Mark Rilling 479

control as related to extended training. Journal of the two continua. Journal of the Experimental Analysis of
Experimental Analysis of Behavior, 1973, 20, 245-252. Behavior, 1963, 6, 223-232. (b)
Seligman, M. E. P., 8c Hager, J. L. (Eds.). Biological Terrace, H. S. Wavelength generalization after discrimina¬
boundaries of learning. Englewood Cliffs, N.J.: Prentice- tion learning with and without errors. Science, 1964,
Hall, Inc., 1972. 144, 78-80.
Shettleworth, S. J. Constraints on learning. In D. S. Terrace, H. S. Behavioral contrast and the peak shift:
Lehrman, R. A. Hinde, 8c E. Shaw (Eds.), Advances in Effects of extended discrimination training. Journal of
the study of behavior. New York: Academic Press, 1972. the Experimental Analysis of Behavior, 1966, 9, 613-617.
Sidman, M. Avoidance behavior. In W. K. Honig (Ed.), (a)
Operant behavior: Areas of research and application. Terrace, H. S. Discrimination learning and inhibition.
Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966. Science, 1966, 154, 1677-1680. (b)
Skinner, B. F. The behavior of organisms. New York: Terrace, H. S. Stimulus control. In W. K. Honig (Ed.),

Appleton-Century-Crofts, 1938. Operant conditioning: Areas of research and applica¬

Skinner, B. F. Are theories of learning necessary? Psycho¬ tion. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966. (c)
logical Review, 1950, 57, 193-216. Terrace, H. S. Discrimination learning, the peak shift, and
Skinner, B. F. Why we need teaching machines. Harvard behavioral contrast. Journal of the Experimental Analy¬
Educational Review, 1961, 31, 377-398. sis of Behavior, 1968, 11, 727-741.
Skinner, B. F. Stimulus generalization of an operant: A Terrace, H. S. Escape from S—. Learning and Motivation7

historical note. In D. I. Mostofsky (Ed.), Stimulus gener¬ 1971, 2, 148-163.

alization. Stanford: Stanford University Press, 1965. Terrace, H. S. By-products of discrimination learning. In
Skinner, B. F. Contingencies of reinforcement: A theoret¬ G. H. Bower (Ed.), The psychology of learning and moti¬
ical analysis. Englewood Cliffs, N.J.: Prentice-Hall, Inc., vation (Vol. 5). New York: Academic Press, 1972.
1969. Terrage, H. 5. Conditioned inhibition in successive dis¬
Spear, N. E. Forgetting as retrieval failure. In W. K. Honig crimination learning. Transactions of the New York
8c P. H. R. James (Eds.), Animal memory. New York; Academy of Sciences (Series II), 1973, 35, 39-50,
Academic Press, 1971. Thomas, D. R. The effects of drive and discrimination
Spence, K. W. The nature of discrimination learning m training on stimulus generalization. Journal of Experi¬
animals. Psychological Review, 1936, 43, 427—449. mental Psychology, 1962, 64, 24-28.
Spence, K. W. The differential response in animals to Thomas, D. R. The use of operant conditioning techniques
stimuli varying within a single dimension. Psychological to investigate perceptual processes in animals. In R. M.
Review, 1937, 44, 430-444. Gilbert 8c N. 5. Sutherland (Eds.), Animal discrimination
Staddon, J. E. R. Inhibition and the operant. A review learning. New York: Academic Press, 1969.
of sensory inhibition by G. V. Bekesy & Mach Bands; Thomas, D. R. Stimulus selection, attention, and related
Quantitative studies on neural networks in the retina matters. In J. H, Reynicrse (Ed.), Current issues in ani¬
by F. Ratliff. Journal of the Experimental Analysis of mal learning, Lincoln. University of Nebraska Press,
Behavior, 1969, 12, 481-489. 1970.
Staddon, J, E. R. Temporal control, attention, and mem¬ Thomas, D. R., 8c Barker, E. G. The effects of extinction

ory. Psychological Review, 1974, 81, 375-391, and “central tendency” on stimulus generalization m
Staddon, J. E. R,, 8c Simmelhag, V. L. The “superstition” pigeons. PsychoYlomic Science? 1964, 199—12L
experiment; A reexamination of its implications for the TnoMAg, D. R., & Setzer, J. Stimulus generalisation gradi¬
principles of adaptive behavior. Psychological Review, ents for auditory intensity in rats and guinea pigs. Psy-
1971, 78, 3-43. chonomic Science, 1972, 28, 22-24.
Thomas, D. R„ & Switalski, R. W. Comparison of stimulus
Stevenson, J, G. Stimulus generalization; The ordering and
spacing of test stimuli. Journal of the Experimental generalization following variahle-ratio and variable-inter¬
val training. Journal of Experimental Psychology, 1966,
Analysis of Behavior, i960, 9, 457-468,
71, 236-240
Stoddard, L. T. k Sidman, M, The effects of errors on
Thomas, D. R., & Williams, J. L. A further study of stiim
children’s performance of a circle-ellipse discrimination.
ulus generalization following three-stimulus discrimina¬
Journal of the Experimental Analysis of Behavior, 1967,
tion training. Journal of the Experimental Analysis of
10, 261-270.
Behavior, 1963, 6, 171-176.
Stoddard, L. T. & Sidman, M. The removal and restoration
Vieth, H. A., 8c Killing, M. E. Comparison of timeout and
of stimulus control. Journal of the Experimental Analy¬
extinction as determinants of behavioral contrast: An
sis of Behavior, 1971, 16, 143-154. analysis of sequential effects. Psychonomic Science, 1972,
Switalski, R. W., Lyons, J., 8c Thomas, D. R. Effects of
27, 281-282.
interdimensional training on stimulus generalization. Wagner, A. R., 8c Rescorla, R. A. Inhibition in Pavlovian
Journal of Experimental Psychology, 1966, 72, 661-666. conditioning application of a theory. In R. A. Boakes 8c
Taus, S. E. 8c Hearst, E. Effects of intertrial (blackout) M. S. Halliday (Eds.), Inhibition and learning. New
duration on response rate to a positive stimulus. Psy- York: Academic Press, 1972.
chonomic Science, 1970, 19, 265-267. Weisman, R. G. Some determinants of inhibitory stimulus
Terrace, H. S. Discrimination learning with and without control. Journal of the Experimental Analysis of Be¬
errors. Journal of the Experimental Analysis of Behav¬ havior, 1969, 12, 443-450.
ior, 1963, 6, 1-27. (a) Weisman, R. G. Factors influencing inhibitory stimulus
Terrace, H. S. Errorless transfer of a discrimination across control: Differential reinforcement of other behavior
 480 STIMULUS CONTROL AND INHIBITORY PROCESSES

during discrimination training. Journal of the Experi¬ tion of interresponse time. Animal Learning and Behav¬
mental Analysis of Behavior, 1970, 14, 87-91. ior, 1973, 1, 297-301.
Weisman, R. G., 8c Palmer, J. A. Factors influencing in¬
Wilkie, D. M. The peak shift and behavioral contrast:
hibitory stimulus control: Discrimination training and Effects of discrimination training with delayed rein¬
prior nondifferential reinforcement. Journal of the Ex¬ forcement. Psychonomic Science, 1972, 26, 257-258.
perimental Analysis of Behavior, 1969, 12, 229-237. Wilkie, D. M. Stimulus control of responding during a
Weisman, R. G., 8c Ramsden, M. Discrimination of a re¬
fixed interval reinforcement schedule. Journal of the
sponse-independent component in a multiple schedule. Experimental Analysis of Behavior, 1974, 21, 425-432.
Journal of the Experimental Analysis of Behavior, 1973, Wilkie, D. M., 8c Ramer, R. G. Errorless discrimination
19, 55-64.
established by differential autoshaping. Journal of the
Weiss, S. J. Discrimination training and stimulus com¬ Experimental Analysis of Behavior, 1974, 22, 333-340.
pounding consideration of nonreinforcement and re¬ Wilton, R. N. 8c Clements, R. O. Behavioral contrast as a
sponse differentiation consequences of SA. Journal of function of the duration of an immediately preceding
the Experimental Analysis of Behavior, 1971, 15, 387- period of extinction. Journal of the Experimental Analy¬
402. sis of Behavior, 1971, 16, 425-428.
Weiss, S. J. Stimulus compounding in free-operant and Winton, J. S. W. 8c Beale, I. L. Peak shift in concurrent
classical conditioning: A review and analysis. Psycho¬ schedules. Journal of the Experimental Analysis of Be¬
logical Bulletin, 1972, 78, 189-208. (a) havior, 1971, 15, 73-81.
Weiss, S. J. Free-operant compounding of high- and low-
Yarczower, M. Behavioral contrast and inhibitive stimulus
rate discriminative stimuli: An interresponse time analy¬ control. Psychonomic Science, 1970, 18, 1-3.
sis. Learning and Motivation, 1972, 3, 469-478. (b) Yarczower, M. Discriminative effects of massed extinction.
Weiss, S. J. Discriminative and incentive interactions in Journal of the Experimental Analysis of Behavior, 1974,
composite-stimulus control of free operant behavior. 22, 161-168.
Paper presented at the 82nd Annual Convention of Yarczower, M., 8c Curto, K. Stimulus control in pigeons
the American Psychological Association, 1974. after extended discrimination training. Journal of Com¬
Weiss, S. J. 8c Van Ost, S. L. Response discriminative and
parative and Physiological Psychology, 1972, 80, 484-489.
reinforcement factors in stimulus control of performance Yarczower, IVf., Dickson, J. F., 8c Gollub, L. R. Some
on multiple and chained schedules of reinforcement. effects on generalization gradients of tandem schedules.
Learning and Motivation, 1974, 5, 459-472. Journal of the Experimental Analysis of Behavior, 1966,
Wessells, M. G. Errorless discrimination, autoshaping, and 9, 631-639.
conditioned inhibition. Science, 1973, 182, 941-943. Yarczower, M., 8c Evans, G. “Combined cue” test of condi¬
Wessells, M. G. The effects of reinforcement upon the pre¬
tioned inhibition in pigeons. Journal of Comparative
pecking behaviors of pigeons in the autoshaping experi¬ and Physiological Psychology, 1974, 87, 261-266.
ment. Journal of the Experimental Analysis of Behavior, Yarczower, M., Gollub, L. R„ 8c Dickson, J. F. Some
1974, 21, 125-144.
effects of discriminative training with equated frequency
Wheatley, K. L., 8c Thomas, D. R. Relative and absolute
of reinforcement. Journal of the Experimental Analysis
density of reinforcement as factors influencing the peak of Behavior, 1968, 11, 415-423.
shift. Journal of the Experimental Analysis of Behavior, Yarczower, M., & Switalski, R. Stimulus control in the
1974,22, 409-418. goldfish after massed extinction. Journal of the Experi¬
White, K, G. Post-discrimination generalization as a func¬
mental Analysis of Behavior, 1969, 12, 565-570.
 16

Stimulus Control
attentional factors*

N. J. Mackintosh

INTRODUCTION spons@£ will be reinforced according to a particular

schedule.

Experimental subjects do not respond, nor do ex¬ If the experimental situation or the experimenter's
perimenters arrange contingencies of reinforcement, discriminative stimulus or CS is changed in one or
in a vacuum. In experiments on both classical and more ways, it is common to observe an apparently
operant conditioning, the experimenter delivers rein¬ correlated change in the subject's behavior, A pigeon
that receives food on a variable-interval schedule for
forcers only in the presence of a specific set of stim¬
uli. This is most obviously true of classical experi¬ pecking at a key illuminated with green light will
ments, where the availability of reinforcement is peck at a lower rate if the color of the light is changed
always signaled by the presentation of a specific to red. A dog salivating upon every presentation of a
conditional stimulus (CS). But it is equally true of op¬ 1,000-Hz tone signaling the delivery of food may
erant experiments, for operant responses are rein¬ salivate less profusely if a 2,000-Hz tone is presented.
forced only when they occur in a specific situation. At If a change in a particular stimulus is always followed
the least they are reinforced only in the experimental by a change in the probability, amplitude, latency, or
chamber. Often, they are reinforced only during cer¬ rate of a particular response, we may say that this
tain periods of an experimental session, with these stimulus exercised some control over that response. The
being marked by the presentation of explicit dis¬ term stimulus control has come to be used as a con¬
criminative stimuli signaling that some class of re- venient shorthand expression for describing such an
observed relationship between changes in external
stimuli and changes in recorded behavior.
* The preparation of this chapter was supported by grants
from the National Research Council of Canada and the U.K. In the hands of some writers, the term stimulus
Science Research Council. I am indebted to Vicky Gray for control has been characterized as “relatively neutral”
permission to cite the results of unpublished research and to A. and as one to be preferred to “traditional concepts of
Dickinson, G. Hall, and the editors of this volume for their
extensive and valuable comments on an earlier draft.
generalization and discrimination” (Terrace, 1966, p.

481
482 STIMULUS CONTROL: ATTENTIONAL FACTORS

271). It is, of course, possible to reserve the term stim¬ to vertical and horizontal lines presented on black
ulus control solely to refer to the slope of a generaliza¬ backgrounds that it was impossible to detect any
tion gradient, but it is neither clear that it is partic¬ evidence of significant control by the orientation of
ularly profitable to do so nor obvious that it has the line. Other tests, however, revealed a substantial
always been used in this purely descriptive sense. In difference in the readiness to respond to vertical and
the present chapter, at any rate, the term will not be horizontal lines (see also Zentall, 1972).
used in this strictly neutral way. To say that a par¬ Conversely, a subject with a very high probability
ticular stimulus has acquired control over a subject’s of responding in the training situation may continue
behavior, I shall assume, is tantamount to saying that to respond on all test trials, and thus show a flat
this stimulus has been established as a signal for rein¬ gradient of generalization because of a ceiling effect.
forcement, or as a signal that a certain class of re¬ This may happen only rarely in experiments which
sponses will be reinforced.1 Evidence that a stimulus study rate of key pecking in pigeons, for rate of re¬
has been successfully established as a signal for rein¬ sponding is a relatively unbounded measure of
forcement may be provided in a variety of ways. One response strength. Where more bounded measures are
way is to show that changes in some features of the used, such as the proportion of trials on which a re¬
stimulus result in correlated changes in behavior. sponse occurs, or measures of conditioned suppression
Another way would be to show that the removal of in aversive conditioning, there is clear evidence that
the stimulus resulted in the cessation of the subject’s ceiling effects may obscure the control actually gained
responses. Yet other measures of control are possible: by a stimulus, which is only displayed during the
the rate of subsequent discrimination learning, when course of an extended series of test trials (e.g.. Gray 8c
the original stimulus continues to signal reinforce¬ Mackintosh, 1973; Hoffman 8c Fleshier, 1961).
ment and a second stimulus signals nonreinforcement, A second reason why the control gained by one
would be equally acceptable measures of the control stimulus may not be revealed in a generalization test
gained by the stimulus. is that other aspects of the experimental situation may
The slope of a generalization gradient, therefore, is have gained even stronger control over responding. If
only one of several potential measures of stimulus these other stimuli remain unchanged during the
control. It is not even necessarily the best or most course of testing, they may maintain a constant rate
sensitive measure. Thus if an experimenter observes a of responding and thus "mask” the control actually
flat gradient of generalization when he varies some acquired by the stimulus varied during testing. The
aspect of the training situation, he is not entitled to concept of masking is one that will play a central role
conclude that this aspect had gained no control over in later sections of this chapter. For the present, it
his subject’s behavior. He may say, if he wishes, that will be sufficient to provide a brief example. Newman
this aspect exerted no control over responding on this and Baron (1965) trained pigeons to peck a response
series of test trials, but this, of course, is no more than key illuminated with a white vertical line on a green
a redescription of the outcome of the data. The flat background. When given a generalization test to other
gradient is not necessarily evidence that this stimulus orientations of the line, still shown on a green back¬
failed to acquire control over behavior; it may simply ground, the pigeons responded at a relatively constant
imply that the testing procedure is inadequate to rate to orientations of the line as far as 45° on either
demonstrate such control. Flat gradients are often, for side of vertical. Several later studies, however, while
example, consequences of ceiling or floor effects. confirming this finding, have shown that under condi¬
Farthing and Hearst (1970) trained pigeons on a dis¬ tions where a significant level of responding can be
crimination between a vertical line displayed on a maintained in the absence of the colored background,
blue background and a horizontal line on a green a reliably sloping gradient of generalization can be
background. When given a series of nonreinforced observed when different orientations of the line are
test trials to the component stimuli as well as to shown on a black background (Freeman 8c Thomas,
various compounds, they responded at so low a rate 1967; Newman 8c Benefield, 1968; Thomas, Svinicki, 8c
Svinicki, 1970).
i A stimulus may also gain control if it is established as a The flat gradient in the first case reflects more
signal for the omission of a reinforcer or as a signal that certain
responses will not be reinforced. The phenomena of inhibitory about the control over responding acquired by the
control, discussed by Rilling in chapter 15, are not specifically colored background than about the lack of control
discussed here. It seems reasonable, however, to expect that acquired by the line. A particular feature of the rein¬
many of the principles derivable from studies of excitatory
stimulus control will apply equally to the case of inhibitory forced stimulus display may indeed acquire control
control. over responding, but this control may be masked dur-
N. J. Mackintosh 483

ing a subsequent test, because the unchanged presence this excitation would spread to other stimuli in pro¬
of another set of features insures a uniform rate of portion to their similarity to the training stimulus.
responding on all test trials. Sloping gradients were an automatic consequence of
The behavior of a subject in a generalization test, reinforcement in the presence of a particular stimulus.
therefore, does not necessarily provide a simple or Although this view can be seriously attributed
direct measure of the control acquired by the stim¬ neither to Hull nor to Spence, it is fair to acknowl¬
ulus during earlier training. This point has a number edge that Lashley and Wade’s paper, by stressing the
of important implications. It may not always be easy, point that sloping gradients might not be an auto¬
for example, to determine whether differences in the matic consequence of the delivery of reinforcement in
slope of a generalization gradient following different the presence of a particular stimulus, did provide
experimental treatments reflect differences in the con¬ valuable impetus in initiating research designed to
trol acquired as a consequence of those treatments, or uncover the conditions necessary and sufficient for the
the effects of those treatments on the host of other establishment and demonstration of control by any set
variables that may affect test performance. If different of stimuli.
treatments produce substantially different rates of re¬
sponding, differences in test performance may reflect
Intrinsic Differences in the Salience of Stimuli
ceiling or floor effects influencing one gradient more
than the other. If different treatments produce sub¬ These are important questions, Perhaps even more
stantial differences in resistance to extinction, then, important, however, is the realization that they may
since it is known that gradients become progressively not admit of any one, general answer. There is no
steeper during the course of extinction (Hoffman & reason to suppose that a single set of conditions,
Fleshier, 1961; Jenkins & Harrison, 1960; Thomas &* sufficient and necessary for the establishment of stim¬
Barker, 1964), their effect on generalization gradients ulus control in one case* will hold for all stimuli, re¬
may be simply attributed to this factor rather than sponses, reinforcers, or subjects. The control exercised
to their effect on stimulus control per se. by a particular stimulus over the behavior of a par¬
Examples of one or more of these possibilities will ticular subject will obviously depend on that subject's
recur in what follows. It will be important to remem sensory apparatus^ pigeons are more likely than ratsj

ber that generalization gradients provide but one of for example, to be controlled by the wavelength of a
several methods of measuring stimulus control and discriminative stimulus. Equally, the amount of train¬
that the assessment of stimulus control is necessarily ing required to establish control by a particular stim¬
an indirect affair. We do not observe stimulus control ulus will surely vary from stimulus to stimulus, and

in the data of a generalization test. We may infer that in a situation where a number of different stimuli are
a stimulus has acquired control over a subject's be¬ equally correlated with reinforcement, some will ac¬
havior by noting a correlation between changes in quire greater control over responding than others.
stimuli and changes in responding. But the inference These are not surprising observations. No one would
is not always easy. deny that some stimuli appear to be more effective for
some subjects than are others. It may even be useful
to characterize such differences as consequences of
CONDITIONS AFFECTING THE differences in the “salience” of particular stimuli to
ESTABLISHMENT OF STIMULUS CONTROL particular subjects, provided that the circular nature
of the definition is appreciated. There are, however,
The study of stimulus control has for a long time less obvious, and therefore rather more interesting,
centered around the question of the sufficient and constraints on the generality of possible answers
necessary conditions responsible for the observation of (Shettleworth, 1972).
a sloping gradient of generalization when some
feature of the training situation is varied. As Terrace
Nature of Response and Reinforcer
(1966) noted, one aspect of this issue was the objection
raised by Lashley and Wade (1946) against what they Dobrzecka, Szwejkowska, and Konorski (1966)
called a “Pavlovian theory of generalization.” Lash¬ have shown that the features of an auditory stimulus
ley and Wade argued that Hull and Spence had fol¬ which come to control the responses of a dog may de¬
lowed Pavlov in supposing that the reinforcement of a pend upon the nature of the response required by the
response in the presence of a particular stimulus was experimenter. Dogs were placed in a stand and ex¬
sufficient to establish a center of excitation, and that posed to two discriminative stimuli, a metronome in
484 STIMULUS CONTROL: ATTENTIONAL FACTORS

(Garcia 8c Ervin, 1968; Revusky 8c Garcia, 1970; Rozin

8c Kalat, 1971). Rats appear to associate the flavor of
food or water with subsequent poisoning, and the
visual or other external features accompanying its in¬
gestion with a reinforcer such as electric shock. In a
study that is, in effect, rather similar in design to that
of Dobrzecka et al., Garcia and Koelling (1966) gave
rats the opportunity to drink water having a partic¬
ular flavor and whose ingestion was accompanied by
a particular set of visual and auditory stimuli. One
group received an electric shock, either immediately
METRONOME BUZZER METRONOME BUZZER or after a delay, contingent on drinking this water; a
BEHIND AHEAD BEHIND AHEAD
second group was made sick either by an injection of
Fig. 1. Results of experiment by Dobrzecka et al. (1966). In the lithium chloride or by X-irradiation. The results of a
left panel are shown the results from dogs trained to respond series of test trials, in which the flavor and the visual
with different legs to qualitatively different auditory stimuli, and auditory cues were separately presented, are
located in different positions. When tested with the stimuli in
reversed positions, they reverse their responses. The right panel shown in Figure 2. The shocked animals showed a
shows the results from dogs trained either to raise or not to marked reduction in their consumption of water ac¬
raise their right leg. When tested with the position of metro¬ companied by these visual and auditory stimuli, but
nome and buzzer reversed, they continue to respond as before.
no aversion to the specific flavor used in training;
poisoned rats, on the other hand, showed an aversion
front and a buzzer sounded from behind. One group to the flavor of the water they had been exposed to,
was required to raise their right foreleg in response to but none to the visual and auditory stimuli accom¬
one stimulus and their left foreleg in response to the panying its ingestion.
other; a second group was trained on a go-no go dis¬ The specific features of food or drink that are
crimination, being required to raise their right foreleg associated with subsequent poisoning may differ from
to one stimulus and to refrain from responding in the one group of animals to another. Predatory birds, for
presence of the other. Subjects were then tested by example, associate the visual characteristics of their
reversing the positions of the metronome and buzzer. prey with its unpalatable taste (Brower, 1969); this is,
The results of these test trials are shown in Figure 1. of course, a feature of their behavior responsible for
It can be seen that if animals were required to learn the evolution of visual Batesian mimicry among prey
which foot to raise, the location of the signal had ac¬
quired control over responding, while animals re¬
INDUCED SICKNESS ELECTRIC SHOCK
quired to learn the go—no go discrimination had
learned to respond to the metronome and not respond
to the buzzer, regardless of the position from which
they were sounded.
Thus although the dogs were perfectly well able to
discriminate both between the locations of the buzzer
and of the metronome, and between the quality of the
sound produced by each source, responding was con¬
trolled in one case only by the qualitative difference,
and in the other case only by the difference in loca¬
tion. The selective control observed cannot be attrib¬
uted to differences in the salience of the two cues, if
this is understood to refer to the physical character¬
X-RAYS LITHIUM IMMEDIATE DELAYED
istics of a stimulus and to the subject’s sensory capac¬ CHLORIDE
ities.
The work of Garcia, Revusky, Rozin, and others Fig. 2. Rate of drinking during test trials when rats have been
punished for drinking water with a particular flavor and whose
has suggested that when a particular reinforcing event ingestion was accompanied by a particular set of audiovisual
follows the ingestion of food, the feature of the food stimuli. The flavor of the water was established as the effective
which will be established as a signal for reinforce¬ signal when induced sickness was the aversive reinforcer, but
the audiovisual stimuli became the effective signal if electric
ment will depend on the nature of the reinforcer shock was the aversive reinforcer. (After Garcia & Koelling, 1966.)
N. J. Mackintosh 485

species. Other birds have been shown to associate the changes in auditory or visual stimuli, or that an adult
visual characteristics of water, rather than its taste, pigeon has an extensive experience of correlations be¬
with subsequent poisoning (Wilcoxon, Dragoin, 8c tween changes in visual stimuli and changes in the
Krai, 1971). How far these differences between differ¬ probability of food, but no correlation between audi¬
ent species may be attributed merely to differences in tory changes and the availability of food. Before
sensory capacity is not at present certain. The impor¬ accepting Garcia and Rozin’s inference, therefore, it
tant point is that in Garcia and Koelling’s experi¬ is important to see how far prior experience can affect
ment, as in that of Dobrzecka et al., one component of the establishment of stimulus control.
a compound stimulus acquired control over behavior
under one condition, while the other component ac¬
quired control under a second condition. Prior Experience

Garcia’s findings reflect an apparent dependency be¬

Although numerous experimenters have exposed
tween specific stimuli and specific reinforcers, but they
animals to a variety of different experiences with par¬
should not necessarily be regarded simply as a conse¬
ticular sets of stimuli, before training them to respond
quence of the specialized nature of the system regu¬
to those, or similar, stimuli, they have only rarely been
lating food intake. Foree and LoLordo (1973), for
concerned to provide answers to the questions raised
example, found that the elements of a compound by the results discussed in the preceding s^cbgn. ft
stimulus that gained control over responding in
will be necessary to survey rapidly some of the other
pigeons depended on whether responding was being
questions posed, and answers provided, before return¬
reinforced by the presentation of food or by the avoid¬
ing to this issue.
ance of electric shock. On a series of discrete trials,

signaled by the combined illumination of a red light

Early Experience: Lashley and
and the presentation of a 440-Hz tone, pigeons were
Wade’s Hypothesis
trained to press a treadle either to obtain food or to
avoid shock. When tested with these components in One of Lashley and Wade’s theses was that a
isolation, subjects reinforced with food tended to re¬ change in some aspect of the training situation would
spond only in the presence of the light, while those a correlated change in the subject’s behavior
reinforced by the avoidance of shock responded more only if that subject had some prior experience of
to the auditory than to the visual component. At the variation along this stimulus dimension. “The 'dimen¬
very least, these results suggest that the well-known sions’ of a stimulus/’ they wrote, “are determined by
of establishing control over food-reinforced
difficulty comparison of two or more stimuli and do not exist
key pecking in pigeons by an auditory stimulus (see for the organism until established by differential
below) cannot be entirely attributed to defects in the training” (Lashley & Wade, 1946, p. 74). The implica¬
birds’ sensory system. tion that has been investigated in a number of studies
Garcia and Ervin (1968) and Rozin and Kalat is that animals deprived of all experience of variations
(1971) have argued that the rat’s readiness to associate along a particular stimulus dimension by restricted

flavors with illness is a prime example of an adaptive conditions of rearing will produce flat gradients of
specialization in learning. One can hardly doubt that generalization along that dimension.
such learning is of adaptive significance in the life of Some results reported by Peterson (1962) appeared
a relatively omnivorous animal, but to point to the to provide some initial support for this suggestion.
adaptive significance of a particular characteristic is Peterson trained two groups of young ducklings to
not the same as specifying the causal factors respon¬ peck a key illuminated with sodium light of 589 nm
sible for its appearance in any particular individual. and then tested them for generalization to other wave¬
There may be genetically determined constraints on lengths. Two birds reared in normal illumination
the probability that different stimuli will become sig¬ showed orderly and sloping wavelength gradients with
nals for different reinforcers. It is also possible, how¬ a peak at 589 nm. Four other birds, however, had
ever, that a subject’s prior experience of correlations been reared in individual cages diffusely illuminated
between events in its environment may affect the by sodium light, and had thus received little or no
probability of certain events being established as experience of variations in wavelength before the
signals for others. It is possible, for example, that an generalization test. All four birds showed flat wave¬
adult rat has had a lifetime’s experience in which length gradients. Peterson’s results have been thought
gastric changes have been correlated with changes in to imply that “a necessary condition for obtaining a
recently experienced tastes, but uncorrelated with generalization gradient of wavelength whose slope is
 486 STIMULUS CONTROL: ATTENTIONAL FACTORS

greater than zero is prior exposure to white light. ment, presented data suggesting that monkeys reared
White light presumably allowed differential reinforce¬ in complete darkness might initially generalize a
ment with respect to different wavelengths to occur trained response relatively completely to wavelengths
prior to the generalization test” (Terrace, 1966, p. other than that to which they were trained to respond.
279). Dark-reared and control subjects were trained to press
As I shall argue below, it is questionable whether a response key for sucrose solution while exposed to
this conclusion necessarily follows from Peterson’s monochromatic light projected to their right eye.
results. For the moment, however, it will be enough to When tested with other wavelengths, interspersed
question the reliability and generality of those results. with reinforced retraining trials with the original
Tracy (1970) has attempted to replicate Peterson’s stimulus, control subjects showed an orderly, sloping
experiment with a larger number of subjects, but with gradient from the first day of testing, while those
at best only partial success. He did, indeed, find that reared in the dark produced an initially flat gradient,
birds reared in monochromatic sodium light showed which gradually became steeper during the course of
somewhat flatter gradients than those reared in successive test sessions. The results of the first test
normal illumination, but the main results of his session, therefore, suggest that dark rearing may
experiment, shown in Figure 3, leave no doubt that initially flatten generalization gradients in monkeys.
their responding was moderately well controlled by Several later experiments, however, employing
changes in wavelength. Tracy further showed that the other subjects, such as chicks or Japanese quail, have
difference between the gradients of these two groups completely failed to substantiate Ganz and Riesen’s
may have been partly a consequence of the effect of or Peterson’s results. Neither rearing in monochro¬
rearing on preferences for different wavelengths. The matic light nor rearing in total darkness has been
right-hand panel of Figure 3 shows wavelength gra¬ found to have any significant effect on the slope of
dients of control and experimental subjects following wavelength gradients in these subjects (Malott, 1968;
reinforcement in the presence of a white vertical line Rudolph Sc Honig, 1972; Rudolph, Honig, Sc Gerry,
on a black key. It can be seen that even without any 1969).
prior reinforcement in the presence of 589 nm, the The conclusion must be, then, that prior experi¬
sodium-reared birds showed a greater preference for ence of variation along a particular stimulus dimen¬
shorter wavelengths than did the controls. It is in this sion is not a necessary condition for the establishment
region of the spectrum, as the left-hand panel of of control by a stimulus falling on that dimension.
Figure 3 shows, that their gradients were flatter than Extended rearing under restricted conditions, of
those of controls following wavelength training. course, might interfere with the normal development
Ganz and Riesen (1962), in a study with monkeys of the neural mechanisms underlying stimulus anal¬
published at about the same time as Peterson’s experi- ysis, and it is possible that such an effect might be
more pronounced in primates than in birds. But as a
general rule, it is clear that artificial restrictions on a
subject’s prior experience do not necessarily disrupt
the normal development of simple perceptual anal¬
ysis, and do not prevent the normal establishment of
stimulus control under appropriate training con¬
ditions.

Nonreinforced Exposure to a Single

Stimulus: Latent Inhibition

In Tracy’s (1970) experiment with ducklings, there

was some suggestion that monochromatic rearing
WAVELE NGTH (nm )
might have had some tendency to flatten the gradient
of wavelength generalization. Although this appears
Fig. 3. Relative gradients of wavelength generalization in duck¬
lings reared either in white light or in monochromatic sodium to have been largely due to a change in the uncondi¬
light (589 nm). The left panel shows gradients obtained after tional preference for different wavelengths, there is
reinforcement for responding to 589 nm. The right panel shows another factor that might have been responsible for
gradients after reinforcement for responding to a vertical white
line. (After Tracy, 1970. © 1970 by the Society for the Experi¬ such an effect. Numerous studies have now established
mental Analysis of Behavior, Inc.) that repeated exposure to a particular stimulus in the
N. J. Mackintosh 487

absence of reinforcement may significantly impair the establishment of control. Monochromatic rearing, as
subsequent establishment of control by that stimulus in Peterson’s (1962) and Tracy’s (1970) experiments,
when it is presented either as a CS or as a discrimina¬ may reduce the slope of gradients of wavelength gen¬
tive stimulus. eralization, not because it abolishes the experience of
This finding was first clearly reported by Lubow variations in wavelength, but because it insures that
and Moore (1959), who termed the effect “latent in¬ the wavelength used in training is not readily estab¬
hibition.” In their experiment, goats and sheep re¬ lished as a signal for reinforcement.
ceived 10 nonreinforced presentations of a stimulus
that later served as a CS signaling shock to the foreleg.
Exposure to Various Correlations
These animals acquired a conditioned flexion re¬
Between Stimuli and
sponse significantly more slowly than control subjects
Reinforcement
who received no preexposure to the CS. Lubow and
Moore suggested that this interference with the ac¬ In experiments stimulated by Lashley and Wade’s
quisition of a conditioned flexion response might have assumptions about the importance of early experience,
been a consequence of the establishment of some in¬ as in studies of latent inhibition, exposure to a par¬
compatible response during preexposure. A subse¬ ticular stimulus is scheduled without any correlated
quent experiment provided little support for this sug¬ exposure to reinforcement. There is? however, another
gestion (Lubow, 1965), and it has been convincingly group of studies which has systematically ^n^ly^d tllS
disproved by the result? of several later experiments effects of exposure to different correlations between
(Halgren, 1974, Reiss 8c Wagner, 1972; Rescorla, stimuli and reinforcers on the subsequent acquisition
1971). In all of these studies, nonreinforced preex¬ of control by those stimuli. Among the earliest and
posure to a stimulus interfered with both excitatory best known of such experiments are those of Lawrence
and inhibitory conditioning to that stimulus. In Res- (1949, 1950) on the acquired distinctiveness of cue§, in
corla’g experiment, for example, rat§ were glYen npn- which transfer between simultaneous and successive
reinforced preexposure to a tone before the start of discriminations in the rat was shown to depend on the
conditioned emotional response (CER) conditioning. relationship between the relevant stimuli of the two
When the tone signaled shock, preexposed subjects problemg. Later studies of lntradiniensional and
showed poorer conditioning than controls; but if a cxtradimcnsional shifts, in which, having learned one
light was used to signal shock, and a tone-light com¬ problem, animals are shifted to a second, where the
pound to signal the omission of shock, preexposed relevant stimuli are either from the game dimemion
animals not only learned to suppress to the light as qx from a different dimension from, those relevant
rapidly as controls, they also continued to suppress to in the first problem, have confirmed that experience
the tone-light compound longer than the controls. of a correlation between a particular set of stimuli
The implication is that nonreinforced exposure to and reinforcement will selectively increase the prob¬
a stimulus may interfere with the establishment of ability that similar stimuli will subsequently gain con¬
that stimulus as a signal either for reinforcement or trol aver responding m a new situation (Shepp &
for the omission of reinforcement. If the effect of such Eimas, 1964; Shepp & Schrier, 1969).
exposure were simply to condition a response incom¬ An experiment by Thomas, Mariner, and Sherry
patible with the required conditioned response (GR), (1969) suggests that the principle of acquired disdnc=
it should obviously facilitate, rather than interfere tiveness may be used to counteract the difficulty of
with, the development of conditioned inhibition. establishing auditory control oyci food-reinforced key
Latent inhibition is, moreover, a quite general phe¬ pecking in pigeons. They confirmed the finding, first
nomenon, having been observed in a variety of class¬ reported by Jenkins and Harrison (1960), that non¬
ical conditioning preparations (Lubow, 1973), as well differential reinforcement of a pigeon’s key pecks in
as in studies of operant discrimination learning the presence of a 1,000-Hz tone would result in essen¬
(Hearst, 1972; Mellgren 8c Ost, 1969). It is certain, tially flat gradients of generalization when the fre¬
therefore, that prior experience with a particular quency of the tone was varied between 300 and 3,500
stimulus will significantly affect the establishment of Hz. For 100 days before the start of key-peck training,
control by that stimulus—although the effect is not however, a second group of pigeons received their
that which Lashley and Wade would have predicted. daily ration of food in their home cages always
Exposure to a particular set of stimuli, so far from signaled by a 1,000-Hz tone. All birds in this group
being a necessary condition for the establishment of showed a steep and orderly gradient of generalization
control by those stimuli, may interfere with the along the auditory frequency dimension, with a peak
488 STIMULUS CONTROL: ATTENTIONAL FACTORS

of responses at 1,000 Hz. It would be of considerable training. Exposure to a positive correlation between a
interest to see whether similar results could be ob¬ stimulus and reinforcer may increase the control
tained if the auditory stimuli established as signals gained by that stimulus; unreinforced presentations
for food during preexperimental treatment were not of a stimulus or exposure to uncorrelated presenta¬
exactly the same as the tone presented during key- tions of a stimulus and reinforcer may decrease the
peck training. This would suggest that the results of control gained by that stimulus when subsequently
Thomas et al. represented a general change in the paired with reinforcement. There is, however, little
probability of auditory stimuli gaining control over reason to accept Lashley and Wade’s contention that
food-reinforced behavior. prior exposure to a set of stimuli, in and of itself,
Foree and LoLordo (1973), it will be recalled, found without regard to the relationship between those
that although a visual stimulus was more likely than stimuli and reinforcement experienced during such
an auditory stimulus to gain control over food-rein¬ treatment, is a particularly important determinant of
forced responding in pigeons, this ordering was re¬ generalization gradients. There is certainly no evi¬
versed when subjects were required to make the same dence to support the view that prior exposure to
response in order to avoid shock. Is it possible that variations along a stimulus dimension is a necessary
this differential sensitivity to visual stimuli as signals prerequisite for the establishment of control by that
for food, and to auditory stimuli as signals for shock, dimension. While the ability of a stimulus to acquire
is related to the normal early experience of the control over a subject’s behavior depends on that
pigeon via a process similar to that observed in experi¬ subject’s past experience, therefore, there is no reason
ments on acquired distinctiveness? For this to be true, to suppose that the perception of stimulus relations is
it would be necessary to assume that experience of a always dependent on exposure to variations in that
correlation between a particular class of stimuli and stimulus. For at least some subjects and some stimulus
a particular class of reinforcer would selectively alter dimensions, the perceptual system is already organized
the distinctiveness of those stimuli as signals for those to respond differentially and in an orderly manner to
reinforcers. It is possible that in the pigeon’s normal variations along that dimension. This is not to say
experience the availability of food is more reliably that the dimensions of stimuli to which animals re¬
signaled by visual than by auditory stimuli (as noted spond correspond exactly to the physical dimensions,
by Jenkins 8c Harrison, 1960), but if this is to help such as wavelength, visual intensity, or auditory fre¬
explain Foree and LoLordo’s results, it must be as¬ quency, which are manipulated by experimenters. It is
sumed that this enhances the distinctiveness of visual obvious that we know very little about the dimensions
stimuli as signals for food, but not as signals for shock. along which animals are capable of classifying their
There is, in fact, some evidence of precisely such a environment.
reinforcer-specific change in the distinctiveness of
particular stimuli. Mackintosh (1973) found that if
rats were exposed to uncorrelated presentations of a EXPERIMENTAL PROCEDURES:
tone and shock, subsequent conditioning between NONDIFFERENTIAL REINFORCEMENT
tone and shock was severely retarded, although the AND DISCRIMINATION TRAINING
tone could be rapidly established as a signal for water.
Conversely, exposure to uncorrelated presentations of Much of the experimental analysis of stimulus
tone and water retarded subsequent tone-water con¬ control in operant experiments has consisted of at¬
ditioning, without having a comparable effect on tempts to specify the training procedures required to
tone-shock conditioning. Thus a stimulus that has in establish control by particular stimuli. The volume
the past signaled no change in the probability of one of research conducted is testimony to the conclusion
reinforcer will be established as a signal for that rein¬ that no single set of conditions appears sufficient and
forcer only with difficulty, but may readily serve as a necessary for all stimuli, subjects, or experimental
signal for another reinforcer. situations. Experimental conditions apparently suffi¬
cient to establish control by visual stimuli over a
Conclusions pigeon’s food-reinforced behavior, as we have seen,
are not sufficient to establish auditory control over
There is, then, evidence that prior exposure to a this behavior. Many investigators have ignored the
particular correlation between a stimulus and a rein¬ possible contribution of differences in salience or past
forcer may affect the control over responding acquired experience, and have attempted to show that these
by that stimulus during subsequent experimental differences in outcome are more apparent than real.
N. J. Mackintosh 489

If, it is argued, visual and auditory stimuli gain con¬ volves differential reinforcement between the presence
trol at different rates, this is because the effective and absence of the CS. The subject is, in effect,
schedules of reinforcement associated with such trained on a discrimination between the experimental
stimuli are not the same. One suggestion, as we shall situation alone, signaling nonreinforcement, and the
see below, is that a localized visual stimulus will gain experimental situation plus CS, signaling reinforce¬
control where a diffuse auditory stimulus will not, be¬ ment. Thus differential reinforcement correlated with
cause the probability of the subject’s being stimulated the presence and absence of the CS may be responsible
by the visual stimulus will be correlated with the for the sloping gradient observed when some aspect
occurrence of responding and therefore with the prob¬ of the CS is varied.
ability of reinforcement, while the auditory stimulus Experiments on instrumental learning do not, it
will impinge on the subject whether or not he is may be thought, necessarily involve any such differ¬
responding. ential reinforcement correlated with the presence and
absence of a discriminative stimulus. The subject may
be placed in the apparatus and responding may be
Nondifferential Reinforcement
reinforced in the continuous presence of some specific
Jenkins and Harrison’s (1960) experiment estab¬
stimulus, which may then be varied in order to test
lished beyond question that the nondifferential rein¬ for generali2ation. Certainly, the classic study of Gutt-
man and Kalish (19£>6) on wavelength generalization
forcement of a subject s responses in the presence of a
particular stimulus was not always sufficient to insure in pigeons at first sight seems to approximate to this
that changes m that stimulus would result in any description. Birds were reinforced for pecking a key
change in the subject’s behavior. This observation has
illuminated with a light of a single wavelength and
been taken by some as their point of departure. Ter¬ were then tested with a series of new wavelengths. It
is, however, not difficult to point to at least two
race (1966), for example, has argued that nondiffer¬
ential reinforcement is never sufficient to establish possible sources of differential reinforcement implicit
stimulus control and that apparent exceptions to this in Guttman and Kalish’s procedure (Heinemann k
rule are always cases where, inadvertently or implicitly, Rudolph, 1963; Terrace, 1966). First, they pro¬
differential reinforcement was in fact scheduled. Al¬ grammed brief, 10-§ec intertrial intervals during
though this position turns out to be rather difficult to which the key was dark and the schedule of reinforce¬
discredit, I shall argue that it is in fact wrong and ment not in effect. Whether or not the birds re¬
that even if we ignore differences in past experience, sponded during these blackouts, it remains true that
the most important cause of differences in stimulus the illumination of the key with light of a given wave¬
control is not any difference in the opportunity for length, during which food wag available, was con
differential reinforcement; but a difference in the ex¬ trasted with the absence of illumination, when food
tent to which such control is masked by the presence was not available. Guttman and Kalish's use of a
of other stimuli. stimulus localised on the pigeon^ response koy may
have permitted a second source of implicit differential
FCinforcCment. Since reinforcement was contingent oii
Tiie Hypothesis of Implicit
pecking the key, it follows that at the moment of rein¬
Differential Reinforcement
forcement subjects must always have just pecked the
One might have thought that there would be nu¬ key, and therefore been exposed to the wavelength
merous examples of sloping gradients of generalization projected onto the key. At times when they were not
obtained without the necessity of programming differ¬ pecking, and therefore not exposed (or not so closely
ential reinforcement by discrimination training. Pav¬ exposed) to this wavelength, reinforcement was never
lov (1927) reported several differences in responding delivered. Implicitly, therefore, reinforcement may
to training and test stimuli in experiments on salivary have been correlated with variations in the subjects’
conditioning in dogs and also observed systematic exposure to wavelength.
changes in rate of salivation to test stimuli progres¬ The first of these suggestions can definitely be ruled
sively less similar to the training stimulus. Subse¬ out. Although the use of a blackout between stimulus
quent experiments have reported reliably sloping presentations may have some effect on the slope of
gradients along such dimensions as auditory frequency generalization gradients, it is not a necessary condi¬
after classical conditioning in pigeons (Hoffman & tion for the establishment of reliable stimulus control.
Fleshier, 1961) and rabbits (Moore, 1972). A classical Thomas, Svinicki, and Svinicki (1970) and Thomas,
conditioning experiment, however, necessarily in¬ Ernst, and Andry (1971), for example, observed rela-
 490
STIMULUS CONTROL: ATTENTIONAL FACTORS

tively steep gradients along the dimension of line cussed later. For the present, it provides another in¬
orientation after pigeons had been reinforced for stance of good control by a stimulus not located on
pecking a key continually illuminated with a vertical the subject’s manipulandum.
line.
The burden of these experiments seems quite clear.
The second suggestions seem intuitively plausible, The localization of a stimulus on or near the subjects’
but although there is some evidence that the localiza¬ manipulandum is not a necessary prerequisite for that
tion of a discriminative stimulus on the subjects’ stimulus to gain control over responding in the ab¬
manipulandum may sharpen generalization gradients, sence of explicit differential reinforcement. The
such results are open to alternative explanations, and theorist who wishes to maintain that implicit differ¬
there is ample evidence that such localization is not ential reinforcement is necessary for the establishment
necessary. Heinemann and Rudolph studied bright¬ of stimulus control must therefore fall back onto a
ness generalization in pigeons after training them to new line of argument. This may not, of course, be
peck a key of a particular brightness. Under ordinary impossible. One might still argue that even when the
conditions, they observed relatively steep gradients; discriminative stimulus is apparently quite diffuse,
but when the entire front wall of the pigeons’ cham¬ subjects are still implicitly exposed to differential
ber was made equal in brightness to the response key, reinforcement for responding in its presence. If a
the brightness gradient was essentially flat. Heinemann pigeon is required to peck a key in the presence of an
and Rudolph attributed this outcome to a reduction overhead light or a tone, for example, the precise
in the opportunity for differential reinforcement, stimuli to which it is exposed will change while it is
since, they argued, during training subjects would executing a response. Thus it could be argued that
have been exposed to a stimulus of the same bright¬ the stimuli impinging on the subject at the moment
ness as the response key even when not pecking. It is of pecking will differ, in some subtle ways, from those
equally possible, however, that it is a consequence of to which it is exposed when not responding: the
the relative indiscriminability of large areas of bright¬ presence of standing waves might cause a discrim-
ness; in the absence of any contrast, changes in bright¬ inable change in the intensity of an auditory stimulus
ness of the entire front wall of the chamber during at the moment of responding.
testing may have been difficult for subjects to detect.
One may wonder whether this possibility is sus¬
It is, at any rate, quite certainly possible to observe ceptible of disproof. Moreover, the claim that the
sloping gradients correlated with changes in relatively establishment of stimulus control requires differen¬
diffuse stimuli not localized on any response key. tial reinforcement within the experimental situation
Hearst (1962^) trained monkeys to press a lever for must not only resort to a considerable amount of spe¬
food in the presence of a continuous overhead light. cial pleading; if pressed too far, it also seems headed
Subsequent variations in the intensity of the light re¬ toward some logical inconsistency. The implication is
sulted in reliably sloping gradients. Mrs. V. Rege, that in the absence of such differential reinforcement,
working in my laboratory, has trained pigeons to peck no stimuli from the experimental situation would
an unilluminated key in the presence of an overhead, ever gain control over responding. And yet the mere
continuously illuminated red or blue light. Subse¬ fact that subjects are reinforced in the experimental
quent generalization tests in extinction revealed reli¬ situation means that differential reinforcement is
ably sloping gradients when the color of the overhead programmed between that situation and their home
light was changed. Finally, Rudolph and Van Houten, cage. 1 hus if no feature of the experimental situation
in an unpublished study, have shown that it is possi¬ could be shown to have acquired control over re¬
ble to obtain reliable control by a diffuse auditory sponding, this would show that differential reinforce¬
stimulus in pigeons without explicit discrimination ment was certainly not sufficient to establish control.
training. They trained pigeons to peck a key illu¬ It may, of course, be impossible to predict which par¬
minated with white light and, once pecking was estab¬ ticular feature or features will acquire control, and
lished, gradually faded out the illumination of the there is obviously no guarantee that the controlling
key until the pigeons were pecking in the dark. Under stimuli will include those which the experimenter
these circumstances, a 1,000-Hz tone, continuously chooses to vary in a subsequent generalization test.
present, could be shown to exert significant control But if differential reinforcement is necessarily pro¬
over responding: they observed reliably sloping gra¬ grammed whenever an animal receives its daily ra¬
dients of generalization when the frequency of the tone tion of food in the experimental situation, and not
was varied over a series of test trials. This study has a outside, then it is surely implausible to suppose that
number of important implications which will be dis¬ further differential reinforcement within the experi-
N. J. Mackintosh 491

mental situation is necessary for the establishment of birds continued to respond at a relatively constant
stimulus control. The difference in schedule of rein¬ rate, with only one bird showing evidence of reliable
forcement between the experimental situation and the control by this stimulus. In a second group, however,
home cage should presumably be sufficient to establish trained to peck an unilluminated key in a dark box,
control over responding by some features of the ap¬ all four birds show excellent control by the airflow
paratus. stimulus in the subsequent generalization test. More¬
over, if birds were required to learn a discrimination
Masking between two different velocities of airflow, they
learned much more rapidly when the chamber was
Since instrumental responses are typically rein¬ dark than when the key was illuminated with white
forced only in a particular situation, some aspects of light. Thus the presence of an illuminated response
that situation should gain control over responding. key would mask the appearance of control by this
Since evidence of control over a pigeon's food-rein¬ nonvisual, relatively unlocalized stimulus.
forced responses by such stimuli as tones is often Rudolph and Van Houten confirmed the conclu¬
hard to come by, it remains to consider why non- sions of this first study in a second experiment using
differential reinforcement within the experimental auditory stimuli. Birds trained to peck an illuminated
situation is not sufficient to establish control by all key m the presence of a 1,000-Hz tone generalized al¬
features of the situation. most completely to other frequencies of tone. This
The simplest solution to this problem is surely that group, therefore, replicated Jenkins and Harrison’s
proposed by Hull (1952, pp. 64-69). If a pigeon’s re¬ (1960) results. As briefly noted above, however, a sec¬
sponses are reinforced in the presence of a set of ond group, trained to peck a dark key in the presence
stimuli S1; S2, S3, . . . Sn, where Sj represents a of a tone, showed a reliable and steep gradient when
1,000-Hz tone, S2 the illumination from the response tested with other frequencies. The results are shown
key, and S3 the illumination from the houselight, a in Figure 4.
generalization test to other frequencies of the tone, It is clear that the presence of visual stimuli may
S/, S/' etc,, will vary only Sj and leave all other stim¬ mask control by stimuli from other modalities in the
uli, S2, S3, . . . Sn constant. To the extent that some pigeon. The general conclusion suggested by these
of these other stimuli have gained control over re¬ experiments, then, is that failures of stimulus Control
sponding, they will continue to control a high rate of are more plausibly attributed to the presence of other
responding on all test trials. Their presence, there¬ stimuli which mask control by the experimenter’s
fore, may mask the control actually gained by the stimulus than to the absence of implicit differential
tone. reinforcement. Variations in entirely diffuse stimuli
As was briefly noted earlier, a pigeon reinforced for may result in sloping generalization gradients, and
pecking at a key containing a white line on a colored
background may show relatively little control by the
line when tested with other orientations shown on the
same-colored background, but produce a steep gradi¬
ent of generalization if the lines are displayed with¬
out colors on a black background (Freeman 8c Thomas,
1967; Newman & Benefield, 1968). Thus some of the
stimuli displayed on the pigeon’s response key may
mask the control gained by other stimuli on the key.
Van Houten and Rudolph (1972) and Rudolph and
Van Houten (unpublished) have extended these ob¬
servations by showing that stimuli presented on a
pigeon’s response key may mask control by such rela¬
tively diffuse stimuli as a flow of air or a tone of
particular frequency. In the former experiment, pi¬
geons were reinforced for pecking a key illuminated
AUDITORY FREQUENCY (HZ)
with white light in the presence of a 30-mph flow of
air from a source behind the response key. When the Fig. 4. Relative gradients of auditory frequency generalization
in pigeons after responses to 1,000 Hz have been reinforced,
speed of this airflow was varied between 30 and 0 mph
either in the dark, or with an illuminated key light. (After
in a subsequent generalization test, three out of four Rudolph & Van Houten, unpublished data.)
492 STIMULUS CONTROL: ATTENTIONAL FACTORS

the removal of potential masking stimuli reliably in¬ group, the color projected onto the key was the same
creases the slope of such gradients. on both positive and negative trials, and the only
stimulus correlated with the availability of reinforce¬
ment was the presence or absence of the line. After
Overshadowing
acquisition, generalization tests were given to other
Rudolph and Van Houten’s studies, as they them¬ orientations of the line, displayed on either a red or a
selves recognized, are open to an alternative interpre¬ black background. The results of these generalization
tation. Control by the frequency of a tone or the tests are shown in Figure 5. The difference in the
velocity of a flow of air may not just be masked by performance of both groups between test trials when
the constant presence of the key light during testing. the lines were shown on a colored background and
The presence of this more salient visual stimulus dur¬ those trials when they were presented on an uncol¬
ing acquisition may have prevented such auditory or ored ground may be taken as evidence of a masking
tactile stimuli from acquiring control in the first effect. The presence of the colored background signifi¬
place. Such a possibility was envisaged by Lashley and cantly decreased the control over responding displayed
Wade (1946) when they argued that a flat gradient of by the line in both groups. Superimposed on this
generalization obtained when one feature of the train¬ effect, however, there is also a clear difference be¬
ing situation was varied might signify that the subject tween the gradients produced by the two groups, re¬
had attended to some other feature of the situation gardless of the type of test trial. Even when the lines
during training. were displayed on a black background, subjects for
We do not need to subscribe without reserve to whom the difference between positive and negative
Lashley and Wade’s theoretical analysis in order to trials in acquisition had been marked both by differ¬
accept the possibility of an effect such as this. That ences in color and by the presence of the line showed
the presence of a more intense or salient stimulus significantly less control by line than did subjects for
may interfere with the acquisition of control by a whom the presence of the line was the only signal for
less intense or salient stimulus was first reported by reinforcement. The presence of the additional wave-
Pavlov (1927, pp. 141-143), who termed the effect
“overshadowing/7 He reported that dogs given clas¬
sical conditioning with a compound CS containing
one intense and one weak component might show
essentially no conditioning to the weak component
presented alone on a test trial, “although it is obvi¬
ous . . . that the ineffective component . . . could
easily be made to acquire powerful conditioned prop¬
erties by independent reinforcement outside the com¬
bination” (p. 142). Evidence of overshadowing has
been reported in a variety of other situations: in CER
conditioning by Kamin (1969), in discrete-trial simul¬
taneous discrimination learning by Love joy and Rus¬
sell (1967), and in discrete-trial successive discrimina¬
tion learning by Miles and Jenkins (1973).
The principle of masking, as defined here, states
that the presence of one stimulus, A, may obscure the
expression of control by a second stimulus, B, even
though it can be shown (by testing with B in the ab¬
sence of A) that B has acquired significant control
over responding. The principle of overshadowing
states that the presence of A may interfere with the
acquisition of control by B. The distinction between
the two can best be illustrated by reference to a con¬ LINE ORIENTATION

crete experiment. Farthing (1972) trained two groups Fig. 5. Relative gradients of line-orientation generalization in
of pigeons on a successive discrimination. For one pigeons trained either on a line-orientation discrimination or
on a compound wavelength and line-orientation discrimination.
group, a vertical line on a red background served as The lines were shown with- or without a colored background.
S+, and a green key light served as S~. For the second (After Farthing, 1972.)
N. J. Mackintosh 493

length cue during training reduced the control ac¬ studies have examined the effects of providing dis¬
quired by the line during this phase of the experi¬ crimination training between two stimuli falling along
ment. a particular dimension on subsequent generalization
In Rudolph and Van Houten’s studies, the illumi¬ to other values of that dimension. Such intradimen-
nation of the key light was the stimulus that inter¬ sional discrimination training has sharpened gradi¬
fered with control by tones or airflows. The nature of ents of auditory frequency generalization in experi¬
this stimulus makes it impossible to determine whether ments on galvanic skin response (GSR) conditioning
they were observing masking, overshadowing, or a in human subjects (Hovland, 1937), eyelid condition¬
combination of the two. For in order to prove that ing in rabbits (Moore, 1972), and key pecking in
the key light had actually overshadowed these other pigeons (Jenkins 8c Harrison, 1962), and similar re¬
stimuli, it would be necessary to show that its pres¬ sults have been reported for brightness generalization
ence on test trials had not merely been masking con¬ in rats (Schlosberg 8c Solomon, 1943) and wavelength
trol. But in order to show this, it would be necessary generalization in pigeons (Honig, 1962: Honig,
to conduct test trials with the key light dark. If Thomas, & Guttman, 1959). The results of the study
pigeons have been trained to peck an illuminated key, by Honig et al. are shown in Figure 6.
however, they will, unless there is another source of The generally accepted explanation of this result

illumination, stop pecking when the key is abruptly has been some version of that proposed by Pavlov
darkened. In the absence of responding, it is impossi¬ himself. Reinforcement in the presence of one stimu¬
ble to assess the degree of control acquired by any lus and nonreinforcement in the presence of another
stimulus. are said to decrease responding to stimuli falling bfi-

This difficulty does not detract from the main con¬ tween S+ and S because the tendency to respond
clusion suggested by this body of research: a major produced by reinforcement at ST which generalizes
reason why some stimuli fail to show significant con¬ to these intervening stimuli, is counteracted by a
trol over responding is that they are either over¬ tendency not to respond, produced by nonreinforee-
shadowed or masked by the presence of more salient ment at S-, which al§c> generalizes to the intervening
stimuli. This conclusion has the further virtue, as we stimuli. The resulting postdiscrimination gradient is
shall see, of explaining why discrimination training a consequence of the interaction between the “excita¬
i§ frequently necessary to establish control by rela¬ tory” gradient centered round SI- and tho “inhibi¬

tively unsalient stimuli, A great deal of research has tory* * gradient centered round S~, an analysis first
been devoted to an examination of the effects of vari¬ formally proposed by Spence in 1937.
ous discriminative procedures on stimulus control, There is, however, good reason to believe that
and it is time to turn to this question.

IntradimensiQnal Dissriminatlon Training

Although, as was argued above, differential rein¬

forcement within the experimental situation may not
be necessary for the acquisition of stimulus control,
this should not be taken to imply that such differen¬
tial reinforcement has no effect on control. On the
contrary, discrimination training has powerful and
important effects on generalization.
Pavlov (1927) reported the most obvious instance
of this effect of discrimination training on generaliza¬
tion. If a particular tone was established as a classical
CS for food, the presentation of other tones would
also elicit salivary CRs. In order to prevent the oc¬
currence of such generalized CRs, Pavlov stated, it was
necessary to continue reinforcement in the presence of Fig. 6. Absolute gradients of wavelength generalization in pi¬
the original CS and to present the other tones without geons. Control subjects received reinforcement for responding
reinforcement. Discriminative conditioning between to S+ (550 nm), while for discrimination subjects, reinforced
trials to S+ were alternated with nonreinforced trials to S~
neighboring stimuli would thus sharpen the gradient (570 nm). After Honig et al., 1959. © 1959 by the American Psy¬
of generalization between them. Numerous other chological Association. Reprinted by permission.)
 494
STIMULUS CONTROL: ATTENTIONAL FACTORS

such an interaction between hypothetical excitatory ment between the presence and absence of a specific
and inhibitory gradients, although of considerable stimulus will increase the slope of a generalization
importance, is not sufficient to account for all features gradient measured when some feature of that stimulus
of the postdiscrimination gradient. As Figure 6 shows, is subsequently varied.
the effect of nonreinforcement at S~ is not only to The important feature of interdimensional training
increase the slope of the gradient between S+ and S~, is that the stimulus correlated with nonreinforcement
it also sharpens the gradient on the other side of during initial acquisition is presumably equidistant
S + . This decline in responding to stimuli so far re¬ from all test stimuli. Thus the increase in the slope
moved from S~ is difficult to attribute to the gen¬ of the postdiscriminative gradient cannot be attrib¬
eralization of any inhibitory tendency not to respond uted to the differential generalization of inhibitory
to S-. A similar effect can be observed in several tendencies to different stimuli. Jenkins and Harrison
other studies of intradimensional discrimination train- accepted that the type of analysis proposed by Hull
ing (e.g., Hanson, 1959; Jenkins & Harrison, 1962; (1952) provided the most plausible explanation of
Moore, 1972). It seems probable that discrimination their data. As noted above, Hull’s argument amounted
training has further effects responsible for this addi¬ to saying that if a pigeon were nondifferentially rein¬
tional sharpening of generalization gradients. This forced for responding in the presence of a 1,000-Hz
conclusion is amply confirmed by studies of interdi- tone, then although the frequency of the tone might
mensional discrimination training, where changes in be established as a signal for reinforcement, the po¬
the slope of postdiscrimination gradients cannot be tential control acquired by the tone might be masked
attributed to any interaction between excitatory and by innumerable other features of the situation, each
inhibitory tendencies. of which was as well correlated with the delivery of
reinforcement as was the tone itself. These other
!nterd{m©n5i‘9n«l Discrimination Training features remain constant during a test for auditory
frequency generalization, and would therefore main¬
Jenkins and Harrison (1960) were the first to use
tain a constant rate of responding. By providing
interdimensioftal training t<? examine the effects of
differential reinforcement between the presence and
differential reinforcement on the slope of a generaliza¬ absence of the tone, the experimenter insures that at
tion gradient, uncomplicated by interactions between
least some of these other features are now less well
excitatory and inhibitory tendencies, T hey trained
correlated with reinforcement than is the frequency
pigeons on a discrimination between the presence and
of the tone. If these differences in the “validity" of
absence of a 1,000-Hz tone and then tested for gen¬ different features result in reliable differences in their
eralization along the dimension of auditory frequency.
control over responding, then interdimensional train¬
Although, as noted above, nondifferential reinforce¬
ing may sharpen generalization gradients because it
ment resulted in a relatively flat frequency gradient,
effectively reduces control by incidental stimuli that
reinforcement in the presence of the tone, randomly
would otherwise mask the stimuli varied by the ex¬
alternated with nonreinforcement in its absence, re¬ perimenter.2
sulted in steep and reliable gradients in all subjects.
Rudolph and Van Houten’s unpublished experi¬
Jenkins and Harrison s results have been confirmed
ment, described earlier, provides the first line of evi¬
and extended in a number of subsequent studies.
dence to support this analysis. In their replication of
Newman and Baron (1965), Switalski, Lyons, and
Jenkins and Harrison’s study they showed that it was
Thomas (1966), and Lyons and Thomas (1967) have
the presence of visual stimuli, such as the key light,
shown that giving pigeons training between the
that prevented the tone from exercising control over
presence and absence of a line or specific wavelength
responding; pigeons trained and tested in the dark
on the response key significantly sharpens generaliza¬
showed orderly and steep gradients of auditory fre¬
tion gradients of orientation or wavelength. Moore quency generalization.
(1972) has reported that rabbits given differential eye¬
The second point that requires investigation is
lid conditioning with a tone as CS+ and a light as
whether interdimensional auditory discrimination
CS~ show a steeper gradient of auditory frequency
training can indeed suppress control by such masking
generalization than a control group simply given
reinforced trials with CS+ alone. Several studies of
2 As noted above, a particular stimulus may fail to control
conditioned reinforcement in rats may be taken to behavior after nondifferential reinforcement not because its
imply a similar effect (e.g., Notterman, 1951). There control is masked by other, unvarying stimuli, but because it is
overshadowed by these other more salient stimuli. This possi¬
is no question, then, but that differential reinforce¬
bility does not, of course, affect the present argument.
N. J. Mackintosh 495

visual stimuli. Miles, Mackintosh, and Westbrook often reinforced and not reinforced. This deteriora¬
(1970) trained pigeons on a discrete-trial discrimina¬ tion in the schedule of reinforcement associated with
tion with a tone as S+ and white noise as S~, but L should, Hull argued, be sufficient to reduce the
with the response key illuminated with light of a par¬ control it acquires.
ticular wavelength on both positive and negative As Jenkins and Harrison (1960) and Wagner (1969)
trials. Key color, therefore, served as a potential mask¬ have pointed out, this is a distinctly implausible sug¬
ing stimulus, and, indeed, when subjects were ini¬ gestion. By comparison with consistent reinforcement,
tially given S+ trials only, they showed strong control it is true, differential reinforcement results in a de¬
by the color of the key, responding consistently on cline in the correlation of an incidental stimulus with
test trials to the color they were trained with and not reinforcement, but it is unlikely that this decline
responding when the color was changed. The effect would in and of itself be sufficient to have any drastic
of auditory discrimination training, however, was to effect on the control such a stimulus acquires. Even
weaken this control by color; after nine sessions of the relevant stimuli in typical free operant schedules
tone-noise discrimination training, subjects responded are only intermittently correlated with reinforcement;
consistently only to the tone and showed a signifi¬ and it is well known that in discrete-trial situations a
cantly greater tendency to respond on test trials to 50% schedule of reinforcement is sufficient to estab¬
the changed key color. lish highly reliably responding. An important §£rk§
The results of Miles et al. haye been replicated in of studies by Wagner, Logan, Haberlandt, and Price
subsequent unpublished experiments by Miles. Once (1968) has confirmed that in such discrete-trial situa¬
again interdimensional auditory discrimination train¬ tions the schedule of reinforcement associated with an
ing led to a significant flattening of the generalkation incidental stimulus common to positive and negative
gradient obtained when the color of the key was trials of a discrimination k not, as §V?dh, gufficklU tO
changed; in this replication, moreover, subjects were explain why such a stimulus fails to acquire control
tested in silence, so that the decrease in control by over responding. Three experiments were conducted,
color could not have been due to any increase in con¬ one employing instrumental discrimination learning
trol by tone. Blough (1969) has also reported data in rats, a second conditioned suppression in rats, and
showing that a stimulus common to both positive the third eyelid conditioning m rabbits; but the basie
and negative trials of a discrete-trial discrimination design of each of these experiments was identical. For
will lose control over responding. It is reasonable, subjects in tbe discrimination group (hereinafter
then, to argue that differential reinforcement between called Croup TD? far true discrimination^ reinforced
any arbitrary pair of stimuli will tend to reduce con¬ trials to a tone-light compound (TiL+) alternated
trol by other stimuli common to both positive and with nonreinforced trials to another tone-light com¬
negative trials, and that the frequently observed pound (TSL“). The light, it should be noted, was
sharpening of generalization gradients resulting from common to both positive and negative trials, and
such discrimination training is a consequence of this when tested with L alone, subjects showed little or no
suppression of control by such incidental stimuli tendency to respond. Discrimination training between
which might otherwise act as effective masking stim¬ Ti and To had apparently prevented L acquiring sig¬
uli. nificant control. Instead of comparing this discrimina¬
Even if this is accepted as an explanation of the tion group with a nondifferentially reinforced group,
effects of interdimensional training, it must be clear however, Wagner et al. used a control group that also
that the principle itself stands in need of explanation. received TXL and T2L trials and also received rein¬
Why should discrimination training between the forcement on only 50% trials. The important differ¬
presence and absence of a tone, for example, reduce ence for this group (hereinafter called Group PD, for
the control exercised by visual stimuli common to pseudodiscrimination training) was that the delivery
both positive and negative trials? Hull argued that of reinforcement was uncorrelated with Tx and T2. In
this was simply a consequence of the new schedule of this PD group, L acquired strong control over re¬
reinforcement correlated with such incidental stim¬ sponding in spite of the fact that the actual schedule
uli. If we schematize the tone as T and the light as of reinforcement associated with L was exactly the
L, then nondifferential reinforcement in the presence same as in Group TD. Thus it is not the schedule of
of the tone consists of a series of TL+ trials, while reinforcement associated with an incidental stimulus
interdimensional training consists of a series of TL+ during discrimination training that suppresses control
trials alternating with L~ trials. In the former case, by such a stimulus, but the fact that there are other
L is consistently reinforced; in the latter, L is equally stimuli better correlated with reinforcement. It is not
496
STIMULUS CONTROL: ATTENTIONAL FACTORS

its absolute validity or correlation with reinforcement the subjects’ behavior. One more salient aspect may
that is the most important determinant of the control still overshadow another. If, therefore, discrimination
gained by a stimulus, but its relative validity com¬ training is given between the presence and absence of
pared to that of other available stimuli. a compound stimulus, not all features of that com¬
This conclusion is of profound importance, for it pound will necessarily acquire good control. The ex¬
implies some interaction between the control over periment by Farthing (1972), described earlier, showed
responding acquired by different stimuli. If incidental that if pigeons receive differential reinforcement be¬
stimuli fail to acquire control, not simply because tween a vertical line on a red key and no line on a
they are imperfectly correlated with reinforcement, green key, the presence of the wavelength difference
but because other stimuli are better correlated, this between positive and negative trials significantly re¬
suggests that stimuli may compete for the acquisition duced the control over responding acquired by the
of control. In the experiments of Wagner et al. the line.
auditory stimuli acquired little control over respond¬ It is even more important to note that overshadow¬
ing in the PD groups, thus enabling the light to gain ing of one aspect of a discriminative stimulus by an¬
substantial control. In the TD groups, on the other other during interdimensional training may not re¬
hand, the auditory stimuli, being perfectly correlated quire the explicit use of a compound stimulus. An
with reinforcement, acquired strong control and thus experimenter may give interdimensional training be¬
prevented the acquisition of control by the light. We tween the presence and absence of a vertical line and
may say that the light failed to gain control because it then test for control by the line by varying its orienta¬
was overshadowed by a better predictor of reinforce¬ tion on a series of test trials. But the line may be
ment.
characterized in many other ways—as having, for ex¬
The term overshadowing is usually used in Pav¬ ample, a particular size, height, width, and bright¬
lov’s original sense to refer to the effect of the pres¬ ness. There is no guarantee that the feature varied by
ence of a salient stimulus on the acquisition of control the experimenter will be the one to have gained con¬
by an equally valid but less salient stimulus. The ex¬ trol over the behavior of the subject. Interdimen¬
tension of the term to cover the case where, of two sional discrimination training may still not insure
equally salient stimuli, the more valid may interfere control by the particular feature varied during the
with the acquisition of control by the less valid im¬ generalization test.
plies a parallel between the two effects. There cer¬ A number of studies illustrate the validity of this
tainly seems to be some resemblance, and, as we shall line of reasoning. Boneau and Honig (1964) found
see later, similar theoretical analyses have been ap¬ that when pigeons were given conditional discrimina¬
plied to both effects. Whether differences in validity tion training with one of the conditional stimuli be¬
have the same effect as differences in salience, how¬ ing the presence or absence of a vertical line on the
ever, is for present purposes less important than the key, they still showed a relatively flat gradient in a
acceptance of the general principle that it is not the subsequent generalization test along the dimension of
absolute validity of a stimulus that determines its line orientation. Williams (1973) found that interdi¬
control, but whether it is accompanied by other, more mensional discrimination training between the pres¬
valid predictors of reinforcement. ence and absence of a series of clicks emitted at a rate
This principle provides the most plausible inter¬ of 2.45 per sec was not sufficient to produce a sloping
pretation of the effects of interdimensional discrimi¬ gradient to other click frequencies. Since pigeons were
nation training on the acquisition of stimulus control. well able to learn a discrimination between two click
Differential reinforcement will result in the over¬ rates, and, having done so, showed a reliable and
shadowing of a potentially wide range of stimuli com¬ steep gradient of generalization when tested with
mon to positive and negative trials, which might other rates, the failure of interdimensional training
otherwise mask, or even themselves overshadow, the to establish control by click rate cannot be attributed
stimuli in which the experimenter is interested. There to an inability to detect such differences. As Williams
is, however, one very important corollary to this analy¬ argued, it is most plausibly regarded as a consequence
sis. Discrimination training between the presence and of overshadowing; other features of the clicks, such as
absence of a discriminative stimulus may indeed re¬ their intensity or individual frequencies, were as well
sult in the overshadowing of all incidental stimuli, correlated with reinforcement, as was their rate, and
but this does not guarantee that the aspect of the by virtue, presumably, of their greater initial salience,
discriminative stimulus subsequently varied in a gen¬ may have overshadowed, or at least masked, control
eralization test will be that which gains control over by rate.
N. J. Mackintosh 497

Unfortunately, neither Boneau and Honig nor differing along the dimension subsequently varied in
Williams tested the prediction that other features of a generalization test. In interdimensional training,
their discriminative stimuli had acquired control over differential reinforcement is correlated with the pres¬
responding. Williams did not measure generalization ence or absence of a specific stimulus, some aspect of
along the dimension of intensity, nor Boneau and which is then varied in a generalization test. But dis¬
Honig along the dimensions of brightness or line crimination training can be programmed between a
length. An experiment by Mackintosh (1965), how¬ pair of stimuli quite unrelated to the set of stimuli
ever, provides some relevant evidence. Rats received varied during generalization testing. One can examine
interdimensional training between the presence and the effect of discrimination training between different
absence of a white circle of a particular size displayed wavelengths on the acquisition of control by a line or
on the window of a jumping stand. Subsequent tests by a tone. Such a test requires, of course, that re¬
between the original circle and one of a different size sponding at some point be reinforced in the presence
revealed relatively poor control by the specific size of of the line or tone. Two procedures which have been
the circle used in original training. The feature of the adopted for the provision of such reinforced experi¬
situation that had gained most control over respond¬ ence are illustrated in Table 1. In the first, successive-
ing was the brightness difference between a door con¬ stage procedure, subjects are initially trained on, say,
taining a white circle and one containing no circle. a wavelength discrimination and are then reinforced

Animals trained with the circle positive, showed a for responding to a vertical line. In the second, con¬

strong preference for the larger (i,e„ brighter) of two current procedure, the line is present during the
circles, regardless of their absolute sizes; while an¬ course of wavelength discrimination training, appear¬
imals trained to respond to a blank door, with the ing on both positive and negative trials. Since the
white circle negative, showed a stronger preference for two procedures may pose rather different problems
the smaller (i.e., less bright) of two circles, again re¬ for theoretical analysis, they will be treated sep¬

gardless of their absolute sizes. arately.

SUGGE55IVE=STAGE EXTRADIMEN5IONAL
Extradimermonal Training
Experiments

In intradimensional training, subjects are exposed That extradimensional training between one pair
to differential reinforcement correlated with stimuli q£ stimuli might enhance the eentrel apparently

Table I Schematic Representation of Two Designs for Studies of iMtradimensional Discrimination

Training and Generalisation

DESIGN GROUP TRAIN TEST

STAGE 1 STAGE 2

Blue -J-
Digcrimm&tidft
Grecn —
LW4-Ovi49itAii3fi

Successive-stage l- Vertical -j-

Genera hzai ton

B1u£ -j- j—
Control
(Pseudodiscrimination) Green +/—

Vertical on Blue -j-

Discrimination
Vertical on Green —
Line-Orientation
Concurrent
Generalization
Vertical on Blue + /—
Control
(Pseudodiscrimination) Vertical on Green + /—
498 STIMULUS CONTROL: ATTENTIONAL FACTORS

gained by an entirely different stimulus was first Table 2 Design of Experiment by Wagner (1969)
clearly shown by Honig (1969). Honig trained one
group of pigeons on a discrimination between differ¬ GROUP TRAIN TEST

ent key colors (Group TD), while a second group

received the same sequence of stimuli uncorrelated TD L,V +, L2v TXV+ Tp T2, etc.
with reinforcement (Group PD). Both groups were PD L1V±, L2V±, TxV+ Tp T2, etc.
then reinforced for pecking at a set of vertical lines
on the response key and finally received a generaliza¬ Lp L2 = Lights; Tp T2 = Tones; V = Vibra¬
tion test along the dimension of orientation. Group tory stimulus

TD showed a significantly steeper gradient than

Group PD. Results clearly related to Honig’s have
stimuli to which subjects were exposed were com¬
been reported by Eck, Noel, and Thomas (1969), who
pounds, LjV, LoV, and TV. Test trials with V alone
found that pigeons given PD training with one set of
indicated that it had acquired less control over re¬
stimuli in stage 1 learned a new discrimination be¬
sponding in Group TD than in Group PD. This
tween a new set of stimuli in stage 2 significantly
feature of Wagner’s design thus enabled him to show
faster than subjects receiving PD training in stage 1.
that the increase in control by T in Group TD was
Similarly, Frieman and Goyette (1973) confirmed that
accompanied by a decrease in control by V.
training on one discrimination would facilitate the
Wagner’s results make it possible to maintain that
learning of a second, independent problem.
discrimination training always increases control by
The argument of the preceding section was that
one set of stimuli by suppressing control by others.
discrimination training sharpens generalization gradi¬
The only new assumption required is that the inci¬
ents by effectively neutralizing incidental stimuli that
dental stimuli suppressed during the course of dis¬
would otherwise interfere with the acquisition or ex¬
crimination training remain suppressed when the
pression of control by the test stimulus. At first sight,
original discriminative stimuli are removed. A minor
Honig’s (1969) results seem inconsistent with any modification of Honig’s design7 however, produces
such analysis; it is hard to see why wavelength dis»
data which are more problematic. Thomas, Freeman,
crimination training should have had any effect on
Svinicki, Burr, and Lyons (1970, Experiments 1 and 2)
control by a subsequently presented stimulus. Wagner
confirmed Honig’s finding that birds given TD color
(1969), however, has shown how this type of analysis training before reinforced exposure to a vertical line
may be relevant to the understanding of extradimen-
would show a steeper gradient to other orientations
sional training. The critical assumption is that there
of the line than a PD group.3 In the experiments of
may be incidental situational stimuli present during
Thomas et al., however, unlike Honig’s, the vertical
all stages of training. If these stimuli are neutralized
line was shown in stage 2 compounded with one of
during initial TD training, and if this effect transfers
the discriminative stimuli from stage 1. Birds initially
to stage 2 of the experiment, they will no longer com¬
given TD or PD training with green and red stimuli,
pete for control with the new set of discriminative
with green signaling a variable-interval (VI) schedule
stimuli manipulated by the experimenter. In PD
for the TD group, were reinforced in stage 2 for re¬
groups7 on the other hand, such situational stimuli
sponding to a vertical line superimposed on a green
will gain control of responding in stage 1 and con¬
background, before being tested for generalization to
tinue to control behavior in stage 2. Wagner (1969)
other orientations of the line on a black background.
reported the results of an experiment on eyelid condi¬
Although this change in procedure appears rela¬
tioning in rabbits, which provided evidence of just
tively minor, it might be expected to have had sub¬
such an effect. The design of his experiment is shown
stantial consequences. In analyzing Honig’s results, it
in Table 2. A TD group was given discrimination
was suggested that TD training might suppress con¬
training between two lights (Lx and L2); on separate
trol by incidental situational stimuli, and that this
trials they also received reinforcement signaled by a
loss of control by potentially competing stimuli might
tone (T). This procedure produced steeper auditory
then enable the vertical line to gain more control
gradients around T than those obtained from a PD
group, which was treated identically in the presence 3 Thomas et al. also obtained similar results when TD and
of T but for whom Lj and L2 were randomly associ¬ PD training were given with two different line orientations, and
subjects were then reinforced for responding to a single wave¬
ated with reinforcement. These results, therefore, length, followed by a generalization test to other wavelengths.
replicated those obtained by Honig. Wagner also, For ease of exposition in what follows, however, it will be
however, provided an explicit incidental vibratory simpler to concentrate on their first experiment and assume
that TD and PD training is given with two wavelengths and that
stimulus (V) common to all trials; thus the actual subsequent training is given with a vertical line.
N. J. Mackintosh 499

over responding than in the PD group. Wagner’s ex¬ eralization test such birds showed reliably less control
periment confirmed that an explicitly manipulated by wavelength than a control group that had re¬
incidental stimulus would indeed gain less control ceived no training on the vertical-horizontal discrimi¬
over responding if presented in conjunction with nation in stage 1. Thus prior discrimination training
stimuli correlated with reinforcement and nonrein¬ on the vertical-horizontal problem reduced rather
forcement (in a TD group) than with stimuli uncor¬ than enhanced the control by stimuli subsequently
related with reinforcement (in a PD group). The compounded with the original discriminative stimuli.
change in procedure introduced by Thomas et al., Although similar blocking effects have not always
however, involved presenting in stage 2 the vertical been observed in similar experiments with pigeons
line itself in conjunction with one of the discrimina¬ (e.g., Farthing 8c Hearst, 1970), there is no doubt that
tive stimuli of stage 1. This might be expected to have the majority of similar studies have reported similar
reduced the control acquired by the vertical line in results (Chase, 1968; Miles, 1970; vom Saal & Jenkins,
the TD group. The principle of overshadowing ex¬ 1970).
emplified in Wagner’s data implies that TD training Why should pretraining on one component of a

will result in the suppression of control by any stim¬ compound sometimes reduce control by a second com¬
ulus presented in conjunction with the relevant dis¬ ponent, as in studies of blocking, and sometimes en¬
criminative stimuli. Although this may include situ¬ hance control by a second component, as in the studies
ational stimuli, it is hard to see why it should not also by Thomas et al.? There are, in fact? several differ¬
have included the vertical line m the experiment of ences between the design of the two type§ gf gxpgja-

Thomas et ah Nevertheless, in that experiment, as in ment. Thomas et al. compared groups given TD or
Honig’s, TD training resulted in an apparent increase PD training in stage 1, while Johnson compared a TD
in control by the vertical line. group with an untreated control group. An experi¬
It is not only a theoretical principle of possibly ment by Freeman (1967, cited by Honigj 1970), how¬

limited importance, such as that of overshadowing, ever, suggests that in this situation at least, this differ¬
and the results of rather different experiments, such ence is of no consequence.4 A second difference is in
as Wagner's, that appear to conflict with the data of the treatment of all subjects in stage % In experi¬
Thomas et ah For the design of their experiments is ments where TD training enhances control by the
in fact very similar to the design of experiments on added component, animals are exposed to a single
“blocking," and blocking has been reliably observed compound stimulus and reinforced for responding in
in studies of free operant discrimination learning in its presence j in experiments where blocking is ob¬
pigeons, with designs extremely similar to that em¬ served, animals receive discrimination training be¬
ployed by Thomas et al. Johnson (1970), for example, tween pairs of compound stimuli. Mackintosh and
initially trained pigeons on a vertical-horizontal dis¬ Honig (1970) have explicitly compared these two
crimination and then gave compound discrimination procedures by running in a single study the two pairs
training with the vertical line superimposed on a of group shown in Table 3. As can be seen from Fig-
blue background and the horizontal line superim¬
posed on a yellow background. In a subsequent gen¬ r Whether this is always true is a question discussed later.

Table 3 Design of experiment by Mackintosh and Honig

GROW STAGE 1 STAGE 2 TEST

TD V+; H-1
Wavelength
Blocking >- VB+, HY-
generalization
Control

TD V+; H-J
Wavelength
Enhancement >- VB +
generalization
Gontrol

V = Vertical line; H = Horizontal line; Y = Yellow; B = Blue

 500
STIMULUS CONTROL: ATTENTIONAL FACTORS

ENHANCEMENT design, therefore, received no discrimination training

CD
LU
If) at any stage of the experiment and, as can be seen
2
O PRETRAINED from Figure 7, gave a flatter gradient than any other
Q_
group in the experiment. If, as we have argued, dis¬
crimination training suppresses control by incidental
< stimuli, it is possible that the flat gradient shown by
h-
O
this control group was a consequence of masking by
these incidental stimuli. The suppression of inciden¬
tal stimuli in the TD group by prior training on the
2
LU line-tilt discrimination, then, may have been more
O
q: than enough to compensate for the partial overshad¬
iu
Q_
owing of the test stimulus that resulted from its pre¬
WAVELENGTH (nm) sentation in conjunction with the more valid line tilt.
In the blocking design, on the other hand, both TD
Fig. 7. Relative gradients of wavelength generalization in pi¬
and control groups received discrimination training
geons. The groups whose data are shown in the left panel
received discrimination training between vertical and horizon¬ m stage 2. In neither group, therefore, should inciden¬
tal lines superimposed on backgrounds of 501 and 576 nm; tal stimuli have succeeded in masking control by
those in the right panel were reinforced for responding to a
wavelength, and the only effect observed was the
vertical line on the 501-nm background. (After Mackintosh &
Homg, 1970. © 1970 by the American Psychological Association. overshadowing of wavelength by the previously trained
Reprinted by permission.) component.
This may well seem an unduly elaborate analysis.
In particular, one could take objection to the argu¬
ure 7, they found it possible to replicate both John¬ ment that enhancement occurs because the overshad¬
son's finding of blocking and the finding of enhance¬ owing of potentially masking situational stimuli is
ment by Thomas et al. When stage 2 involved a sufficient to outweigh the overshadowing of the test
discrimination between two lines shown on a. colored stimulus. Against this, however, it can reasonably be
background, pretraining on the line-tilt discrimina¬ insisted that if such apparently contradictory results
tion reduced the control acquired by color. When as enhancement and blocking depend upon relatively
stage 2 involved the nondifferential reinforcement of minor differences in experimental procedure, an ade¬
responding to a vertical line on a colored background, quate analysis is likely both to be complex and to
however, pretraining on the line-tilt discrimination appeal to a conflict between opposing processes. It is
tended to enhance the control gained by color. It possible, nevertheless, that enhancement is not a con¬
seems probable, therefore, that the decisive factor sequence of differences between the overshadowing of
determining the outcome of such experiments is incidental and test stimuli. It may be necessary to
whether stage 2 involves reinforcement for responding appeal to an entirely new set of principles. Thomas
in the presence of a single compound or discrimina¬ (1970) has argued that enhancement indicates the
tion training between two compound stimuli. operation of a much more general process of attentive¬
The occurrence of blocking is entirely consistent ness which may affect the control acquired by any set
with the principle of overshadowing. In studies of of stimuli. Discrimination training, he suggests, may
overshadowing, the more salient or valid member of a insure that animals learn
compound stimulus may reduce the control acquired
by the other component. In blocking, the validity of
the validity of external stimuli as signifying
one element is increased by previously establishing it events or contingencies of significance for the
as a signal for reinforcement, and such pretraining welfare of the organism. In this way the benefits
reduces the control acquired by the other element. of discrimination training would not be specific
Why, then, should pretraining on one component ap¬ to the dimension or dimensions varied in train¬
parently increase the control acquired by the other ing but might generalise to other aspects of the
in the enhancement design? The answer must be re¬ training stimulus as well. By the same token,
lated to Mackintosh and Honig’s finding that en¬ nondifferential training might serve to teach the
hancement occurred instead of blocking only when animal the insignificance of external stimuli
and/or the futility of behaving differentially in
subjects received nondifferential reinforcement for
their presence, and this learning might gener¬
responding to the stimulus subsequently varied dur-
alise to stimuli not involved in the initial train¬
ing generalization testing. The control group in this ing. (p. 324)
 N. J. Mackintosh 501

Some light on these different interpretations may be Wagner et al. replicated their findings in three
shed by examining a further set of experimental re¬ separate experiments. There is equally no doubt about
sults, those obtained in the second type of extradi- the reliability of the results of Thomas et al. They
mensional study referred to in Table 1. It is time to have been confirmed in subsequent studies by Bresna-
consider these studies. han (1970) and Turner and Mackintosh (1972) and in
a series of unpublished experiments conducted at
Concurrent Extradimensional Dalhousie University by Dr. V. Gray. An understand¬
Experiments ing of the causes of this discrepancy, therefore, is a
necessary prerequisite to any adequate theoretical
In successive extradimensional experiments, the analysis.
stimulus whose control is assessed is presented sep¬ In pursuit of his argument that the major effect of
arately from the extradimensional discriminative stim¬ discrimination training is an increase in general at¬
uli. In concurrent extradimensional experiments, the tentiveness, insuring an increase in control by all stim¬
test stimulus is presented, in conjunction with the uli, Thomas (1970) has sought to dispute the validity
discriminative stimuli, during the course of discrimi¬ of the data given by Wagner et ah and has suggested
nation training. It is, in other words, an explicit, in¬ that their conclusions are a mistaken inference from
cidental stimulus, common to positive and negative an inappropriate test procedure. Wagner et al. as¬
trials. Several recent studies of free operant discrimi¬ sessed the degree of control gained by the incidental
nation learning by pigeons have examined the effect visual Stimuli in their experiments by measuring the
of such extradimensional training on the acquisition amount of responding that occurred when that stim¬
of control by such a stimulus and have yielded rela¬ ulus was presented alone, without the auditory dis¬
tively consistent results, Thomas et al. (1970, Experi¬ criminative stimuli. In the experiments of Thomas et
ments 3 and 4), for example, gave pigeons TD or PD al. and in subsequent replications of their work, the
training with two wavelengths, with a vertical line control exercised by the incidental stimul us has been
appearing on the response key on all trials; they then assessed by varying som? aspect o£ that stimulus and
tested the birds for generalization to other orienta¬ measuring the slope of the resulting generalization
tions of the line presented on a black background. gradient. It is possible that these measures might not
Just as TD wavelength training had enhanced control coincide. Subjects in a PD group might respond at a
by a subsequently presented line, so in these studies higher rate to the incidental §timulu§ pr<j§£j!tcd alone,
ID training enhanced control by a line present dur¬ because they were less disrupted than were TD sub*
ing the course of TD training. jects by the removal of the discriminative stimuli.
Thomas (1970) has argued that this finding pro¬ Simultaneously, however, PD subjects might also re¬
vides definitive evidence that any principle of over¬ spond at a substantially higher rate when some
shadowing must at best be subservient to a much aspect of the incidental stimulus was varied in a gen¬
more important general effect of discrimination train¬ eralization test. The former measure was taken by
ing. There is, of course, no doubt that these results Wagner et al. to imply stronger control by the inci¬
are not what the principle of overshadowing would dental stimulus. The latter might imply less.
lead one to expect. This is hardly surprising, for they Thomas, Burr, and Eck (1970) trained rats in a free
are the exact opposite of the results reported by Wag¬ operant situation with results that appeared to pro¬
ner et al. (1968), which earlier provided the impetus vide some support for this argument. Rats were
for the application of the principle of overshadowing trained to press a lever in the presence of two com¬
to the effects of differential reinforcement. Wagner pound stimuli, TxLi and ToL^; for TD animals
et al. concluded that the control acquired by a stim¬ T]Li signaled a VI schedule of reinforcement and
ulus reinforced on 50% of trials was adversely affected T2Lx signaled extinction; for PD animals each com¬
by the presence of other, more valid signals of rein¬ pound signaled reinforcement and extinction equally
forcement. In their experiments, TD training between often. As is shown in Figure 8, when subjects were
two tones reduced the control displayed by a light tested with Lj alone and with a dimmer light, L2, TD
common to positive and negative trials. The design of animals responded significantly less to Lx than did PD
their studies is exactly the same as that of Thomas animals, thus apparently showing less control by the
et al., with tones instead of wavelengths, and a light light and confirming the results of Wagner et al. How¬
serving as the incidental stimulus instead of a vertical ever, since they also responded very much less to L2
line. And yet the two sets of studies produced dia¬ than did PD animals, they in fact made a higher pro¬
metrically opposed results. portion of their total test responses to Lx than did the
502 STIMULUS CONTROL: ATTENTIONAL FACTORS

a lower absolute level of responding. To say that this

represents a greater difference in the “true” response
strength to Lj and L2 is, in effect, to say that the
difference between 280 and 150 responses is really
greater than the difference between 600 and 480.
There is certainly no a priori reason why this should
be true, and it is not difficult to think of reasons, such
as a ceiling effect obscuring the true response strength
to Lx in Group PD, which would suggest exactly the
opposite conclusion. Furthermore, as is also shown
in Figure 8, when another pair of TD or PD groups
STIMULI
was tested with Lx and L2, but this time presented in
conjunction with Tx and T2, the PD animals now
Fig. 8. Effects of true (TD) and pseudo (PD) auditory discrimi¬ appear to have shown a steeper absolute gradient
nation training on generalization to different light intensities.
The first and second panels show absolute and relative gradients
than the TD animals. Thomas, Burr, and Eck ignore
in rats tested to the lights alone, where L± is the light intensity these data, and merely assert, without citing statistical
used in training. The third panel shows absolute gradients of support, that “subjects responded approximately as
generalization when rats were tested with the lights shown in
conjunction with the auditory stimuli, where T± was S+ and
much to compounds including L2 as they did to those
T2 was Sr for the TD group. (After Thomas et al., 1970. © 1970 including L.v”
by the American Psychological Association. Reprinted by per¬ There are, moreover, several experiments which
mission.)
have shown that the discrepancy between the data of
Wagner et al. and those of Thomas et al. cannot be
PD animals. On this measure, therefore, they showed resolved by pointing to differences in their procedures
a steeper relative gradient along the dimension of for measuring control. Even if the control exercised
light intensity and may be said to have shown more con¬ by an incidental stimulus is assessed by varying some
trol by the light. Thomas, Burr, and Eck thus claimed aspect of it in a generalization test, it is possible to
that there was no real discrepancy between the results confirm the finding by Wagner et al. that TD training
reported by Wagner et al. and those originally re¬ may reduce, rather than enhance, such control. What,
ported by Thomas et al. They further argued that the then, is the basis for this difference in outcome? Per¬
only proper measure of control by an incidental stim¬ haps the most obvious difference between the studies
ulus is the slope of a relative generalization gradient by Wagner et al. and Thomas et al. is that in each of
when some feature of that stimulus is varied, and that their experiments Wagner et al. employed either an
the reason why TD animals in their experiment had instrumental or a classical discrete-trial procedure,
responded at a lower rate to Lx than did PD animals while the studies by Thomas et al. and subsequent
was simply because they were more disrupted by the replications of their results have all employed free
removal of the auditory discriminative stimuli. They operant procedures. Turner and Mackintosh (1972)
thus argued that the results of all of these studies were first suggested that this might be an important factor
consistent with the proposition that TD training in¬ and presented data showing that pigeons given dis¬
creases control by an incidental stimulus common to crete-trial discrimination training might show a flatter
positive and negative trials. gradient than a PD group around a stimulus common
There are, however, features even of their own data to positive and negative trials. Gray and Mackintosh
that suggest some caution in accepting Thomas, Burr, (1973) confirmed this result. They trained pigeons on
and Eck’s conclusion, and the results of other experi¬ a series of discrete trials to peck a key illuminated
ments leave little doubt that neither their arguments with a vertical line on all trials. For TD birds, posi¬
nor their data can be accepted in their entirety. In the tive trials were signaled by a tone and negative trials
first place, there are grave problems involved in the by white noise; for PD birds the tone and noise each
interpretation of relative generalization gradients signaled reinforcement on 50% of trials. The results
when these are based on widely differing absolute of generalization tests, conducted in silence, to other
rates of responding. As Figure 8 shows, in Thomas, orientations of the line are shown in Figure 9. It can
Burr, and Eck’s experiment there is little or no differ¬ be seen that in this experiment all measures agree in
ence between TD and PD groups in the slope of the showing greater control by the line in PD animals
absolute gradient. The reason why the relative gradient than in TD animals: the PD group responded at a
is steeper for the TD group is that it is derived from higher rate to the vertical line and also showed
 N. J. Mackintosh 503

overshadow an incidental stimulus. The S+ of a dis¬

crete-trial discrimination, signaling the immediate
delivery of reinforcement on every trial, is surely a
better predictor of reinforcement than is an S+ asso¬
ciated with a variable-interval schedule in a free oper¬
ant discrimination. Thus TD training should be more
effective in suppressing control by incidental stimuli
in typical discrete-trial studies than in typical free
operant experiments. There is, unfortunately, no
evidence to support or refute these speculations. It
should be noted, moreover, that although it may ex¬
plain the direction of the difference between discrete-
LINE ORIENTATION
trial and free operant experiments, this suggestion
Fig. 9. Absolute and relative gradients of line-orientation gen¬ will not explain why TD training in a free operant
eralization in pigeons following TD or PD training using a experiment should actually increase the control dis¬
discrete-trial procedure. (After Gray & Mackintosh, 1973.) played by an incidental stimulus.
In the preceding section, the argument was ad¬

steeper absolute and relative gradients along the vanced that discrimination training might increase
control by stimulus A because it suppressed control by
dimension of line orientation.
another stimulus B, which would otherwise have over¬
It is clear, therefore, that regardless of the pro¬
cedure used to assess control, discrimination training shadowed or masked control by A, Following this
in a discrete-trial situation may decrease, rather than Ime of reasoning, Turner and Mackintosh (1972)
argued that the repetitive nature of responding on
increase, the control gained by an incidental stimulus
typical free operant schedules might provide a source
common to positive and negative trials. Thomas's re¬
of stimuli that came to control behavior and S9u!d
sults are of less generality than he has supposed.
therefore mask control by an incidental stimulus in
a PD group. If these response-produced stimuli lost
Analysis or the Effects of
Extradimension at Training
control as a consequence of TD training, an extero¬
ceptive incidental stimulus might be able to exercise
If the use of free operant or discrete-trial pro¬ more control over responding. Even If discrimination
cedures is the critical variable determining the effects training resulted in some overshadowing of this inci¬
of discrimination training on control by incidental dental stimulus by the relevant discriminative stimuli,
stimuli, it remains to attempt some interpretation of it would also suppress control by response-produced
this difference. There are numerous differences be¬ stimuli, and this latter unmasking effect might be
tween the two procedures. Which are the important even more important.
ones, and how do they come to affect the outcome of A study by Hall and Honig (1974) provides some
these experiments? support for this suggestion. They first gave pigeons
In free operant discriminations, responses to S+ are TD or PD training with red and green overhead
usually reinforced on variable-interval schedules, lights serving as discriminative stimuli and then rein¬
typically on VI 1-min schedules in the experiments of forced them for pecking at a set of vertical lines on
concern here. In discrete-trial discriminations, on the the response key, before finally testing for generaliza¬
other hand, reinforcement is typically available on all tion to other orientations of the line. One pair of TD
S+ trials. It is possible that this marked difference in and PD groups had been required to peck the key
the schedule of reinforcement associated with S+ has during initial TD and PD training, and these groups
an important effect on the experimental outcome. If confirmed the results of Thomas et ah, in that initial
it is accepted that discrimination training suppresses TD training enhanced control by the lines. The
control by incidental stimuli by insuring that other second pair of groups, however, had received free
stimuli are relatively more valid signals of reinforce¬ reinforcement not contingent on key pecking dur¬
ment, the magnitude of this effect will necessarily ing their initial exposure to the TD or PD schedules.
depend on the difference in the validity of incidental These groups, after being subsequently autoshaped to
and discriminative stimuli. The more precisely the peck the vertical line and then given VI reinforcement
presence of a discriminative stimulus signals the avail¬ for several sessions, showed no difference whatsoever
ability of reinforcement, the more successfully it will in the slope of their generalization gradients when
504 STIMULUS CONTROL: ATTENTIONAL FACTORS

tested with other orientations of the line. The only ulus. Both Hearst (1969) and Ray and Sidman (1970)
obvious difference between the two pairs of groups have gone so far as to argue that free operant sched¬
was that where TD training increased control by the ules of reinforcement may be inappropriate for the
lines, animals were repetitively pecking at a response study of exteroceptive stimulus control. The probabil¬
key during exposure to the TD or PD schedule. When ity of initiating a response (as in a discrete-trial pro¬
this one factor was changed, there was no suggestion cedure) may be a better measure of the role of extero¬
of an enhancement effect. ceptive discriminative stimuli than is the probability
These results are certainly consistent with the idea of continuing to respond.
that TD training may enhance control by some set of Turner and Mackintosh’s argument required only
exteroceptive stimuli only to the extent that such that some set of incidental stimuli, more prevalent
training suppresses control by stimuli associated with in free operant than in discrete-trial procedures,
repetitive instrumental responding. The implication should be responsible for the masking of control by a
is that such response-produced stimuli, if not neutral¬ specific incidental stimulus in PD subjects. Whether
ized by discrimination training, may mask or even or not these masking stimuli are a product of repeti¬
overshadow control by an exteroceptive stimulus. tive responding is still open to question. A further set
There is, of course, nothing novel in the suggestion of results, however, confirms that much of the effect
that free operant schedules of reinforcement may en¬ observed in the experiments of Thomas et al. is un¬
able stimuli associated with the repetitive nature of doubtedly a consequence of such masking of control
responding to gain control over the subject’s behavior. in the PD group (Honig, 1969, 1974; Turner 8c Mack¬
intosh, 1972). The design of Turner and Mackintosh’s
Under a variable interval schedule of reinforce¬ experiment is shown in Table 4. Pigeons initially re¬
ment, for example, the organism often responds ceived TD or PD training between different key
at a nearly constant rate for long periods of colors signaling variable-interval and extinction sched¬
time. All reinforcements therefore occur when
ules, with a vertical line on the key on all trials. In
it is responding at that rate, although this condi¬
the control condition, TD or PD groups were subse¬
tion is not specified by the equipment. The rate
quently reinforced for responding to a plain red key
becomes a discriminative and, in turn, a rein¬
forcing stimulus, which opposes any change to a and were then tested for generalization to other orien¬
different rate. (Skinner, 1966, p. 25) tations of the line on a black background. As in other
studies, the TD group showed the steeper gradient.
The point has been documented by Blough’s (1963) In the experimental condition, however, both TD and
studies of pigeons trained to peck a key illuminated PD subjects received several sessions of TD training
with a given wavelength on variable-interval and with a new pair of wavelengths and with no line on
differential reinforcement of low rate (DRL) sched¬ the key, before being tested for generalization along
ules. The probability of pecking within a very brief the dimension of line orientation. In these groups,
interval of a preceding peck was found to be essen¬ there was no difference between the gradients of sub¬
tially unaffected by changes in the wavelength pro¬ jects initially given TD training and those given PD
jected onto the key. Such pecks, Blough concluded, training. Additional TD training on a new pair of
were controlled more by the occurrence of preceding stimuli increased the control displayed by the line in
pecks than by any exteroceptive discriminative stim¬ PD subjects to the point where their generalization

Table 4 Design of Experiment by Turner and Mackintosh (1972)

GROUP STAGE 1 STAGE 2 TEST

TD BV+, GV-T Orientation

Control l R+
PD BV±, GV±J Generalization

TD BV+, GV-] Orientation

Experimental Lr+, y-
PD BV±, GV± J Generalization

B = Blue; G = Green; R = Red; Y = Yellow; V = Vertical line

N. J. Mackintosh 505

gradient was indistinguishable from that of TD compromise by showing that discrimination training
subjects. might not reduce control by a specific incidental stim¬
That TD training with a new pair of stimuli can ulus, even if masking effects have been controlled.
sharpen, for PD subjects, the gradient around a pre¬ The present argument has been that discrimination
viously presented incidental stimulus implies that the training always tends to suppress control by relatively
typically flat gradient produced by PD subjects in free less valid stimuli but that this effect might not always
operant experiments cannot be a consequence of any be observed because such training has also suppressed
failure of that incidental stimulus to acquire control. control by other, potentially masking stimuli. It fol¬
It must be due to a masking of such control by lows that if discrimination training could somehow
stimuli, which can then be neutralized by subsequent be given to PD subjects so as to neutralize these other
discrimination training. Honig (1969, 1974) has masking stimuli, this basic overshadowing effect
demonstrated the converse of these results. Just as would become apparent. Gray (personal communica¬
subsequent TD training can sharpen the gradient of tion) attempted to test this prediction by training
PD subjects, so subsequent PD training can flatten the pigeons to peck a vertical line superimposed on a blue
gradient of TD subjects. Honig also showed that this or green background. For a TD group, the color of
flattening can itself be reversed by further TD train¬ the background was correlated with the availability
ing. The generality and reliability of these results, PD group, no
of reinforcement; for a such correlation
therefore, leave little doubt that at least part of the existed. For both groups, however, these trials were
difference between TD and PD gradients observed in interspersed with trials on which the key light was
the experiments of Thomas ct al. and in subsequent white (with no line), and responding was not rein¬
experiments represents nothing more than the mask¬ forced. From the outset of the experiment, therefore,
ing of control in PD subjects by other irrelevant the “PD” group received discrimination trainings wh
stimuli. the line on colored backgrounds signaling occasional
Is this a sufficient account of the effects of extra reinforcement and a plain whits key Signaling nom
dimensional discrimination training? Thomas (1970),
remforeement, Since the line was the best single pre¬
as we noted earlier, has insisted that it is necessary to dictor of reinforcement, it should have acquired
appeal to a process of general attentiveness, brought strong control over responding. In the TD group, on
into play by discrimination training, which can affect the other hand, the color o£ the key wag an wen more
control by all stimuli. An increase in general atten reliable predictor of reinforcement and. by the prin¬
tiveness Will increase the control acquired not only by ciple of relative validity, should have overshadowed
relevant stimuli and subsequently presented stimuli, the line. There was no suggestion of such an effect!
but also by incidental stimuli, present but irrelevant tho slope of the line-tilt gradient wa§, indeed, margin¬
during the course of discrimination learning. It is im¬ ally steeper in the TD group than in the PD group.
portant to see whether recourse to such an analysis is
necessitated by the data.
There is good reason to question whether Thomas’s DISCUSSION

analysis is sufficient to account for the data we have

been considering. It is clear that discrimination train¬ The argument to this point has been complex and
ing in discrete-trial situations does not increase con¬ possibly tortuous. Before attempting to assess the fur¬
trol by incidental stimuli. It is equally clear that the ther theoretical implications of this argument, there¬
effect observed in free operant experiments is due not fore, it may be as well to recapitulate briefly the main
so much to an increase in the control actually ac¬ outline.
quired by the incidental stimulus as to an increase in
the probability that the control acquired by such
Recapitulation
stimuli during training will in fact be displayed in the
test situadon. It is possible, nevertheless, that discrim¬ If a change in a particular stimulus results in a
ination training does have the sort of general effect correlated change in a subject’s behavior, we may say
postulated by Thomas, in addition to the more that this stimulus controls the subject’s behavior. In
specific effects suggested here, and that these general any situation, it is obvious some stimuli will gain con¬
effects serve to counteract more selective processes. trol more rapidly than others, while yet other stimuli
The evaluation of this possibility must be a rather may apparently fail to gain control. We may cate¬
problematic affair. Some results obtained with pigeons gorize these differences as differences in salience.
by Gray, however, provide some support for such a The simplest possible account of stimulus control
506 STIMULUS CONTROL: ATTENTIONAL FACTORS

would be to say that the reinforcement of a particular tone, for example, will produce little control by the
response in the presence of a given set of stimuli will tone, because other, more salient stimuli, such as the
insure that all those stimuli will gain control over key light, will either mask or overshadow the tone.
that response at a rate determined by their salience. Differential reinforcement between the presence and
We have seen that this is an oversimplification: a absence of the tone, however, will suppress control by
stimulus may gain control without necessarily display¬ the key light and other situational stimuli and pre¬
ing that control in a particular test situation. This is vent their masking the tone. Whenever animals are
often because a change in one feature of the situa¬ exposed to differential reinforcement, therefore,
tion leaves other controlling features unchanged: the stimuli correlated with these changes in reinforcement
responding maintained by these latter features may wTill tend to acquire control over behavior, and in so
prevent the experimenter from detecting the control doing will prevent other less valid stimuli from gain¬
gained by the former. The unchanged features serve ing control, even if these other stimuli are intrinsi¬
to mask control, which can readily be detected by cally more salient. Since these other stimuli may in¬
testing in their absence. This masking effect is of no clude such features as the shape and size of the
great theoretical significance in itself5; its importance apparatus, a background masking noise, the time of
lies in the fact that the experimenter may fail to day, or the occurrence of previous responses, it is not
recognize its presence and thus fail to appreciate the surprising that differential reinforcement should have
reasons for apparent failures of stimulus control. It is such pervasive effects and should so reliably enhance
clear, for example, that if a pigeon’s key pecks are the control gained by discrete features of the experi¬
nondifferentially reinforced in the presence of a tone, mental situation, such as stimuli on a pigeon’s re¬
the tone is well able to acquire control over respond¬ sponse key.
ing. The absence of control by the tone may simply be Whether this principle is sufficient to explain all
a consequence of masking by the key light. It is also effects of discrimination training on stimulus control
probable that responding on most free operant sched¬ is still an open question. In some situations, discrim¬
ules of reinforcement is at least partially under the ination training, so far from enhancing control by
control of previous responses, which may mask con¬ relevant stimuli only at the expense of irrelevant
trol by exteroceptive stimuli. There is indeed direct stimuli, appears to enhance control by the latter also.
evidence that this is true of some of a pigeon’s key It may be possible to reconcile this observation with
pecks. Experimenters need to remind themselves that the principle of overshadowing by arguing that dis¬
the experimental situations in which they place their crimination training suppresses control by other
subjects contain a multiplicity of features, and that a irrelevant stimuli which might mask the control
failure to detect control by one feature may reflect gained by the particular incidental stimulus manip¬
nothing more than the control gained by others. ulated by the experimenter. It is also possible, how¬
A more salient stimulus might not only mask the ever, that discrimination training has additional, more
expression of control by another, it might also prevent general effects on stimulus control.
the less salient stimulus from acquiring control in the
first place. This principle of overshadowing also ap¬
plies to the case where stimuli differ in validity rather Theoretical Analysis
than in salience: a stimulus which is reliably corre¬
The argument throughout this chapter has been
lated with the occurrence of reinforcement may pre¬
theoretical in the sense that I have not listed a set of
vent one less well correlated from acquiring control
empirical conditions known to affect the slope of gen¬
over responding. Whenever discrimination training is
eralization gradients, but have rather attempted to
programmed, constant features of the experimental
reach an understanding of the principles governing
situation become less well correlated with reinforce¬
those effects. The principles invoked, however, them¬
ment than the discriminative stimuli. The principle
selves stand in need of explanation. In particular, it
of overshadowing, therefore, implies that the discrim¬
is important to consider how overshadowing and gen¬
inative stimuli will prevent these situational stimuli
eral attentiveness can be incorporated into theoretical
from acquiring control, and this may explain the
analyses of learning.
effect of discrimination training on stimulus control.
Nondifferential reinforcement in the presence of a
Overshadowing

5 This is, perhaps, an exaggeration. An adequate explanation There can be no gainsaying the fundamental im¬
of masking requires at least some assumptions about the effects
of adding together different sources of response strength. For one
portance of overshadowing. The presence of a more
set of possible assumptions, see Hull (1952, p. 66). salient or more valid stimulus is apparently able to
N. J. Mackintosh 507

interfere with the acquisition of control by less salient nomenon to require some version of the inverse hy¬
or less valid stimuli. This observation contradicts the pothesis for its explanation. It is not possible, as has
basic assumption, common to most traditional been attempted by Thomas (1970), to dismiss all
theories of learning, that all stimuli present at the instances of overshadowing or blocking as artifacts
moment of reinforcement will gain control over be¬ susceptible of alternative explanation. Nevertheless,
havior, with any differences in the control acquired by Thomas’s point is worth serious consideration, even
different stimuli being a consequence of their own if not always for the reasons he advances. There have,
salience or absolute validity. The occurrence of over¬ in fact, been several reported failures of overshadow¬
shadowing implies, on the contrary, some interaction ing, both in instrumental learning (e.g., Sutherland &
or competition between stimuli for control of be¬ Andelman, 1967) and in classical conditioning (e.g.,
havior. Schnur, 1971). Mackintosh (1974), indeed, has sug¬
One possible explanation of overshadowing is to gested that on the basis of the available evidence
appeal to what Thomas (1970) has called an “inverse overshadowing requires either marked differences in
hypothesis,” The assumption is that there is a fixed salience between overshadowing and overshadowed
upper limit to the control that can be gained by any cues or a difference in their correlation with rein¬
set of stimuli, and that this must be shared between forcement. A weak or poorly correlated stimulus will
all stimuli present in the experimental situation. The be overshadowed by another more salient or more
greater the control acquired by one subset of the avail¬ valid one, but it will not itself detract from condition¬
able stimuli, therefore, the less will be available for ing to the stronger stimulus; nor indeed is there much
others. One expression of this inverse hypothesis is to evidence that two salient or equally correlated stimuli
be found in theories of selective attention (Lovejoy, will overshadow one another. If this conclusion is
1968; Sutherland & Mackintosh, 1971), which assume substantiated by further research, it will require some
an inverse relation between the probabilities or modification of the rather rigid interpretation of the
strengths of attention to different sets of stimuli, The inverse hypothesis postulated by theories of selective
term attention is here used not in the sense employed attention or limited associative capacity.
by Honig (1970) to delimit a class of experimental Before leaving the topic of overshadowing, it i§
operations, nor yet in the sense employed by Terrace worth showing that it provides a simple explanation
(1966) to refer to residual variability in experimental of one result which has not been considered so far.
data. The meaning of the term is quite precisely de¬ We have documented the finding that in free operant
fined, by its use in a formal model, to refer to a Situations an incidental stimulus may §liOW stronger
parameter whose value determines first the amount of control oyer responding following TD training than
change in associative strengtli of a stimulus as a conse¬ after PD training. There is good evidence that a
quence of reinforcement and nonreinforcement and, major part of this difference is due to the difference
secondly, the extent to which the subjects’ behavior between the TD group and subjects receiving simple,
will be actually controlled by that stimulus rather nondifferential reinforcement, even if this is pro¬
than by another at any particular moment. grammed in an unchanging stimulus situation (e.gii
The inverse hypothesis may also be derived from a Bresnahan, 1970; Honig, 1969; Turner & Mackintosh,
theory of associative competition (Rescorla & Wagner, 1972). We have not considered whether PD training
1972; Revusky, 1971). Following a suggestion of would produce any different level of control from that
Kamin (1969), Rescorla and Wagner have argued that produced by simple nondifferential reinforcement.
changes in the associative strength of one component The question is whether, for example, nondifferential
of a compound depend upon the current associative reinforcement of a pigeon’s key pecks in the presence
strength of all other components. This will insure that of a vertical line and an unchanging wavelength
the asymptotic associative strength of one component would result in stronger control by the line than
of a reinforced compound will be inversely related to would the same schedule of reinforcement in the
the asymptotic strength of other components. Over¬ presence of the line, but with random variations in
shadowing of an auditory by a visual component is a wavelength. Although some studies have suggested
consequence of the fact that the visual component, by that these two procedures may have relatively similar
virtue of its greater salience or better correlation with effects on control in such a situation (e.g., Thomas et.
reinforcement, acquires associative strength more al., 1970, Experiment 5), there are other studies which
rapidly than the auditory component and thus re¬ have rather clearly shown that the PD treatment may
duces the associative strength available for condition¬ result in significantly flatter generalization gradients
ing to the latter. (Bresnahan, 1970; Honig, 1969, 1974; Tomie, Davitt,
Overshadowing seems a sufficiently reliable phe¬ & Thomas, 1978).
508 STIMULUS CONTROL: ATTENTIONAL FACTORS

Tomie et al. have inferred from this difference that response key, without this having any apparent effect
animals learn to be inattentive to all stimuli when ex¬ on the outcome of the experiment. In one experiment,
posed to variations in one set of stimuli uncorrelated Thomas et al. (1970) gave pigeons TD or PD training
with variations in reinforcement. It is equally possible, with different floor tilts serving as the discriminative
however, to interpret such a result in terms of Res- stimuli and found that TD training resulted in reli¬
corla and Wagner’s (1972) account of overshadowing. ably better control by the wavelength projected onto
If the flat gradients of PD subjects are a consequence the key. Hall and Honig (1974) found differences in
of control by some set of masking stimuli, then the the control gained by a set of lines projected onto the
more control such stimuli acquire, the flatter these response key after pigeons had received TD or PD
gradients will be. But if stimuli compete for the acqui¬ training between different overhead lights. Conversely,
sition of control, then the control acquired by mask¬ Gray (personal communication) has shown that if
ing stimuli must be inversely related to the control pigeons are given TD or PD training with different
acquired by any other stimuli explicitly manipulated wavelengths, with a tone present on all trials, TD sub¬
by the experimenter. A single constant wavelength jects will show better control by the tone in a subse¬
will acquire control more rapidly than will a ran¬ quent generalization test.
domly varying pair of wavelengths. In the former case, Thomas (1970) noted that results such as these
therefore, potential masking stimuli will have less ruled out the possibility that enhancement could be
chance to acquire control and will be less able to mask simply due to the establishment of appropriate observ¬
control by other stimuli subsequently varied in a gen¬ ing responses and concluded that “a central atten-
eralization test. tional mechanism seems required” (p. 327). If subjects
are exposed to a correlation between changes in
General Attentiveness stimuli and changes in reinforcement, the assumption
seems to be that they will come to expect that future
A large part of Thomas’s argument against the changes in stimulation will also be correlated with
inverse hypothesis was based on the assertion that changes in reinforcement. There is a certain plausibil¬
evidence for a process of selective attention is far out¬ ity to this idea, but it should be recognized as no more
weighed by evidence for a process of general attentive¬ than an intuitive and disturbingly vague proposal.
ness. Although this claim may be exaggerated, there Certainly no attempt has yet been made to specify a
are sufficient problems with attempts to attribute all formal model with the required properties. Until such
effects of discrimination training to the overshadow¬ a day, it may be worth seeing whether more prosaic
ing of incidental stimuli so that it becomes important processes are sufficient to account for the observed
to see whether any more precise characterization can data.
be provided of this concept of general attentiveness. A more neutral description of those data may help
What are the factors which operate to prevent the to suggest one possible set of ideas. Extradimensional
relevant stimuli of a discrimination problem from enhancement is said to occur if subjects, given a gen¬
overshadowing, and thus reducing control by, inci¬ eralization test after TD training between two wave¬
dental stimuli common to both positive and negative lengths with a vertical line always present, respond
trials? A simple suggestion, which should be consid¬ relatively infrequently to other orientations of the
ered, even if it must also in the end be dismissed, is line, whereas subjects given PD training continue to
that discrimination training establishes a set of observ¬ respond at a relatively constant rate regardless of the
ing or orienting responses which increase control by orientation of the line. If subjects are to show good
other stimuli located in proximity to the discrimina¬ control by the stimuli varied in a generalization test,
tive stimuli. Pigeons trained to discriminate between they must either never respond, or rapidly stop re¬
two wavelengths projected onto a response key will sponding, to the majority of test stimuli. Since it is
learn to look at the key and will thus learn about any only rarely that subjects do not respond at all to any
other stimuli, such as a line, that also appear on the test stimulus, one determinant of the slope of most
key. Venerable as this suggestion may be, in the generalization gradients will be the rate at which re¬
present context it is neither plausible nor empirically sponding extinguishes to such stimuli. This point is
substantiated. It is hard to see how, even in the ab¬ documented by the observation, noted earlier, that
sence of explicit discrimination training, a pigeon generalization gradients become progressively steeper
could learn to direct responses at a key without look¬ in the course of testing in extinction.
ing at it. Moreover, several studies of extradimen- It may therefore be parsimonious to attribute at
sional transfer have used stimuli not located on the least some of the difference between TD or PD gra-
N. J. Mackintosh 509

clients to a simple difference in the resistance to extinc¬ when exposed to nonreinforcement in a new situa¬
tion engendered by the two schedules. There is, in tion.6
fact, independent evidence that discrimination train¬ fn this analysis, the specific discriminative stimuli,
ing may result in the more rapid extinction of re¬ in whose presence animals receive partial or consistent
sponding to S+ than does partial reinforcement in the reinforcement, are viewed as providing a context in
presence of that stimulus (Jenkins, 1961), and several which habits of persistence or its converse are estab¬
reasonably well-defined theoretical analyses are avail¬ lished. Thus examples of extradimensional transfer of
able to explain such an effect. Amsel’s theory of par¬ TD or PD training are explained not by saying that
tial reinforcement and persistence (Amsel, 1962, 1972) animals attribute significance to external stimuli and
is readily applied to the present set of data. Animals generalize this to other stimuli, but rather by saying
exposed to a PD schedule may be assumed to have that limited changes in contextual stimuli may not
learned that responding in the face of occasional disrupt patterns of behavior that have previously been
periods of nonreinforcement will be later reinforced. established under particular conditions of reinforce¬
Responding will therefore persist in the fact of a series ment. It seems probable that there must be some
of nonreinforced test trials. After TD training, on the limits to this generalization. There may be better
other hand, subjects may learn that responding dur¬ transfer of persistence to a nonreinforced generaliza¬
ing periods of nonreinforcement will not be rein¬ tion test, if animals have been partially reinforced in
forced, and they will therefore stop responding during the presence of uncorrelated variations in external
a generalization test. Although Amsel has not applied stimuli, than if such training has been given in an un¬
his analysis specifically to the case of generalization changing stimulus situation. This would provide an
tests, he has presented an account of discrimination alternative explanation of the previously noted find¬
learning which implies that prior exposure to a schecb ing that PD training may result in even flatter gen¬
ule of partial reinforcement will retard the acquisition eralizationgradients than exposure to a comparable
of a successive discrimination by retarding the extinc¬ schedule of reinforcement in the presence of a single,
tion of responses to S_ (Amsel Sc Ward, 1965). Amsel unchanging stimulus.
and Ward's original analysis assumed that such effects
would be confined to situations where the discrimina¬
Concluding Comments
tive stimuli remained unchanged between discrim¬
ination training and prior exposure to partial rein It is worth stressing one point in conclusion. What¬
forcement. Several subsequent experiments, however, ever the merits of rival explanations of stimulus selec¬
have shown that this is incorrect and that differences tion and overshadowing on the one hand, and of rival
between partial reinforcement and either consistent or explanations of general attentiveness on the other, the
differential reinforcement transfer virtually without distinction between the principles of general and
loss even when the discriminative stimuli are changed selective changes in controlling stimuli may be diffi¬
(e.g., Flaherty & Davenport, 1972; Galbraith, 1973). cult to maintain both in fact and in logic. To say that
Such transfer is readily explained by an extension discrimination training produces 3 “set to discrim¬
of the analysis applied to the “generalized partial re¬ inate,’* which encourages differential responding to
inforcement effect” (Amsel, Rashotte, 8c MacKinnon, all stimulus dimensions, may seem to imply some
1966; Brown 2c Logan, 1965). Animals exposed to theoretical assumptions quite distinct from those im-
partial and consistent reinforcement in different alleys
respond persistently over a series of extinction trials 6 Amsel himself has always assumed that the subject’s emo¬
even if extinction is conducted in the previously con¬ tional reactions to nonreinforcement provide the source of
sistently reinforced alley. Although animals that re¬ stimulation which controls persistent behavior. This identifica¬
tion does not affect the logic of the argument. In discrete-trial
ceive only consistent reinforcement in one alley situations, where responding on one trial is reinforced after a
rapidly stop running if extinguished in that alley, ex¬ preceding nonreinforced trail, it may be as appropriate to assume
posure to partial reinforcement in another situation is that the memory of such an outcome, rather than the frustration
conditioned by that outcome, comes to control behavior (Capaldi,
sufficient to insure persistent responding in extinction. 1967). In the present context, it might be more appropriate to
Amsel refers to this finding as -a case of mediated gen¬ make some entirely different identification. The important point
eralization. The events that control behavior in ex¬ is that a free operant PD or nondifferential schedule reinforces
a steady rate of responding, even after relatively prolonged
tinction are assumed to be stimuli arising from the periods of nonreinforcement. As soon as TD subjects learn the
delivery or omission of reinforcement: animals rein¬ discrimination, they do not receive reinforcement after respond¬
forced for responding persistently in the face of non¬ ing through such prolonged periods of nonreinforcement. It
does not matter how such periods of nonreinforcement are
reinforcement in one situation will continue to do so detected.
 510 STIMULUS CONTROL: ATTENTIONAL FACTORS

plied by theories of overshadowing. It is, however, crimination and discrimination training upon stimulus
possible to argue that evidence of a general effect of control. Journal of Experimental Psychology, 1970, 86
discrimination training is always a consequence of the 155-156.
Brower, L. P. Ecological chemistry. Scientific American,
suppression of control by other stimuli which were
1969, 220, 22-29.
not measured by the experimenter. Brown, R. T., 8c Logan, F. A. Generalised partial reinforce¬
In the preceding section, an analysis of general ment effect. Journal of Comparative and Physiological
attentiveness was proposed which suggested that PD Psychology, 1965, 60, 64-69.
Capaldi, E. J. A sequential hypothesis of instrumental
subjects show less control by stimuli varied in a gen¬
learning. In K. W. Spence 8c J. T. Spence (Eds.), The
eralization test than do TD subjects, because they
psychology of learning and motivation (Vol. 1). New
have been reinforced for responding at a steady rate York: Academic Press, 1967.
through periods of nonreinforcement and continue to Chase, S. Selectivity in multidimensional stimulus control.

do so during testing. In effect, this amounts to saying Journal of Comparative and Physiological Psychology,
1968, 66, 787-792.
that they show relatively little control by the test
Dobrzecka, C., Szwejkowska, G., 8c Konorski, J. Qualita¬
stimuli, because their behavior is controlled by other
tive versus directional cues in two forms of differentia¬
events. This analysis, therefore, would seem to be a tion. Science, 163, 87—89. © 1966 by the American As¬
special case of the general proposition advanced sociation for the Advancement of Science.
earlier: procedures which increase the control exer¬ Eck, K. O., Noel, R. C., 8c Thomas, D. R. Discrimination

learning as a function of prior discrimination and non¬

cised by one set of stimuli achieve their effects by de¬
differential training. Journal of Experimental Psychol¬
creasing the control exercised by others. If, as a conse¬ ogy, 1969, 82, 156-162.
quence of discrimination training, subjects are ready Farthing, G. W. Overshadowing in the discrimination of
to attribute changes in reinforcement to changes in successive compound stimuli. Psychonomic Science, 1972,
28, 29-32.
external stimuli, as Thomas has suggested, this may
Farthing, G. W., 8c Hearst, E. Attention in the pigeon:
simply be because they have learned that such changes
Testing with compounds or elements. Learning and
are not dependent on the time of day, their own Motivation, 1970, 1, 65-78.
motivational state, the occurrence of preceding re¬ Flaherty, C. F., 8c Davenport, J. W. Successive brightness

sponses or reinforcers, or yet other events which the discrimination in rats following regular versus random
experimenter has not even suspected. intermittent reinforcement. Journal of Experimental
Psychology, 1972, 96, 1-9.
Foree, D. D., 8c LoLordo, V. M. Attention in the pigeon:
Differential effects of food-getting versus shock-avoid¬
REFERENCES
ance procedures. Journal of Comparative and Physio¬
logical Psychology, 1973, 86, 551-558.
Amsel, A. Frustrative nonreward in partial reinforcement Freeman, F. The effect of extradimensional discrimination
and discrimination learning.Psychological Review, 1962, training on the slope of the generalization gradient.
69, 306-328. Unpublished master’s thesis, Kent State University, 1967.
Amsel, A. Inhibition and mediation in classical, Pavlovian, Freeman, F., 8c Thomas, D. R. Attention vs. cue utilization
and instrumental conditioning. In R. A. Boakes & M. S. in generalization testing. Paper presented at Midwestern
Halliday (Eds.), Inhibition and learning. London: Aca¬ Psychological Association, Chicago, 1967.
demic Press, 1972. Frieman, J., 8c Goyette, G. H. Transfer of training across
Amsel, A., Rashotte, M. E., 8c MacKinnon, J. R. Partial stimulus modality and response class. Journal of Experi¬
reinforcement effects within subject and between sub¬ mental Psychology, 1973, 97, 235-241.
jects. Psychological Monographs, 1966, 80 (20, Whole Galbraith, K. Fractional anticipatory goal responses as
No. 628). cues in discrimination learning. Journal of Experimental
Amsel, A., 8c Ward, J. S. Frustration and persistence: Resist¬ Psychology, 1973, 97, 177-181.
ance to discrimination following prior experience with Ganz, L., 8c Riesen, A. H. Stimulus generalization to hue
the discriminanda. Psychological Monographs, 1965, 79 in the dark-reared macaque. Journal of Comparative
(4, Whole No. 597). and Physiological Psychology, 1962, 66, 92-99.
Blough, D. S. Interresponse time as a function of con¬ Garcia, J., 8c Ervin, F. R. A neuropsychological approach
tinuous variables: A new method and some data. Journal to appropriateness of signals and specificity of rein¬
of the Experimental Analysis of Behavior, 1963, 6, 237- forcers. Communications in Behavioral Biology, 1968, 1,
246. Part A, 389-415.
Blough, D. S. Attention shifts in a maintained discrimina¬ Garcia, J., 8c Koelling, R. A. Relation of cue to conse¬
tion. Science, 1969, 166, 125-126. quence in avoidance learning. Psychonomic Science,
Boneau, C. A., 8c Honig, W. K. Opposed generalization 1966, 4, 123-124.
gradients based upon conditional discrimination train¬ Gray, V. A., 8c Mackintosh, N. J. Control by an irrelevant
ing. Journal of Experimental Psychology, 1964, 66, 89- stimulus in discrete-trial discrimination learning by pi¬
93. geons. Bulletin of the Psychonomic Society, 1973, 1, 193—
Bresnahan, E. L. Effects of extradimensional pseudodis¬ 195.
N. J. Mackintosh 511

Guttman, N., 8c Kalish, H. I. Discriminability and stimulus of inhibition following auditory discrimination learn¬
generalization. Journal of Experimental Psychology, ing. Journal of the Experimental Analysis of Behavior,
1956, 51, 79-88. 1962, 5, 435-441.
Halgren, C. R. Latent inhibition in rats: Associative or Johnson, D. F. Determiners of selective stimulus control

nonassociative? Journal of Comparative and Physiolog¬ in the pigeon. Journal of Comparative and Physiological
ical Psychology, 1974, 86, 74-78. Psychology, 1970, 70, 298-307.
Hall, G., 8c Honig, W. K. Stimulus control after extra- Kamin, L. J. Predictability, surprise, attention, and condi¬

dimensional training in pigeons: A comparison of re¬ tioning. In B. Campbell 8c R. Church (Eds.), Punishment
sponse contingent and noncontingent training pro¬ and aversive behavior. Englewood Cliffs, N.J.: Prentice-
cedures. Journal of Comparative and Physiological Hall, Inc., 1969.
Psychology, 1974, 87, 945-952. Lashley, K. S., 8c Wade, M. The Pavlovian theory of gen¬

Hanson, H. M. Effects of discrimination training on stim¬ eralization. Psychological Review, 1946, 53, 72-87.
ulus generalization. Journal of Experimental Psychology, Lawrence, D. H. Acquired distinctiveness of cues, I: Trans¬

1959, 58, 321-354. fer between discriminations on the basis of familiarity

Hearst, E, Concurrent generalization gradients for food- with the stimulus. Journal of Experimental Psychology.
controlled and shock-controlled behavior. Journal of the 1949, 39j 770-784.
Experimental Analysis of Behavior, 1962, 5, 19-31. Lawrence, D. H. Acquired distinctiveness of cues, II: Selec¬

Hearst, E. Aversive conditioning and external stimulus tive association in a constant stimulus situation. Journal
control. In B. A. Campbell & R. M. Ghurch (Eds.), of Experimental Psychology, 1950* 10, 175-188.
Punishment and aversive behavior. Englewood Cliffs, Love joy, E, Attention in discrimination learning, San Fran¬
N.J.: Prentice-Hall, Inc., 1969. cisco: Holden-Day, 1968.
HEARST, E. Some persistent problems in the analysis of con¬ Lovejoy, E., & Russell, D. B. Suppression of learning
ditioned inhibition. In R. A. Boakes 8a M. S. Halliday about a hard cue by the presence of an easy cue. Fsy-
(Eds.), Inhibition and learning. London: Academic Press, chonomic Science, 1967. 8, 365=366.
1972. Lubow, R. E. Latent inhibition: Effects of frequency of
Heinemann, E. G., & Rudolfh, R. L. The effect of discrim¬ nonrdnforccd preexposurs of the GS. Journal of Com¬
inative training on the gradient of stimulus generaliza¬ parative and Physiological Psychology j 1965, 60 j 454—
tion. American Journal of Psychology; 1963, 76, 653-658. 459.
Hoffman, H. S., 8a Fleshler, M. Stimulus faeters in aver¬ Lubow, R. E. Latent inhibition. Psychological Bulletin,

sive controls: The generalization of conditioned sup¬ 1973, 79, 398-407.

pression. Journal of the Experimental Analysis of Be¬ Lubow, R. E., & Moore, A. U. Latent inhibition: The
havior, 1961, 4, 371-378. effect of nonreinforced preexposure to the conditioned
Honig, W. K. Prediction of preference, transposition, and stimulus. Journal of Comparative and Physiological Psy¬
transposition-reversal from the generalization gradient. chology, 1959, 52, 415-419.
Journal of Experimental Psychology, 1962, 64 > 239-248. Lyons, j„ Sc Thomas, D* R. Effects of interdimensional
Honig, W. K.Attcntional factors governing the slope of training on stimulus generalization, II. Within-subjects
the generalization gradient. In R. M. Gilbert & N. S. design. Journal of Experimental Psychologys 1967, 73s
Sutherland (Eds.), Animal discrimination learning, Lon¬ 572-574.
don: Academic Press, 1969. Mackintosh, N. J. Transposition after single-stimulus pre¬
Honig, W, K. Attention and the modulation of stimulus training. American Journo? I of Psychology} 1965, 7$, 116-
control. In D. Mostofsky (Ed.), Attention; Contempo¬ 119.
rary theory and analysts. Englewood Cliffs, N.J.; Pren¬ Mackintosh, N. J. Stimulus selection: Learning to ignore

tice-Hall, Inc., 1970. stimuli that predict no change m reinforcement. In R.

Honig, W. K. Effects of extradimensional discrimination A. Hmde 8c J. Stevenson-Hinde (EdsJ, Constraints on
training upon previously acquired stimulus control. learning. London! Academic Press, 1973.
Learning and Motivationf 1974, 5, 1-15. Mackintosh, N. J. The psychology of animal learning.
Honig, W. K., Thomas, D. R., & Guttman, N. Differential London! Academic Press, 1974.
effects of continuous extinction and discrimination Mackintosh, N. J., 8c Honig, W. K, Blocking and atten-

training on the generalization gradient. Journal of Ex¬ tional enhancement in pigeons. Journal of Comparative
perimental Psychology, 1959, 58, 145-152. and Physiological Psychology, 1970, 73, 78-85.
Hovland, C. I. The generalization of conditioned responses, Malott, M. K. Stimulus control in stimulus-deprived
I: The sensory generalization of conditioned response chickens. Journal of Comparative and Physiological Psy¬
with varying frequencies of tone. Journal of General chology, 1968, 66, 276-283.
Psychology, 1937, 17, 125-148. Mellgren, R. L., 8c Ost, J. W. P. Transfer of Pavlovian
Hull, C. L. A behavior system. New Haven: Yale Univer¬ differential conditioning to an operant discrimination.
sity Press, 1952. Journal of Comparative and Physiological Psychology,
Jenkins, H. M. The effect of discrimination training on 1969, 67, 390-394.
extinction. Journal of Experimental Psychology, 1961, Miles, C. G. Blocking the acquisition of control by an
61, 111-121. auditory stimulus with pretraining on brightness. Psycho-
Jenkins, H. M., 8c Harrison, R. H. Effect of discrimination nomic Science, 1970, 19, 133-134.
training on auditory generalization. Journal of Experi¬ Miles, C. G., 8c Jenkins, H. M. Overshadowing in operant
mental Psychology, 1960, 59, 246-253. conditioning as a function of discriminability. Learning
Jenkins, H. M., 8c Harrison, R. H. Generalization gradients and Motivation, 1973, 4, 11-27.
 512
STIMULUS CONTROL: ATTENTIONAL FACTORS

Miles, C. G., Mackintosh, N. J., 8c Westbrook, R. F. Re¬ Comparative and Physiological Psychology, 1971 76
distributing control between the elements of a com¬ 123-130.
pound stimulus. Quarterly Journal of Experimental Psy¬ Shepp, B. E., 8c Eimas, P. D. Intradimensional and extra-
chology, 1970, 22, 478-483.
dimensional shifts in the rat. Journal of Comparative
Moore, J. W. Stimulus control: Studies of auditory general¬ and Physiological Psychology, 1964, 57, 357-361.
ization in rabbits. In A. H. Black 8c W. F. Prokasy (Eds.), Shepp, B. E., 8c Schrier, A. M. Consecutive intradimen¬
Classical conditioning, Vol. 2: Current research and
sional and extradimensional shifts in monkeys. Journal
theory. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1972. of Comparative and Physiological Psychology, 1969 67
Newman, F. L., 8c Baron, M. R. Stimulus generalization 199-203.
along the dimension of angularity. Journal of Compara¬ Shettleworth, S. J. Constraints on learning. In D. S.
tive and Physiological Psychology, 1965, 60, 59-63.
Lehrman, R. A. Hinde, 8c E. Shaw (Eds.), Advances in
Newman, F. L., 8c Benefield, R. L. Stimulus control, cue
the study of behavior. New York: Academic Press, 1972.
utilization, and attention. Effects of discrimination train¬ Skinner, B. F. Operant behavior. In W. K. Honig (Ed.),
ing. Journal of Comparative and Physiological Psychol¬
Operant behavior: Areas of research and application.
ogy, 1968, 66, 101-104.
Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966.
Notterman, J. M. A study of some relations among aperi¬
Spence, K. W. The differential response in animals to
odic reinforcement, discrimination training, and sec¬
stimuli varying within a single dimension. Psychological
ondary reinforcement. Journal of Experimental Psychol¬ Review, 1937, 44, 430-444.
ogy, 1951, 41, 161-169.
Sutherland, N. S., 8c Andelman, L. Learning with one
Pavlov, I. P. Conditioned reflexes. Oxford: Oxford Uni¬
and two cues. Psychonomic Science, 1967, 15, 253-254.
versity Press, 1927.
Sutherland, N. S., 8c Mackintosh, N. J. Mechanisms of
Peterson, N. Effect of monochromatic rearing on the con¬ animal discrimination learning. New York: Academic
trol of responding by wavelength. Science, 1962, 136, Press, 1971.
774-775.
Switalski, R. W., Lyons, J., 8c Thomas, D. R. Effects of
Ray, B. A., & Sidman, M. Reinforcement schedules and
interdimensional training on stimulus generalization.
stimulus control. In W. N. Schoenfeld (Ed.), The theory Journal of Experimental Psychology, 1966, 72, 661-666.
of reinforcement schedules. Englewood Cliffs, N.J.: Terrace, H. S. Stimulus control, In W. K. Honig (Ed.),
Prentice-Hall, Inc., 1970.
Operant behavior: Areas of research and application.
Reiss, S.j 8c Wagner, A. R. CS habituation produces a Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966.
“latent inhibition effect” but no active “conditioned Thomas, D. R. Stimulus selection, attention, and related
inhibition.** Learning and Motivation, 1972, 3, 227-245. matters. In J. H. Reynierse (Ed.), Current issues in
Rescorla, R. A. Summation and retardation tests of latent animal learning. Lincoln: University of Nebraska Press,
inhibition. Journal of Comparative and Physiological 1970.
Psychology, 1971, 75, 77-81. Thomas, D. R., 8c Rarker, E. C. The effects of extinction
Rescorla, R, A., 8c Wagner, A. R. A theory of Pavlovian and ‘central tendency” on stimulus generalization in
conditioning: Variations in the effectiveness of reinforce¬ pigeons. Psychonomic Science, 1964, 1, 119-120.
ment and nonreinforcement. In A. H. Black & W. F. Thomas, D. R., Burr, D. E. S., 8c Eck, K. O. Stimulus selec¬
Prokasy (Eds.), Classical conditioning, Vol. 2: Current tion in animal discrimination learning: An alternative
research and theory. Englewood Cliffs, N.J.: Prentice- interpretation. Journal of Experimental Psychology,
Hall, Inc., 1972. 1970, 86, 53-62.
Revusky, S. The role of interference in association over a Thomas, D. R., Ernst, A. J., & Andry, D. K. More on mask¬
delay. In W. K. Honig 8c P. H. R. James (Eds.), Animal ing of Stimulus control during generalization testing.
memory. New York: Academic Press, 1971. Psychonomic Science, 1971, 23, 85-86.
Revusky, S., 8c Garcia, J. Learned associations over long Thomas, D. R., Freeman, F., Svinicki, J. G„ Burr, D. E. S.,
delays. In G. H. Bower (Ed.), The psychology of learn¬ 8cLyons, J. Effects of extradimensional training on

ing and motivation (Vol. 4). New York: Academic Press, stimulus generalization. Journal of Experimental Psy¬
1970. chology Monograph, 1970, 83 (1, Part 2).
Rozin, P., & Kalat, J, W. Specific hungers and poisoning Thomas, D. R., Mariner, R. W., & Sherry, G. Role of

as adaptive specializations of learning. Psychological preexperimental experience in the development of stim¬

Review, 1971, 78, 459-486. ulus control. Journal of Experimental Psychology 1969
79, 375-376.
Rudolph, R., 8c Honig, W. K. Effects of monochromatic
Thomas, D. R., Svinicki, M. D., 8c Svinicki, J. G. Masking
rearing on spectral discrimination learning and the
of stimulus control during generalization testing. Journal
peak shift in chicks. Journal of the Experimental Analy¬
of Experimental Psychology, 1970, 84, 479-482.
sis of Behavior, 1972, 17, 107-111.
Tomie, A., Davitt, G. A., 8c Thomas, D. R. Role of stimu¬
Rudolph, R. L., Honig, W. K., 8c Gerry, J. E. Effects of
lus similarity in equivalence training. Journal of Experi¬
monochromatic rearing on the acquisition of stimulus
mental Psychology, 1973, 101, 146-150.
control. Journal of Comparative and Physiological Psy¬
Tracy, W. K. Wavelength generalization and preference
chology, 1969, 67, 50-58.
in monochromatically reared ducklings. Journal of the
Schlosberg, H., 8c Solomon, R. L. Latency of response in
Experimental Analysis of Behavior, 1970, 13, 163-178.
a choice of discrimination. Journal of Experimental
Turner, C., 8c Mackintosh, N. J. Stimulus selection and
Psychology, 1943, 33, 22-39.
irrelevant stimuli in discrimination learning by pigeons.
Schnur, P. Selective attention: Effect of element preexpo¬ Journal of Comparative and Physiological Psychology
sure on compound conditioning in rats. Journal of 1972, 78, 1-9.
N. J. Mackintosh 513

Van Houten, R., 8c Rudolph, R. The development of Stimulus selection in animal discrimination learning.
stimulus control with and without a lighted key. Journal Journal of Experimental Psychology, 1968, 76, 171-180.
of the Experimental Analysis of Behavior, 1972, 18, 217- Wilcoxon, H. C., Dragoin, W. B., 8c Kral, P. A. Illness-
222. induced aversions in rat and quail: Relative salience
vom Saal, W., 8c Jenkins, H. M. Blocking the development of visual and gustatory cues. Science, 1971, 171, 826-828.
of stimulus control. Learning and Motivation, 1970, 1, Williams, B. A. The failure of stimulus control after pres¬
52-64. ence-absence discrimination of click-rate. Journal of the
Wagner, A. R. Incidental stimuli and discrimination learn¬ Experimental Analysis of Behavior, 1973, 20, 23-27.
ing. In R. M. Gilbert 8c N. S. Sutherland (Eds.), Animal Zentall, T. Attention in the pigeon: Novelty effects and
discrimination learning. London: Academic Press, 1969. testing with compounds. Psychonomic Science, 1972, 27,
Wagner, A. R., Logan, F. A., Haberlandt, K., 8c Price, T. 31-32.
 17

Animal Psychophysics*

Donald Blough
and
Patricia Blough

INTRODUCTION physical data from the same species, because a com¬

plete understanding of sensory functioning within a
This chapter concerns the assessment, by operant species requires interlocking anatomical, physiological,
methods, of the sensory and perceptual capacities of and behavioral data. A broad understanding of sen¬
nonverbal animals. Though behavioral methods have sory functioning also requires cross-species compari¬
been used for such purposes since early in the century, sons. For example, when systems differ character¬
istically in structure, psychophysical data may reveal
modern operant techniques have markedly increased
their efficiency. Many methodological variations have corresponding functional differences. Already such
appeared; instrumentation has been vastly improved, correspondences have contributed a good deal to our
and the number of species and problems studies has understanding of the way sensory systems work, and
they suggest fascinating possibilities for future re¬
multiplied. Experimenters have successfully answered
search.
questions that would have been out of reach just a
This chapter will focus on research and methods
few years ago, and further rapid development is in
prospect. that seek information regarding sensory and percep¬
tual systems. Problems in some related areas of stim¬
This burst of activity in animal psychophysics has
ulus control, although they are sometimes considered
been prompted in part by the rapid expansion of re¬
search on the anatomy and physiology of sensory proc¬ part of “animal psychophysics/’ are treated elsewhere
in this volume, and we shall not dwell on them. We
esses. For such experiments, nonhuman organisms
shall stress studies that seek a functional relationship
have, of course, been popular subjects. The result¬
between a carefully defined response and a carefully
ing findings have called for corresponding psycho-
controlled stimulus dimension, and our emphasis will
be on methods that have been successful in describing
* Preparation of this chapter was partially supported by such relationships. Thus more consideration will be
USPHS grant MY-02456. We thank Dr. Charles Shimp and the
Psychology Department, University of Utah, for providing the
given to studies that show how a response measure
facilities that we used during the writing of this chapter. varies across a range of stimulus values than to studies

514
Donald Blough and Patricia Blough 515

that simply inquire about an animal’s ability to dis¬ With nonverbal subjects, such studies usually begin
criminate between two values. The more limited ex¬ by establishing a discrimination between very differ¬
periments have helped in the development of be¬ ent stimuli. Threshold determination involves the
havioral techniques, and have been useful in answer¬ examination of the discriminative response to a range
ing such questions as “Do cats have color vision?” of stimulus strengths, and the threshold itself is taken
However, detailed functional relationships may an¬ to be the stimulus value or difference that yields some
swer not only the question “Do they?” but also “What arbitrary criterion of discriminability.
kind?” and “How much?” The next two sections examine methods of achiev¬
Although this chapter will concentrate on operant ing the stimulus-response relationship upon which
techniques, it is well to remember that other be¬ threshold determination rests. First, we consider vari¬
havioral methods, involving reflexes or classical con¬ ous response measures and their associated reinforce¬
ditioning, may best serve particular purposes. Indeed, ment contingencies, and then we deal with methods of
for some species, operant methods simply may not stimulus presentation.
work. In the frog, for example, psychophysical data
are badly needed to supplement rapidly mounting
data on visual anatomy and physiology, but as yet Response Movseres
frogs have proven refractory to positive reinforcement
techniques. Methods That Use a Single

Within the operant paradigm, the choice of a Response Manipulanpum

method often hinges on the achievement of maximal
Single-response methods may be chosen for several
stimulus control and simple, quantitative indices of
reasons. They are behaviorally and conceptually
this control. This means that operant methods favored
simple; they are relatively easy to instrument; and
for other purposes may be inappropriate in psycho¬
they keep the subject in a reasonably constant posi¬
physical research. For example, the rate of a free oper¬
tion. We shall distinguish between “go/no—go”
ant as exhibited on a cumulative record will rarely be
methods, where the stimulus dimension controls a dis¬
seen in studies cited here; many experiments use dis¬
criminated operant that is maintained by appropriate
crete-trial methods in some ways resembling the
reinforcement contingencies, and the “conditioned
mazes and jumping stands of earlier decades. How¬
suppression” methods, where the stimulus is paired
ever, the modern use of older methods involves a
with an aversive stimulus and controls the suppression
more thorough analysis of the situation than did
of an ongoing operant.
earlier applications. Extraneous cues arising from
many sources are carefully eliminated; competing Go/no—go Methods. To an operant psychologist,
sources of control are minimized, and appropriate an attractive form of go /no—go procedure might be a
orienting and fixating responses are often an integral simple multiple schedule in which discrimination is
part of the behavioral picture. assessed from the relative response rates during posi¬
In the following pages we shall consider first some tive stimulus (S+) and negative stimulus (S~) condi¬
techniques used to measure sensory thresholds in tions. Unless special precautions are taken, however,
animals and some approaches to the many problems this method has at least two important d£f£<5t§i (1) tho
that arise in this area. We shall next deal with percep¬ occurrence of reinforcement in an S4- period may act
tual and scaling studies. This research involves supra- as a cue for responding, and (2) the appearance of the
threshold stimuli that raise special problems of stim¬ S+ may act to reinforce responses in the preceding S
ulus control. Finally, we shall consider the role of period. A study by Raslear, Pierrel-Sorrentino, and
signal detection theory in animal psychophysics, out¬ Brissey (1975) shows how reinforcement cues may con¬
lining relevant methods and examining the applic¬ found stimulus effects when variable interval and
ability and the usefulness of this approach. extinction alternate in a multiple schedule (mult VI
EXT). This research examined discriminations of
auditory intensity in the chinchilla, using the absence
MEASURING SENSORY THRESHOLDS of sound as the S+ and its presence, at varying inten¬
sities, as the S~. Since the design included unrein¬
In threshold studies, the psychophysicist seeks an forced as well as reinforced presentations of the S+,
index of his subject’s sensory capacity either in terms the authors were able to assess the role of reinforce¬
of the minimum perceptible stimulus strength or the ment as a cue. At large intensity differences, a measure
minimum perceptible difference between two stimuli. based on unreinforced S+ periods showed about the
 516 ANIMAL PSYCHOPHYSICS

same discriminability as a measure based on rein¬ was more explicitly defined in terms of its topography
forced S+ presentations. As intensity difference de¬ and also differed in its temporal relation to signal on¬
creased, however, reinforcement cues appeared to as¬ set, since it could occur any time during the 3-sec
sume more importance, so that the measure based on stimulus presentation and still produce reinforcement.
reinforced S+ presentations continued to show differ¬ The discriminated avoidance response has also
ential responding even at very small intensity differ¬ been used in psychophysical studies, especially with
ences. species that are not readily amenable to positive rein¬
The go/no—go paradigm may also be used in forcement procedures. In go/no—go avoidance a re¬
discrete-trial methods that program very short stim¬ sponse is maintained because it removes a stimulus
ulus presentations and measure the probability that a signaling shock or some other aversive event. Varia¬
single response will occur on each trial. This prob¬ tions in the stimulus along some dimension yield
ability measure is often favored over response rate be¬ corresponding changes in response probability, which
cause of its relative simplicity and because response describe a psychophysical function. A nice example
rate appears to be affected by complex nonstimulus of this procedure has been reported by Clack and Her¬
factors. Furthermore, reinforcement cues cannot con¬ man (1963), who used it to obtain a set of auditory
found stimulus effects, since reinforcement terminates thresholds in the monkey. Their sessions were divided
the trial. into trials, each starting with the presentation of a
As a rule, go/no—go methods employ the traditional white light. On some trials a tone followed the onset
reinforcement contingencies; that is, they program re¬ of this light; the tone signaled shock, and lever presses
inforcement of responses in the presence of the stim¬ in its presence avoided the shock. A lever press in the
ulus and their extinction in its absence. However, presence of the tone was a hit, and was reinforced by
since subject chambers and manipulanda are usually shock avoidance. A press in the presence of the white
designed with a view to facilitating the response, these light but in the absence of the tone was a false alarm
procedures often favor excessive “false positive” re¬ and was punished by shock as, of course, were misses
sponses. That is, subjects tend to err in the direction —failures to respond during the tone. A similar pro¬
of responding when the stimulus is absent, rather than cedure has been used by Saunders (1969) in a study of
failing to respond when it is present. Signal detection auditory intensity discrimination in the cat, a species
theory has focused attention on these two types of known for its finicky attitude toward food reinforcers.
error (“false alarms” and “misses”) and suggests that Although the use of electric shock has been effec¬
an imbalance between them indicates a “biased” tive, it has certain disadvantages. Some species-
subject. pigeons, for example—require wires that may inter¬
An experiment by Terman and Terman (1972) fere with the animaFs freedom of movement. More
shows how a go/no—go situation may be modified in important is the possible disruptive effect of punish¬
an attempt to discourage response bias. In a study of ment on the subject’s behavior. Nonetheless, one com¬
auditory intensity discrimination in rats, these authors parison between positive and negative reinforcement
introduced symmetrical contingencies for the two types techniques (Sidley, Sperling, Bedarf, & Hiss, 1965) re¬
of error. The rats were required to press a nose key in ports that, while the positive reinforcement method
the presence of a standard stimulus and to withhold yielded more “cooperative” subjects, similar spectral
the response in the presence of a comparison intensity. sensitivity functions were generated by the two pro¬
Time-outs followed both failures to press during the cedures.
standard stimulus presentation and the occurrence of
presses during the comparison intensities. Positive Conditioned Suppression Method. Under suitable
reinforcement was also symmetrical; that is, it fol¬ conditions an aversive signal will suppress the rate of
lowed both “hits” (key responses to the standard stim¬ an operant response, and the degree of this suppres¬
ulus) and “correct rejections” (withholding the re¬ sion may be used as a psychophysical response
sponse during the comparison stimulus presentations). measure. The method involves the superposition on a
Despite these contingencies, Terman and Terman base line operant of signals that terminate after a
found that their subjects tended to be biased in the minute or so with a shock or other aversive event. As
direction of responding, for false alarms were more a result of such pairings, the signal itself acquires sup¬
probable than misses. The authors suggest that differ¬ pressing properties. The amount of suppression is
ences in the topographies of the response for hits measured by the change in the baseline response rate,
(“press”) and correct rejections (“don’t press”) might and it usually is stated in terms of a ratio composed
account for some of the bias. The nose key response of rates before and after the introduction of the
Donald Blough and Patricia Blough 517

signal. A stable baseline is important, since pauses in withhold the response on others. A comparable pro¬
the absence of the signal can be recorded as “false cedure with animals requires a manipulandum for
alarms.” Variable-interval and variable-ratio schedules each response alternative. An advantage of such
are frequently used to maintain such a baseline. (For symmetrical response requirements is that the experi¬
an extensive discussion of the suppression method and menter can then distinguish “failures to respond”
related data see Blackman, Chapter 12 in this from “incorrect responses” and is thus unlikely to
volume.) confuse poor attention, “freezing,” and so forth with
The psychophysical function resulting from the poor stimulus discriminability. Response bias may
suppression method describes the relationship be¬ also be minimized to the extent that the alternative
tween some dimension of the aversive signal and the responses are comparable with one another and rein¬
amount of suppression that the signal produces. Many forcement contingencies are symmetrical.
such functions have been acquired by Smith (1970)
and his colleagues for a variety of species and sensory Forccd-choicc Procedures. The most straightfor¬
modalities. One study describes olfactory sensitivity in ward of the multiresponse techniques is the two-re¬
the pigeon (Henton, 1969). Using as a baseline the sponse forced-choice method, used extensively in visual
key-pecking response maintained on a variable-inter¬ psychophysics, Here, each of two stimuli appear near
val schedule, delivery of controlled amounts of amyl their associated response manipulanda, and the ani¬
acetate was paired with shock at the birds’ pubis mal is reinforced for making the response correspond¬
bones. When a strong concentration of the odorant ing to the “correct” stimulus, which might be defined
had acquired good suppressing properties, the concen¬ as “the one that is illuminated,” “the one that has
tration was gradually decreased. A psychophysical the stripes,” or perhaps “the one that is the brighter.”
function related the odorant concentration to the sup¬ The position of the correct stimulus should vary, of
pression ratio. course, in such a manner that consistent responding to
A series of studies on the auditory sensitivity of one of the manipulanda results in chance perfor¬
neurologically mutant mice (Ray, 1970; Sidman, Ray, mance. Although this method is most frequently used
Sidman, & Klinger, 1966) examined the reliability and for problems in vision, Wilson, Stamm, and Pribram
validity of the suppression method. This research (1960) successfully investigated tactual discrimination
produced such data a§, for example, the finding that in monkeys by training their subjects to choose the
mice with inner-ear defects failed to suppress to audi¬ coarser of two grades of sandpaper.
tory stimuli, although suppression to visual stimuli Frequently, a third, “observing” response is added
was still present. These studies support the reliability to the forced-choice method, as in a series of studies
of the suppression method; the data replicated well on goldfish color vision described by Yager and
within subjects, and agreement between subjects was Thorpe (1970). These investigators used tanks that
also good. It should be noted, however, that while contained two response keys at one end and an ob¬
conditioned suppression is measured in terms of oper¬ serving key at the other, A press on the illuminated
ant response rate, the phenomenon is generally con¬ observing key presented the stimulus and set up rein¬
sidered an instance of classical conditioning (see forcement on one of two response keys* the fish then
Chap. 12). As with other differences between some of swam to the other end of the tank and made its choice
the methods we describe, we do not know the extent response. In their work on spectral sensitivity, Yager
to which thresholds measured by suppression might and Thorpe caused the observing response to illu¬
differ from thresholds based on a discriminated oper¬ minate one of the two choice keys with monochromatic
ant. Even were such comparisons done, they may be light, and a response to this lighted key brought food.
difficult to interpret, because obtained thresholds may Following the choice response was an intertrial inter¬
depend on so many aspects of the procedures used to val during which the entire tank was illuminated so as
measure them. to produce chromatic adaptation conditions desired
by the experimenters. Using a similar method, P.
Methods That Use Two or More Blough (1971) assessed pigeon visual acuity at con¬
Response Manipulanda trolled target distances. Here, one purpose of the ob¬
serving response was to force the bird to retreat a
In most human psychophysics, the subject is in¬ considerable distance from the target stimuli before
structed to give a specifically defined response on each each trial. Again, operation of an observing key at one
trial—“yes” or “no,” “brighter” or “dimmer,” or the end of the experimental chamber set up stimuli at the
like—rather than to respond on some trials and to other end. One stimulus key was striped, the other
 518 ANIMAL PSYCHOPHYSICS

in succession, as in most auditory tasks, human sub¬

jects may be instructed to choose the earlier or later
stimulus, but a corresponding task is probably too
difficult for most nonhuman animals. For such prob¬
lems, the two-response “yes-no” task is a popular al¬
ternative. Here, a response to one manipulandum is
reinforced if a specified condition is present (the “yes”
.
Fig. 1 An experimental chamber illustrating the use of an response) and to another if the condition is absent
observing response combined with a two-key forced-choice task.
To initiate a trial, the bird pecks the observing key, thus (the “no” response). In this task, of course, response
opening the shutter and illuminating the two stimulus keys. As preferences may affect the data by altering the a
the bird enters an alley leading to one of the two keys, it
priori ability of “yes” or “no” responses. A bias toward
breaks a photocell beam, thus recording its choice and initiating
the appropriate consequences. If the choice is correct, the the “yes” key tends to drive the “threshold” down,
shutter remains open until the bird pecks the stimulus key for while a bias toward the “no” key drives it up. Work
food reinforcement. An incorrect choice closes the shutter at
of Irwin and Terman (1970) on auditory sensitivity in
once. The purpose of the hurdles is to help delay the choice
response. (From P. Plough, 1971. © 1971 by the Society for the rats illustrates the use of this method. To start a trial,
Experimental Analysis of Behavior, Inc.) the rat positioned^itself so as to break a photocell
beam. Tins caused the lighting of two choice keys and
blank, and pecks at the striped key produced rein¬ also produced either noise alone or a tone added to
forcement. Figure 1 shows that a vertical partition noise. The animal was to press the left key for noise
and photocell arrangement forced the bird to make its alone and the right key for noise plus tone. Electrical
choice while it was still at the far end of the chamber. brain stimulation reinforced a correct response to
Although it complicates the apparatus, a three- or either key; a time out followed each error. One of

four-response forced-choice procedure has desirable the two rat subjects reached approximately 95% cor¬
aspects. Additional choices extend the range between rect responses when the tone signal was strong; the
perfect and chance performance; for example} while other rat’s best performance was about 85% correct.
Despite the symmetrical reinforcement contingencies,
chance performance is 50% for a two-response method,
it falls to 25% for a four-response method. Three or both rats showed position preferences that increased
more responses also allow the experimenter to ask as the signal became more difficult to detect.
Pigeons can learn a variation of the yes-no method
that a subject choose the '‘different” stimulus. This
more general paradigm may be useful for a variety of that is reminiscent of DeValois's and Jacobs’s use of
problems, at least for subjects capable of responding forced choice with oddity problems. Birds respond
to one key if two stimuli match and to another if the
to the “difference” concept. DeValois and his col¬
stimuli differ. Wright (1972) used this method to
leagues have collected psychophysical data on monkey
color vision with the multiresponse procedure, illus¬ study pigeon wavelength discrimination. A center ob¬
trated by a study of increment thresholds reported by serving key contained a bipartite field whose halves
could be illuminated by the same or by different
Jacobs (1972). A tungsten source illuminated three
response panels, and a monochromatic light was added monochromatic lights. A peck on this key illuminated
to this background on one of the three panels. Squirrel two side keys, and a right-key response was then cor¬
monkey subjects were reinforced for responding to rect if the stimulus lights were the same, while a left-
the panel with the added monochromatic light, what¬ key response was correct if they differed. Both types
of correct responses were reinforced, and errors pro¬
ever its wavelength. Each of the three panels was
correct equally often. By varying wavelength and in¬ duced time-outs. During each session, a single refer¬
tensity of the added light, Jacobs measured sensitivity ence wavelength appeared with each of a number of
under these conditions across a broad range of wave¬ comparison wavelengths. Unfortunately, it is not clear
lengths. how well the birds discriminated “difference” on the
center key as a concept apart from the specific wave¬
“Yes-no” Procedures. Forced-choice procedures us¬ length differences to which they responded. That is,
ing two or more keys may not be as subject to the the report does not indicate whether the birds learned
effects of response bias as are less symmetrical para¬ each new discrimination more quickly than the last.
digms. Unfortunately, successful use of forced-choice Evidently the task was difficult, however, for 6 out of
with animals probably necessitates the simultaneous 12 birds were eliminated from the study because of
presentation of two or more stimuli that can be ade¬ poor performances.
quately separated in space. When stimuli must occur A variation of the yes-no procedure, described by
 Donald Blough and Patricia Blough 519

Stebbins (1970) and some of his colleagues, has been are, however, rather difficult to interpret, since many
particularly successful in auditory psychophysics. Here, aspects of each situation are chosen arbitrarily (e.g.,
one manipulandum has both an observing function stimulus placement, response force required, etc.),
and a “no” function. Responses to this lever produce and effects of these parameters could obscure real
the auditory signal on a variable-interval or variable- methodological effects.
ratio schedule. The subject must switch to another
lever (“yes”) during the signal to be reinforced; how¬ Adapting the Response to the Species
ever, switching to this second lever either before or
after the signal brings a time-out. Occasional “catch” The animal psychophysicist often wishes to study
trials, when no tone is presented, measure the prob¬ species other than the rats, pigeons, or monkeys for
ability of false alarms. Methods such as this have which reinforcers, manipulanda, and general meth¬
yielded dear auditory data for monkeys (Gourevitch, odology have become standardized. Some species, such
1970; Stebbins, 1970) and bats (Dalland, 1970). The as the cat or the turtle, may be chosen because exten¬
procedure has also been used to measure latencies, as sive sensory physiological data are available. Others,
we shall see m a later section. Figure 2 shows a set of such as the bat or tbe sea lion, are of interest because

psychophysical functions obtained by this method of the special characteristics of a sensory system, Often,
with monkey subject. such “new” species are found to perform well in rela¬
As we have indicated, the measures described above, tively standard situations: for example, turtles, squir¬
though all useful, differ in their suitability for par¬ rels, seals, and chinchillas all press keys or levers for
ticular sensory modalities, in their ease of instrumen¬ appropriate consummatory rewards. But some species
tation, and very likely in their suitability for particu¬ do not adapt wHl to such familiar tasks, and the Ex¬
lar species. Unfortunately, we are generally unable to perimenter must apply his imagination and his knowl¬

say whether they also differ with respect to the edge e>£ the response repertoire of his subjects to the
threshold measures that they yield. A few studies sug= design of appropriate experimental arrangements. Re-
gest that methodological differences may not be great; search with bats, described by Dalland (1970), ex¬
the experiments of Sidley et al. (1961?), mentioned emplifies this point. The experiment., inspired fey the
above, showed similar findings from a method based bad§ y£ echofocatfon in detecting prey} involved
on positive reinforcement and one based on aversive the assessment of the auditory response to very-high*
control. Mentzer’e (1966) research also failed to reveal frequency tones. Since the nonflying movements of
any clear difference among thresholds obtained from bats are minimal and poorly adapted to the usual
yes-ng, two-key forced-choice, and four-key forced- manipulanda^ Dalland chose to measure gross bodily

choice methods. Studies that make such comparisons movements, An observing response required the bat
to position itself on a platform in such a way as to
break a photocell beam. In front of this platform wa§
a trihe fr^m which the auditory signal emanated.
Thus the observing response harmonized with the
bat’s natural tendency to orient toward sounds, and it
also ensured constant position with regard to the
Hearing

stimulus. In this procedure, the bat was reinforced, in

the presence of sound, for walking to a food cup. Fig¬
of

ure 3 illustrates this arrangement. Since bats do not

Frequency

readily walk, this response had the advantage of re¬

ducing the probability of false alarms. Although ex¬
tended response shaping was required, Dalland suc-
cessfuly measured tone thresholds through a range of
high frequencies.
Naturalistic considerations also suggested a useful
response in a study of visual acuity of sea lions. These
animals emit a “barking” vocalization which the ex¬
perimenters, Schusterman and Balliet (1970), brought
Fig. 2. A set of psychophysical functions for a single monkey under visual control, at first by associating the visual
showing the relation between frequency of hearing and sound
intensity. Each curve describes this relationship for a different
stimulus with a situation that tended to elicit the
tone frequency. (From Stebbins, 1970.) bark. Eventually they were able to test visual acuity
520 ANIMAL PSYCHOPHYSICS

A report by Nye (1973) suggests, for example, that the

pigeon’s key peck is poorly controlled by laterally
placed visual stimuli, although performance on an
identical discrimination is excellent when the stimuli
are located in the bird’s frontal field of view. This
find, if confirmed, illustrates the subtlety of species
idiosyncrasies with which the animal psychophysicist
may have to cope.

Methods of Stimulus Presentation

The experimenter’s goal, in threshold studies, is

typically to estimate the stimulus value that can be
detected 50% of the time. Preliminary work is usu¬
ally necessary so that stimuli may be chosen over a
range that will include the threshold value and across
which stimulus and response values will systematically
covary. In working with human subjects, experimen¬
Fig. 5. Apparatus for obtaining audibility data for the bat.
The bat positioned itself on the platform at the rear of the ters commonly omit stimuli far above or below
cage in such a way as to break a photocell beam. When a tone threshold, since such presentations are inefficient. In
occurred the subject crossed the cage for food reinforcement.
animal work, however, some signals must be of known
The tone came from the tube beyond the platform, (From
Dalland, 1970.) detectability if reinforcement contingencies are to be
effective in training the animal and in maintaining its
behavior during testing. Thus there must be a num¬
by requiring the sea lions to vocalize when a grating
ber of “wasted” trials, during which the stimulus is
stimulus appeared but to remain silent when the
well above or below threshold. Experimenters have
stimulus was blank. An important advantage of
adapted, with this modification, most of the well-
vocalization as a response is that it is not tied to a
known human psychophysical methods, which pre¬
spatial location. The authors thus were able to test
scribe the spacing of stimuli around threshold and the
acuity at various controlled target distances.
order of stimulus presentation.
Not only an animal’s response repertoire, but also
the conditions under which the responses ordinarily
Testing with a Broad Range
occur may bear on the choice of psychophysical meth¬
of Stimulus Values
odology. The cat, for example, has a reputation for
recalcitrance in operant situations, but this reputation To obtain complete psychophysical functions, ex¬
may be due more to obtuse experimenters than to perimenters usually choose a stimulus set that ex¬
Obstinate animals. Recent studios indicate that posi¬ tends in regular steps from values that are almost
tive reinforcement methods are quite feasible if the always detected to values that are very rarely de¬
cat is sufficiently hungry and if its task is properly de¬ tected. Once chosen, these stimuli may be presented
signed. Berkley (1970), for example, found that the in ascending order of strength (or increasing differ¬
natural tendency for the cat to use its paws, though ence between standard and comparison, when a differ¬
apparently ideal for lever pressing, was actually an ence threshold is sought), descending order, or in
obstacle to visual control, since cats tend to watch random order. In animal work, descending order is
their paws as they use them. Berkley’s successful ap¬ perhaps the most popular, for it seems best to start
paratus employed a nose key, which the cat could with an easy discrimination and proceed to those more
press only after it had placed its head through a hole difficult. This procedure thus incorporates a “shaping”
too small to include the paws. The author reported or “fading” aspect. Although this method resembles
rapid success in training visual discriminations with the psychophysical method of limits, stimuli tend to
this apparatus, and found it more easily automated be more widely spaced than in a human experiment,
than some earlier methods used with cats. and each stimulus value is apt to occur in blocks of
Less obvious, but possibly of great importance, are several trials before a new value is introduced. The
considerations of stimulus-response associability re¬ block method is desirable because performance may
lated to the “preparedness” notion of Seligman (1970). change somewhat over trials, and a number of trials
Donald Blough and Patricia Blough 521

may be necessary to yield a reliable indication of easy discriminations are mixed with more difficult
performance. In the study of pigeon acuity, illustrated ones. In the monkey auditory work of Stebbins, which
in Figure 1, trial blocks were very large; altogether we described earlier, audiometric functions from the
each block included 256 trials and occupied two ex¬ method of limits and the method of constant stimuli
perimental sessions. As is true of many animal studies, appear very similar. The pigeon acuity study (P.
this procedure required the birds to reach a criterion Blough, 1971) compared a random stimulus method
level of performance at a strong signal (wide stripes) with a descending series method, and again the two
before proceeding with progressively weaker signals. yielded similar thresholds. However, performance on
The research affirmed the importance of these easy the easy discriminations was somewhat worse for the
sessions, for performance on easy discriminations random stimulus method, despite the fact that warm¬
tended to be poor following sessions at the most diffi¬ up sessions and gradual introduction of less detectable
cult discriminations. stimuli preceded sessions based on random presenta¬
Hodos and Bonbright (1972) describe a procedure tion. Another disadvantage of random ordering is that
that employed much smaller blocks of descending it tends to commit the experimenter to a set of
stimuli. The study neatly incorporates warm-up trials stimulus values that may later turn out to be inap¬
and checks on the subjects discrimination perfor¬ propriate,
mances, Pigeon subjects discriminated a standard light As we have said, a threshold is generally defined as
from lights of varying comparison intensities. Each a stimulus value associated with a criterion response
session began with a 20-trial warm-up with a large probability. Because the criterion is rarely met ex¬
intensity difference (.8 log units). The next 20 trials actly, the actual threshold value usually comes from
constituted an “assessment” period, at the same large interpolation on a graph that relates the stimulus and
intensity difference, -to determine the subjects basic respond msasura. In methods that use a broad rang?
performance level. If the bird’s error rate was peater of sdmtilus valueSi this graph will usually show a wide
than 10% during this period, no new comparison range of response probabilities, extending from chance
values were introduced, and the entire session was to nearly perfect performance. To arrive at a threshold
devoted to training. If performance met criterion, figui-e, many pOrirriOutors simply connect the two
however, the data were recorded and further sets of points on cither side of their threshold enterion and
20-block trials occurred, each at a smaller intensity interpolate appropriately. A second procedure, which
difference than the last. After testing at the smallest makes use of more of the data, is to fit a function to
difference, the warm-up condition at .8 log-unit differ¬ all of the pointg and then proceed with the interpola¬
ence was repeated. Then a final descending series be¬ tion. (A transformation which makes the function
gan. The authors state that “the warm-up period be¬ approximately linear is helpful for this purpose, since
tween the first and second descending series of stimuli a straight-line fit is relatively easy to achieve either
served to dissipate extreme response biases that de¬ statistically or by eye,) This second procedure is more
veloped . , . during the later, more difficult discrimi¬ complicated, but is worth the trouble when the data
nations of the first sequence and would otherwise be are variable, since inclusion of all points contributes
carried over into the earlier discriminations of the to the reliability of the threshold estimate.
second sequence” (p. 473).
Some animal studies have used series of ascending
Threshold Tracking
stimulus values, usually along with descending series.
Used alone, the ascending order may generate a high The methods just described are inefficient, in a
rate of false alarms (Dalland, 1970). It may also yield sense, because they include many trials on which the
higher thresholds than the descending series (e.g., stimulus is well above threshold. In tracking methods
Mishkin 8c Weiskrantz, 1959; Terman, 1970), although (also known as “staircase,” “up-down,” or “titration”
Smith (1970) found similar thresholds by the two methods), most stimuli presented are near threshold,
methods in his conditioned suppression procedure. As so there are fewer wasted trials. However, tracking
in human psychophysics, it seems appropriate to aver¬ methods may require long training and relatively
age the results of ascending and descending series complex apparatus.
when both are used. Tracking has been used with a number of sense
A set of stimulus values may also occur in random modalities in a variety of species. An early applica¬
order. This procedure has yielded orderly psycho¬ tion in animal psychophysics produced data on dark
physical functions, and we might expect it to main¬ adaptation in pigeons (D. Blough, 1958, 1961). The
tain the basic discrimination most effectively, because paradigm was essentially two-manipulandum, yes-no,
522 ANIMAL PSYCHOPHYSICS

although the session was not divided into trials. Pecks single-trial staircase method, and one comparison of
on the “no” key (key B) were reinforced when a stim¬ the methods reports that the block procedure indeed
ulus patch was dark, and pecks on the “yes” key (key yields less variable data (Moskowitz 8c Kitzes, 1966).
A) were correct when the stimulus was illuminated. The staircase method, however, provides more thresh¬
Correct key B responses were reinforced with food, old indications per unit time, and this may be an
and key A responses were maintained because they overriding factor when sensitivity is to be followed
occasionally turned out the stimulus light, thus setting through time.
the occasion for key B reinforcement. Variable-interval Are thresholds found by the tracking method com¬
and variable-ratio requirements helped to prevent parable to thresholds from other methods? In Blough’s
control of pecking by temporal or numerical factors; method, reinforcement contingencies for the two keys
that is, reinforcement had no close relationship to were asymmetrical, since food was contingent only on
elapsed time or to number of responses emitted. “no” responses (Saslow, 1967). Since “no” responses
Threshold tracking resulted from this procedure be¬ increase the signal's strength, the resulting thresholds
cause responses on the “no” key raised the intensity could be spuriously high. Some confirmation of such
of the stimulus and responses on the “yes” lowered it; an effect comes from Clack and Harris (1963), who
thus a well-trained subject kept the stimulus light noted that their rats tended to maintain the auditory
near its threshold value, and a continuous record of stimulus at suprathreshold levels. Rosenberger (1970)
intensity followed the threshold through time. identified another possible difficulty with tracking:
The tracking procedure raises special problems of when the animal controls the signal, responses may
stimulus control, some of which have been discussed not be reinforced with equal probabilities at all signal
by Blough (1958) and more recently by Clack and strengths. Thus the tendency to switch from key A to
Harris (1963) and Rosenberger (1970). It is necessary, key B could become stronger in the presence of cer¬
for example, to discourage the subject from simply tain signals than for others, with a distortion of
responding alternately to the keys until reinforcement threshold data resulting.
occurs. An effective means of discouraging this strategy Direct comparisons between tracking and other
is to have incorrect responses increase the ratio by methods are rare. Symmes (1962), in his studies of
which reinforcement is programmed. Another poten¬ flicker discrimination in monkeys, reported that a
tial problem is the deterioration of stimulus control technique much like Blough’s generated lower critical
that may occur when a discrimination remains diffi¬ flicker-fusion frequencies (higher thresholds) than did
cult over a long period of time. To compensate for a go/no—go metnod using comparable target param¬
this. Clack and Harris increase signal strength to a eters. However, Stebbins (1970) found that auditory
relatively high level following reinforcement to pro¬ thresholds generated by his version of the tracking
vide a “warm-up” period. method were very close to those yielded by a method
The tracking idea is readily adaptable to several of of limits and a method of constant stimuli. Perhaps
the response paradigms that we have previously de¬ Stebbins’s failure to find differences had to do with his
scribed. For example, a combination of tracking and modification of the technique. While Blough and
conditioned suppression is described by Ray (1970) Symmes provided direct reinforcement only for re¬
and by Rosenberger (1970). Here, degree of response sponses appropriate to the signal’s absence, Stebbins
suppression controlled signal strength. When suppres¬ also reinforced with food responses appropriate to the
sion exceeded some criterion, signal strength de¬ signal’s presence, whatever its strength, and punished
creased; when suppression no longer met the criterion, (by a time-out) failure to report a signal. These con¬
signal strength increased. A threshold could be de¬ tingencies perhaps favor a stronger “yes” response and
rived from the stimulus values, since these were cor¬ thus a lower threshold than the less symmetrical rein¬
related with response suppression. forcement procedure. Such differences among various
In trialwise tracking methods, the stimulus value versions of the tracking method seem to preclude any
may change following each presentation (the “stair¬ blanket statement comparing this method with others.
case” method) or following a block of trials. Thorpe The definition of threshold is somewhat less stan¬
(1971) used the latter procedure in a study of goldfish dardized in the tracking method than in the other
spectral sensitivity. Stimulus intensity decreased when methods we have described. Basically, the experimen¬
the number of correct responses per block exceeded a ter seeks the stimulus value at which the probability
criterion and increased when the number fell below of switching from the “yes” to the “no” response, and
the criterion. The block method bases each stimulus vice versa, is greatest. This may be done visually, by
change on more response information than does the drawing horizontal lines that bisect various subsets of
Donald Blough and Patricia Blough 5 23

data and averaging the stimulus values represented turn on the tone. As we have seen, in his audiometric
by these lines (Clack & Harris, 1963). Alternatively, works Dalland (1970) required his bats to break a
reversal points—those stimulus values at which the photocell beam to produce the tone stimulus (see
subject switches from key A to key B—may be aver¬ Figure 3). Because tone intensity varied with distance
aged. Rosenberger (1970) has described a method for from its source, the control of the subject’s position
treating data from discrete-trial procedures. during stimulus presentation was crucial in this case.
The training of visual fixation was an important
Some Methodological Problems
feature of studies by Scott and Milligan (1970) on the
motion aftereffect in monkeys. This effect requires a
Sensory data are most informative when variations preexposure condition in which the subject fixates on
in the response measure depend solely on changes the center of a rotating target for a few seconds. Hu¬
in stimulus conditions or sensory states. Though a mans perform this task readily with the aid of verbal
certain amount of variability from other sources is instructions. With monkeys, however, lengthy train¬
inevitable, the contribution of nonstimulus factors ing was required. Using infrared light reflected from
may be reduced considerably by careful experimental the subject’s cornea, an observer monitored eye posi¬
design. The sections below outline some stimulus con¬ tion and gradually shaped the required fixation. Al¬
trol problems and methods that have been successful though this method apparently succeeded, the con¬
ih coping with them. We shall consider these prob¬ stant presence of a human observer makes it inefficient
lems in the relatively well-defined context of threshold for many types of research. Automated techniques
research, but it will be evident that most of the dis¬ that precisely control eye position will be a boon to
cussion is equally applicable to the suprathreshold visual psychophysics with animals.
and scaling research to be covered later. In some in¬ Although various fixating and orienting responses
stances, we shall return to these maters in the later
can be trained, good stimulus control requires, by
discussion. definition, an attentive subject. Thus in psychophys^
ical studies, a subject’s failure to respond to a signal
Fixation and Attention may result from a failure of attention as well a§ fail¬
The achievement of good control by an accurately ure of the stimulus to exceed threshold. One pro¬
specified stimulus requires that a subject be appropri¬ cedure that appears to favor attention is the program¬
ately oriented and attentive. In most human psycho¬ ming of aversive consequences for incorrect responses.
physics the subject’s physical orientation is adjusted Such a CQn§CquCIlCC i§ th^ time-outj a period during
with the utmost care, but with animals such control is which stimuli do not occur and responses are ineffec¬
often quite crude. Nonetheless, researchers have tive. Time outs are usually programmed to follow
worked out methods for achieving at least some con¬ false positive responses, but in an effort to achiCvC
trol over this variable. An obvious method is physical symmetrical consequences, experimenters may also
restraint (as in the monkey restraining chair), but we program timeouts to follow failures to report the
stress behavioral methods here. positive stimulus. Occasionally a stronger punishment
When a single manipulandum is used, an animal appears to be helpful. Gourevitch (1970) sometimes
subject’s orientation may remain roughly constant. In combined shock with time-out in order to improve
an olfactory study with rats, Goff (1961) required that stimulus control in his auditory experiments with
the rats press the response lever only with their left monkeys. Shock should be used with caution, how¬
paws; this condition helped to maintain the rat in a ever, because it can cause serious response disruption.
relatively constant position with respect to a device Still another method that may make the subject “stop
that delivered the odorant. More explicit control of and look” is the elimination of reinforcement for any
orientation may be achieved through the use of an response that follows stimulus onset with very short
observing response, upon which stimulus presentation latency. In two-key designs, a changeover delay op¬
is contingent. In her study of pigeon acuity, for ex¬ erates in a similar fashion to discourage rapid alterna¬
ample, P. Blough (1971) used an observing response tion between keys. In this procedure, reinforcement is
to position the birds at an appropriate distance from withheld if the time between operation of the keys is
the stimulus targets. Other researchers have required less than some minimum. Many experimenters also
their subjects to insert their heads through holes in cause responses to require a certain amount of effort,
order to view a visual stimulus, and, in some auditory as in the substitution of a short fixed ratio for a single
work, orientation has been controlled by requiring “yes” or “no” response. Observing responses, when
the animal to press an appropriately placed lever to made to the critical stimulus, may be programmed to
524 ANIMAL PSYCHOPHYSICS

require multiple responses, presumably resulting in Stimulus preferences may also confound psycho¬
an increased exposure to the appearance of the stim¬ physical data. For example, pigeons tend to respond
ulus. more to some colors than to others, although the pre¬
Despite the effectiveness of such techniques, most ferred color appears to differ among birds. In the
animal subjects are not “perfectly attentive”—that is, auditory modality, the intensity dynamism effect may
on some trials they make errors even on easy discrimi¬ be considered a type of stimulus preference. Sadow-
nations. Psychophysical functions reflect such errors sky (1966) has shown, for example, that rats discrimi¬
by approaching an asymptote below 100% correct at nate more accurately when the positive stimulus is the
the strongest signal values. Criteria of stable perfor¬ more intense of two sounds. This effect appears to be
mance, measured during warm-up trials or “catch” consistent enough to require controls. Thus in their
trials, help the experimenter to estimate the degree of study of the auditory intensity difference limen, Ras-
such inattention and, if necessary, take measures to lear et al. (1975) made silence the positive stimulus so
reduce it. In addition, the experimenter may use a that dynamism effects could not account for the dis¬
correction for attention, such as that suggested by crimination in this go/no—go task. An experimenter
Heinemann, Avin, Sullivan, and Chase (1969). This may cope with less predictable preferences by using
correction, based on the upper and lower asymptotes a reasonable number of subjects, a high training cri¬
of a sigmoidal psychophysical function, assumes that terion, and insightful data analysis.
the subject is inattentive on a certain proportion of Response bias is one of the worst plagues of the
the trials and that the probability of being inattentive animal psychophysicist. In multiresponse designs, for
does not vary with signal strength. The second part of example, position preferences are almost always a
this assumption is somewhat questionable as a general problem. These may be minimized by a correction
rule, since there is evidence that performance on easy procedure, such that incorrect responses are followed
discriminations tends to deteriorate after a series of by the same stimulus or stimulus array until a correct
trials on difficult discriminations. Perhaps behavior response occurs. Analysis must omit these correction
associated with attentiveness extinguishes during diffi¬ trials, of course, since they are nonrandom. The cor¬
cult discriminations. rection procedure may reduce but it does not elimi¬
nate effects of position preference, which appear to
Extraneous Cues grow more pronounced as signal strength decreases
(P. Plough, 1971; Terman, 1970). In a forced-choice
Animal subjects have an annoying ability to devise
design, the effects of such preferences are relatively
response strategies different from those intended by
benign, since the correct stimulus var/es in position.
the experimenter. These strategies are sometimes based
In a yes-no task, however, preferences may affect the
on extraneous cues provided by poorly designed ap¬
threshold by driving it up (if the preference is for the
paratus. Stimulus systems must be free of transients
“no” response) or down (if the preference is for
such as sounds accompanying the onset and offset of
the “yes” response). Similarly, biases that favor re¬
the stimulus; auditory apparatus must control for the
effects of standing waves; equipment must be designed sponding or not responding may affect thresholds
determined by a go/no—go procedure.
so that easily confounded dimensions, such as visual
luminance and wavelength, may be varied indepen¬ Sequential dependencies may also serve as un¬
dently. The rise time of stimuli can act as a discrimin- wanted sources of control. A strictly random determi¬
able cue, so the onset and offset characteristics of the nation of stimulus order is usually undesirable, since
stimulus event should remain constant across stimulus this procedure may generate a long series of identical
values. Improved equipment has been an important trials, during which a position preference may become
factor in the growing quality of animal psychophys¬ strengthened. Unfortunately, however, modified ran¬
ical studies, but there are occasions when even the dom series may include contingencies that some an¬
most careful experimenter cannot be sure that he has imals discriminate. Thus Terman (1970) reported that
perfect control over an extraneous variable. For ex¬ rats apparently discriminated the constraint that a
ample, in studies involving stimulus wavelength, even given stimulus array could not occur more than three
small luminance differences between monochromatic times in succession. When performance in a psycho¬
lights may confound the data. In such cases, the con¬ physical task fails to reach chance level even at very
founding factor may be varied at random over a small low signal strengths, the experimenter may suspect
range in such a way that no particular value will be (as Terman did) the operation of sequential or other
well correlated with reinforcement. nonstimulus cues.
Donald Blough and Patricia Blough 525

Reinforcement Contingencies in steepness for many months. Mishkin and Weis-

krantz (1959) also reported learning effects that lasted
We have discussed previously the role of special
for a considerable time. The extent of such effects no
contingencies in maximizing attention and the role
doubt relates to the difficulty of the task. Experimen¬
that reinforcement may inadvertently play as an ex¬
ters should be alert to this problem and, when run¬
traneous discriminative stimulus. Other uses of rein¬
ning a sequence of conditions, either counterbalance
forcement may also involve difficulties. For example,
for order or first ascertain that their functions repre¬
most threshold procedures program reinforcement for
sent a stable asymptote.
correct detections of the signal, regardless of its
strength. This procedure avoids selective reinforce¬
Abstraction of the Stimulus
ment of particular signal strengths and would seem
to aid the subject in discriminating weak stimuli. Animal subjects appear to differ in their ability to

However, when the stimulus is present but sub¬ form concepts based on relational or abstract proper¬
threshold,1 the procedure may also provide reinforce¬ ties of stimuli. The efficiency of many experiments
ment for responses that are, functionally, false alarms, could be improved if, for example, all species could
transfer readily among specific Instances of “match¬
Little research has been directed specifically at this
ing” or “oddity,” but only primates, after long train¬
problem, though Nevin (1970) found that increased
ing, hav£ b^en cfearly to do this. Ev£ii matching
relative reinforcement at weak signals produced an
increased false alarm rate. Several studies indirectly or oddity within a specific dimension (color? for ex¬

suggest that reinforcement of positive responses to ample) may be learned as a series of specific problems
subthreshold signals may not seriously affect threshold (Gumming & Berryman, 1901). However, Honig (1905)
was able to establish “wavelength stimulus difference”
data. The tracking procedure used by Stebbins (1970)*
as a controlling relation, apparently independent of
for example, included a large proportion of weak
wavelength stimuli. In this study, pigesns
signals and must have resulted in occasionally rein¬
forced "yes” responses to sounds below threshold. Yet, learned to peck the right one of two keys if both keys

as we have noted, these audibility functions were very were illuminated by the same wavelength and to peck
the left key if the wavelengths differed. The birds
much like those from methods that included a greater
could not base their discrimination on any absolute
proportion of more intense tones. In connection with
Wavelength difference because each wavelength ap¬
the conditioned suppression method, Ray (1970) dis¬
cussed the effect of “unsignaled” shoeks that pre¬ peared equally often on both keys and was paired
sumably occurred following §ubthrc§hoId sounds and equally often with itself and with a different wave¬
length. The birds also performed correctly with new
reported that, when kept to a minimum, these did not
seem to affect the data. Blough's version of the track¬ wavelengths. Malott and Malott (1970) report similar

ing method (1958) seemingly avoided the reinforce¬ generalized matching in single-key tests. There is no
ment of perceptually inappropriate responses, yet we good evidence, however, that this “same-different con¬
have seen that this method may introduce other sorts cept” can transfer to a new stimulus dimension in
of bias. Perhaps the advantages of reinforcing correct the pigeon.
responses to weak but superthreshold signals out¬ Because of the abstraction problem and others that
weigh the disadvantages of reinforcing “false alarms” we have outlined, the measurement of sensory thresh¬
to subthreshold stimuli. olds is apt to be time-consuming and a continual
challenge to the experimenter's ingenuity. Nonethe¬
less, as we have seen, the patient application of well-
Learning Effects
tailored operant methods can yield remarkably de¬
Performance on some psychophysical tasks may tailed data on the sensory capacities of an animal
continue to improve for an exasperatingly long time. subject. The use of such methods in the study of more
In the acuity work previously mentioned (P. Blough, complex perceptual phenomena raises exciting possi¬
1971), successive psychophysical functions increased bilities, but has produced many fewer data. We con¬
sider such research in the next section.
i We recognize that the “theshold” concept is questioned in
modern psychophysics. In this discussion, however, we find it
convenient to speak of “subthreshold” stimuli rather than de¬ SUPRALIMINAL STIMULI
veloping a complex argument that assumes a continuum of
stimulus effects to which response criteria are applied. The
practical impact of the present argument remains essentially
The following sections will consider research in
unchanged after transposition to the signal detection format. psychophysical scaling and perception. The nature of
 526
ANIMAL PSYCHOPHYSICS

these problems is more complex than those involving ulus suffice, or must training isolate one stimulus as¬
threshold determination, and successful studies are pect or dimension and associate the response measure
relatively few. Such experiments often involve ab¬ with this alone? We turn now to examples of research
stract or poorly defined stimulus variables, and they in which these questions have been answered in vari¬
tend to be modeled after human research that de¬ ous ways.
pends on relatively complicated verbal instructions
and responses. Translating such problems into para¬ Reaction Time Methods
digms suitable for animal experimentation has been
difficult compared to the task of designing threshold When stimulus intensity increases, the response,
experiments. In a threshold study, for example, the whether reflex or learned, decreases in latency. This
experimenter may program reinforcement on the relationship has been used to equate stimuli for in¬
basis of the physical presence or absence of the signal. tensity, in much the same manner that lights may be
In suprathreshold experiments, however, there is a equated for luminance by finding intensities that
broad range of stimuli for which only the subject can leave the pupil a constant diameter. Stebbins (1966)
define “correct” and “incorrect” responses. We cannot used the reaction time method to determine equal¬
tell a subject in a scaling experiment what sound in¬ loudness contours for the monkey, and in the same
tensity is “twice as loud” as another, nor can we tell manner Moody (1969) found equal-brightness con¬
him in a perceptual study at what relative distance tours for the rat. In these experiments, the animals
two sizes should look equal. Such stimulus control learned to release a lever as soon as a sound or light
must carry over from training in which performance occurred. For a set of different frequencies (or wave¬
can be defined and reinforced. In a size-distance ex¬ lengths), those intensities that yielded constant reac¬
periment, for example, the subject might be trained tion times were called equally loud or equally bright.
with equal physical sizes at equal distances. Test The use of several criterion reaction times gave con¬
stimuli, which would include a variety of size and dis¬ tours at several intensity levels. A set of curves from
tance conditions, might either follow training presen¬ Moody (1970) appears in Figure 4.
tations or be interspersed with them. We might look at one of these reaction time ex¬
periments in more detail, noting the steps taken to
clarify the stimulus-response relationship. Moody
Perceptual Studies
(1969) trained his rats to enter a viewing tunnel set in
What does an animal “see" when subjected to com¬ the wall of an experimental chamber. At the far end
plex suprathreshold stimulation? As with human sub¬ of the tunnel, lights of various wavelengths could
jects, the question can often be translated, “What
different sets of stimulus conditions yield the same
response?” For example: “For what combinations of
input parameters do two sounds seem equally loud?”
(equal-loudness functions), “What combinations of
arrowheads make two lines elicit the same response?”
(Muller-Lyer illusion), “After exposure to which wave¬
lengths does the subject respond to a neutral stimulus
as though it were 550 nm?” (colored afterimage). In¬
formation is also gained from the degree to which re¬
sponses are the same in different situations; for exam¬
ple, if a monotonic response measure is used, it may
be possible to scale the similarity of a set of stimuli.
It is apparent that perceptual studies, as just
formulated, are fundamentally studies of generaliza¬
tion or transfer, and operant generalization methods
have been used increasingly in this area (Malott 8c
Malott, 1970; Thomas, 1969). However, the precise
method to use is not usually obvious. Shall the re¬ Wavelength (nanometers)
sponse measure be relatively built-in, like reaction
Fig. 4. Equal-brightness functions of the rat determined by the
time, or more subject to conditioning, like response
latency method. The criterion latency is indicated to the right
rate? Will simple training in the presence of one stim¬ of each curve. (From Moody, 1970.)
Donald Blough and Patricia Blough 527

appear on a frosted glass screen. The rat encountered each stimulus value during the extinction test allows
a response lever about halfway down the tunnel, and the values to be ranked according to their similarity
above this lever was a loudspeaker. The rat learned a to the training stimulus.
response chain as follows: in the presence of white This simple generalization procedure frees the ex¬
noise from the loudspeaker, press the lever; when a perimenter to choose the most convenient response
light appears on the glass screen, release the lever. and also the most informative set of test stimuli. The
Lever releases to the light produced water reinforce¬ stimuli need not come from a physically defined “di¬
ment. Since the light did not appear unless the lever mension” such as intensity; indeed, the generalization
was held down for an interval (.2 to 3.0 sec), the rat test may be designed to identify possible stimulus di¬
was in a relatively constant position when the light mensions. Exemplifying the flexibility of the method
came on and was presumably ready to react. Stebbins with regard to response measures and test stimuli is
(1966) assured constant conditions from trial to trial a study of taste by Tapper and Halpern (1968). Rats
by restraining his monkeys in a chair and delivering were irradiated with gamma rays as they drank a
reinforcements directly into the monkey’s mouth. chemical solution. Such irradiation makes the animal
Variability in reaction time may be further reduced sick, and under suitable conditions a single exposure
by adding a “limited hold” to the reinforcement con¬ induces an aversion to the taste of the substance as¬
tingency, such that if the subject waits too long to re¬ sociated with irradiation. Thus in this case the re¬
lease its lever, no reinforcement is forthcoming even sponse was a decrease in drinking. The test stimuli
for “correct” responses. Moody discusses this and were a number of different chemical solutions; aver¬
other methodological matters in his review of the sion (decreased drinking) generalized to some of these
method (Moody, 1970). but not others. Tapper and Halpern suggest that the
The reaction time method has the advantage that method may be a way to determine the dimensions
the relationship between the stimulus variable and along which taste quality is represented in the rat,
response is built in. The necessary training, designed The work of Thomas and his colleagues (Thomas,
to shorten reaction time and reduce its variability, 1969) on perception of the vertical exemplifies the
serves only to exhibit this unlearned relationship use of generalization tests to clarify complex interac¬
more clearly, The method thus avoids the difficult tion of perceptual variables. In one such study, pi¬
task of teaching an animal an arbitrary stimulus- geons were trained to peck at a vertical line projected
response relationship and maintaining that relation¬ on a response key in a lighted box. When the birds
ship under novel test conditions. However, the reac¬ were exposed successively to a number of lines set at
tion time method suffers the disadvantage that the different angles, they responded most to the line that
unlearned stimulus-response relationship may not pro¬ was “visually vertical” (i.e., at right angles to the ceil¬
vide the information that the experimenter seeks. It ing) even when the box was tilted around the bird
has thus far been applied only to intensitive dimen¬ while the floor remained horizontal. Birds trained in
sions, and on these it has not permitted one to say by a dark box while standing on a horizontal floor, but
how much one stimulus is louder or brighter than tested in the dark on a floor tilted 24° to the left,
another, but only to indicate stimuli that are equal. responded most to a line tilted approximately the
It may be possible to surpass both of these limita¬ same amount to the left. The experiments suggest that
tions; Moody (1970) suggests that modified choice of visual cues predominate in the birds’ perception of
discriminative situations may make reaction times vertical, but when these are absent postural or kines¬
applicable to qualitative dimensions. The possibility thetic cues may be used. Thomas relates these results
that a metric scale may be based on reaction time to a similar work on “sensory-tonic” and “field-
data is touched on below. dependency” effects in human subjects.
The simple generalization test, with single-stimulus
training followed by extinction, has serious limita¬
Simple Generalization Methods
tions, however. Since the data are obtained during
At the present writing, generalization methods are extinction, responses are relatively few and data
the most widely used operant techniques in the study points are variable. The procedure singles out no
of animal perception. Of these, the simplest involves particular aspect of the stimulus, so the aspect to be
merely training an animal to respond in the presence tested may control the measured response weakly or
of a given stimulus and then systematically varying not at all. For example, after simple exposure train¬
the stimulus as the response extinguishes (Guttman ing, the wavelength of light on a pigeon’s pecking
8c Kalish, 1956). The number of responses emitted to key does control response probability; line tilt also
 528
ANIMAL PSYCHOPHYSICS

does, though less strongly; tone frequency does not, went unreinforced (S~). During S~ periods, pecks also
without additional discrimination training. We turn delayed the return of the 45° S + . Following such
now to the uses of such additional training. training, “angle of tilt” would be expected to control
pecking more than other aspects of the line, such as
Generalization After color, that were the same during both S+ and S~
Differential Reinforcement periods.
After this training period, Mello tested her birds
Perceptual studies commonly incorporate differ¬ with a series of line tilts. When tested with their
ential reinforcement, in order to improve upon the trained eye uncovered, the birds responded most (as
relatively weak and nonspecific control often gen¬ expected) to the 45° line to which they had been rein¬
erated by single-stimulus training. The simplest way forced. With the untrained eye uncovered, however,
to strengthen control is to reinforce responses in the they responded most to a tilt of 135°, the mirror
presence of the stimulus and omit reinforcement in image of the training pattern. This result could have
the absence of the stimulus. Jenkins and Harrison interesting implications for the nature of the pigeon’s
(1960) found this sufficient to bring the pigeon’s peck¬ stimulus processing mechanisms; Mello’s comments
ing response under the control of tone frequency. on this matter concerned the representation in the
When an interrupted tone (“beep-beep-beep . . .”) brain of patterns viewed monocularly.
was on continuously throughout a pigeon’s key-pecking However, the interpretation of the mirror image
experience, the rate of pecking did not change when data is open to conflicting interpretations, and these
the frequency of the tone departed from its training exemplify hidden assumptions that may underlie per¬
value. However, when birds were trained to peck only ceptual studies. Mello assumed, quite naturally, that
when the tone was on and not when it was off—a dis¬ angle of line tilt controlled behavior in the training
crimination presumably irrelevant to the frequency and test situations. Corballis and Beale (1970) sug¬
dimension—subsequent tests revealed strong control gested, however, that the controlling stimulus in this
of response rate by tone frequency.
situation may best be defined as “up” versus “down,”
In the Jenkins and Harrison experiment, reinforce¬ rather than angle of tilt. A bird trained with its left
ment in the presence of a stimulus, contrasted with eye covered has most of its left field of view occluded;
nonreinforcement in its absence, may be said to “call if such a bird attended only to the right half of the
the subject’s attention” to the stimulus. However, no
stimulus key, a 45° line would be “stimulus in the
particular aspect of the stimulus was singled out. Jen¬
upper part of the field” and a 135° line would be
kins and Harrison would probably have observed “stimulus in the lower part of the field.” If, when the
response changes to variations, for example, in the right eye was covered, the left eye similarly attended
intensity or harmonic content of their tone stimuli, only to the left half of the key, the 135° line would
had these been tested. To increase control by a spe¬
now be “upper” and the 45° line “lower.” Thus a
cific stimulus aspect, one may employ differential discrimination between “upper” and “lower,” per¬
reinforcement of stimuli differing in this aspect alone. formed with either eye, would cause the apparent
Such differential reinforcement during training may mirror-image reversal reported by Mello. Corballis
be accompanied by equal reinforcement of irrelevant
and Beale did tests with other sorts of line stimuli
stimulus variations. We mentioned an instance of this that seem to support their contention. We may con¬
above: the equal reinforcement of randomly varying
clude that the controlling aspects of even seemingly
stimulus luminances, in studies where wavelength was
simple stimuli cannot be taken for granted; the ex¬
crucial but luminance was irrelevant.
perimenter may have one stimulus aspect in mind,
Studies of interocular transfer of line tilt illustrate and the animal subject another.
these methodological points. They also illustrate the
The interocular transfer experiments also suggest
uncertainty about the nature of the controlling stimu¬ the importance of possibly unlearned perceptual fac¬
lus which may remain, even following differential tors in determining the nature of controlling stimuli.
reinforcement of a seemingly obvious aspect. Mello Another example from recent research concerns “fea¬
(1966) studied transfer of a tilt discrimination from tures,” or salient portions of visual patterns, which
one eye to the other by fitting pigeons with goggles seem crucial to pigeons in determining the course of
that restricted vision to the anterior visual field of pattern discrimination learning (Sainsbury, 1971).
one eye. She then reinforced presentations on the This feature-dependent control would surely affect
pigeon’s pecking key of a 45° line (S+) while presenta¬ research on pattern perception in which birds peck
tions of the line tilted in the opposite direction (135°) at complex visual targets.
Donald Blough and Patricia Blough 529

In some cases, such as the Tapper and Halpern

taste research cited above, the dimensions of complex
stimuli are admittedly unknown. The object of differ¬
ential training then is not to call the subject’s atten¬
tion to a supposedly known dimension, but rather to
establish control by known physical stimuli. Some
experimenters then use generalization tests to identify
the manner in which the subject classifies or dimen-
sionalizes the stimuli. A good example of such work
is a study by Sutherland (1969) on shape perception
in rats, octopuses, and goldfish. Sutherland taught his
subjects to discriminate between a square and either
a horizontally oriented parallelogram or a vertically
oriented parallelogram. He then presented a variety
of shapes and recorded the percentage of trials on
which the subjects gave the “square” response. These
percentages were used to rank the test shapes with Fig. 5. Pmportion of left-lever (“contracting”) responses as a
respect to their similarity to the square. From his function of rate of change of circle diameter, following pre¬
exposure to three spiral conditions; stationary—open circles;
inspection of this ranking, Sutherland concluded that counterclockwise rotation (“contracting”)—filled cird es; clock¬
octopuses seem to discriminate squares from parallelo¬ wise rotation (“expandingq-triangles. (From Scott & Powell,
grams on the basis of the “presence or absence of thin 196S. @ 1963 by the American Association for the Advancement
of Science.)
horizontal or vertical segments/’ while rats discrimi¬
nated on the basis of “oblique contours running in
the same direction as the contours of the original ever, the use of monkey subjects enabled the investi¬
parallelogram.” (Goldfish seemed somewhere in be¬ gators to carry out a subsequent study of the effects
tween.) Sutherland incorporated these conclusions in of certain brain lesions on this perceptual phenome¬
a tentative receptive-field model of shape discrimina non.
tion. The spiral aftereffect experiments provide a good
Generalization following differential training may example of the problems caused by the experimenter’s
not be simply a matter of recording the rate or prob¬ inability to define the “correct” test responses and
ability of response to a set of test stimuli, Several hence his inability to maintain reinforcement during
manipulanda may be involved, and whole patterns of testing. To minimize the difference in reinforcement
discriminative response may transfer to the test situa¬ conditions between training and testing, Scott and
tion. A case in point is the measurement, by Scott and Milligan (1970) sometimes reinforced correct re¬
his co-workers (1963, 1970) of the spiral aftereffects in sponses during testing on trials when the circle was
monkeys. We discussed previously the measures these expanding or contracting so rapidly that its motion
investigators took to ensure the monkey’s attention would override any aftereffect. One monkey appar¬
to and fixation upon a preexposure target. The train¬ ently detected this contingency: “He seemed to dis¬
ing task was a discrimination between an expanding cover that when the spiral was rotating, difficult dis¬
bright circle (right-hand lever correct) and a con¬ criminations never resulted in shock, and the animal
tracting circle (left-hand lever correct). Every second responded with a stereotyped left lever response to all
correct press produced a food pellet, while presses of circles moving at less than 1.7 minarcs per second”
the wrong lever yielded a mild shock. Following this (p. 354). The maintained generalization procedure,
differential reinforcement, a series of test stimuli next to be described, accepts the development of dis¬
yielded a psychometric function relating the percent¬ crimination in the testing situation as the price for a
age of left-lever responses to rate and direction of more copious flow of test data.
change in circle diameter. Preexposure to an “expand¬
ing” or “contracting” spiral shifted this function, as
Maintained Generalization
shown in Figure 5. Notably, the method provided a
Procedures
quantitative measure of the aftereffect; the rate of
circle expansion or contraction at which the monkey A generalization test in extinction is often ade¬
judged the circle “stationary” was affected by pre¬ quate to suggest the major characteristics of generali¬
exposure in much the same way as in humans. How- zation gradients, especially if group averages are used.
 530 ANIMAL PSYCHOPHYSICS

However, for meaningful individual gradients show¬ requires a substantial data base. The procedure is
ing some detail, it is necessary to increase overall advantageous also for the investigation of discrimina¬
responding by incorporating reinforcement into the tive processes such as stimulus summation and atten¬
test procedure. The simplest way to do this is to inter¬ tion, which are considered elsewhere in this volume
sperse some reinforced training stimulus presentations (Chapter 16; cf. D. Blough, 1969, 1972). A serious dis¬
among randomized test stimulus presentations. Usu¬ advantage of the maintained procedure is that re¬
ally, the reinforced trials can occur rather infre¬ sponse to stimuli highly discriminable from S+ may
quently and still maintain responding. fall rapidly to an uninformative zero level. Thus the
Recently, P. Blough (1972) used the maintained procedure is most useful with sets of quite similar
procedure to investigate a number of regions on the stimuli.
visual wavelength continuum. Her data showed
marked differences among spectral regions in the
Scaling
shape of the generalization gradients; the steepness of
the gradient slopes were reasonably consistent with With one or two exceptions, meaningful scales of
wavelength discriminability data (cf. Wright, 1972, stimulus continua produced by animal subjects are
discussed above). A set of these gradients appears in still more a hope than a reality. We consider scaling
Figure 6. The figure also shows an important feature here because of promising results from a few experi¬
of the maintained procedure: the gradual sharpening ments, interesting methodological beginnings, and the
of gradients around the reinforced stimulus. As an potential usefulness of such data. Adequate scales may
experiment continues over many hours, gradient simplify our conception of the processes of behavioral
steepness gradually approaches an asymptote that de¬ control by aiding us to distinguish influences at¬
pends on the discriminability among the test stimuli. tributable to sensory or perceptual systems from those
The maintained procedure thus represents a transition attributable to other variables. For example, we are
from “pure generalization” data, where the subject’s in a better position to study discrimination learning
response is relatively unconstrained by differential if we can present our subject with sets of stimuli that
reinforcement, to “discriminability” data, where re¬ are equally different, since the effects of other vari¬
sponse is maximally constrained and limits are set by ables may then be isolated with more success (for
factors such as sensory acuity. such a use of scaling see D. Blough, 1972). Scales also
Because the maintained generalization gradient tell us about the functioning of sensory processes
provides a continuous stream of quantitative data, it themselves. For example, sensory transduction must
is useful for scaling and signal detection work, which be consistent with a power law of sensory magnitude,
if that law is correct; similarly, color coding mech¬
anisms must be consistent with the “cqlor circle" such
as that determined by Schneider for the pigeon
(Schneider, 1972; see below).
Luce (1972) identifies three kinds of psychophysical
measurement that we may use as a guide to animal
work. First, and currently most promising in animal
work, are studies that derive scales from ordinal data.
In this kind of scaling, the response measure (for ex¬
ample, rate or probability) is used only to provide in¬
formation about order; that is, for example, if stimu¬
lus A elicits 30 responses, B elicits 15 responses, and C
elicits 14 responses, the method uses only the informa¬
tion that responses to A > responses to B > responses
to C. Many quite different data sets (such as 30, 27, 1)
would of course yield the same order. The method has
Fig. 6. Maintained wavelength generalization gradients around the great advantage that nothing is assumed about
reinforced stimuli in six spectral regions. Responses to the value
indicated by the arrow in each panel were intermittently rein¬
the response measure except that it is monotonically
forced. Other stimuli were presented in extinction. Data are related to the similarity or psychological distance be¬
averaged across three birds at two different stages of training. tween the corresponding stimuli.
The points connected by solid lines are spaced at 4-nm steps
were obtained somewhat earlier than those connected by dashed
Shepard, a major developer of this method (e.g.,
lines. (From P. Blough, 1972.) Shepard, 1966), applied his analysis to existing wave-
 Donald Blough and Patricia Blough 531

split field lighted the side keys. If the two wavelengths

were the same, a peck at the right key was correct, but
if they were different, a peck at the left key was
correct. Correct responses were always signaled by a
feeder light flash, with food coming on 20% of the
correct trials. The measure of dissimilarity of two
stimuli was the percentage of left key pecks given
when these stimuli appeared together.
Schneider used 12 wavelengths (66 pairs) spaced
across the spectrum in one experiment and 15 wave¬
lengths (105 pairs) in a second. The percentages of
left-key pecks to all the pairs were ranked, the ranks
Fig. 7. Generalization gradients based on data from Guttman
averaged across birds, and the averages used to derive
and Kalish (1950) (solid lines) and from Blough (1961) (broken
lines). Note the uneven spacing of the wavelengths^ their relative a two-dimensional spatial representation of the dis¬
distances have been adjusted to increase the uniformity of the tance between the various stimuli. This representation
gradients. (From Shepard, 1905,)
appears in Figure 8. An appreciation of the relation
between the raw data and this figure may b^ aidsd if
length generalization data from pigeons (Shepard, one imaging straight linos drawn across the diagram
1965). He showed that a transformation of the wave¬ to connect various pairs of wavelengths. A long line
length scale would convert these gradients, which signifies that the pair elicited many “different” peeks',
come from overlapping regions of the spectrum, to if the lengths of these lines were ordered from greatest
approximately the same shape. Some converted gra¬ to least, their order would (ideally) match die order of
dients appear in Figure 7; note that wavelengths on the
the ranked key-peck data. A provocative feature of
abscissa are irregularly spaced, for unit§ gn thi§ axis this method is the extent to which mere rank-order
now represent psychological distance or similarity. information can determine the distance between
The ordinate in Figure 7 is response frequency, but points m a diagram such as Figure 8. The form of
the wavelength scale would be unaffected by any Schneider’s function is also very interesting, for
y-axis transformation that preserved the order among human ratings of color similarity algo yield a roughly
the points. For example, after a logarithmic trans¬ circular configuration, with long and short wave¬
formation of the ordinate, each gradient would still lengths perceptually similar. The diagram suggests
be the same shape a§ the others, cycu though this that, despite important anatomical differences? the
common shape would change. Thus such a trans»
formation would not call for any change in the stim¬
ulus scale. 5*Z
It is helpful to think of distance along the abscissa
in Figure 7 as representing similarity, to the pigeon,
among the various wavelengths. However, this par¬
ticular scale does not actually represent similarity
among hues very accurately. For one thing, stimulus
luminance was not controlled in the experiments
upon which the curves are based. More importantly, a
single dimension is inadequate to represent the sim¬
ilarity among hues judged by human subjects, and
evidently this is also true of pigeons. A more adequate
two-dimensional map of hue for the pigeon has been
developed by Schneider (1972), in the most extensive
scaling job to be found in the animal literature to
date. Schneider based this map on wavelength differ¬
ence ratings from pigeons, using a method much like
that employed by Honig (1965) to study the wave¬
length difference dimension. In Schneider’s study, two
wavelengths appeared as a split field on the center key Fig. 8. A two-dimensional color space for the pigeon, based on
the rank-ordering of wavelength pairs by “same-different” pecks.
of a three-key pigeon box. An observing peck at the See text. (From Schneider, 1972.)
532 ANIMAL PSYCHOPHYSICS

color mechanisms o£ the pigeon and man may be account for individual differences in the data from
similar in some ways. some of Boakes’s birds.
Direct scaling methods use more information from Because operant response probability is easily con-
the response measure than simple rank order. In ditionable and is subject to so many nonsensory in¬
human psychophysics, magnitude estimation exempli¬ fluences, it seems unlikely that it can be used con¬
fies these methods. In animal work, attempts have veniently as a direct scaling measure. The relation
been made to use response rate for the same purpose. between reaction time and intensity, on the other
For example, Herrnstein and Van Sommers (1962) hand, seems largely built into the organism, and
used rate in an attempt to scale visual stimulus inten¬ this measure may provide information on the trans¬
sity to the pigeon. In this study, pecking rate corre¬ duction of intensity. As we have already seen, equal
sponded roughly to a human subject’s assignment of loudness and equal brightness contours have been
numbers to stimulus intensities in the magnitude successfully based on equal reaction times (Moody,
estimation procedure. To bring rate under the control 1969, 1970; Stebbins, 1966). Stebbins suggested that
of intensity, these investigators first trained birds to loudness for the monkey might be scaled in units of
respond at different rates to several different visual reciprocal response latency. Recent work with human
intensities with increased rates required for higher subjects (Mansfield, 1970; Vaughan, Costa, 8c Gilden,
intensities. New stimuli, in between the training 1966) suggests that human reaction time is a power
stimuli in intensity, were found to elicit intermediate function of intensity, just as are numerical magnitude
rates. These rates, taken as estimates of the magnitude estimates. Aikin (1973) compared reaction time scales
of the intermediate stimuli, were roughly predicted by with magnitude estimation scales for the same sets of
a power function relating intensity and magnitude. auditory and visual stimuli and found them very
As Boakes (1969a) points out, the Herrnstein and similar. Perhaps reaction time scales may serve the
Van Sommers method for direct sensory scaling seems function in animal psychophysics that magnitude esti¬
to depend on the establishment of a general relation mation scales serve in human psychophysics, but much
(“the brighter the light, the faster I go”) rather than remains to be done before this correspondence can be
on the attachment of specific rates to specific inten¬ accepted.
sities. In other words, for the method to be effective, Biases introduced by training conditions and rein¬
the response measure must change continuously and forcement contingencies clearly hamper direct scaling
monotonically with the stimulus dimension under to animal studies. Probabilistic models of information
study. Furthermore, the function representing this processing, such as signal detection, hold out the hope
change must not be tied to specific parameters of that the effects of bias may be extracted from psycho¬
training, such as the particular response rates rein¬ physical data, and “pure” sensory scales might result.
forced. It is not clear that either of these conditions is The d' value of detection theory, for example, could
met in the Herrnstein and Van Sommers work; the serve as the unit for a sensory scale which, if the
results might, for example, represent the generaliza¬ assumptions of the theory were met, would be inde¬
tion to the test stimuli of the specific rates controlled pendent of reinforcement and other nonsensory biases.
by nearby training stimuli. In light of these objec¬ Though such scaling has not been attempted with
tions, Boakes (1969a) recommends the indirect method animals, signal detection theory has played an in¬
of bisection, which makes fewer assumptions, and he creasingly important role in other areas of animal
has used this method with some success in the bisec¬ psychophysics, and we shall consider this matter in
tion, by pigeons, of a brightness interval (1969b). In the following sections.
this case, pigeons learned to peck right for a bright
stimulus and to peck left if the stimulus was dim. The
stimulus that produced equal pecking on each key was SIGNAL DETECTION THEORY IN
then assumed to bisect the brightness interval. ANIMAL PSYCHOPHYSICS
Boakes’s data suggest that a power function describes
the relation between visual intensity and subjective Experimenters who study sensory processes usually
magnitude in his pigeons. However, the prevalence of seek data that reflect stimulus effects unconfounded by
position preferences, which we discussed earlier, could variables that affect behavior through other channels,
seriously interfere with this method. Since these pref¬ such as motivation, attention, or response bias. We
erences tend to be most pronounced during difficult have already discussed some procedural measures de¬
discriminations, they might be expected to affect re¬ signed to minimize the impact of these other vari¬
sponse choice for intermediate stimuli. This effect may ables. Despite such techniques, however, confounding
Donald Blough and Patricia Blough 5 33

effects remain, and efforts to isolate them through

appropriate data analysis continue to be part of psy¬
chophysical research. The theory of signal detection is
a relatively recent mode of analysis that attempts to
separate sensory from other influences, which are
lumped together as “bias.”
The theory of signal detection became popular
largely because it seemed to account more adequately
than did classical theory for the pattern of detection
responses to weak stimuli. In the present operant con¬
text, we can express a major difference between signal
detection theory and classical theory in terms of their
assumptions about the nature of stimulus control,
Classical theory says, in effect, that detection responses
are entirely under stimulus control on most trials. A
weak stimulus sometimes is below threshold, leading
to a “no77 response. Classical theory further assumes
that on some percentage of the bel ow-threshold trials
(which experimenters try to minimize) unspecified
nonstimulus variables cause the subject to emit a
“yes.77 These “false alarms77 are pure guesses, unre¬
lated to the stimulus. Signal detection theory, on the
other hand, holds that the stimulus controls response
on all trials, but this control is shared with other
(bias) variables. The theory incorporates a statistical
scheme by which stimulus effects are, in effect, added $, .idealized ROC (isosensitiyity) curves for two different
Signal intensities. The points along each curve repr^§«jj|£ dlffsiT-
to bias effects to determine response outcome, ent respongg biases, which modify the p£V££M&^e <s£ responses
The assumptions of detection theory lead to the both to 'noise” (no signal)»-plotted on the abscissa—and to the
prediction, widely confirmed in human experiments, signal—plotted oH e vrdinate, Panel A shows untransformed
response probabilities; panel B shows the same data? hut
that changes in signal strength on the one hand, and spacing on the coordinates is in terms of normal s soores.
changes in bias on the other, affect the probability of
a detection response in characteristic ways. The out¬
come of these changes is clarified by the sort of graph index d' measures the stimulus effect; it represents the
common to signal detection presentations, in which distance of a curve from the diagonal. Wc cannot go
correct detections (“yesses” to a signal) are plotted further into the basic theory in this chapter; the
against false alarms (“yesses77 to non-signals). When a reader unfamiliar with signal detection theory 15
signal of constant strength is presented many times, urged lo consult a reference such as Engen (1971) or
while one or more other factors are varied (e.g., the the more comprehensive Green and Swets (19GG).
relative reinforcement of “yes” and responds), The primary yariablc5 that affect performance, ac¬
points along a "receiver operating characteristic77 cording to detection theory, are signal strength, signal
(ROC) or “isosensitivity77 curve result. If signal probability, and reinforcement contingencies. Since
strength changes, a new ROC curve is produced. these variables are easily manipulated in animal dis¬
Examples of such curves are shown in Figure 9. The crimination experiments, and because such studies
most common form of signal detection theory predicts may feasibly involve many thousands of responses,
that these ROC curves will have the form shown in animal experimenters have been increasingly inter¬
Figure 9A and that they will become straight lines, as ested in using this sort of analysis. This has been done
in Figure 9B, when the coordinates are transformed not only to clarify sensory processes, but also to in¬
into standard deviation units (“z-scores”). These dia¬ vestigate the assumptions of detection theory and to
grams separate graphically the two kinds of variables, find out more about the actual interactions of the
signal and bias, with which the theory deals: signal variables controlling behavior.
changes move a data point from ROC curve to ROC The use of animal subjects has led to some re¬
curve, toward or away from the main diagonal; bias visions in the rationale behind assumptions of detect¬
changes slide the point along a curve. The sensitivity ability theory, if not in their form. For example, the
534 ANIMAL PSYCHOPHYSICS

human observer is usually assumed to be “fully in¬ detection theory predicts that the ROC curve for each
formed” about the distribution of input events that he stimulus condition will be parallel to the main
will encounter, and he is thought to make a rational diagonal as shown in Figure 9B. The distance be¬
decision leading to such a goal as “maximum number tween the ROC curve and the diagonal then may be
of correct responses.” An analysis more congenial to summarized by a single number, d', which represents
most animal investigators is provided by Boneau and the subject’s sensitivity to the stimulus that yielded
Cole (1967), who show that, if a subject adjusts its re¬ the line. However, most of Wright’s data, like most
sponses to “most food for least work,” its contact with results from human subjects, are best fit by lines that
the reinforcement contingencies can provide informa¬ converge, so no single distance index can describe the
tion comparable to that available to the “fully in¬ bird’s sensitivity for a given stimulus condition. A
formed” human observer. Detection theory also as¬ variety of measures have been used to cope with this
sumes that trials are statistically homogeneous; that is, situation (cf. Green 8c Swets, 1966). Wright adopted as
essentially the same statistical processes control detec¬ his measure the distance of the ROC curve from the
tion on every trial. This assumption is rather rarely major diagonal at the point of “zero bias,” that is,
mentioned in human studies, but must be carefully where the curve crosses a minor diagonal in a diagram
considered in animal work, where occasional “lapses such as that of Figure 9. From the d' values de¬
of attention” seem almost inevitable. We shall return termined in this way, Wright constructed psycho¬
to this matter below. metric functions relating df to wavelength difference
at each reference wavelength that had been tested.
From each of these functions he picked off that wave¬
Some Animal Studies Using Signal
length difference that yielded a d' of 2.0, and this set
Detection Analysis
of wavelength differences yielded detailed discrimin-
A number of recent studies have subjected animal ability functions.
data to signal detection analysis. Of these, however, In a more recent article, Wright (1974) used these
few have used the analysis as an aid in determining results to develop a model of discrimination that
sensory functions. One of the few is the wavelength brings signal detection theory together with more
discrimination study by Wright (1972), whose method classic views of the psychometric function. He argues
was previously described. In this study, the pigeon that when discrimination is measured under equal-
made a choice peck at one key if two wavelengths bias conditions the psychometric function becomes a
were the same, and at another key if they differed. In straight line if plotted with df as a function of stim¬
order to provide the bias changes that are necessary to ulus difference. Further, this straight line passes
generate ROC curves, Wright varied the relative prob¬ through the origin, so its slope provides the appro¬
ability of reinforcement for responses to the two priate index of discriminability. Wright argues fur¬
choice keys. Families of such curves arose from the use ther that this conceptually simple picture has been
of a number of wavelength differences (corresponding obscured, in much psychophysical work, by procedures
to different “signal strengths”) at each reference wave¬ that introduce strong response bias. For example,
length. human subjects have often been strongly cautioned
Wright organized his data by separating correct de¬ not to give “false alarms” and thus have a strong bias
tections (pecks on the “different” key when the pro¬ against “yes” responses. We cannot delve further here
jected wavelengths differed) from false alarms (pecks into Wright’s provocative article, but it clearly illus¬
on the “different” key when the wavelengths were the trates two central points: (1) the great significance of
same). This pair of values was obtained for each set response bias and (2) the general utility of using
of wavelengths under each reinforcement condition, animals in psychophysical work. It is most unlikely
and such a pair provided the coordinates for a single that Wright could have obtained from humans the
point on a ROC diagram. Variation of reinforcement very extensive results upon which his theory is based.
probability, with wavelength difference constant, pro¬ Studies by Clopton (1972) and Elsmore (1972)
vided the set of points for an isosensitivity (ROC) exemplify the manipulation of bias through changes
function; each wavelength difference provided a new in the probability of signal presentation rather than
function, as exemplified in Figure 4 by the data from the probability of reinforcement. Clopton’s monkeys
Moody (1970). In Wright’s data, the curves shift discriminated increments in noise intensity, and his
toward the upper left corner of the diagram as wave¬ ROC curves are well fit by functions based on Ray¬
length difference increases. leigh distributions rather than on the more usual
When correct detections and false alarms are scaled normal distributions of noise and signal plus noise.
on normalized coordinates, the most common form of The monkeys in Elsmore’s experiment were trained to
 535
Donald Blough and Patricia Blough

signal was presented,” it may still suffice to make him

% LEFT RESPONSEj S<100 SEC
say, “Yes, a signal might have been presented.” Each
of these degrees of certainty, or ratings, represents the
effect of a different bias and generates a point on the
ROC curve for a given signal strength. D. Blough
(1967) applied this method to animal subjects, treat¬
ing the pigeon’s rate of pecking as an index reflecting
its rating of a stimulus. This study used stimuli on a
wavelength continuum; the stimulus set included a
single S+ value, presentations of which were inter¬
spersed with many S~ presentations. If response rate
was very low, the pigeon was considered to be “quite
sure” that the reinforced stimulus was not present,
while if the rate was high, the pigeon was “sure” that
the reinforced stimulus was present. As in other
methods, a single ROC curve corresponds to one par¬
ticular stimulus (“signal”) that differs from the S+
(“noise”). Each point along such a curve is based on
Fig. 10. Duration discrimination data from two monkeys. The
animals discriminated 60, 80, or 90 sec from 100 sec. The point? the proportion of trials that yielded a given rate of
along each line represent different response biases induced by response to the S4 (the ^-coordinate of the point) or
alterations in the relative probability that the ShOIt OT long to the stimulus in question (the ^-coordinate of the
durations would be presented. (From Elsmore, 19*72. @ 1972 by
the Society for the Experimental Analysis of Behavior, Ine.) point). When plotted on normali2ed coordinates^ these
values produced quite linear ROC curyc§ (Figure 11),
press one lever for a long stimulus duration and hut again the curves converged somewhat, rather than
another lever for shorter durations. By changing the remaining parallel to the main diagonal. Response
relative probability of the long and short durations, latency has also been used as a rating measure m
Elsmore successfully changed the bias toward the two animal work (e.g., Yager & Duncan, 1971).
levers, producing the rather nice data exemplified in Even in the absence of data necessary to generate
Figure 10. Elsmore also analyzed the interaction be¬ complete ROC curves, the signal detection format,
tween signal probability and the probability of rein¬ with “hits” plotted against “false alarms,” can be a
forcement on the two levers. This analysis exemplifies convenient way to display response bias and to esti¬
the sort of interactions of variables that may operate mate its interaction with apparent sensitivity. Torman
in a detection situation, and we shall illustrate the (1970), for example, reinforced rats equally on two
main idea by a simple example. Suppose that a long-
duration stimulus (the “signal”) occurs on many trials
and a short-duration stimulus occurs on relatively
few. If the reinforcement schedule for the two stimuli
is the same (as in this case), more reinforcements will
be obtained for pressing the lever indicating “long
duration.” Elsmore showed that the “optimal response
bias” that maximizes reinforcement probability under
these circumstances is very close to the actual bias that
he observed.
To obtain ROC curves through experimental
manipulation of the subject’s criterion, as exemplified
above, one requires a great many trials at a number 0:0! 0.05 0.15 0.3 0.5 0.7 0.85 0.95 0.99
P ([RESPONSES TO SD ] * i)
of values of the biasing variable. In human psycho¬
physics, the “rating method” has been used to produce Fig. 11. ROC (isosensitivity) functions for a discrimination be¬
ROC curves more quickly. This method assumes, in tween a wavelength of 582 nm (considered “noise”) and each
of several other wavelengths (“signals”). The curves are pro¬
effect, that a subject has a number of criteria at the duced by the “rating” method, where the coordinates of each
same time, each controlling a response that indicates point represent the probability that a given number of response
a degree of certainty about the signal. Thus if a signal (i) or fewer were made to the stimulus in question. When i is
set at successively greater values, a set of points along one curve
is not strong enough to make the subject say, “Yes, I results. (From D. Blough, 1967. © 1967 by the American Associ¬
am very sure a signal was presented,” or even, “Yes, a ation for the Advancement of Science.)
536 ANIMAL PSYCHOPHYSICS

of signal detection analysis? This question raises two

more specific issues, which we shall consider in turn.
First, does the interaction of “sensory” and “bias”
factors that is implicit in the detection model corre¬
spond to the nature of discriminative control in psy¬
chophysical settings, and does signal detection isolate
a measure of sensitivity that is truly independent of
nonsensory variables? Second, is this measure practical
to use, in terms of the time or the number of observa¬
tions required to estimate it?
As we suggested above, a major impetus for detec¬
tion analysis in human psychophysics was a failure of
psychophysical data to correspond to the classical view
that on any given trial a detection response is con¬
PER CENT RIGHT | STD trolled either by sensory input, or by some error fac¬
(false alarms)
tor, but not by both. As our review suggests, animal

Fig. 12. Isobias functions for discriminations between a 100-db

data agree with human results in this respect. Animal
standard and comparison stimuli from 80 to 100 db. The rats ROC curves clearly suggest that the detection response
differ in their degree of bias, but the amount of bias is rela¬ is a joint function of sensory and other variables, as
tively consistent in each. ROC curves cannot be plotted here
because bias was not varied for fixed signal strengths. (From
proposed in the detection model. For this reason, the
Terman, 1970. (c) 1970 by the Society for the Experimental detection paradigm may be useful to researchers inter¬
Analysis of Behavior, Inc.) ested in clarifying the nature of stimulus control. This
sort of research is beyond the scope of this chapter,
levers for “correct detections” and “correct rejections,” but see, for example, Nevin (1967, 1970), D. Blough
obtaining the “isobias” functions shown in Figure 12. (1967, 1972), Heinemann et al. (1969), and Chase and
Only signal intensity varied here, for there was no Heinemann (1972).
manipulation of bias upon which ROC curves could The available literature in this area suggests, how¬
be based. Nonetheless, it is evident that each of the ever, that indices like d' must t>e interpreted with
three rats, although run under symmetrical reinforce¬ great care. The data of Heinemann et al. (1969), for
ment conditions, had a relatively constant bias toward example, indicate that animals may be inattentive to
one lever or the other. Rat T16, for example, was the stimulus on a significant proportion of trials. This
strongly biased toward the right lever, as indicated by is equivalent to saying that the classical view of detec¬
its high rate of false alarms at weak signal values. De¬ tion is partly right, for, on some trials, sensory input
spite such biases, Terman found that the rats gave plays no role in controlling the response. Thus a cor¬
very similar functions when the sensitivity index d' rection like the classical correction for false reports
was plotted against stimulus intensity. must be estimated and applied to the data, in effect
Nevin, one of the first to apply the signal detection removing the “inattentive” trials from the subsequent
method to animal data, has suggested that the analysis.
paradigm may clarify the interactions of variables Available animal data, like most results from
other than signal and bias (Nevin, 1970; Nevin, human subjects, fail to follow the simplest version of
Olson, Mandell, 8c Yarensky, 1975). As we mentioned detection theory, which predicts that ROC curves
earlier, he reports cases in which d' appears to vary plotted on normalized coordinates will be straight
with reinforcement contingencies—despite the com¬ lines parallel to the major diagonal. Instead, these
mon assumption that this index is an unbiased index lines converge in such a way that the sensitivity index
of sensory and input factors. Such research suggests d' cannot be determined unambiguously. As we saw
the question with which we close this chapter: to what above, Wright chose to estimate dr from the point at
extent, and in what ways, is signal detection analysis which his ROC curves crossed the minor diagonal; this
of use in animal experimentation? method had the advantage of representing detection
under conditions of “no bias”—that is, when equal
numbers of incorrect responses were occurring on
The Usefulness of Signal Detection Analysis
both keys. Many other solutions to the convergence
in Animal Work
problem may be found in the detection literature.
Under what circumstances, if any, should a re¬ Assumptions about the form of underlying distribu¬
searcher plan animal experimentation around the use tions of sensory events may be altered, or nonpara-
Donald Blough and Patricia Blough 5 37

metric measures, such as the area under the ROC biased yes-no situation, the practical effects of non-
curve, may be employed (cf. Green & Swets, 1966). sensory variables on a threshold estimate may be
Though various means may be employed within rather small. Irwin and Terman (1970) point out that,
the framework of detection theory to deal with the assuming an animal’s sensitivity (d') to be constant,
convergence of ROC curves, one explanation for the even a rather strong response bias will yield only a
phenomenon is rarely considered, for it strikes at a relatively small change in percentage correct. For
fundamental assumption of the theory. This explana¬ example, if a subject incorrectly presses the "no” lever
tion questions the independence of signal and bias five times as often as he incorrectly presses the “yes”
variables. In other words, detection theory assumes lever, his percentage correct for a given signal strength
that a given signal will produce a central effect which, will be at most 4% lower than his percentage correct
though it varies statistically from moment to moment, had he no response bias. The impact of this relatively
is not affected by the variables that control bias. Un¬ small shift will be even less if bias remains relatively
fortunately, the correctness of this assumption has not constant, for it is the difference in threshold across
been firmly established either for human or animal conditions, rather than its absolute value, that is
data, and alternatives to it have been suggested (D. usually of interest.
Blough, 1972; Pike, 1973). It appears, then, that in operant work conc^rn^d
The work of Nevm and Ills co-workers (1970, 1975) strictly with sensory processes, the determination of
suggests procedural limits on the applicability of the ROC curves probably does not add enough informa¬
detection paradigm. In these studies, the usual rein¬ tion to be worth its cost. Rather than focusing on the
forcement contingencies produced a good separation ROC curve, or on the dr mea$ure? the experimenter
of sensitivity and bias effects, as implied by the theory. should concentrate qu the minimisation of bias, and
However, when the usual correlation of reinforcement use the forced-choice procedure if possible. For work
with stimuli was abandoned, this separation broke not strictly sensory, however, the signal detection
down. Such was the case, for example, when a “yes" paradigm may comprise a convenient framework,
response was occasionally reinforced in the presence This framework may further the analysis of discrim¬
of "noise” as well as in the presence of "signal.” In inative processes in animals, and animal experiments
such a circumstance, it is hardly surprising that the may provide efficient means for testing the detection
measure of sensitivity suffers since these ambiguous model and its assumptions.
reinforcement contingencies lead to an ambiguous
definition of the “correct” response. Such work is
valuable in providing a broader context within which
REFERENCES
the area of applicability of signal detection may be
identified.
Aikin, L. G. A comparHQn of reaction time and magnitude
Even if one assumes that d' is a relatively good
estimation methods for brightncss and loudnejj, Un¬
measure of sensitivity under given conditions, one may published doctoral thesis. Brown University, 1373.
ask if it i§ sufficiently better than other measures to Berkley, M; A, Visual discriminations in the cat. In YG G.
justify its use. The most accurate way to estimate dr, Stubbing (Ed,), Animal psychophysics. Englewood Cliffs.
and a necessary procedure for some nonparamchric Prentice Hall, Inc„ 13?$,
Blough, D; §: A method for obtaining psychophysical
indices, is to determine experimentally a large por¬
thresholds from the pigeon, Journal of ihc Experimental
tion of an ROC curve. This takes a great deal of time. Analywa of Bchaviort 1358, 1s 31-43.
It seems that, biased though it may be, percentage Blqugh, D, 6. Animal psychophysics. Scientific American,
correct remains an adequate measure for most studies 196b 205, 113-122.
Blough, D. 5. Stimulus generalization as signal detection in
having to do with sensory function. If the experi¬
pigeons. Science, 1967, 158, 940-941.
ment can be cast into a forced-choice paradigm, in Blough, D. S. Attention shifts in a maintained discrimina¬
which the two responses are symmetrically related to tion. Science, 1969, 166, 125-126.
the stimulus (e.g., “push the lever under the brighter Blough, D. S. Recognition by the pigeon of stimuli varying
panel”), and response preference is minimized, d' and in two dimensions. Journal of the Experimental Analysis
of Behavior, 1972, 18, 345-367.
percentage correct are virtually equivalent. To quote
Blough, P. M. The visual acuity of the pigeon for distant
Green and Swets (1966): “If the observer’s preference targets. Journal of the Experimental Analysis of Be¬
among alternatives is indeed negligible, ... a satis¬ havior, 1971, 16, 57-61.
factory index of sensitivity is the percentage of correct Blough, P. M. Wavelength generalization and discrimina¬
response, or the value of d' that corresponds to this tion in the pigeon. Perception and Psychophysics, 1972,
12, 342-348.
percentage” (p. 408).
Boakes, R. A. Response continuity and timing behavior. In
Even under less favorable circumstances, such as a R. M. Gilbert 8c N. S. Sutherland (Eds.), Animal dis-
 538 ANIMAL PSYCHOPHYSICS

crimination learning. New York: Academic Press, 1969. on the basis of stimulus differences. In D. I. Mostofsky
(a) (Ed.), Stimulus generalization. Stanford, Calif.: Stanford
Boakes, R. A. The bisection of a brightness interval by University Press, 1965.
pigeons. Journal of the Experimental Analysis of Be¬ Irwin, R. J., 8c Terman, M. Detection of brief tones in
havior, 1969, 12, 201-209. (b) noise in rats. Journal of the Experimental Analysis of
Boneau, C. A., & Cole, J. L. Decision theory, the pigeon, Behavior, 1970, 13, 135-143.
and the psychophysical function. Psychological Review, Jacobs, G. H. Increment-threshold spectral sensitivity in the
1967, 74, 123-135. squirrel monkey. Journal of Comparative and Physio¬
Chase, S., 8c Heinemann, E. G. Choices based on redundant logical Psychology, 1972, 79, 425-431.
information: An analysis of two-dimensional stimulus Jenkins, H. M., 8c Harrison, R. H. Effect of discrimination
control. Journal of Experimental Psychology, 1972, 92, training on auditory generalization. Journal of Experi¬
161-175. mental Psychology, 1960, 59, 246-253.
Clack, T. D., 8c Harris, J. D. Auditory thresholds in the rat Luce, R. D. What sort of measurement is psychophysical
by a two-lever technique. Journal of Auditory Research, measurement? American Psychologist, 1972, 27, 96-106.
1963, 3, 53-63. Malott, R. W., 8c Malott, M. K. Perception and stimulus
Clack, T. D., 8c Herman, P. N. A single-lever psychophys¬ generalization. In W. C. Stebbins (Ed.), Animal psycho¬
ical adjustment procedure for measuring auditory thresh¬ physics. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1970.
olds in the monkey. Journal of Auditory Research, 1963, Mansfield, R. J. W. Intensity relations in vision: Analysis
3, 175-183. and synthesis in a nonlinear sensory system. Unpub¬
Clopton, B. M. Detection of increments in noise intensity lished doctoral thesis, Harvard University.
by monkeys. Journal of the Experimental Analysis of Mello, N. K. Interocular generalization: A study of mirror-
Behavior, 1972, 17, 473-481. image reversal following monocular training in pigeon.
Corballis, M. C., 8c Beale, I. L. Monocular discrimination Journal of the Experimental Analysis of Behavior, 1966,
of mirror-image obliques by pigeons: Evidence for later- 9, 11-16.
alized stimulus control. Animal Behavior, 1970, 18, 563- Mentzer, T. L. Comparison of three methods for obtaining
566, psychophysical thresholds from the pigeon. Journal of
Cumming, W. W., 8c Berryman, R. Some data on matching Comparative and Physiological Psychology, 1966, 61,
behavior in the pigeon. Journal of the Experimental 96-101.
Analysis of Behavior, 1961, 4, 281-284. Mishkin, M., 8c Weiskrantz, L. Effects of cortical lesions in
Dalland, J. I. The measurement of ultrasonic hearing. In monkeys on critical flicker frequency. Journal of Com¬
W. C. Stebbins (Ed.), Animal psychophysics. Englewood parative and Physiological Psychology, 1959, 52, 660-666.
Cliffs, N.J.; Prentice-Hall, Inc., 1970. Moody, D. B. Equal brightness functions for supra-thresh-
Elsmore, T. F. Duration discrimination; Effects of proba¬ old stimuli in the pigmented rat: A behavioral deter¬
bility of stimulus presentation, Journal of the Experi¬ mination. Vision Research, 1969, 9, 1381-1389.
mental Analysis of Behavior, 1972, 18, 465-469. Moody, D. B. Reaction time as an index of sensory func¬
ENGEN, T. Psychophysics, I: Discrimination and detection. tion. In W. C. Stebbins (Ed.), Animal psychophysics.
Ill J. W. Kling 8c L. A. Riggs (Eds.), Woodworth and Englewood Cliffs, N.J.; Prentice-Hall, Inc., 1970.
Schlosberg's experimental psychology (3rd ed.). New Moskowitz, H., & Kitzes, L. A comparison of two psycho¬
York: Holt, Rinehart & Winston, 1971. physical methods using animals. Journal of the Experi¬
Gorr, W. R. Measurement of absolute olfactory sensitivity mental Analysis of Behavior, 1966, 9, 515-519.
in rats. American Journal of Psychology, 1961, 74, 384- Nevin, J. A. Effects of reinforcement scheduling on simul¬
393.
taneous discrimination performance. Journal of the Ex¬
Gourevitch, G. Detectability of tones in quiet and in perimental Analysis of Behavior, 1967, 10, 251-260.
noise by rats and monkeys. In W. C. Stebbins (Ed.), Nevin, J. A. On differential stimulation and differential
Animal psychophysics. Englewood Cliffs, N.J.: Prentice- reinforcement. In W. C. Stebbins (Ed.), Animal psycho¬
Hall, Inc., 1970. physics. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1970.
Green, D. M., 8c Swets, J. A. Signal detection theory and Nevin, J. A., Olson, K., Mandell, C., 8c Yarensky, P. Rela¬
psychophysics. New York: Wiley, 1966. tive reinforcement and signal detection. Journal of the
Guttman, N., 8c Kalish, H. I. Discriminability and stimulus Experimental Analysis of Behavior, 1975, 24, 355-367.
generalization. Journal of Experimental Psychology, Nye, P. W. On the functional differences between frontal
1956, 51, 79-88.
and lateral visual fields of the pigeon. Vision Research,
Heinemann, E. G., Avin, E., Sullivan, M. A., 8c Chase, S. 1973, 13, 559-574.
Analysis of stimulus generalization by a psychophysical Pike,R. Response latency models for signal detection. Psy¬
method. Journal of Experimental Psychology, 1969, 80, chological Review, 1973, 80, 53-68.
215-224. Raslear, T. G., Pierrel-Sorrentino, R., 8c Brissey, C. Con¬
Henton, W. W. Conditioned suppression to odorous current assessment of schedule and intensity control
stimuli in pigeons. Journal of the Experimental Analysis across successive discriminations. Journal of the Experi¬
of Behavior, 1969, 12, 175-185. mental Analysis of Behavior, 1975, 23, 247-254.
Herrnstein, R. J., 8c Van Sommers, P. Method for sensory Ray, B. A. Psychophysical testing of neurologic mutant
scaling with animals. Science, 1962, 135, 40-41. mice. In W. C. Stebbins (Ed.), Animal psychophysics.
Hodos, W., 8c Bonbright, Jr., J. C. The detection of visual Englewood Cliffs, N. J.: Prentice-Hall, Inc., 1970.
intensity differences by pigeons. Journal of the Experi¬ Rosenberger, P. B. Response-adjusting stimulus intensity.
mental Analysis of Behavior, 1972, 18, 471-479. In W. C. Stebbins (Ed.), Animal psychophysics. Engle¬
Honig, W. K. Discrimination, generalization, and transfer wood Cliffs, N.J.: Prentice-Hall, Inc., 1970.
Donald Blough and Patricia Blough 5 39

Sadowsky, S. Discrimination learning as a function of of equal loudness contours for the monkey. Journal of
stimulus location along an auditory intensity continuum. the Experimental Analysis of Behavior, 1966, 9, 135-142.
Journal of the Experimental Analysis of Behavior, 1966, Stebbins, W. C. Studies of hearing and hearing loss in the
9, 219-225. monkey. In W. C. Stebbins (Ed.), Animal psychophysics.
Sainsbury, R. S. Effect of proximity of elements on the Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1970.
feature-positive effect. Journal of the Experimental Anal¬ Sutherland, N. S. Shape discrimination in rat, octopus, and
ysis of Behavior, 1971, 16, 315-325. goldfish: A comparative study. Journal of Comparative
S as low, C. A. Animal psychophysics: An examination of the and Physiological Psychology, 1969, 67, 160-176.
field. Seattle: University of Washington, Report PRP- Symmes, D. Self-determination of critical flicker frequencies
37N, 1967. in monkeys. Science, 1962, 136, 714-715.
Saunders, J. Behavioral discrimination of click intensity in Tapper, D. N., & Halpern, B. P. Taste stimuli: A be¬
cat. Journal of the Experimental Analysis of Behavior, havioral categorization. Science, 1968, 161, 708-709.
1969, 12, 951-957. Terman, M. Discrimination of auditory intensity by rats.
Schneider, B. Multidimensional scaling of color difference Journal of the Experimental Analysis of Behavior, 1970,
in the pigeon. Perception and Psychophysics, 1972, 12, 13, 145-160.
373-378. Terman, ML, I erman, J. S. Concurrent variation of
Schusterman, R,, 8c Balliet, R, F. Conditioned vocaliza¬ response bias and sensitivity in an operant-psychophysi¬
tions as a technique for determining visual acuity thresh¬ cal test. Perception and Psychophysics, 1972, 11, 428-432.
olds in sea lions. Science, 1970, 169, 498—501. Thomas, D. R. The use of operant conditioning technique
Scott, X. R., & Milligan, W. L. The psychophysical study to investigate perceptual processes in animals, In R, M.
of visual motion aftereffect rate in monkeys. In W. C. Gilbert 8c N. S. Sutherland (EdsJ Animal discrimination
Stebbins (Ed.), Animal psychophysics. Englewood Cliffs, learning. New York: Academic Press, 1969.
NJ.; Prentice-Hall, Inc., 1970. Thorpe, S. A. Behavioral measures of spectral sensitivity
Scott, T, R,, & Powell, D. A, Measurement of a visual of the goldfish at different temperatures. Vision Re¬
motion after-effect in the rhesus monkey, Science, 1963, search, 1971, 11, 419-433.
140, 57-59.
Vaughan, Jr., H. G., Costa, L. D., & Gilben, L. The func¬
Seligman, M. E. P. On the generality of the laws of learn¬
tional relation of visual evoked response and reaction
ing. Psychological Review, 1970, 71, 406-418.
time to stimulus intensity. Vision Research, 1966, 6, 654-
Shepard, R. N. Approximation to uniform gradients of
656.
generalization by monotone transformations of scale. In
Wilson, M., Stamm, J. S., & Pribram, K. H. Deficits in
D. I. Mostofsky (Ed.), Stimulus generalization. Stanford,
roughness discrimination after posterior parietal lesions
Calif.: Stanford University Press, 1965.
Shepard, R. N. Metric structures in ordinal data. Journal
in monkeys. Journal of Comparative and Physiological
of Mathematical Psychology, 1966, 3, 287-315. Psychology, 1960, 6, 535-539.
WRICHT, A. A. Psychometric and psyclroph ysical hue dis¬
Sidley, N. A., Sperling, H. G., Bedarf, E. W., & Hiss, R. H.
Photopic spectral sensitivity in the monkey: Methods for crimination functions for the pigeon, Vision Research}
determining and initial results. Science. 1965, 130, 1837- 1972, 12, 1447-1464.
1839. Wright, A. A. Psychometric and psychophysical theory
Sidman, M., Ray, B. A., Sidman, R. U, & Klinger, J. M. within a framework of response bias. Psychological Re¬
Hearing and vision in neurological mutant mice: A view, 1974, 813 322-347.
method for their evaluation. Experimental Neurology, Yaoer, D., 8c Duncan, I. Signal-detection analysis of lump
1966, 16, 377-402. nance generalization in goldfish using latency as a
Smith, J. Conditioned suppression as an animal psycho¬ graded response measure. Perception and Psychophysics,
physical technique. In W. G. Stebbins (Ed.), Animal 1971, 9, 353-355.
psychophysics. Englewood Cliffs, NJ.: Prentice=Hall, Inc., Yager, D., k Thorpe, 5. Investigations of goldfish color
1970. vision. In W. C. Stebbins (Ed.), Animal psychophysics.
Stebbins, W, G. Auditory reaction time and tlie derivation Englewood Cliffs, N.J.: Prentice-Hall, Inc.. 1970.
 18

Operant
Behavioral Pharmacology

Travis Thompson
and
John J. Boren

INTRODUCTION However, due in large part to these early clinical dis¬

coveries, further interest in laboratory research in be¬
Scientific interest in the behavioral actions of drugs havioral pharmacology took a sharp upswing. In the
is not new. For decades investigators have studied late 1950s and early 1960s, the primary interest cen¬
whether certain drugs could alter behavior in open tered around two problems: (1) How can one discover
field tests, T-mazes, straight alleys, pole climbing new drugs with useful applications in psychiatry? (2)
apparatuses, shuttle boxes, key pecking situations, and Is it possible to arrive at a more thorough understand¬
lever pressing devices. Often such studies attempted to ing of these drugs through a laboratory analysis?
elucidate drug effects on such phenomena as memory, It became clear early on that operant techniques
learning, anxiety, or drives. Two problems became were among the most sensitive for measuring the be¬
apparent from this body of research: (1) More sensi¬ havioral action of drugs (Boren, 1966; Cook 8c Kel-
tive and reliable behavioral procedures were needed leher, 1963; Dews 8c Morse, 1961; Gollub 8c Brady,
to assess the behavioral actions of drugs. (2) Many of 1965; Sidman, 1959). The profusion of studies using
the early questions asked were clearly premature, if operant techniques that followed led to the emergence
not misguided. An objective and operationally based of several journals to accommodate the burgeoning
conceptual framework within which to interpret the literature. In addition, textbooks on drugs and be¬
actions of drugs on behavior had to be formulated be¬ havior were written (e.g., Iverson 8c Iverson, 1975;
fore such issues could begin to be considered. Rech 8c Moore, 1971; Thompson 8c Schuster, 1968)
Early infrahuman experiments were given impetus which involved major emphasis on the experimental
in the 1950s when chlorpromazine and reserpine, the analysis of operant behavior.
first tranquilizers, were reported to be useful in treat¬ That operant baselines are sensitive indicators of
ing certain psychiatric patients. These findings were drug action now seems uncontested. However, it is
made in the clinic and came more as a surprise than one thing to show that a behavioral system is sensitive
as the logical outcome of planned laboratory research. to the manipulation of an independent variable (in

540
Travis Thompson and John J. Boren 541

this case, drug administration), and it is another to ask, “What is the behavioral mechanism by which
show that the resulting data are meaningful and this drug effect is brought about?”
significant. One of the major theses of the present
chapter is that operant behavioral pharmacology has,
The Problem of Sample Size
by and large, succeeded in satisfying the two major
requisites of a scientific domain concerned with the A common feature of the operant approach to be¬
analysis of drug actions on behavior: (1) The pro¬ havioral pharmacology involves the intensive study of
vision of sensitive and reliable behavioral procedures; a single individual subject. The emphasis is on close
and (2) the provision of an objective, operationally observation and firm experimental control. If the
based conceptual framework within which to inter¬ experiment is successful, a subject will behave pre¬
pret the results of experiments on the behavioral dictably from session to session and even from minute
actions of drugs. It is our contention that one of the to minute. Thus, when an effective drug is adminis¬
main purposes of operant behavioral pharmacology is tered in the middle of a session, a change from the
to interpret the behavioral mechanisms by which these dependable baseline behavior should be readily ap¬
behavioral changes are brought about; that is, to ex¬ parent in an individual subject. Furthermore, on
press the scientific significance of findings concerning different sessions, a range of drug dosages can be
behavioral actions of drugs in terms of a more general studied in the same subject with a sound basis for
set of principles. comparison.
Behavioral pharmacology, which has grown out of The intensive study of the individual subject will
the integration of experimental psychology and be emphasized in this chapter and in other discussions
pharmacology, is concerned with the behavioral of operant techniques. This might be labeled the “N
actions of drugs. Its primary goal is the description of of One” approach, and has sometimes been misunder¬
the behavioral mechanisms by which drugs alter be¬ stood. The approach is not simply to use a small num¬
havior. A prerequisite to such a description is an ber of subjects for its own sake. For example, someone
understanding of the factors that control the behavior might attempt a drug-behavior experiment with three
in question. To understand the way in which drugs subjects stabilized on a behavioral baseline. The first
alter behavior, it is first necessary to understand the subject is given dose #1, the second subject is given
factors which control behavior. It follows that a de¬ dose #2, and the third subject is given dose #3. From
scription of the interaction of drug variables with be¬ such an experiment, one can do little more than esti¬
havioral variables is an essential first step in under¬ mate crudely the nature of the dose-response curve.
standing the behavioral mechanisms of drug actions Differences among subjects will be confounded with
(Thompson and Schuster, 1968). In general pharma¬ differences produced by the drug dosage so that one
cology, the “mechanism of action” of a drug refers to cannot tell which is which. A more informative use
some “basic” process, typically physiological or bio¬ of three subjects would be to administer each dose on
chemical, which mediates a drug’s effect upon a par¬ separate sessions to each of the subjects. Then with
ticular response. In pharmacology, the term “re¬ each individual, one could determine how the differ¬
sponse” refers to any change in the organism which ent doses modified this subject’s behavior.
can be reliably produced by a drug on repeated oc¬ There is no simple rule for deciding on the number
casions. Within behavioral pharmacology such re¬ of subjects. The critical question is whether the ex¬
sponses can be properly related only to behavioral periment can be replicated, and the most straight¬
mechanisms of action—that is, mechanisms describable forward answer comes from successful replications. If
in terms of basic behavioral processes. unknown and/or uncontrolled variables are operating
It is important to keep in mind the principle that a in the experiment, replication will be difficult, and
drug cannot cause a biological system to respond in a the experimenter will be aware of trouble with his
qualitatively new way. That is, a drug may increase or technique. Thus, when the experimenter has reason
decrease values of dependent variables but may not to suspect uncontrolled variables, he is wise to use
cause a fundamental change in the operation of the more than one subject.
biological system. As a consequence, we must ask our¬ There are circumstances where a single subject is
selves, “With which of the existing systems that regu¬ adequate. Although opinions may differ on this mat¬
late behavior is a drug interacting to produce the ter, the circumstances are approximately as follows:
observed behavioral change?” This is the fundamental (1) The experimenter is working in a well-controlled
question to which we must address ourselves when we experimental situation with a thorough knowledge of
542 OPERANT BEHAVIORAL PHARMACOLOGY

his techniques and his subjects. Thus, the experi¬ chemicals (Sparber, 1972; Spyker, Sparber, Sc Gold¬
menter probably has the problem of uncontrolled berg, 1972) administered during gestation or chron¬
variables well in hand. (2) The results from the single ically in the adult animal (Weiss and Laties, 1969).
subject are in accord with previous data and fit plau¬ Further, the seemingly evergrowing incidence of drug
sibly into a well-understood body of knowledge. (3) abuse has posed an enormous research problem for
The experimenter has studied other subjects with behavioral pharmacologists.
related procedures, and the results are consistent. Sid-
man’s (1960) more extensive discussion of this matter Analysis and Verification
under the topic of “systematic replication’’ is recom¬ of Clinical Findings
mended to the interested reader.
A drug which is useful in the clinic is interesting
Under these circumstances the probability of suc¬
to the research worker for several reasons. He may
cessful replication is high, so that additional subjects
wish to understand more fully the mechanism respon¬
are unnecessary. The same amount of experimental
sible for the clinical effect than can be done conve¬
time might be better spent in studying variations on
niently and without danger in human patients. In the
the experiment rather that replicating. By the same
laboratory with animal subjects, he can readily per¬
token, if these circumstances do not hold, the use of
form surgery, administer toxic doses, or implant elec¬
additional subjects is indicated.
trodes in an effort to understand the drug’s action.
In another case he may use the drug to determine
Why Experiment with Drugs?
whether the laboratory procedures are relevant to a
The reasons for selecting a drug for an experi¬ clinical problem. For example, suppose a researcher
mental study are no less complicated than the reasons has found a way to disrupt the complex conditioned
for investigating any variable which affects behavior. behavior of guinea pigs by injecting an extract from
However, at least a few of the more common reasons the blood of psychotic patients and he wants to know
can be indicated: if his experimental situation is related to the psychoses
of human patients. If Drug A is known to affect the
Curiosity about the Behavioral behavior of psychotic patients favorably, and if Drug
Effects of Drugs A also reduced the extract-induced disruption of the
How will atropine affect conditioned avoidance be¬ behavior of the guinea pigs, then the researcher has
havior? Is behavior maintained by a fixed-interval reason to suspect that his behavioral technique might
schedule more sensitive to drugs than behavior main¬ be useful in studying anti psychotic agents, such as
tained by a fixed-ratio schedule? Such questions are drugs. However, he may be incorrect. Drug A may
typical of those which a behavioral phamacologist alter the disrupted behavior of the guinea pigs for
might find interesting. Drugs can be powerful vari¬ entirely different reasons from those responsible for
ables; they can eliminate behavior altogether or in¬ changing the behavior of psychotic patients, and the
crease it manifold. Any drug might have an interest¬ laboratory situation may bear only a superficial re¬
ing or unusual action in a behavioral situation, and semblance to clinical psychosis.
the fact that such a situation occasionally arises may
Analyzing Behavioral Processes
be quite enough to maintain the activity of a scientist.
A drug can occasionally be found which has a cer¬
Practical Utility of Drugs in
tain main effect without seriously disrupting second¬
Human Affairs
ary effects. This drug can then be used as an analytical
The use of drugs in treating human ills represents tool. For example, Dews (1955a) showed that behavior
the most socially valuable application of pharmacol¬ maintained by a fixed-interval (FI) schedule of rein¬
ogy. As a result, a great deal of research is directed forcement was much more sensitive to the effects of
toward potential applications of drugs. For example, pentobarbital than was behavior maintained by a
scientists in government and industrial laboratories fixed-ratio (FR) schedule. In a related study, Herrn-
study thousands of newly synthesized organic chem¬ stein and Morse (1956) examined the effect of pento¬
icals every year in the hope that they will discover a barbital on similar tandem FI FR performance, where
medically useful drug. There are also a number of the behaviors generated by the two components of the
other practical implications of drug research. In re¬ schedule were joined in a single performance and
cent years, for example, increasing attention has been could not be easily disentangled. When pentobarbital
given to problems of behaviorally toxic effects of was given in a high dosage, the pause characteristic of
Travis Thompson and John J. Boren 5 43

fixed-interval behavior was sharply changed while the It is sometimes a hallmark of powerful control by
fixed-ratio behavior remained unchanged. Thus, the determining variables which may make it hard to pro¬
drug experimentally separated the two behaviors and duce any departure. Simple escape behavior to an in¬
gave the experimenters additional evidence that the tense electric shock (e.g., the rat presses a lever to
complex tandem performance could indeed be prop¬ turn off a shock) is a case in point. This behavior is
erly analyzed into the simple components. As another quite stable and typically occurs less than a second
example, Thompson and Pickens (1970) compared pat¬ from the shock onset. However, a drug must produce
terns of self-administration of stimulants and opiates a massive effect, such as making the rat severely ataxic,
by infrahuman subjects. They found that stimulants before the escape responding is reduced.
such as the amphetamines and cocaine were self- Another case where stability assumes secondary im¬
administered in a highly regular pattern with ex¬ portance involves behavior which is interesting partly
tremely narrow distributions of inter-response times. because it is inappropriate to or fails to meet the cur¬
Opiates, on the other hand, engender a bimodal dis¬ rent environmental contingencies. Such behaviors are
tribution of inter-response times which is far flatter not likely to remain stable. However, a drug study of

and more variable. individual subjects in such situations is not necessarily

difficult since the behavior frequently is temporarily
stable or undergoes slow, systematic change. A drug
Baseline Stability
which rapidly brings the behavior under the control
The ideal behavioral baseline should be stable. of the current contingencies (a “therapy” effect) pro¬
Stable means that the behavior remains about the vides an interesting outcome. Morse and Herrnstein

same from one observation period to another (i.e., (1956) described a pigeon which was required to peck
from session to session or from hour to hour). For a key 160 times (FR 160) for a food reinforcement.
example, if an animal’s lever-pressing rate to avoid a For the conditions of the experiment the number of
shock remained between 9.5 and 10.5 responses per responses required was overly large, so that the bird
minute over 20 sessions, then the behavior would sometimes paused half an hour or more before work¬
surely be regarded as stable because of the low vari¬ ing for a reinforcement (a “strained” fixed-ratio per¬
ability. One would have considerable confidence that formance). Methamphetamine not only greatly in¬
the response rate in the next session would remain creased the bird’s pecking rate immediately but the
between 9.5 and 10.5 responses per minute. If a drug rate remained high the next day when the drug was
were given prior to this session and the response rate no longer present. The high rate in the next session
went up as little as 20 percent (to 12 responses/min¬ was presumably due to the unusually large number of
ute), one would still conclude that the drug had in¬ reinforcements made possible by the drug’s action on
creased the rate because of the clear departure from the previous day. Baseline stability, while it makes
the usual variability of the baseline. drug work more convenient and exact, need not be a
Note the relation between the degree of stability critical consideration. Semi-stable procedures may per¬
and the magnitude of effect with which the experi¬ mit useful observations which are not possible with
menter can work. The greater the stability, the the more conventional techniques.
smaller the effect which can be reliably studied. For
example, if the mean of 20 control values is 10.0
responses/minute and the range is =±.l response/min- PRINCIPLES OF DRUG ACTION
ute (a very stable baseline), then a 10 percent increase
to 11 responses/minute following a drug injection While the student of operant behavior is usually
would be considered a reliable effect. On the other aware of behavioral factors in designing drug-be¬
hand, with the same mean and a range of ±5 re¬ havior research, he may be unfamiliar with basic
sponses/minute (a less stable baseline), a 10 percent pharmacological variables. In attempting an under¬
“increase” above the mean would be well within the standing of the principles of pharmacology, it is well
normal variation. Thus, a drug dose which was in¬ to keep in mind several basic classes of variables:
jected before the session would be considered ineffec¬
tive. Statistical tests of significance can be used for a A. The type of drug
more formal analysis of this issue. By any analysis, B. The route of administration
however, greater baseline stability makes for easier C. The relationship between the dose of drug ad¬
evaluation of small drug effects. ministered and the magnitude of response
Extreme baseline stability can be a mixed blessing. D. Absorption and distribution
 544 OPERANT BEHAVIORAL PHARMACOLOGY

E. Time course of drug effects peutic use, and a common starting point for the
F. Distribution synthetic program is a known useful (and salable)
G. Fate drug. Thus, after the success of the first barbiturates
H. Multiple administrations of the same drug. (barbital and phenobarbital), over two thousand
variations were synthesized. Furthermore, industrial
The Type of Drug pharmacologists screened the compounds for the abil¬
ity to put mice or rats to sleep. Largely as the result
It would clearly be useful if the many behavioral of such activity, a number of hypnotic barbiturates
drugs could be arranged on the basis of common were made available to the physician and to the
properties into a small number of categories. Thus, pharmacologist. However, it would be utterly incor¬
we could study, understand, and remember significant rect to think that all compounds with a barbiturate
facts about the small number of categories rather than nucleus are hypnotic drugs. Indeed, there are count¬
being confused by a mass of particulars. The only less barbiturates which are inactive as hypnotics or
problem would seem to be the selection of appro¬ are toxic at hypnotic doses.
priate criteria for the various categories. Unfor¬ A similar case can be made for the phenothiazines,
tunately, no classification scheme turns out to be for which chlorpromazine is the model compound.
really satisfactory. For example, one logical choice for After the clinical success of chlorpromazine in treat¬
a classification criterion might be the mechanism of ing schizophrenic patients, countless phenothiazines
action, such as the locus of action within the CNS and were synthesized, and screening programs selected out
the neurotransmitter involved. However, the informa¬ the compounds which had chlorpromazine-like ac¬
tion on most drugs is much too scanty and speculative tions. The selected drugs were then usually tested in
to permit classification on this basis. Another more the same clinical situation for which chlorpromazine
molar possibility might be to try to classify drugs as had been proven useful. As a result of this carefully
stimulants or depressants on the basis of whether the biased selection procedure (and, of course, partly be¬
drug increases or decreases the response rate on some cause variations of the phenothiazine structure yielded
standard schedule of reinforcement. The problem compounds with antipsychotic activity), we now have
with such categories is that drugs usually have more a group of structurally similar drugs with clinically
complex effects. For example, pentobarbital can in¬ similar effects. Nevertheless, there are many inactive
crease fixed-ratio response rates at low doses and de¬ phenothiazines which did not reach the market and
crease rates at high doses (Waller and Morse, 1963). are hardly known outside drug company laboratories.
Is pentobarbital to be classed a stimulant or a depres¬ There are other difficulties with classification by
sant? Atropine increases the rate in the initial section chemical structure. Compounds with similar effects
of fixed-interval scallops and decreases the rate in the (such as amphetamine and methylphenidate, or
terminal section (Boren and Navarro, 1959). Should chlorpromazine and haloperidol) have quite different
the atropine effect on the initial section be classed structures. Sometimes two compounds with only
stimulant and on the terminal section depressant? Or minor chemical differences (for example, one methyl
should atropine, based on the entire effect, be called a group or one chlorine atom more or less) have sub¬
disruptant? These examples illustrate an inherent stantially different pharmacological activity. Further¬
problem with drugs: They have multiple and complex more, the action of certain molecules separately may
effects and they therefore resist classification into any bear no relation to their action when combined. For
one category. example, the antipsychotic drug, perphenazine, is
Drugs are sometimes categorized by chemical struc¬ composed largely of joining phenothiazine to piper¬
ture. Classification by structure can be useful when azine, both of which are used to destroy intestinal
several compounds with a similar structure have worms. For reasons such as these, it has not proved
similar effects. The classical case for CNS drugs is the feasible to classify most drugs by chemical structure.
barbiturate structure. Amobarbital, pentobarbital, The common categorization of drugs is on the basis
secobarbital, etc., are all used to induce sleep, and of their therapeutic effect. Table 1 lists a number of
therefore, discussing these compounds as “barbitu¬ drugs affecting behavior and groups them in catego¬
rates” is meaningful. However, we sometimes forget ries based upon therapeutic usage. These categories
that drugs used in medical practice are often selected are subject to the advantages and disadvantages dis¬
in a thoroughly biased way. In the laboratories of cussed above. However, the table serves to list repre¬
pharmaceutical manufacturers, organic chemists syn¬ sentative behavioral drugs grouped according to a
thesize a great many compounds for possible thera¬ widely used classification.
Travis Thompson and John J. Boren 545

TABLE 1 Representative Behavioral Drugs Classified According to Therapeutic Usage*

I. ANTIPSYCHOTIC DRUGS V. HYPNOTICS

chlorpromazine (Thorazine) pentobarbital (Nembutal)

triflupromazine (Vesprin) secobarbital (Seconal)
trifluoperazine (Stelazine) amobarbital (Amytal)
perphenazine (Trilafon) phenobarbital (Luminal)
fluphenazine (Permitil, Prolixin) methaqualone (Quaalude, Sopor)
thioridazine (Mellaril) chloral hydrate (Somnos)
haloperidol (Haldol)
VI. HALLUCINOGENS (PSYCHOTOMIMETICS)
II. ANTIANXIETY DRUGS
LSD
meprobamate (Miltown, Equanil) mescaline
chlordiazepoxide (Librium) psilocybin
diazepam (Valium)
VII. ANALGESICS
III. ANTIDEPRESSANT DRUGS
morphine
imipramine (Tofranil) meperidine (Demerol)
amitryptaline (Elavil) methadone
isocarboxazid (Marplan) heroin
nialamide (Niamid)
phenelzine (Nardil) VIII. OTHER PSYCHOTROPIC DRUGS

atropine
IV. STIMULANTS
scopolamine
dl-amphetamine (Benzadrine) cocaine
methamphetamine (Desoxyn, Methedrine) reserpine
methylphenidate (Ritalin) lithium carbonate
magnesium pemoline (Cylert) tetrahydrocannabinol
caffeine

* The generic or nonproprietary name is listed first, and the trade or proprietary name is listed sec¬
ond in parentheses.

Routes of Administration* In subcutaneous (SC) injections the tip of the

needle is inserted immediately under the skin where
The pathway by which a drug is introduced into an
a solution is expelled. Intramuscular (IM) injections
organism is called route of administration. The more
are accomplished by inserting the needle deep into a
commonly used routes can be categorized as either
muscle mass and expelling a solution or suspension.
oral or parenteral (any route outside the alimentary
Intraperitoneal (IP) administration is perhaps the
tract). The oral route is often used in infrahuman
most commonly used route in rats. The needle is in¬
research because it readily allows the administration
serted directly into the peritoneal cavity, providing
of insoluble or irritating compounds and because it
rapid drug absorption. Intravenous (IV) administra¬
facilitates comparison with human studies, which
tion is used when immediate action and maximal cer¬
typically use the oral route.
tainty of dosage is required. The tip and shaft of the
With parenteral administration, the drug is in¬
needle are inserted into the lumen of a vein, and the
jected directly into the desired site. Although there
drug is directly expelled into the vein. Generally only
are other routes for parenteral administration, the
aqueous solutions, which will not damage blood and
ones most widely used in infrahuman research are sub¬
its constituents or produce local vascular irritation
cutaneous, intramuscular, intraperitoneal, and intra¬
may be used.
venous. Since none of the drug can be lost by vomit¬
ing or destroyed by gastrointestinal fluids, the dosage
is more certain in parenteral administration than in
The Dose-Response Relationship
oral. In addition, the rate of absorption is usually
more rapid. According to a common notion, every drug has a
“just right” dose which is standard, appropriate, and
* This section is based largely on Chapter 2 of Thompson
and Schuster, 1968. physiologically active. This notion may have some
546 OPERANT BEHAVIORAL PHARMACOLOGY

justification where a specific effect of a drug is desired.

For example, when a physician must treat a patient
for an acute bacterial infection, he will probably ad¬
minister a “standard” dose of an antibacterial drug.
On the basis of extensive experience he has probably
selected this dose as being large enough to have a
therapeutic effect on most patients and small enough
to avoid undesirable side effects. Even in this re¬
stricted situation, however, the notion of the “just
right” dose may be inappropriate where the dose hap¬
pens to be too low for a particular infection or too
large for a particular patient who is sensitive to the
toxic side effects.
In a more general sense, the concept of the “just
right” dose is quite misleading. It overlooks the fact
0 _L J
that drugs, like most variables, can be applied at
c 2 3 4 6
different levels under varying conditions to produce
different effects. At the extreme ends of the dose DOSE IN MG
range, every drug has a dose which is so low that it is
Fig. 1. The effects of pentobarbital on rate of responding on
ineffective and one which is so high that it is lethal. FR 30. The dose is plotted on a log scale while the rate is
Between these two extremes are dose levels which are plotted on a linear scale. The dose-effect curves of two indi¬
generally appropriate for a pharmacological study. vidual pigeons (B-2 and B-10) are shown. The points above “S”
show the rate after saline injections (mean of two observations).
An observation fundamental to all pharmacology is The points above “C” show the non-injection control rates
the quantity of drug administered is related in an (mean of six to eight observations). The points for each dosage
orderly way to the magnitude of the effect produced. are the mean of two observations. The dosage is given in terms
of number of mg injected. Since the birds weighed slightly more
The relation between the dose and the magnitude of than 400 g, the dosage in mg/kg can be readily calculated (From
effect is called the dose-response (or dose-effect) rela¬ Waller & Morse, 1963. (c) 1963 by the Society for the Experimen¬
tionship. An example is shown in Figure 1. The data tal Analysis of Behavior, Inc.)

are from an experiment by Waller and Morse (1963)

and show how pentobarbital affects two pigeons’ key- high dosages. The selection of one particular dose
pecking rates. The pecking response was maintained does not permit a valid statement of what the drug
by reinforcement on a fixed-ratio (FR 30) schedule does. To return to the example of Figure 1, if one
(i.e., every thirtieth peck produced grain). The dose- selected 3 mg of pentobarbital to study, one would be
effect curves for both birds show an orderly increase convinced that the drug was a “stimulant” which
in the response rate as a result of the intramuscular increased response output. If one chose 6 mg, one
injection of 2 and 3 mg (total dose per pigeon). Three would be equally sure that the drug was a “depres¬
mg seemed to produce the maximum rate while the sant” which decreased response output. If one picked
largest dose (5.6 mg) substantially decreased the re¬ .01 mg, the drug would be classed inactive, and if one
sponse output. The dose-effect curve, taken as a whole, picked 100 mg, it would appear to be a deadly toxin.
shows how pentobarbital over an effective dosage Even if two dosage levels were studied, the con¬
range quantitatively affects the FR 30 response rate. clusion might be misleading. For example, one might
Why is it important to determine a dose-response choose two dosages, one on each side of a maximal
curve for a drug? Perhaps the major reason is that level, and then find that they had almost the same
drugs often have different effects at different doses. effect. A logical but erroneous conclusion might be
Therefore, full knowledge of a drug’s effects can be that increased dosage levels of this drug do not cause
attained only if a full dosage range is studied. In the a greater effect and that the dose-response curve is
data shown in Figure 1, pentobarbital both increased relatively flat. Furthermore, it would be easy to select
and decreased the response rate—depending on the two doses from the left or the right side of the max¬
dose. As a general rule, any drug which will increase imum. Thus, one might conclude that larger doses
behavioral output at some intermediate dose will either increase the behavioral output or decrease it.
surely decrease output at some higher dose. The de¬ One can guard against such conditions only by study¬
crease will occur, if for no other reason, because a ing a number of doses distributed over the effective
toxic effect can always be produced by excessively dosage range of the drug.
Travis Thompson and John J. Boren 547

As another example of confirmation by other doses, 2 mg/kg and less potent at 8 and 16 mg/kg. In such
suppose that the results from a low dose suggested cases expression of potency in terms of the median
ambiguously that the subject’s ability to discriminate effective dose (ED50) of the two drugs is arbitrary and
between a steady light and a flashing light had de¬ misleading. Statistical or computational devices do
teriorated slightly. To clarify this matter, the investi¬ not help since the ambiguity is inherent in the data.
gator might consider two alternative procedures. One The most convenient solution is to end the search for
is to study the low dose again to see if the small effect a single number which relates the potency of each
on the discrimination could be observed. If so, he drug and simply to recognize the characteristics of
would have greater confidence in the effect. In all the two dose-response curves.
probability, however, the effect will be as ambiguous Note the likelihood of error if Drugs A and C were
as before (within the range of extreme control effects), to be compared at a single dose of each instead of the
so that a definite conclusion may still be difficult. A full dose range. If 1 mg/kg were used, one would defi¬
second alternative is to increase the dose (as one nitely conclude that Drug G was more potent; if 16
would normally do in determining a dose-response mg/kg were used, one would conclude equally defi¬
relationship) to determine whether the effect on the nitely that Drug A was more potent. To gain com¬
discrimination is increased. If the effect now emerges plete and accurate information about a drug there 16
as large and clear, the investigator’s confidence in the no substitute for a study which determines a full dose-
effect increases a great deal. This principle of experi¬ response curve.
mental design is not limited to drug-behavior experi¬
ments, In the study of any independent variable
where the effects are small, it is often more efficient to Absapptidn emd Distribution

enlarge the effect by intensifying the variable than

The amount of drug at the site of action determines
simply by replicating the small effect.
the effect produced. Once a drug has been introduced
A further use of the dose-response curve is in the
into an organism, it is absorbed and distributed to
quantitative comparison of two or more drugs. The
many parts of the body, including the site of action.
pharmacologist will frequently want to know which of
The amount of drug reaching the site of action is pri¬
two drugs is the more potent, or synonymously, which
marily dependent on the amount administered, its
is the more active. In other words, he wants to know
physical state, the character of the membranes the
which drug produces a given effect at the smaller
drug must pass, and the route that it must take to get
dosage. The first step is to determine a dose response
from the site of administration to the site of action.
curve for each drug. Figure 2 illustrates several pos¬
The time taken to get from the site of administration
sible outcomes (assumed values) for Drugs A, B, and
to the site of action is largely determined by the rate
G. The figure might, for example, represent the de¬
of absorption.
creases in the rate of avoidance responding produced
The absorption rate, in turn, is primarily deter¬
by three depressant drugs. Comparison of Drugs A
mined by the route of administration, the physical
and B is easy. Drug A is clearly more patent than
properties of the drug preparation, and the rate of
Drug B in the sense that equivalent effects are pro¬
duced at lower doses by Drug A. The comparison is
easy partly because the effective dosage ranges over¬
lap very little but largely because the dose-effect
curves are parallel. The parallel feature permits one
to reach the same conclusion about potency regardless
of the size of the effect. In Figure 2, Drug A is about
eight times more potent than Drug B, regardless of
whether the comparison is based on a 50 percent
effect, a 25 percent effect, etc. Furthermore, because
of the parallel curves, it is possible to calculate a
single value for each drug which represents its po¬
tency.
The comparison of Drugs A and C is considerably
more difficult. The dose-response curves are not paral¬
lel, and the effective dose ranges overlap. To be more
Fig. 2. Three possible dose-effect curves illustrating the compari¬
specific. Drug C appears to be more potent at 1 and son of potency.
548 OPERANT BEHAVIORAL PHARMACOLOGY

administration. These are the manipulable factors will last. For this sort of information concerning the
that determine how rapidly a drug reaches the site of drug’s time course, the basic source is generally ex¬
action. The absolute absorption rate can be expressed perimental and determined individually with each
in terms of the change in concentration of the drug at behavioral preparation.
the site of application over time. Figure 3 illustrates Every drug has its own time course. As shown in
how theoretical absorption and excretion curves pro¬ Figure 4, there is a delay in onset of action^ (with i.p.
duce the concentration curve of the amount of drug and i.m. routes) an increasing effect up to a peak, and
at the site of action. This theoretical curve is modi¬ finally a decreasing effect until the predrug state is
fied by a set of variables, not all of which are readily again reached, where the drug has been matabolized
controllable. The absolute rate of drug absorption or excreted. To illustrate such a relation, consider the
from the site of administration can be considered a study by Grove and Thompson (1970) dealing with
physicochemical relation between the drug and the the effects of pentobarbital on food-reinforced FR 120
transporting medium. The transporting medium schedule performance by rats. The top frame in Figure
(blood) is the primary factor regulating the absolute 5 shows baseline FR 120 performance on the left side,
absorption rate. followed by the effects of a saline injection on the right
Drugs tend to move from sites of high concentra¬ side. A pause occurred, even after saline, as indicated
tion to areas of lower drug concentration. However, at C on the cumulative record. The three sets of
movement of a drug from the site of administration cumulative records below are labelled 5, 10, and 20,
along a concentration gradient very seldom limits ab¬ corresponding to the dosage in milligrams/kilogram.
sorption rate. Most frequently, the amount of blood In general, pentobarbital suppressed the overall re¬
flowing through tissue determines how rapidly a drug sponse rate by successively increasing pausing. At 5
will be absorbed from surrounding tissue. Therefore, mg/kg the drug increased pausing immediately after
anything that modifies circulation—exercise, tempera¬ reinforcement; in addition, a series of pauses and bursts
ture, presence of other drugs—also alters rate of ab¬ of responding occurred in later ratios (e.g., at E). At 10
sorption. In intravenous administration, of course, mg/kg a pause in responding of 45 min duration
absorption is not a limiting factor, and blood levels
of a drug reach their maximal concentration immedi¬
ately. Figure 4 presents the comparative durations of
action curves for intramuscular, intravenous, sub¬
cutaneous, and oral routes of administration, in which
serum concentrations of penicillin were determined
for various periods following administration. Clearly,
intramuscular administration most closely approx¬
imates intravenous administration, while oral admin¬
istration and subcutaneous routes, though very slow,
maintain serum drug levels for a longer period. Al¬
though intraperitoneal administration is associated
with a duration curve similar to that obtained with
the intramuscular route, it provides a slower but
longer lasting peak drug concentration.

Time Course of Drug Effects

Many behavioral variables can be applied and re¬

moved almost instantaneously. A light can be turned
on and off; a shock can be delivered and removed.
Such events are public and easily observed so the
Fig. 3. Theoretical absorption and excretion curves, yielding a
experimenter knows when the variable is present and curve of concentration at the site of action. Although sym¬
at what intensity. The situation is not as simple with metrical curves of this type are theoretically possible, almost
all decay curves are hyperbolic and bear little resemblance to
a drug. Although the experimenter knows he has in¬
the absorption curves. Similarly, the resultant curve of the
jected an animal with a drug, he does not know in actual amount reaching the site of action is never symmetrical
advance when the drug will take effect, the drug con¬ and is modified by numerous factors, as discussed in the text
(From Marsh, D. F., Outline of Fundamental Pharmacology,
centration at the site of action, or how long the effect 1950. Courtesy of Charles C Thomas, Publisher, Springfield, Ill.)
Travis Thompson and John J. Boren 5 49

occurred following the first reinforcement. Then about

50 responses were completed at an intermediate rate of
about 30 responses per min. Responding was almost 5
D
completely suppressed by the 20 mg/kg dose with slight cr
LxJ
ay
responding resuming toward the end of the session. As
UL
can be seen from the foregoing example, the duration of O
action of a drug can be indicated by a period of dis¬ 8
ruption or alteration of the ongoing baseline perfor¬ ££
UJ
mance. In this case the length of disruption and the CL
dosage of the drug were closely correlated.

Distribution
£
When the drug concentration in the blood equals U_
O
the concentration at the. site of administration, ab¬
30
sorption is said to be complete. This does not, how¬ a.
ever, imply that the drug has been equally distributed
to all tissues of the body. The factors determining
HOURS ROST ADMINISTRATION
differential distribution are poorly understood, but
some variables are known to be important. Drug
molecules vary greatly in size, from methanol, with a Fig. 4. Intravenous (IV), intramuscular (IM), oral (PO) and
subcutaneous (5C) routes of administration and serum concen¬
molecular weight of 32, to some of the biological
trations of penicillin. Three milligrams of penicillin G per
macromolecules with molecular weights of up to kilogram of body weight were administered at various times to

4 X 107 (Bernal, 1958). Obviously such variability in one individual, and the amounts of penicillin activity in the
serum determined at time intervals. It is readily apparent that
molecular size is reflected in differential rates of dis¬
certain routes of administration greatly influence uptake and
tribution. also elimination. Much drug is wasted by some routes, and

The solubility properties of the drug comprise an¬ more frequent administration of the drug is necessary if effective
blood levels are to he maintained (From Marsh. D. F., Outline
other factor known to alter distribution. For example,
of Fundamental Pharmacology > 1950. Courtesy of Gharles G
thiobarbiturates are very fat soluble, and therefore Thomas, Publisher, Springfield, Ill.)

Fig. 5. Cumulative records illus¬

trating the effects of saline, 5.0,
10.0, and 20.0 mg/kg ip of pen¬
tobarbital on food-reinforced
FR 120 schedule performance.
The primary effect of pento¬
barbital on FR performance was
to exaggerate pausing, with
minimal effects on running rate.
(From Grove & Thompson,
1970.)
550 OPERANT BEHAVIORAL PHARMACOLOGY

tend to be rapidly distributed in adipose tissue. Other are desirable, the investigator would much prefer
compounds tend to have affinities for proteins of minimal interaction between successive administra¬
blood plasma, not on the basis of their solubility but tions of the same drug. At times, however, it is found
because of their protein-binding properties. Finally, that the dose required to produce the same effect
distribution to tissues depends on the presence and must be increased on successive administrations. Or,
concentrations of the same or similar drugs in those when the same dose is repeated, the effect becomes
tissues. Addition of the same drug or of its antag¬ smaller with each dose. When this occurs, it is said
onists may lead to no increase in concentration in that tolerance has developed. Multiple administra¬
given tissues if receptor sites for that drug are already tions of the narcotic analgesics, barbiturates, and
saturated. amphetamines are particularly likely to lead to the
development of tolerance. Certain other chemically
Fate related drugs can, when administered in place of the
original drug, produce a very similar response; at
Following absorption, a drug may undergo trans¬ times, they can substitute for the original drug. Usu¬
formation in the body and be ultimately excreted ally, as tolerance develops to the original drug, tol¬
either unchanged or as a biotransformation product. erance also develops to substitute drugs so that suc¬
The biotransformations that a drug undergoes and cessively higher doses of such drugs are also required
the mechanism of its excretion are referred to as the to produce the original effect. Under these conditions,
fate of a drug. Figure 3 presented the theoretical ex¬ it is said that cross tolerance has developed.
cretion curve, revealing the assumption that the mech¬ If discontinuing a drug which has been adminis¬
anisms of drug excretion are diametrically opposite to tered repeatedly and regularly precipitates a charac¬
those of absorption. As a matter of fact, the routes of teristic syndrome of illness (often including vomiting,
excretion are very seldom the simple inverse of those diarrhea, convulsions, and even death), the animal has
of absorption. It is worthwhile to consider briefly the become physically dependent on the drug. If an an¬
most common routes of excretion; the kidney, the imal reliably self-administers the drug when provided
lungs, the skin, the bile duct, and the intestines. with the opportunity, the term behavioral dependence
Volatile agents, such as the anesthetic gases and applies. Thus, an organism that is physically depen¬
alcohol, are excreted across the pulmonary membrane. dent may be behaviorally dependent as well, though
We are all well aware that sodium chloride is ex¬ the converse is not necessarily true. Humans who ex¬
creted in part across the skin; however, few other hibit behavioral dependence without physical depen¬
compounds of significance are found on the skin sur¬ dence are said to be habituated to the drug.
face. Organic arsenicals are among the drugs ex¬ Another problem arises when a drug is readmin¬
creted across the bile duct, and certain agents like istered before the effects of the previous dose have
quinine, as well as some sterols, are excreted in the disappeared. When a drug has not been entirely ex¬
feces. The vast majority of drugs are excreted by the creted or has not undergone complete transformation
kidneys. Because of the central role of the kidneys in before a second dose is administered, cumulation re¬
removing drugs from the body, proper functioning of sults. Such factors as the presence of the necessary
these organs is of extreme importance in drug re¬ enzymes to carry out the transformation reaction, the
search. Renal damage may increase a drug’s duration normal functioning of the excretory mechanism (e.g.,
of action; it may even have lethal consequences at a excretion by the kidney), or storage can affect the like¬
dosage that would otherwise be well within a toler¬ lihood of cumulation. Under these conditions, the
able range. concentration of the drug in various tissues and fluid
Some drugs, such as the inhalant anesthetics, are compartments progressively increases. In general, if a
excreted from the body in unchanged forms. Most drug is administered repeatedly, a portion is trans¬
drugs, however, undergo some chemical changes prior formed and excreted, but a certain amount remains.
to excretion. The transformation of a drug with a Cumulation rate depends on the interval between
specific biological action to an inactive form, or to a administrations and the dose. A typical cumulative
form with different effects, is called biotransformation. effect is illustrated in Figure 6, where data from five
administrations of the same dose of a drug are pre¬
Multiple Administrations of the Same Drug sented. While partial recovery occurs following each
dosing, the level following the last administration is
In behavioral pharmacology research, where sev¬ well above that seen on the first injection.
eral replications of a procedure on the same animal Obviously, a major consideration in gauging cu-
Travis Thompson and John J. Boren 551

plied and removed and repeated. This requires a high

degree of reproducibility of the behavioral baseline.
There are basically two approaches to the problem of
the reliability of the baseline. One is the use of the
ABA design in which the manipulation is repeatedly
applied to the same baseline. It is sometimes
difficult to re-establish the baseline once it has
been shifted, such as under conditions of transition
states (Sidman, 1960). In this case, another alternative
is commonly used, involving multiple baselines. Two
or more schedules may be conditioned under distinc¬
tive stimulus conditions which may be presented
sequentially (such as in chains or multiple schedules),
Fig. 6. The cumulation of the blood concentration of atro- or concurrently (as in concurrent schedules or non-
lactamide with time when the drug was readministered to a
reversible options). For example, during one session a
dog at 24-hour intervals. Note that on the fourth administra¬
tion an essentially steady state has been achieved (From Histor¬ given performance can be evaluated by presenting the
ical background and general principles of drug action, by J. A. discriminative stimulus for that behavior (e.g., FR 40
Wells. In V. A. Drill (Ed.), Pharmacology in Medicine. © 1958
on the right lever), while on another session a similar
by McGraw-Hill Book Company. Used with permission.)
performance (FR 40 on the left lever) can be exam¬
ined in the presence of a different stimulus to deter¬
mulative effects is interadministration interval. By mine whether the measured effect can be replicated.
spacing successive administrations sufficiently far Such schedules may be presented repeatedly with high
apart, it is usually possible to avoid cumulation. How¬ reliability of performance during each schedule com¬
ever, the disappearance of the active compound or its ponent. These procedures are not without their diffi¬
metabolites does not necessarily indicate that a drug culties, because there may be interactions among com¬
effect may still not exist. For example, the drug may ponents.
cause morphological or biochemical changes in cells Assuming an effect of a drug is measured at a given
which may far outlast the presence of the active agent, dose of compound on a given behavioral baseline, the
fn these cases, cumulative effects are not defined by the question then arises “Is the effect unique to that dose,
presence or absence of the drug but by the changes in or does it also occur at other doses?” Hence, as in any
the measured drug effect. other area of pharmacology, it is necessary to admin¬
ister at least three doses of the drug in question,
typically using a logarithmic dosage regimen. A paral¬
ANALYZING BEHAVIORAL MECHANISMS lel question is: “Is the observed effect specific to the
OF DRUG ACTION schedule value chosen?” Behavior maintained by rein¬
forcement schedules which generate high response
rates may be affected differently by a given drug, than
Specificity of Drug Action
schedules which generate low rates (cf. Dews, 1955a).
A fundamental problem in analyzing the ways in Hence, it may be necessary to evaluate the effects of a
which drugs alter behavior is to determine the degree drug at more than one schedule value. For example,
to which effects observed are specific to the drug and Thompson, Trombley, Luke, and Lott (1970) studied
to the specific set of conditions investigated. In the effects of morphine on rats responding for food on
following pages we will discuss some of the minimal FR 10, FR 20, and FR 40 schedules (Figure 7). Had a
conditions that must be satisfied in order to determine single schedule been studied (e.g., FR 10), it might
the specificity of drug action. have been concluded that in the dosage range in¬
As indicated earlier, typically a small sample size vestigated (1 to 6 mg/kg IP) morphine generates a
will be used in analyzing the actions of drugs in op¬ rather flat dose-response curve. However, when a dif¬
erant behavioral pharmacology. The emphasis in these ferent schedule value is investigated (FR 40) a clearly
investigations is the reversibility of drug effects on the inverse curvilinear function emerges. Hence, it is es¬
established behavioral baseline. In general, within- sential to study both a range of drug dosages and
subject control procedures are used (that is, A B A a range of schedule parameter values to obtain a
designs will be employed) in which a given variable, complete picture of drug action.
such as a drug dose or a schedule value, will be ap¬ Another useful control procedure involves experi-
 552
OPERANT BEHAVIORAL PHARMACOLOGY

Finally, it may be that drugs affect two behaviors

differently because their baseline rates differ. For ex¬
ample, Dews (1958) has shown that amphetamines
affect schedule-controlled performances differently, not
so much because the reinforcement schedules differ
per se, but because different schedules generate differ¬
ent patterns of inter-response times. In one study such
a necessary control was accomplished by manipulating
the baseline rate during one component of concurrent
operants. Cherek and Thompson (1973) studied con¬
current key pecking reinforced by access to food and
by access to a target bird which could be attacked. In
an initial study it was found that A9 THC, the active
ingredient in marijuana, had a more marked effect on
the key pecking maintained by access to an attackable
object. However, since the baseline rate of the two
operants differed, it was necessary to complete a final
manipulation in which the rate of pecking maintained
by food was driven down by adding a DRL contin¬
gency to the FI food reinforced performance. Even
when the rates were equated, A9 THC had a much
MORPHINE DOSE IN MG/KG (i.p.) more marked effect on the operant reinforced by
access to an attackable object (Figure 8).
Fig. 7. Mean overall response rates on FR 10, FR 20, and FR 40
schedules treated with saline, 1.0, 3.0, and 6.0 mg/kg of mor¬
phine sulfate ip. Each mean is based on three values, and
ranges are indicated by vertical lines. (From Thompson et al,
1970.)

mentally varying the rates of responding, while hold¬

ing the reinforcement density constant. For example,
subjects might be conditioned on a fixed-ratio sched¬
ule and the mean interreinforcement time is deter¬
mined. Then the same animals would be conditioned
on a multiple schedule including a variable-interval
component having the same overall reinforcement
density but which maintains a lower rate of respond¬
ing (i.e., Mult FR VI). Under such conditions, differ¬
ences might be presumed to be due to some property of
the performance, such as rates of responding, rather
than reinforcement density.
Yet another kind of control procedure may be nec¬
essary to evaluate whether the drug effects are specific
to the particular behavioral consequence employed.
For example, behavior maintained by food, water, or
sexual reinforcement may not be affected in the same
way by all drugs. Recent research on behavior main¬
tained by electric shock presentation suggests that, at
times, the nature of the consequence is less important
than schedule considerations (Kelleher 8c Morse, 1968;
see also chapter 7 in this volume). However, other
Fig. 8. Effects of tetrahydrocannabinol on key pecking main¬
research dealing with drug-maintained responding tained by access to food (solid) and a target bird (dashed) in
suggests that the generalization cannot be applied to three pigeons. Baseline rates of key-pecking maintained by the
two reinforcers were equated prior to drug administration (From
all reinforcers (Thompson 8c Pickens, 1972). Cherek & Thompson, 1973.)
Travis Thompson and John J. Boren 553

Behavioral Mechanisms By Which Drugs May Act crimination was learned with errors. Birds learning
the discrimination in the errorless fashion exhibited
Arriving at an understanding of the mechanisms
no errors whatever in a dosage range from 1.0 to 17.0
by which drugs modify behavior is as complex as the
mg of the two drugs.
permutations and combinations of variables which
A second class of antecedent variables involves in¬
interact at any moment in time to engender a par¬
teractions of drug effects with deprivation states.
ticular performance. An approach to analyzing this
Singh and Manocha (1966) studied interactions of
quagmire of variables has been suggested elsewhere
deprivation conditions and past reinforcement history
(Thompson and Schuster, 1968). The rate, pattern,
in determining the effects of chlorpromazine. They
and form of current operant behavior are determined
found the effect of a given dose of chlorpromazine de¬
by certain antecedent factors, the current stimulus
pended upon both the degree of deprivation and the
conditions, and the maintaining consequences of be¬
amount of reinforcement history. That is, extensive
havior. Drugs, as independent variables, interact with
past histories and high deprivation levels attenuated
any or all of these classes of factors to determine the
the effects of chlorpromazine. Deprivation states can
particular behavioral outcome. A better grasp of the
be relevant wherein drugs serve as maintaining con¬
meaning of a particular finding in behavioral pharma¬
sequences as well. Meisch and Thompson (1973)
cology can be had if one asks, “With which of these
studied the effects of food deprivation on lever press¬
factors that regulate behavior has the drug inter¬ ing reinforced by ethanol in rats, Figure 9 shows the
acted?” Has the drug altered the deprivation state (an
effect of increasing food deprivation levels on the dis¬
antecedent variable), stimulus control (a current stim=
position to respond for ethanol. In general, high rates
ulus variable), response topography (a property of the
of ethanol-reinforced responding were maintained as
con¬
response), or the reinforcer or schedule control (a
a function of increasing food deprivation, which also
sequent variable)? In the succeeding pages a number varied ft function of FR value. Woods, Downs, and
of studies are discussed within the foregoing frame¬
Villarreal (1973) studied two methods of drug depriva¬
work. This is intended not to provide an exhaustive
tion: Withdrawal of the drug and administration of
literature review, but rather an illustrated outline of
a drug antagonist. The subjects were physically de¬
research dealing with a particular class of variables.
pendent rhesus monkeys who could self-ad minister a

narcotic by pressing one lever and receive food by

Antecedent Variables
pressing a second lever. Both methods of inducing
Among the more important antecedent variables opiate withdrawal evoked nearly identical behavioral
which an organism brings to an experimental situa¬ changes on food-reinforced responding, While food-
tion are its past history, and its deprivational state reinforced behavior was disrupted by both drug
established by manipulations prior to the experiment. deprivation procedures, responding on the drug lever
Terrace (1963a) demonstrated that pigeons were was increased by both methods of induction, Thu§j
capable of acquiring a discrimination of color and the deprivation variables as antecedent procedures can be
orientation of a line in an “errorless” fashion. An powerful in determining the actions of the drug.
“error” was defined as the failure to respond to a stim¬ As discussed earlier, on repeated administration of
ulus correlated with reinforcement (an SD) or a re¬ certain drugs, tolerance develops—that is, a higher
sponse to a stimulus correlated with non-reinforcement dose of the drug is required to produce the same
(an SA). Errorless learning was established by starting effect. Tetrahydrocannabinol (THG) is one such com¬
discrimination training immediately after the response pound. McMillan and co-workers (1970, 1971) have
to SD had been conditioned, and by progressively re¬ studied the development of behavioral tolerance to
ducing the difference between the SD and the SA from THC in pigeons. The degree of past history of THC
an initial large difference to a relatively small, final treatment can profoundly effect the degree to which
difference. In a subsequent study Terrace (1963b) the drug suppresses ratio-reinforced responding. Al¬
showed that pigeons that had learned a discrimination though a dose of 1.8 mg/kg of A9 THC may be suffi¬
without errors, which appeared superficially the same cient to totally suppress responding during the first
as that established by the more typical method (in¬ 3-5 days of administration, after 30 days of adminis¬
volving the occurrence of many errors), responded in tration of gradually increasing doses, pigeons’ re¬
a dramatically different way to imipramine and chlor- sponse rates following administration of 10 mg/kg
promazine. Chlorpromazine and imipramine disrupted may be close to normal control values.
the pigeons’ performances on a discrimination be¬ A final class of antecedent variables involves the
tween vertical and horizontal lines only if the dis¬ superimposition of a history of classical conditioning
 554
OPERANT BEHAVIORAL PHARMACOLOGY

dependent monkeys were first trained to press a lever

for food reinforcement on an FR 10 schedule. A tone,
initially neutral, was aperiodically presented five min¬
utes before the intravenous injection of nalorphine.
Nalorphine, because it is a morphine antagonist, can
suddenly produce withdrawal symptoms. After several
sessions, conditioned suppression of the food rein¬
forced operant was observed during tone presenta¬
tion prior to the administration of nalorphine. In a
later study, Goldberg, Woods, and Schuster (1969) dem¬
onstrated increases in responding for morphine in¬
jections in physically dependent monkeys when a
stimulus pair with nalorphine administration was pre¬
sented. Using rhesus monkeys formerly dependent on
morphine, Goldberg and Schuster (1969) demonstrated
an increased sensitivity to nalorphine’s operant sup¬
pressing effect as compared with control monkeys hav¬
ing no prior history of morphine exposure. Within
the dosage range employed, nalorphine injections pro¬
duced hypersensitivity in formerly dependent monkeys
but not in controls. Such effects were observed to
occur for as long as 60-120 days of complete abstinence
from morphine, long after any possible residual effects
of the drug could have persisted, and after physical
dependence had disappeared.

Stimulus Variables

Environmental factors altering the stimulus control

Fig. 9. Effect of FR size on responses per 6-hr session. Ordinate: of operant behavior have been extensively studied and
responses plotted on a linear scale. Abscissa: FR size plotted on are described elsewhere in the present volume. The
a logarithmic scale. Different scales were used on the ordinate.
precise manner and the degree to which various classes
Open triangles: mean ethanol responses during food depriva¬
tion (n - 2); filled triangles: mean ethanol responses during food of drugs alter or participate in stimulus control is a
satiation (n-3). Open circles: mean water responses during matter of some conjecture. That the drugs are capable
food deprivation (n = 4); filled circles: mean water responses
of altering stimulus control is widely known. For
during food satiation (n = 6). The height of the vertical lines
indicates the range; absence of a vertical line at a particular example, LSD alters a visual stimulus discrimination
point indicates that the range was within the area occupied by in pigeons in a dose-dependent fashion (Becker, Ap¬
the symbol. The results for FR 1 on the left were obtained after-
completing the sequence of increasing FR values. (From Meisch
pel, 8c Freedman, 1967). LSD is also known to effect
8c Thompson, 1973.) the shape of a stimulus generalization gradient in rats
(Dykstra and Appel, 1972). Further, the degree to
which stimulus control is altered by a drug varies
on operant baselines. The widely noted placebo effect
with the complexity of the discriminative stimulus
involves, at least in part, a conditioned effect due to
(Dews, 1955b). However, it is one thing to show that
environmental variables paired with drug administra¬
a drug produces a dose-dependent change in the de-
tion. Pickens and Crowder (1967) studied the effects of
giee of stimulus control and another to describe the
a history of amphetamine injections on locomotor
behavioral mechanism by which such an effect was
activity. After several pairings of the amphetamine in¬
brought about. One attempt at such an account was
jection with increased locomotor activity, the injec¬
provided by Dykstra and Appel (1972) in which it
tion of saline was capable of producing a similar
was shown that the shape of a stimulus generalization
increase in general activity. Goldberg and Schuster
gradient was changed after the administration of LSD.
(1967) studied conditioned suppression by a stimulus
The authors noted that a dose of LSD which produces
paired with nalorphine administration in rhesus mon¬
a change in the gradient did so only at relatively high
keys physically dependent upon morphine. Physically
rates of responding, suggesting that the change pro-
Travis Thompson and John J. Boren 555

duced by the drug was more a rate-dependent effect of On drug sessions animals pretreated with various be-
the drug than an effect specifically on stimulus con¬ haviorally active drugs are reinforced for turning one
trol. direction in the maze, and on vehicle control days for
Laties and Weiss (1966) have suggested that the turning the opposite direction. Certain CNS drugs,
degree to which behavior is affected by drugs de¬ including the barbiturates and minor tranquilizers,
pends to a considerable degree on how much the exercise strong stimulus control, while phenothiazines
baseline behavior is controlled by exteroceptive as op¬ such as chlorpromazine exercise rather weak stimulus
posed to interoceptive stimuli. They studied per¬ control. Rats readily learn the correct turn at the
formance of pigeons on fixed-interval schedules in choice point in a T-Maze when pretreated with a
the presence of five one-minute discriminative stimuli barbiturate, alcohol, or chlordiazepoxide, while re¬
(i.e., a clock condition). Their primary dependent quiring a relatively long period to acquire stimulus
variable was the distribution of responses over succes¬ control when pretreated with a phenothiazine deriva¬
sive minutes of a clock as a function of doses of am¬ tive. Taking this line of research one step further,
phetamine, scopolamine, pentobarbital, chlorproma- Barry and Kubena (1972) studied discriminative stim¬
zine, and promazine. All of the drugs used produced ulus characteristics of alcohol, marijuana, and atro¬
substantial changes in the FI response distribution pine in rats. In their procedure rats were reinforced
when the pigeons had no exteroceptive discriminative on a FR 5 schedule of food reinforcement for pressing
stimulus correlated with elapsed time. Providing the one lever, while if they pressed the other lever they
birds with an exteroceptive clock, however, modified received painful foot shock. Under ethyl alcohol, atro¬
the response distribution greatly and decreased sensi¬ pine, and A1 THC, rats rapidly learned to press the
tivity to amphetamine, scopolamine, and pentobarbi¬ food lever under the appropriate drug or control con¬

tal, although the sensitivity to chlorpromazine and dition and to avoid the lever which produced shocks,
promazine was largely unchanged. These findings sug¬ Tests for stimulus generalization were then conducted,
gest that the source of discriminative stimuli control¬ in which rats trained under one drug condition were
ling the performance is important in determining the tested with various doses of the same ol* other drugs.

reaction to drugs, and further that it is relatively more The experimental question was “how similar are the
important for some drugs than others. In another stimulus properties of drug X to the drug used in
study (Laties, 1972), pigeons were trained on a chained training?” Lower doses of the drug used in training
and tandem FR 8 FR 1 reinforcement schedule in generally produce intermediate percentages of correct
which eight pecks on one response key were followed responding. However, when other drugs were admin¬
by a single peck on the second key which produced istered, the results were more complex. Animals that
access to grain. If the bird switched keys before the had been trained under pentobarbital tended to re¬
count of eight, the series of responses had to be started spond correctly when tested under other depressants,
again. During one condition, no external stimulus such as chlordiazepoxide. Animals trained using atro¬
change occurred following the eighth response (i.e., pine as the discriminative stimulus tended to respond
tandem condition). During the other condition a stim¬ correctly when receiving doses of other anticholinergic
ulus change invariably occurred following the eighth drugs such as scopolomine. The A1 THC animals
response (i.e., chain condition). The addition of the responded correctly only when other marijuana ex¬
stimulus change made the subjects much more effi¬ tracts were administered. If alcohol trained animals
cient in meeting the required minimum count before were administered chlorpromazine or d-amphetamine
switching to the reinforcement key. That is, the under generalization test conditions, incorrect re¬
chained schedule condition generated more efficient sponding tended to occur. When A1 THC trained an¬
performance than the tandem schedule condition. Re¬ imals were administered various depressant, stimulant,
sponse rate, however, remained about the same. When or hallucinogenic drugs, incorrect or control respond¬
a discriminative stimulus was not present (i.e., the ing tended to occur.
tandem condition), chlorpromazine, d-amphetamine, One of the earliest studies using an operant tech¬
and scopolamine led to premature switching to the nique to analyze the discriminative stimulus proper¬
reinforcement key. The addition of the external dis¬ ties of drugs was conducted by Cook, Davidson, Davis,
criminative stimulus attenuated the effects of scopola¬ and Kelleher (1960). Dogs, surgically prepared with
mine and d-amphetamine most; chlorpromazine and intravenous catheters, were intravenously administered
promazine least. various doses of epinephrine, norepinephrine, or
Overton (1971) has studied the discriminative stim¬ acetylcholine prior to a painful shock to the dog’s
ulus properties of an array of drugs using a T-Maze. limb. The dog could avoid the shock by lifting his
556
OPERANT BEHAVIORAL PHARMACOLOGY

limb during the discriminative stimulus period pre¬ according to which they are presented appear to be
ceding the shock. Acetylcholine served as a highly fundamental determinants of behavior (Morse and
effective discriminative stimulus, while 1-epinephrine Kelleher, 1970). Hence, one would expect that the
served as a relatively weak discriminative stimulus, effects of drugs would depend critically upon how the
taking approximately twice as long to establish con¬ drugs interact with consequences and their schedules
trol to a criterion of 100% correct responding. of presentation. Dews and Morse (1961) and Kelleher
One of the more elegant studies of discriminative and Morse (1969) have argued the type of consequence
control of operant behavior by a drug was conducted is a relatively unimportant factor in determining the
by Schuster and Brady (1964). In that study rhesus behavioral actions of drugs, whereas the schedule ac¬
monkeys, also surgically prepared with an intravenous cording to which various types of consequences are
catheter, were infused with various doses of epineph¬ presented is of primary importance. There are several
rine in the presence of which lever pressing was lines of evidence supporting this thesis.
reinforced on a fixed-ratio schedule of food presenta¬ Laties and Weiss (1963) studied the effects of am¬
tion. When saline was injected, lever pressing had no phetamine, chlorpromazine, and pentobarbital on the
consequence. The independent variable was the dos¬ behavioral regulation of temperature. The rats, after
age of epinephrine administered, and the dependent being placed in a cold compartment, were trained to
variable was the percentage of epinephrine and saline warm themselves by pressing a lever that turned on a
control trials during which the subject met the re¬ heat lamp. Amphetamine, at a dose level that by itself
sponse requirements for reinforcement. Figure 10 did not increase the rate at which body temperature
shows the acquisition curves. By one animal’s 25tli fell in the cold, increased the frequency with which
session, epinephrine exercised substantial stimulus the rats turned on the lamp even though the skin
control over the animal’s behavior.
temperature was driven above normal. Chlorproma¬
Harris and Balster (1971) first trained rats on mul¬ zine, at a dose level that accelerated heat loss in the
tiple and mixed reinforcement schedules using ex¬
cold, decreased the frequency with which the lamp
teroceptive discriminative stimuli. Subsequently, the was turned on. Pentobarbital produced only a transient
same schedules were used in training with various
depression directly correlated with base level. Be¬
drugs as discriminative stimuli instead of exterocep¬
havioral thermoregulation was impaired by both am¬
tive stimuli. Their findings essentially corroborated
phetamine and chlorpromazine, the former by increas¬
those of Overton, in that depressants such as chlordi-
ing and the latter by decreasing the optimal frequency
azepoxide and ethyl alcohol were capable of establish¬
of bursts of heat.
ing strong stimulus control (in this case over multiple
Waller and Waller (1962) studied the effects of
schedule performance), whereas phenothiazine deriva¬
chlorpromazine on behavior maintained by food rein¬
tives exercised little or no stimulus control. Similarly,
forcement and shock avoidance in a multiple rein¬
hallucinogens such as psilocybin and LSD were very
forcement schedule. There was no evidence that
ineffective as discriminative stimuli.
chlorpromazine had a differential effect on avoidance

Consequence Variables
behavior or on food-reinforced behavior as a function
of the type of reinforcer. At low doses, rates of re¬
Type of consequence: Positive reinforcement. The sponding on the food reinforcement component in¬
types of consequences of behavior and the schedules creased slightly whereas rates on the avoidance com-

Fig. 10. Percentage of epine¬

phrine and saline control trials
in which the subject met the
response requirements for rein¬
forcement. (From Schuster &
Brady, 1964.)
 557
Travis Thompson and John J. Boren

ponent remained relatively unchanged. At higher (1955), and Beach (1957) were among the first to pre¬
doses both components showed an approximately sent experimental evidence that drugs could serve as
equal depression of responding. Kelleher and Morse reinforcers. These early studies set the stage for later
(1964) studied the effects of d-amphetamine sulfate experiments demonstrating more conclusively the rein¬
and chlorpromazine on rates of responding under forcing properties of an array of compounds. This
multiple FI FR food and shock-escape schedules. literature has been reviewed recently, and the array of
Three squirrel monkeys were studied at each multiple drugs that serves as reinforcing consequences is ap¬
schedule. Each drug was given IM immediately before proximately the same as those that are associated with
the beginning of the 2i/? hour session. Both amphe¬ drug dependence in man (Schuster and Thompson,
tamine and chlorpromazine had similar effects on 1969; Thompson and Pickens, 1969; 1970). Among the
behavior maintained on a given schedule regardless of drugs that have been shown to be self-administered
the type of consequence, i.e., whether it was shock are the narcotic analgesics, barbiturates, certain cen¬
avoidance or food reinforced. Similarly Cook and tral nervous system stimulants such as amphetamine,
Catania (1964) studied the effects of drugs on food- cocaine, and cannabis, and some hallucinogens.
reinforced and escape behaviors. They studied the The development which was of critical importance
performance of squirrel monkeys under FI 10-min in fostering research on drug reinforcement was de¬
schedules of food reinforcement or electric shock velopment of the technology necessary to permit
termination. In one group of food-deprived monkeys, chronic intravenous injections of drug solutions in

the first key-pressing response after 10 min led imme¬ unrestrained or partially restrained animals (Pickens

diately to food presentation. In the second group of 8c Thompson, 1975; Schuster & Thompson, 1969). The
monkeys, an intermittent electric shock of low in¬ intravenous route is especially important because it
tensity was continuously delivered to the grid floor of minimizes the delay of reinforcement between the oc¬
the experimental chamber. The first response after 10 currence of the operant and the onset of drug effect.
min terminated the electric shock. These schedules of The fact that drugs serve as reinforcers for infra¬
food presentation and shock termination are formally human subjects may be viewed by some to be an inter¬
similar, and both engendered patterns of responding esting, if somewhat curious, finding. However, the
characteristic of FI schedules. With both types of skeptical reader may ask why the reinforcing proper¬

reinforcers, chlordiazepoxide, meprobamate, imipra- ties of drugs are of any general interest to those in the
mine, and chlorpromazine had similar effects. All of field of operant conditioning. There are three rea¬
the foregoing data would seem to argue that the pre¬ sons, two practical and one theoretical. First, the basic
cise consequence of the behavior may be less impor¬ processes which are involved in drug reinforcement in
tant in determining the drug effect. infrahuman subjects may be functionally comparable
The foregoing remarks have to be modified under to those in humans who use and abuse drugs. If so, our
certain circumstances. For example, animals physically approach to the problems associated with human
dependent on morphine derivatives and treated with drug dependence changes dramatically. Nearly all
antagonist drugs such as nalorphine or naloxone re¬ drugs which are commonly abused by humans are self-
spond differently from animals that are not physically administered and serve as effective reinforcers for
dependent. Naloxone and nalorphine have entirely infrahuman subjects (Schuster and Thompson, 1969;
different effects on morphine-reinforced responding in Thompson and Pickens, 1969). This suggests that the
animals than on food-maintained responding. Simi¬ principles and knowledge concerning the control of
larly, Jacobs (1958) studied the effect of exogenous operant behavior by other reinforcers can now be
insulin on the choice between a 10% glucose solution brought to bear in the human situation, and so lead
and a 35% solution. The 10% solution was typically to a better understanding of the controlling variables
preferred by untreated rats. However, when insulin of drug-maintained behavior in man.
was administered prior to choice testing, the prefer¬ A second and related reason why drug reinforce¬
ence shifted to the 35% solution. That is, the more ment is an interesting phenomenon is the possibility
concentrated glucose solution was shown to be a more that the infrahuman drug self-administration labora¬
powerful reinforcer depending upon the insulin in¬ tory may serve as a testing ground for future abuse
jection. Under such circumstances the type of conse¬ potential of drugs introduced for human therapeutic
quence is of critical importance. purposes. As indicated above, thus far, all drugs which
Another class of reinforcers which has received par¬ have been tested in infrahuman laboratory and which
ticular attention in behavioral pharmacology has are actively self-administered by infrahuman subjects
been drugs themselves. Spragg (1940) and Masserman (e.g., rats and/or monkeys) are also known to be com¬
and Yum (1946), Headlee, Coppock, and Nichols monly abused by man. In the years to come, as new
558 OPERANT BEHAVIORAL PHARMACOLOGY

compounds are manufactured and about to be intro¬ what is the role of conditioned reinforcement in the
duced into the clinic, testing in infrahuman drug overall control of drug-maintained responding? Once
self-administration laboratories may prove to be use¬ answers can be provided for the foregoing questions,
ful in predicting which compounds have the highest it may be possible to begin to alter the degree to
abuse potential and should be subject to special regu¬ which drugs as reinforcers control behavior. In other
lation. words, solutions to problems of drug dependence may
A third reason for being interested in the drug depend, to a significant degree, upon an understand¬
reinforcement phenomena has to do with understand¬ ing of the basic mechanisms by which drug reinforce¬
ing basic mechanisms controlling behavior. Drugs as ment operates.
reinforcers have certain unusual, if not unique prop¬
erties, which permit them to be used to study rein¬ Punished Responding. Morse (1964) studied the
forcement phenomena difficult to study using other effects of amobarbital and chlorpromazine on pun¬
reinforcers. It is easy to obtain infrahuman subjects ished behavior of pigeons. Key pecking which was
with no previous experience of drug reinforcers, so maintained by a variable-interval schedule of food
that interpretation of effects is not complicated by an reinforcement was also punished by brief electric
unknown past history. In addition, infrahuman drug shocks. Under this simultaneous food reinforcement
self-administration provides us with a way of studying and shock punishment schedule, responding was de¬
mechanisms controlling behavior in the laboratory, pressed to a low and fairly uniform rate that was in¬
including biochemical and physiological mechanisms, versely related to punishment intensity. Morse found
as well as an array of environmental variables, includ¬ that amobarbital partially restored responding sup¬
ing reinforcement contingencies, stimulus control, and pressed by punishment, while chlorpromazine had no
other classes of variables discussed throughout the tendency to attenuate the suppressing effects of pun¬
present volume. ishment. Other investigators using a variety of species,
When research on drug self-administration by in¬ including the rat and monkey, have shown similar
frahuman subjects was first initiated, the main ques¬ effects—namely that barbiturates and minor tranquiliz¬
tions that were asked dealt with the types of drugs ers generally decrease ,the suppressing effects of punish¬
that served as reinforcers. It is now clear that it is the ment while amphetamines, morphine, chlorpromazine,
rule rather than the exception that drugs serve as and trifluoperazine usually do not attenuate the sup¬
powerful primary reinforcers for most infrahuman pressing effects of punishing shock (Geller and Seifter,
subjects. Now the questions that must be answered in¬ 1960, 1962; Kelleher and Morse, 1964, 1968; Morse,
clude the following: (1) Under what conditions do 1964; Wuttke and Kelleher, 1970). McMillan (1973) at¬
various classes of compounds serve as reinforcers and tempted to discover the mechanism by which various
gain the greatest control over behavior? (2) To what drugs attenuate the effects of punishment. The effects
extent do drug reinforcers have the same properties as of a variety of compounds on key pecking responses
other reinforcers? For example, do drugs lead to the punished by electric shock in a multiple FI 5 FI 5
establishment of schedule control in much the same punishment schedule were investigated using pigeons.
fashion as other reinforcers (Thompson and Pickens, Most of the drugs studied increased low rates of both
1969)? To what extent do manipulations of reinforce¬ punished and unpunished responses, while increasing
ment magnitude yield effects similar to the magnitude higher rates or decreasing them. However, low rates
of other reinforcers such as food, water, or brain stim¬ of punished responding were sometimes increased
ulation? Do histories of intermittent reinforcement or more by pentobarbital, diazepam, and clilordiazepox-
reinforcement with certain types of drugs affect the ide than were matched rates of unpunished respond¬
resistence to extinction? Similarly, do all drugs, con¬ ing. In contrast d-amphetamine and chlorpromazine
trolling for the number of reinforcements, generate usually increased low rates of unpunished responding
the same degree of resistance to extinction? What is more than matched rates of punished responding.
the role of deprivation conditions in determining the Thus, the effects of drugs on punished responding
control of behavior by various classes of drug reinforc¬ appeared to depend upon the control rate of punished
ers (e.g., those which produce physical dependence responding; however, the rate-dependent effect of
such as morphine, as contrasted with those which do drugs on punished responding is not always the same
not produce physical dependence such as cocaine)? To as for unpunished responding.
what degree are behaviors maintained by drug rein¬ A series of experiments conducted by Morse and
forcement subject to stimulus control in the same way Kelleher (1970) and McKearney (1972) have cast con¬
as behaviors maintained by other reinforcers? Finally, siderable doubt upon what has commonly been termed
Travis Thompson and John J. Boren 55 9

a “motivational” interpretation of the effects of drugs tributed to changes related to the reinforcer, but
on operant behavior. In these experiments, squirrel rather must be interpreted in terms of the reinforce¬
monkeys were trained on schedules of electric shock ment schedules.
presentation. Monkeys exposed to various reinforce¬ In another study, Dews (1958) demonstrated sched¬
ment histories (typically unsignaled shock avoidance, ule-dependent effects using methamphetamine. The
but sometimes shock elicitation or variable-interval number of responses made by pigeons in a fixed pe¬
food-reinforced responding) were exposed to response- riod of time was greatly increased by methampheta¬
contingent painful electric shock. Under these condi¬ mine when the birds were conditioned using FI
tions, after sufficient exposure to the schedule, 15-min and FR 900 schedules. However, the rates
lever-pressing performance stabilized and typical fixed- were only slightly increased when the birds were con¬
interval performance emerged. That is, a shock which ditioned using other schedules (VI 1-min and FR 50).
under other certain circumstances would serve as an The fact that the effect of drugs depends critically on
effective punisher appears to be maintaining behavior. the type of schedule and the schedule parameter is
Morse and Kelleher (1970) have explored the implica¬ now widely recognized. For a number of years it was

tions of schedules of shock presentation for a general thought that the schedule per sc was a fundamental

understanding of the concept of reinforcement and determinant of the behavioral actions of drugs* How¬
reinforcement schedules. It has generally been found ever, Dews (1958) suggested that perhaps the mecha¬
that drugs have effects on responding maintained by nism by which schedule-dependent drug effects were
schedules of electric shock presentation that are in¬ brought about concerned the number and length of

distinguishable from those on schedules of food-rein¬ intcr-rcsponsc times generated by a given schedule.

forced responding or water-maintained responding. That is, schedules which generate short inter-response
This striking result was not only unexpected but is times, such as fixed-ratio schedules, will lead to rate
obviously incompatible with any simple notion one decrements following amphetamine administration,
might have about “tranquilmng” or “anxiety reduc¬ while schedules which engender long intsr-respome

ing” effects of drugs as being the primary basis for times will tend to be associated with rate increases
determining their behavioral actions. One would as¬ following amphetamine administration. This notion,
sume that the motivational state associated with a which has come to be called the “rate dependency
schedule of shock presentation would not be at all hypothesis/7 has considerable support involving at!

like that under a schedule of food or water presenta¬ array of drugs* Kelleher and Morse (1969) hayc sum¬
tion. marized the findings pertaining to rate dependent
drug effects as follows*. “The net effect of ampheta¬
mines on the average rate of responding under a
Schedule Considerations
schedule can be analyzed in terms of effects of rates of
The last class of variables which will be considered responding in different temporal periods of the sched¬

in the present context concerns the schedules accord¬ ule. . . . Whether amphetamine increases or decreases
ing to which response consequences are presented. responding depends upon the pre-drug rate of re¬
Dews (1955a) studied differential sensitivity to pento¬ sponding as well as the dose. Evidence . . . indicates
barbital of key-pecking performance by pigeons using that pre-drug rates of one response or more per sec¬
fixed-interval and fixed-ratio reinforcement schedules. ond are only decreased by increasing doses of am¬
Dews found that a dosage of pentobarbital which had phetamine. Pre-drug rates of less than one response
a rate increasing effect on fixed-ratio 50 performance per second increased to a maximum and then de¬
produced a markedly rate reducing effect on FI per¬ creased after increasing doses of amphetamine” (Kelle¬
formance. Although Dews employed either an FI or her and Morse, 1969). Rate dependent drug effects
an FR schedule throughout an entire session, other have been reported for the amphetamines, barbitu¬
investigators have used multiple schedules, in which rates, minor tranquilizers such as meprobamate and
FI and FR components occurred randomly through¬ chlordiazepoxide and morphine (Kelleher, Fry, Dee-
out each session (Morse and Herrnstein, 1956). With gan, 8c Cook, 1961; Richelle, Xhenseval, Fontaine, 8c
a multiple schedule the same subject can be studied Thone, 1962; Smith, 1964; Thompson, et al., 1970).
under drug conditions in which responding is main¬ Some of the more convincing research done dealing
tained by the same reinforcer but based upon differ¬ with the rate dependency hypothesis involves detailed
ent reinforcement schedules within a single session. examination of performance within a given simple
Thus, differential effects of a given drug on these fixed-interval schedule. Smith (1964) studied the effects
alternating patterns of responding can hardly be at¬ of d-amphetamine (.01-10.0 mg/kg IM) on a FI 5-min
560 OPERANT BEHAVIORAL PHARMACOLOGY

performance by pigeons. Specifically, Smith studied creasing effect on responding in the presence of visual
the effects on response rate during the first and last stimulus in which key pecking had no programmed
minute of each FI 5-min component of the schedule. consequence.
He found that d-amphetamine markedly increased Thus, while the rate dependency hypothesis ap¬
the low rates of responding characteristic of the first pears to apply to a wide array of experimental find¬
minute of the schedule and decreased the high rates ings, there are certain situations in which it does not
of responding characteristic of the fifth minute of the hold. The fact that rate dependency does occur under
schedule. At a dose of 3 mg/kg the maximum overall a number of circumstances, does not necessarily in¬
rate increase was observed. The rate of responding in dicate the mechanism by which the rate dependency
the first minute was significantly increased and was phenomenon is engendered. For example, ampheta¬
significantly lower during the last minute. A dose of mine may attenuate or in some way alter the degree
10 mg/kg decreased overall response rates; however, of stimulus control (Dews, 1955b; Laties and Weiss,
this dose produced a greater increase in rate in the 1966). In a related study Hearst (1964) has examined
first minute than did the 3 mg/kg dose, but also pro¬ the effects of d-amphetamine on avoidance responses
duced a more marked decrement in rate during the in monkeys. Hearst found that amphetamine flattened
last minute. In short, the change in overall respond¬ the generalization gradient, once again suggesting an
ing produced by d-amphetamine was the net result of alteration in stimulus control. Hill (1970) has pre¬
its rate increasing effects early in the interval and its sented data suggesting that one of the mechanisms by
rate decreasing effect during the latter portion of the which amphetamines altered behavior is by changing
fixed interval. reinforcing properties of stimuli paired with uncon¬
Although the rate dependency hypothesis appears ditioned reinforcement. In these investigations Hill
to apply to a considerable range of schedule-controlled has presented evidence suggesting that amphetamine
phenomena, there are several noteworthy exceptions. increases the conditioned reinforcing properties of
Responding that is maintained at a relatively low such stimulus events.
rate under punishment conditions does not seem to
follow the rate dependent effect to the same degree as
responding which is not under the control of punish¬ TRADITIONAL PROBLEMS FORMULATED

ment. Responding that is punished and therefore WITHIN AN OPERANT FRAMEWORK

maintained at a low rate is often further decreased

by amphetamine (Geller and Seifter, 1960). The rate Acquisition and Extinction—Learning
dependency hypothesis would predict an increase in
these low response rates. Under conditions in which In the introduction to this chapter it was suggested
the responding has not previously occurred or has no that early research dealing with'drugs and behavior
programmed consequence, amphetamine may have was sometimes misguided. Investigations designed to
little tendency to enhance responding, Verhave (1958) deal with drug effects on such phenomena as learning,
studied bar pressing by untrained rats during 12 daily motivation (e.g., fear, anger, hunger, etc.), or percep¬
one-hour sessions in which responding had no pro¬ tion were often formulated so that no matter what the
grammed consequence. During the first session, all rats experimental outcome, it would be impossible to de¬
responded and the mean number of responses was termine the mechanisms involved. The development
15.7. Over successive sessions, the mean number of re¬ of behavioral pharmacology over the past decade has
sponses showed an orderly decrement, such that by the begun to make it more profitable to ask experimental
seventh session the mean response rate was only .8 questions pertaining to these very complex issues. For
responses per hour, and three of the six rats did not example, when one is concerned with the effects of
respond at all. Methamphetamine (2 mg/kg S.C.) was drugs on learning, as Dews has pointed out (1970),
administered before the eighth session. The rate de¬ one is interested in more than a trivial change of be¬
pendency hypothesis would predict increases in these havior of a student in a classroom, such as the differ¬
low response rates. However, the mean response rate ence between being awake and sleeping during a
remained at .8 responses per hour and four of the six learning task. Instead, operant behavioral pharmacol¬
subjects did not respond at all. Finally, Dews (1955b) ogy has focused attention on questions such as “In
studied the effects of methamphetamine on an an¬ what way do various drugs effect the transition from
imal which received extensive training on a simple one steady state to another?”
discrimination, and found that methamphetamine in As an example of an operant approach to the study
the dosage range .1 to 3 mg/kg IM had no rate in¬ of drug effects on “learning,” Stolerman (1971) studied
Travis Thompson and John J. Boren 561

the acquisition of lever pressing by rats on a contin¬ tions when they are substituted for water in a sched¬
uous reinforcement schedule under saline, chlorpro- ule-induced polydipsia procedure (Meisch 8c Thomp¬
mazine, and chlordiazepoxide. He found that while son, 1971). In Meisch’s experiment, sessions were run
both chlorpromazine and chlordiazepoxide reduced using a 4-hour polydipsia period preceded by a 2-hour
the rate of acquisition of the lever pressing response, period during which concurrent food reinforcement
there were significant differences both in the degree of was not available. Responding for water, or for
the reduction and the mechanism underlying the ethanol on experimental days, could occur at any time
differences. Heise and Lilie (1970) studied the effects during the 6-hour session. Little or no responding
of scopolamine, atropine, and d-amphetamine on occurred for water during the first 2-hour component,
elimination of responding on nonreinforced trials in a while high rates of water-reinforced responding were
discrete-trial situation. Under one set of conditions, an obtained during the subsequent 4-hour period. Fol¬
exteroceptive stimulus indicated periods when re¬ lowing establishment of this pattern of water-rein¬
sponses would go unreinforced (i.e., SA) while under forced responding, an 8% ethanol solution was pre¬
other conditions there was no external SA. Scopola¬ sented on days alternating with water control days.
mine impaired performance (that is, reduced the per¬ Using the pattern of water responding as a baseline,

centage of trial responses that were reinforced) to Meisch was able to show the development of ethanol
about the same extent when an external stimulus in¬ as a reinforcer by comparing the rate of ethanol re¬
dicated a non-reinforcement as when the stimulus was sponding during the first 2-hour component with the
absent. D-amphetamme, on the other hand, impaired rate of water responding. Using this procedure it was
performance only when there was no exteroceptive $A, possible to establish ethanol as an effective remforcer
Barry and Kubena (1971) studied the effects of THC with five exposures to the drug solution.
on acquisition of shock avoidance by rats. They found Griffiths and Thompson (1973, 1974) studied the
that acquisition of an avoidance response was im¬ effects of pentobarbital on the elimination of food-
proved when performed under the acute effects of remforced fixed-ratio responding by rats during ex¬

high daily doses of THC beginning at an early stage tinction. In a series of studies, it was demonstrated
of training. Facilitation of the acquisition of avoid¬ that the administration of immobilizing doses of
ance has also been found with various other drugs pentobarbital on the first day of extinction markedly
which, in common with THC, have a predominantly reduces the overall number of response? to extinction.

behavioral depressant effect, but impair well-estab In a number of control procedures it was shown that
fished avoidance responses oniy at very high doses it did not matter whether a series of pentobarbital
(Barry and Buckley, 1966). Meisch (1972) studied the doses was administered which decreased abruptly or
development of ethanol as a reinforcer for rats, de¬ very gradually, but rather merely whether the rate
scribing a procedure for establishing rapid acquisition of responding was markedly suppressed during the
of ethanol-controlled behavior. It has been shown first part of the first session of extinction, Figure 11
previously that rats will self-administer ethanol solu¬ shows responses in extinction by animals treated with

Fig. 11. Effects of 20 mg/kg i.p.

pentobarbital on responding
during extinction following
matched FR 20 food reinforce¬
ment history (rats). Pentobarbi¬
tal was administered at the
arrow prior to extinction ses¬
sion, with no further drug ad¬
ministrations. Vertical bars at
the right side of each curve in¬
dicate range of variability for
each group (Griffiths & Thomp¬
son, 1973.)
 562 OPERANT BEHAVIORAL PHARMACOLOGY

pentobarbital prior to the beginning of a 5-hour ex¬ stimulus control was discussed with reference to drug
tinction session. As can be seen, there was a very sub¬ effects. As can be seen, the locus of analysis is not
stantial difference in total responding to extinction as within the central nervous system or the mind, but is
a function of a high immobilizing dose of pentobar¬ expressed in terms of covariation between certain ex¬
bital during the first portion of the extinction session. ternal stimulus events and systematic changes in be¬
The foregoing studies indicate that it is possible to havior.
study phenomena falling within the rubric of “learn¬
ing” but expressed in terms of concepts understand¬
able within an operational experimental analysis of FUTURE OF BEHAVIORAL PHARMACOLOGY
behavior. Transition states, such as acquisition and
extinction or performance shift from one schedule to The future of behavioral pharmacology appears to
another, are subject to analysis within the framework lie in two primary directions.
of an experimental analysis of behavior.

A Fine-Grained Laboratory Analysis of Behavioral

Motivational Factors Mechanisms of Drug Actions

Motivational variables have long been a major Weiss (1970) and Weiss and Gott (1972) have pro¬
focus of many investigators in psychopharmacology. vided a micro-analysis of drug effects on fixed-ratio
Psychiatrists and other practitioners working in performance and the temporal structure of behavior.
psychopharmacology are interested in reducing anx¬ Weiss and Gott found that amphetamine and imipra-
iety, aggression, altering sex drives, and so forth. mine shortened all inter-response times within a FR
Laboratory investigators had hoped to find drugs 30 reinforcement schedule while pentobarbital length¬
which would selectively alter hunger or thirst. Within ened them. The effects observed were related to what
the framework of an experimental analysis of the be¬ Weiss and Gott called “the relatively unitary charac¬
havioral actions of drugs, such phenomena are profit¬ ter of fixed-ratio performance and its inherent co¬
ably approached by manipulating the type of conse¬ hesiveness.” Plots of the incidence of inter-response
quences controlling behavior. Behavior reinforced by time greater than one second suggest that ampheta¬
access to a target which can be attacked (Cherek, mine and imipramine alter fixed-ratio cohesiveness
Thompson, 8c Heistad, 1973; Hutchinson, Azrin, 8c whereas pentobarbital enhances it (i.e., either the
Hunt, 1968) provides a mechanism for studying the ratio performance consisted of extremely short and
effects of drugs on aggressive behavior. Similarly, extremely long inter-response times, or was totally dis¬
drugs’ effects on food reinforced responding, on re¬ rupted at very high doses).
sponding maintained by water, and responding to
avoid painful shocks, are methods by which one can
Applied Implications of Behavioral Pharmacology
begin to understand the degree to which drugs have
any selective effects on these motivational states. As
Behavioral Toxicology
indicated in our earlier discussion, considerable doubt
has been cast upon the proposition that drug effects In 1969, Weiss and Laties wrote the first major re¬
are primarily determined by the consequence con¬ view dealing with behavioral toxicology. They stated,
trolling behavior, but are to a far greater degree de¬ “Many studies of the behavioral effects of drugs can
pendent on the way in which those events are sched¬ be conceived of as attempts to determine selective
uled to be presented or avoided. toxicity in the context of a therapeutic aim. Be¬
havioral toxicology is the study of the selective toxic¬
Sensation and Perception ity as the direct aim.” (p. 320) A number of investiga¬
tions have attested to the sensitivity of operant pro¬
A third major category of investigations concerning cedures in assessing behavioral actions of toxins.
drug effects pertains to the way in which drugs alter Armstrong, Leach, Belluscio, Maynard, Hodge, and
perception or sensation. Drugs such as the hallu¬ Scott (1963) studied the effects of mercury vapor on
cinogens, depressants, and even to some degree stim¬ performance on a multiple reinforcement schedule in
ulants, are said to alter an organism’s perception. An pigeons, and found it was possible to produce re¬
effort to understand the way in which drugs alter per¬ versible changes in the behavioral baseline before
ception can be formulated, to a considerable degree, they could detect any overt pathology or gross be¬
in terms of stimulus control. In an earlier section havioral disruption. Beard and Wertheim (1967)
Travis Thompson and John J. Boren 563

studied the effects of carbon monoxide on temporal drinks per hour, a great reduction in alcohol con¬
discrimination in human subjects. They were able to sumption occurred. Pickens, Bigelow, and Griffiths
detect the effect of relatively low concentrations of (1973) studied alcohol consumption in a chronic
carbon monoxide (50 parts/million over a 90-min alcoholic under controlled ward conditions. They
exposure). They also studied the effect of low concen¬ studied the effects of a time-out contingency and the
trations of carbon monoxide on responding during a establishment of stimulus control on drinking by a
spaced reinforcement schedule in rats. The amount of chronic alcoholic over a one-year period. Using these
carbon monoxide in the atmosphere necessary to pro¬ procedures, it was possible to reduce the amount of

duce a given change in performance was a function of alcohol consumed from 1.5 ounces per minute to ap¬
the pause between the responses demanded of the proximately .2 to .3 ounces per minute.
schedule. That is, when a 30-sec pause was required, One of the more impressive applied outgrowths of
about 10 min of exposure to 100 parts/million was an operant interpretation of drug dependence has
enough to diminish response rate more than two been Hunt and Azrin?s (1972) work with chronic
standard deviations below the control. With a 10-sec alcoholics in an outpatient setting, Using a combina¬

pause required, about 40 min of exposure was neces¬ tion of positive reinforcement for non-drug related
sary to produce a comparable decrement in perfor¬ behavior and time-out from positive reinforcement
mance. contingent on drinking, they have been able to estab¬
lish extensive control over alcohol consumption by

Drug Abuse chronic alcoholics. In addition, the patients with

whom they worked maintained a high degree of em¬
Another area of behavioral pharmacology with sig¬ ployment and maintained something approximating
nificant implications for applied matters is drug abuse. a normal home life following treatment. Uiebson,
Thompson and Schuster (1968), Schuster and Thomp¬ Bigelow, and Flamer (1973) have used methadone as
son (1969), and Thompson and Pickens (1969) have a reinforcer for consuming disulfuram, in combined
provided a conceptual framework within which to alcoholics and heroin addicts. Disulfuram blocks
better understand human drug dependence. That alcohol metabolism and leads to yomiting following
drugs serve as powerful reinforcers is now widely ac¬ consumption of alcohol. In their technique, alcoholics
cepted, and effort^ to understand and modify human were reinforced with methadone contingent on con¬
drug dependence are emerging based on an operant suming their disulfuram, Boudin (1972) has developed
interpretation. Much of the published research has a large scale program based on a system of positive
dealt with alcoholism although some workers have reinforcement for participating in treatment programs
begun to deal with problems of modifying the use of involving a variety of contingency management con¬
opiate compounds as well. Mello and Mendelson trol techniques for violation of the system.
(1970) have studied drinking patterns during work-
contingent and non-contingent alcohol acquisition m
Pre-Clinical and Clinical
human alcoholics. Bigelow and co-workers (1972) in¬
Extensions of Laboratory Findings
vestigated factors influencing alcohol consumption by
chronic alcoholics. Bigelow, Cohen, Liebson, and Fail- Behavioral pharmacology has now reached the
lace (1972) studied establishment of controlled drink¬ point where it is meaningful to approach some ques¬
ing by alcoholics. Volunteer chronic alcoholics were tions concerning clinical therapeutic effects of be-
given access to substantial quantities of alcohol in haviorally active drugs. Physicians have often used
situations where they earned the opportunity to par¬ behaviorally active drugs based on the assumption
ticipate in an enriched ward environment contingent that a drug functions independently of the environ¬
on controlled drinking. The subjects overwhelmingly mental conditions under which the drug is admin¬
chose to drink moderately. In another study, Bigelow istered.
and Liebson (1972) examined response-cost factors In recent years interest has grown in the use of
controlling alcoholic drinking. Once again the sub¬ operant techniques in conjunction with drug therapy
jects were given access to alcohol under experimental in various applied settings. Lindsley (1962) first
conditions. When a high ratio requirement was estab¬ demonstrated the applicability of operant condition¬
lished for access to alcohol, alcohol drinking dropped ing techniques in the measurement of drug-behavior
to near zero. In another manipulation, when the num¬ interactions in an applied human setting. Subse¬
ber of tokens required to purchase drinks was mark¬ quently Hollis (1968) demonstrated a technique for
edly increased, if the subject drank more than two measurement of differential behavioral effects of
 564 OPERANT BEHAVIORAL PHARMACOLOGY

baseline placebo and 12.5 and 25 mg doses of diphen¬

hydramine. Both response-contingent consequences
(loss of candy and self-recording) suppressed grimac¬
ing to near zero rate per minute, whereas diphen¬
hydramine, which also had a rate reducing effect, did
so to a considerably lesser degree. In another experi¬
ment, Sulzbacher (1972) studied the effects of d-am-
phetamine and methylphenidate on classroom be¬
havior of children with learning difficulties. Figure 15
shows the effects of placebo, 5, and 10 mg of d-
amphetamine on talking out in class and out-of-seat
behavior during class. Figure 16 shows the effect of
d-amphetamine on academic performance. Ampheta¬
mine had a marked rate-reducing effect on the inap¬
propriate behavior, while producing a slight disrup¬
tive effect on academic responding. Methylphenidate,
on the other hand, had a much less striking effect on
talking out in class, a slight rate increasing effect on
writing performance in the classroom, but little or no
Fig. 12. Effects of chlorpromazine on rate of ball-manipulanda
effect on arithmetic and reading behavior.
responding under fixed-ratio 25 schedule of positive reinforce¬
ment. Curves represent standard deviation units of change in Recently attention has been given to the interac¬
response rate from control to drug sessions for subjects DG tion between reinforcement contingencies and drug
and KL. The dashed line at 0 units indicates the mean control
treatment in purely applied settings, such as in state
level. (From Hollis & St. Omer, 1972.)
mental hospitals. Paul, Tobias, and Holly (1972)
studied the effects of a variety of behaviorally active
drugs on psychotic behavior of chronic mental hos-
chlorpromazine in human retardates. More recently
Hollis and St. Omer (1972) studied the effects of
chlorpromazine (0.5-3.0 mg/kg) on operant respond-
ing by retardates in a controlled experimental situa¬
tion. Figure 12 shows the effect of chlorpromazine on
Leg-Manipulanda
DISCRIMINATED OPERANT
the rate of FR 25 responding for M & M reinforce¬
ment by two retarded subjects. The response rates are
expressed as deviations in standard scores from the Jb
baseline rates. The data demonstrate a direct relation £
between the dose of chlorpromazine administered and
the amount of response rate suppression of fixed-ratio I
performance. In another manipulation, Hollis and St.
Omer studied the effects of chlorpromazine on re¬
sponse rates and response latency using a FR 400
Is
§
schedule of positive reinforcement in two additional
*
subjects. Figure 13 shows dose response curves of FR
400 performance across the dosage range of .25 to 1.0
I
&
mg/kg. As can be seen, once again there is an orderly fc!
relation between dose of chlorpromazine administered
and suppression of operant responding, a finding com¬
parable to those obtained with infrahuman subjects.
Strong, Sulzbacher, and Kirkpatrick (1973) studied
the effects of diphenhydramine on facial grimacing in
a classroom by a five-year-old boy, deficient in lan¬ Fig. 13. Effects of chlorpromazine on cumulated latency and
rate of left-leg responding under conditions of a simple dis¬
guage and learning ability. Figure 14 shows the rela¬
criminated operant on a fixed-ratio 400 schedule of positive
tionship between several manipulations of conse¬ reinforcement for subjects TK and BP. Curves represent stan¬
quences following occurrence of grimacing in Experi¬ dard deviation units of change in response latency and rate from
control to drug sessions. The dashed line at 0 units indicates
ment 1. In Experiment 2, data are presented from a the mean control level. (From Hollis & St. Omer, 1972.)
Travis Thompson and John J. Boren 565

Fig. M. Rate per minute of facial grimaces. Experiment 1 shows

the effects of three different reinforcement contingencies. Ex¬
periment 2 compares the effects of two dosage levels of diphen¬ Fig. 15. Effeets Of d-aMpliet&Mine On Ralphfs classroom be¬
hydramine with baseline (no drug) and placebo conditions. havior. The heavy lines connect the means of each condition.
These data on drug effect were gathered about 2 hours after the The light bars around each mean indicate the range of daily
medication was administered. (From Strong et al.7 1974.) rates. (From Sulzbaehcr, 197^-)

pital patients under conditions of milieu therapy and

behavior modification in a state mental hospital. Al¬
though behavioral contingencies seem to have a sig¬
nificant effect in improving the behavior of psychotic
patients, none of the drug therapies in any of the dos¬
ages administered had any statistically significant
effect in improving behavior. McConahey and
Thompson (1972) and McConahey (1973) studied the
effect of chlorpromazine on the behavior of twenty-
two moderately to severely retarded women in a state
hospital. A multiple schedule was used in which token
reinforcement was presented contingent upon adap¬
tive behavior during a portion of the day, while dur¬
ing a comparable period of the day no programmed
consequences were arranged. During alternate 28-day
periods the residents received, in a randomly assigned
order, either chlorpromazine or an identically appear¬
ing placebo. No overall significant differences were
found between drug and placebo treatments in any of
25 behaviors recorded on all 22 patients over time. On A R! THMETIC WR/ TING

the other hand, very large statistically significant

Fig. 16. Effects of d-amphetamine on the academic performance
differences were obtained on 22 of the 25 behavioral of Ralph. (From Sulzbacher, 1972.)
 566
OPERANT BEHAVIORAL PHARMACOLOGY

measures across all residents, comparing the reinforce¬

ment periods and the non-reinforcement periods of
the multiple schedule. Figure 17 shows the total fre¬
quency of observations during which residents were
sitting at a table working constructively at a learning
task during periods when they were being reinforced
(contingencies) as opposed to periods when they were
not being reinforced (no contingencies), while being
treated with placebo or chlorpromazine. There were
no apparent differences due to the chlorpromazine
and placebo treatments in the tendency to work con¬
structively at a task, while there were very large differ¬
ences between periods in which patients were rein¬
forced with tokens as compared with those when they
were not. Figure 18 shows similar data with the total
frequency of residents raising their voices, a behavior
frequently occurring preceding physical aggression.
As can be seen there were no appreciable differences
between the frequency of raising the voice during
placebo and chlorpromazine treatments either with

-I- NO CONTINGENCIES [

Fig. 18. Total frequency of instances of raising voice (a be¬

havior frequently preceding physical aggressive acts) under
the same four treatment conditions as indicated in Figure 17.
(From McConahey 1972.)

or without reinforcement. However, when reinforce¬

ment occurred, there was an enormous reduction in the
amount of raising voices both during placebo and
chlorpromazine treatments.
Data from the foregoing studies suggest that it is
possible to analyze some of the interactions between
reinforcement conditions and drug-treatment condi¬
tions in a variety of applied settings involving dis¬
turbed children, children with learning difficulties, re¬
tardates and hospitalized psychiatric patients. The
introduction of behaviorally active medication in a
variety of applied settings must be predicated upon
more careful attention to the interactions between
drug treatment and the prevailing environmental
contingencies.
4- NO CONTINGENCIES L 1

Fig. 17. Total frequency of instances of constructively working

at a task (seated) by 22 retarded women during daily 45 minute REFERENCES
periods under four treatment conditions: Token reinforcement
(contingencies) plus placebo; token reinforcement plus chlorpro¬
mazine (CPZ); no programmed contingencies plus placebo; no Armstrong, R. P., Leach, L. J., Belluscio, P. R., Maynard,
programmed contingencies plus chlorpromazine. (From Mc- E. A., Hodge, H. C., 8c Scott, J. K. Behavioral changes
Conahey, 1972.)
in the pigeon following inhalation of mercury vapor.
Travis Thompson and John J. Boren 567

American Industrial Hygiene Association Journal, 1963, Dews, P. B. Drugs in psychology: A commentary on Travis
24, 366-375. Thompson and Charles R. Schuster’s Behavioral Phar¬
Barry, H. 8c Buckley, J. P. Drug effects on animal per¬ macology. Journal of the Experimental Analysis of Be¬
formance and the stress syndrome. Journal of Pharma¬ havior, 1970,13, 395-406.
ceutical Sciences, 1966, 55, 1159-1183. Dews, P. B. 8c Morse, W. H. Behavioral pharmacology.
Barry, H. 8c Kubena, R. K. Discriminative stimulus charac¬ Annual Review of Pharmacology, 1961, 1, 145-174.
teristics of alcohol, marihuana and atropine. In J. Singh, Dykstra, L. A. 8c Appel, J. B. Lysergic acid diethylamide
L. Miller, 8c H. Lai (Eds.), Drug Addiction: Experimental and stimulus generalization: Rate-dependent effects.
Pharmacology. North Miami, Florida: Symposia Special¬ Science, 1972, 177, 720-722.
ists, 1972. Geller, I., Kulok, Jr., J. T., 8c Seifter, J. The effects of
Beach, H. D. Morphine addiction in rats. Canadian Jour¬ chlordiazepoxide and chlorpromazme on a punishment
nal of Psychology, 1957, 11, 104-112. discrimination. Psychopharmacologia, 1962, 3, 374-385.
Beard, R. R. & Wertheim, G. A. Behavioral impairment Geller, I. & Seifter, J. The effects of meprobamate, barbi¬
associated with small doses of carbon monoxide, Amer¬ turates, d-amphetamine and promazine on experimen¬
ican Journal of Public Health, 1967, 57, 2012-2022. tally induced conflict in the rat. Psychopharmacologia,
Becker, D. I., Appel, J. B., & Freedman, D. X. Some effects 1960, 1, 482-492.
of lysergic add diethylamide on visual discrimination in Geller, I. & Seifter, J. The effect of mono-uretKans, di-
pigeons. Psychopharmacologia, 1907, 11, 354-364. urethans and barbiturates on a punishment discrimina¬
Bernal, J. D. Structure arrangements of macromolcculeSi tion. Journal of Pharmacology and Experimental Thera¬
Discussions of the Faraday Society, 1958, 25, 6.
peutics, 1962, 136, 284-288.
Bigelow, G., Cohen, M., Liebson, I. & Faillace, L. A. Goldberg, S. R. 8c Schuster, G. R, Conditioned suppression
Abstinence or moderation? Choice by alcoholics. Be¬ by a Stimulus associated with nalorphine in morphine-
havior Research and Therapy, 1972, 10, 209-214. dependent monkeys. Journal of the Experimental Analy¬
Bigelow, G. & Liebson, I. Cost factors controlling alcoholic sis of Behavior, 1967, 10, 235-242.
drinking. The Psychological Record, 1972, 22, 305-314. CoLDDERGi 5. R. & Schuster, G R. Nalorphine: Increased
Boren, J. J. The Study of drugs with operant techniques.
sensitivity g£ monkeys formerly dependent on morphine.
In W. K. Honig (Ed.), Operant Behavior: Areas of Re¬ Science ? 1969, 166, 1548-1549.
search and Application. Englewood Cliffs, NT: Prentice- Goldberg, S. R., Woods, J. IT. & Schuster, G. R. Morphing
Hall, Inc., 1966.
Conditioned increases in self-administration in rhesus
Boren, J. J, & Navarro, A, P. The action of atropine, monkeys. Science, 1969,166, 1306-1307.
benactyzine, and. scopolamine upon fixed interval and. Gollub, L. R. & Brady, J. V. Behavioral pharmacology.
fixed ratio behavior. Journal of the Experimental Analy¬ Annual Review of Pharmacology, 1965, J, 235-262.
sis of Behavior, 1959, 2, 107-115.
Griffiths, R, R, & Thompson, Y, Effects of chlorpromas
Boudin, H. M. Contingency contracting as a therapeutic zme and pentobarbital on pattern and number of re¬
tool in the deceleration of amphetamine use. Behavior sponses in extinction. Psychological Reports. 1973, ??t
Therapy} 1972, 3, 604-608. 323-334. 1 J ' '
Ceierek, D. 8c Thompson, T. Effects of tetrahydrocannabi¬ Griffiths, R. R., & Thompson, T. Pentobarbital facilitated
nol on schedule-induced aggression. Pharmacology, Bio¬ extinction: Effects of different schedules of drug with¬
chemistry and Behavior, 1973,1, 493-500. drawal. Pharmacology, Biochemistry and Behavior, 1974,
Cherek, D. R., Thompson, T. & Heistad, G, T. Respond¬ 2, 331-338.
ing maintained by the opportunity to attack during an Gr<?ye, R, & Thompson, T. The effects of pentobarbital on
interval food reinforcement schedule. / ournal of the performance maintained by an interlocking fixed*ratio
Experimental Analysis of Behavior, 1973, 19} 113-123. fixed-interval reinforcement schedule. In T. Thompson,
Cook, L. & Catania, A. C. Effects of drugs on avoidance
R. Pickens, 8c R, Mdsch (Eds.), Readings in Behavioral
and escape behavior. Federation Proceedings 1964, 2? Pharmacology, Englewood Cliffs, N.J.: Prentice*Hall,
818-835. InCr, 1970.
Cook, L. C., Davidson, A., Davis, D. J. & Kelleher, R, T.
Harris, R. T. 8c Balster, R. L. An analysis of the function
Epinephrine, norepinephrine, and acetylcholine as con¬ of drugs in the stimulus control of operant behavior. In
ditioned stimuli for avoidance behavior. Science, 1960 T. Thompson 8c R. Pickens (Eds.), Stimulus Properties of
^ 131, 990-991.
Drugs. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1971.
Cook, L. 8c Kelleher, R. T. Effects of drugs on behavior. Headlee, C. P., Coppock, H. W. 8c Nichols, J. R. Ap¬
Annual Review of Pharmacology, 1963, 3, 205-222. paratus and technique involved in a laboratory method
Dews, P. B. Studies on behavior: I. Differential sensitivity of detecting the addictiveness of drugs. Journal of the
to pentobarbital of pecking performance in pigeons American Pharmaceutical Association, 1955, 44, 229-231.
depending on the schedule of reward. Journal of Phar¬ Hearst, E. Drug effects on stimulus generalization gradients
macology and Experimental Therapeutics, 1955, 113, in the monkey. Psychopharmacologia, 1964, 6, 57-70.
343-401. (a)
Heise, G. A. 8c Lilie, N. Effects of scopolamine, atropine
Dews, P. B. Studies on behavior: II. The effects of pento¬ and d-amphetamine on internal and external control of
barbital, methamphetamine, and scopolamine on per¬ responding on non-reinforced trials. Psychopharmaco¬
formances in pigeons involving discrimination. Journal logia, 1970, 18, 38-49.
of Pharmacology and Experimental Therapeutics, 1955 Herrnstein, R. J. 8c Morse, W. H. Selective action of
115, 380-389. (b)
pentobarbital on component behaviors of a reinforce¬
Dews, P. B. Studies on behavior: IV. Stimulant actions of ment schedule. Science, 1956, 124, 367-368.
methamphetamine. Journal of Pharmacology and Ex¬ Hill, R. T. Facilitation of conditioned reinforcement as a
perimental Therapeutics, 1958, 122, 137-147. mechanism of psychomotor stimulation. In E. Costa 8c S.
568 OPERANT BEHAVIORAL PHARMACOLOGY

Garattini (Eds.), Amphetamines and Related Com¬ Behavior modification, drug therapy and their combina¬
pounds. Proceedings of the Mario Negri Institute for tion. In T. Thompson and J. Grabowski (Eds.), Be¬
Pharmacological Research, Milan. New York: Raven havior Modification of the Mentally Retarded. New
Press, 1970. York: Oxford University Press, 1972.
Hollis, J. H. Chlorpromazine: Direct measurement of a McConahey, O. L. Effects on concurrent behavior modi¬
differential behavioral effect. Science, 1968, 159, 1487— fication and chlorpromazine administration in mentally
1489. retarded women. Unpublished doctoral dissertation, Uni¬
Hollis, J. H. 8c St. Omer, V. V. Direct measurement of versity of Minnesota, December, 1973.
psychopharmacologic response: Effects of chlorpromazine McConahey, O. L. 8c Thompson, T. Concurrent behavior
on motor behavior of retarded children. American Jour¬ modification and chlorpromazine therapy in a popula¬
nal of Mental Deficiency, 1972, 76, 397-407. tion of institutionalized mentally retarded women.
Hunt, G. M. 8c Azrin, N. H. Community reinforcement Paper presented at the Annual Meeting of the American
approach to alcoholism. Presented at the Annual Meet¬ Psychological Association, Washington, D.C., 1972.
ing of the American Psychological Association, Hono¬ McKearney, J. W. Schedule-dependent effects: Effects of
lulu, Hawaii, Sept., 1972. drugs, and maintenance of responding with response-
Hutchinson, R. R., Azrin, N. H. 8c Hunt, G. M. Attack produced electric shocks. In R. M. Gilbert 8c J. D. Keehn
produced by intermittent reinforcement of a concurrent '(Eds.), Schedule Effects: Drugs, Drinking, and Aggres¬
operant response. Journal of the Experimental Analysis sion. Toronto: University of Toronto Press, 1972.
of Behavior, 1968, 11, 489-495. McMillan, D. E. Drugs and punished responding I: Rate-
Iverson, S. 8c Iverson, L. Behavioral Pharmacology. Lon¬ dependent effects under multiple schedules. Journal of
don: Oxford University Press, 1975. the Experimental Analysis of Behavior, 1973, 19, 133—
Jacobs, H. L. Studies on sugar preference: The preference 145.
for glucose solutions and its modification by injections McMillan, D. E., Dewey, W. L. 8c Harris, L. S. Character¬
of insulin. Journal of Comparative and Physiological istics of tetrahydrocannabinol tolerance. Annals of the
Psychology, 1958, 51, 304-310. New York Academy of Sciences, 1971, 191, 83-99.
Kelleher, R. T., Fry, W., Deegan, J. 8c Cook, L. Effects of McMillan, D. E., Harris, L. S., Frankenheim, J. M. 8c
meprobamate on operant behavior in rats. Journal of Kennedy, J. \-L^-transtetrahydrocannabinol in pi¬
S.
Pharmacology and Experimental Therapeutics, 1961, geons: Tolerance to the behavioral effects. Science, 1970,
133, 271-280. 119, 501-503.
Kelleher, R. T. 8c Morse, W. H. Escape behavior and Meisch, R. Rapid development of ethanol as a reinforcer
punished behavior. Federation Proceedings, 1964, 23, for rats. Proceedings, 80th Annual Convention, The
808-817. American Psychological Association, 1972, 851-852.
Kelleher, R. T. 8c Morse, W. H. Determinants of the Meisch, R. A. 8c Thompson, T. Ethanol intake in the
specificity of the behavioral effects of drugs. Ergebnisse absence of concurrent food reinforcement, Psychophar-
der Physiologie, 1968, 60, 1-56. macologia, 1971, 22, 72-79.
Kelleher, R. T. 8c Morse, W. H. Determinants of the Meisch, R. 8c Thompson, T. Ethanol as a reinforcer: Effects
behavioral effects of drugs. In D. H. Tedeschi 8c R. E. of fixed-ratio size and food deprivation. Psychopharma-
Tedeschi (Eds.), Importance of Fundamental Principles cologia, 1973, 28, 171-183.
in Drug Evaluation. New York: Raven Press, 1969. Meisch, R. A. 8c Thompson, T. Ethanol as a reinforcer: An
Laties, V. G. The modification of drug effects on behavior operant analysis of ethanol dependence. In J. M. Singh
by external discriminative stimuli. Journal of Pharma¬ 8c H. Lai (Eds.), Drug Addiction, Vol. 3: Behavioral and
cology and Experimental Therapeutics, 1972, 183, 1-13. Clinico-Toxicological Aspects. Miami, Florida: Symposia
Laties, V. G. 8c Weiss, B. Effects of amphetamine, chlorpro¬ Specialists, 1974.
mazine and pentobarbital on behavioral thermoregula¬ Mello, N. K. 8c Mendelson, J. H. Experimentally induced
tion. Journal of Pharmacology and Experimental Thera¬ intoxication in alcoholics: A comparison between pro¬
peutics, 1963, 140, 1-7. grammed and spontaneous drinking. Journal of Pharma¬
Laties, V. G. 8c Weiss, B. Influence of drugs on behavior cology and Experimental Therapeutics, 1970, 173, 101—
controlled by internal and external stimuli. Journal of 116.
Pharmacology and Experimental Therapeutics, 1966, Morse, W. H. Effect of amobarbital and chlorpromazine on

152, 388-396. punished behavior in the pigeon. Psychopharmacologia,

Liebson, I., Bigelow, G. 8c Flamer, R. Alcoholism among 1964, 6, 286-294.
methadone patients: A specific treatment method. Amer¬ Morse, W. H. 8c Herrnstein, R. J. Effects of drugs on

ican Journal of Psychiatry, 1973, 4, 483-485. characteristics of behavior maintained by complex sched¬
Lindsley, O. Operant conditioning techniques in the mea¬ ules on intermittent positive reinforcement. Annals of
surement of psychopharmacological response. In J. H. the New York Academy of Sciences, 1956, 65, 303-317.
Nodine 8c J. H. Moyer (Eds.), Psychosomatic Medicine. Morse, W. H. 8c Kelleher, R. T. Schedules as fundamental

Philadelphia: Lea and Febiger, 1962. determinants of behavior. In W. N. Schoenfeld (Ed.),

Marsh, D. F. Outline of Fundamental Pharmacology: The The Theory of Reinforcement Schedules. Englewood
Mechanics of the Interaction of Chemicals and Living Cliffs, N.J.: Prentice-Hall, Inc., 1970.
Things. Springfield: Charles C. Thomas, 1950. Overton, D. A. Discriminative control of behavior by drug

Masserman, J. H. 8c Yum, K. S. An analysis of the influence states. In T. Thompson and R. Pickens (Eds.), Stimulus
of alcohol on experimental neurosis. Psychosomatic Properties of Drugs. Englewood Cliffs, N.J.: Prentice-
Medicine, 1946, 8, 36-52. Hall, Inc., 1971.
McConahey, O. L. A token system for retarded women: Paul, G., Tobias, L. L. 8c Holly, B. L. Maintenance psycho-
Travis Thompson and John J. Boren 56 9

tropic drugs with chronic mental patients in the pre¬ “errors”. Journal of the Experimental Analysis of Be¬
sence of active treatment programs: A “triple blind” havior, 1963, 6, 1-27. (a)
withdrawal study. Archives of General Psychiatry, 1972, Terrace, H. S. Errorless discrimination learning in the
27, 106-115. pigeon: Effects of chlorpromazine and imipramine.
Pickens, R., Bigelow, G. 8c Griffiths, R. An experimental Science, 1963, 140, 318-319. (b)
approach to treating chronic alcoholism: A case study Thompson, T., Griffiths, R. 8c Pickens, R. Drug self¬
and 1 year follow-up. Behavior Research and Therapy, administration by animals: Some implications for hu¬
1973, 11, 321-325. man drug dependence. In F. Hoffmeister (Ed.), Psychic
Pickens, R. 8c Crowder, W. F. Effects of CS-US interval on Dependence, Berlin: Springer, Verlag, 1973.
conditioning of drug response, with assessment of speed Thompson, T. 8c Pickens, R. Drug dependence and condi¬
of conditioning. Psychopharmacologia, 1967, 11, 88-94. tioning. In H. Steinberg (Ed.), Scientific Basis of Drug
Pickens, R. 8c Thompson, T. Intravenous preparations for Dependence. London: J. A. Churchill, 1969.
self-administration by animals. American Psychologist, Thompson, T. 8c Pickens, R. Stimulant self-administration by
1975, 30, 274-276. animals; Some comparisons with opiate self-administra¬
Rech, R. H. 8c Moore, K. E. (Eds.), An Introduction to tion. Federation Proceedings, 1970, 29, 6-12.
Psychopharmacology. New York: Raven Press, 1971. Thompson, T. 8c Pickens, R. Drugs as reinforcers: Schedule
Richelle, M., Xhenseval, B., Fontaine, O., 8c Thone, L. considerations. In R. M. Gilbert 8c J. D. Keehn (Eds.),
Action of chlordiazepoxide on two types of temporal Schedule Effects: Drugs, Drinking, and Aggression. Tor¬
conditioning in rats. International Journal of Neuro- onto; University of Toronto Press, 1972.
pharmacology, 1962, 1, 381-391. Thompson, T. 8c Schuster, G. R. Behavioral Pharmacology,
Schuster, C. R. 8c Brady, J. V. The discriminative control Englewood Cliffs, New Jersey: Prentice-Hall, Inc., 1968.
of a food-reinforced operant by interoceptive stimula¬ Thompson, T., Trombley, J., Luke, D. 8c Lott, D. Effects
tion. Pavlov Journal of Higher Nervous Activity, 1964, of morphine on behavior maintained by four simple
11, 448-458. food-reinforcement schedules. Psychopharmacologia,
Schuster, G. R. 8c Thompson, T, Self-administration of and 1970,17, 182-192.
behavioral dependence on drugs. Annual Review of VerhAve, T. The effect of methamphetamine on operant
Pharmacology, 1969, 9, 483-502. level and avoidance behavior. Journal of the Experi¬
Sidman, M. Behavioral pharmacology. Psychopharmaco¬ mental Analysis of Behavior, 1958, 1, 207-219.
logia, 1959, 1, 1-19. Waller, M. B. 8c Morse, W. H. Effects of pentobarbital on
Sidman, M. Tactics of Scientific Research. New York; Basic fixed-ratio reinforcement. Journal of the Experimental
Books, 1960. Analysis of Behavior, 1963, 6, 125-130.
Singh, S. D. 8c Manocha, S. N. The interaction of drug Waller, M. B. 8c Waller, P. F. Effects of chlorpromazine
effects with drive level and habit strength. Psychophar¬ on appetitive and aversive components of a multiple
macologia, 1966, 9, 205-209. schedule. Journal of the Experimental Analysis of Be¬
Smith, C. B. Effects of d-amphetamine upon operant be¬ havior, 1962, 5, 259-264.
havior of pigeons: Enhancement by resperine. Journal of Weiss, B. Amphetamine and the temporal structure of

Pharmacology and Experimental Therapeutics, 1964, behavior. In E. Costa 8c S. Garattini (Eds.), Ampheta¬
116, 167-174. mines and Related Compounds. New York: Raven Press,
Sparber, S. B. Effects of drugs on the biochemical and be¬ 1970.
havioral responses of developing organisms. Federation Weiss, B. 8c Gott, C. T. A microanalysis of drug effects on

Proceedings, 1972, 31, 74-80. fixed-ratio performance in pigeons. Journal of Pharma¬

Spragg, S. D. S. Morphine addiction in chimpanzees. Com¬ cology and Experimental Therapeutics, 1972, 180, 189—
parative Psychology Monographs, 1940, 15, No. 7.
202.
Spyker, J. M., Sparber, S. B. 8c Goldberg, A. M. Subtle
Weiss, B. 8c Laties, V. G. Behavioral pharmacology and
toxicology. Annual Review of Pharmacology, 1969, 9,
consequences of methylmercury exposure: Behavioral de¬
297-326.
viations in offspring of treated mothers. Science, 1972,
Wells, J. A. Historical background and general principles
177, 621-623.
of drug action. In V. A. Drill (Ed.), Pharmacology in
Stolerman, I. P. A method for studying the influences of
Medicine. New York: McGraw Hill Book Company,
drugs on learning for food rewards in rats. Psychophar¬
1958.
macologia, 1971, 19, 398-406.
Woods,J. H., Downs, D. A. 8c Villarreal, J. E. Changes in
Strong, C., Sulzbacher, S. I. 8c Kirkpatrick, M. A. Use of
operant behavior during deprivation- and antagonist-in¬
medication versus reinforcement to modify a classroom duced withdrawal states. In L. Goldberg 8c F. Hoff¬
behavior disorder. Journal of Learning Disabilities, meister (Eds.). Psychic Dependence. Berlin: Springer-
1974, 7, 214-218. Verlag, 1973.
Sulzbacher, S. I. Behavior analysis of drug effects in the Wuttke, W. 8c Kelleher, R. T. Effects of some bezodiaze-
classroom. In Semb, G. (Ed.), Behavior Analysis and pines on punished and unpunished behavior in the
Education. University of Kansas, 1972. pigeon. Journal of Pharmacology and Experimental
Terrace, H. S. Discrimination learning with and without Therapeutics, 1970, 172, 397-405.
 19

Central Reinforcement
a bridge between
brain function and behavior*

Gordon Mogenson
and
Jan Cioe

INTRODUCTION ments occurred. The demonstration of central rein¬

forcement, made possible by the use of stereotaxic
surgical procedures for the implantation of chronic
It was nearly 60 years after the brain was first
stimulating electrodes and by the use of operant tech¬
stimulated electrically that it was shown that motiva¬
niques, provided the major impetus for the study of
tional effects could be elicited by such stimulation.
brain mechanisms of reinforcement and more gen¬
This long delay following the historic experiments of
erally for the study of brain-behavior relationships.
Fritsch and Hitzig, who stimulated the motor cortex
Positive central reinforcement was first reported by
in 1870, was due in part to the facts that most of the
J. Olds and Milner (1954), who observed that rats re¬
experiments during this period were in anesthetized
ceiving electrical stimulation of the septum returned
animals and that most investigators stimulated the
to the place in an open field where they had been
cerebral cortex, which is motivationally neutral (Doty,
stimulated (Figure 1). When the animals could initiate
1969). In the early 1930s Hess performed important
the brain stimulation by pressing a lever they made
pioneering experiments in unanesthetized, freely mov¬
this operant response at high rates for long periods of
ing animals. Although he observed that a variety of
time. Central reinforcement, inferred from the brain
motivated behaviors could be elicited by stimulation
self-stimulation phenomenon, immediately excited the
of the hypothalamus and other subcortical structures,
interest and imagination of psychologists and other in¬
more than two decades passed before further develop-
vestigators. During the next few years central rein¬
forcement was demonstrated with electrical stimula¬
# The authors wish to express their appreciation to Miss
tion of a number of subcortical structures in a variety
Anne Baxter and Mrs. Marianne Jeffery for typing the several
drafts of the manuscript and to D. Baran, J. P. Huston, W. J. of species, including man. Although the possible prac¬
McClelland, P. M. Milner, A. G. Phillips, Ann Robertson, P. tical application of central reinforcement was recog¬
Russell, B. B. Schiff, and T. B. Wishart who read and commented
nized early (in 1955 McClelland suggested, in a hu¬
on earlier versions of this article. The authors’ research is sup¬
ported by grants from the National Research Council of Canada morous yet prophetic vein, that brain self-stimulation
and the Medical Research Council of Canada. might be made readily available to perk us up, as an

57 0
Gordon Mogenson and Jan Cioe 571

Central reinforcement has also stimulated other

areas of research. The self-stimulation procedure has
been widely used in studying the effects of drugs on
behavior, and research using central reinforcement has
had a strong impact on the new disciplines of neuro¬
pharmacology and psychopharmacology. A number of
studies have indicated that catecholaminergic neurons
subserve brain self-stimulation. Self-stimulation is
markedly reduced by drugs which inhibit the synthesis
of catecholamines, deplete catecholamines from nerve
terminals, or destroy catecholaminergic neurons, and
is increased by drugs, such as amphetamine, which in¬
crease the synaptic release and prevent the reuptake
of catecholamines (for a review see German 8c Bow¬
den, 1974). Several years ago Stein postulated the
noradrenergic hypothesis of reward and subsequently
demonstrated that noradrenalin was released during
self-stimulation, apparently due to the activation of
noradrenergic fibers (Stein 8c Wise, 1969), Stein, Wise,
and Berger (1972) have proposed that schizophrenia is
the result of a biochemical disturbance (a deficit in
the enzyme dopamine-^-hydroxylase) in which nor¬
adrenergic “reward” fibers are destroyed by the pro¬
duction of 6-hydroxydopamine, a neurotoxin,
The first successful conditioning of visceral re¬
sponses was obtained with central reinforcement and
opened the field of bipfeedback. Central reinforce¬
ment enabled experimenters to deliver a reinforcement
for the desired visceral response even though the
animal was paralyzed and so was unable to approach
Fig. 1. This picture illustrates the discovery of central rein¬
forcement by James Olds and Peter Milner. (F rom Hebb, 1972.) or consume conventional reinforcers. It is necessary to
induce paralysis in order to avoid the possibility that
the visceral response is merely an artifact of the con¬
“exotic coffee break”) and although the possibilities ditioning of skeletal movements. Di Cara and Miller
for its use in the alleviation of abnormal behaviors (1968) demonstrated that blood pressure, blood flow,
clearly exist (Crichton, 1972), the major developments intestinal contractions, and other visceral responses
have been experimental and theoretical. Psychologists were conditioned using stimulation of reinforcing
saw the relevance of central reinforcement to many of sites in the hypothalamus. This field, although contro¬
the issues and problems of the psychology of learning, versial, is an active area of investigation which may
and they were intrigued by the possibility that self¬ contribute eventually to an understanding and treat¬
stimulation would permit the direct study of the ment of psychosomatic disorders.
neural substrates of reinforcement and lead eventually Finally it should be mentioned that central rein¬
to an understanding of the basic mechanisms of moti¬ forcement has stimulated research at the interface be¬
vation and learning. They were interested in compar¬ tween psychology and ethology. This resulted from
ing central reinforcement with conventional rein¬ the observations that feeding (Hoebel 8c Teitelbaum,
forcers to determine whether the self-stimulation 1962; Margules 8c Olds, 1962), drinking (Mogenson 8c
phenomenon was interpretable in terms of drive re¬ Stevenson, 1966), object carrying (Phillips, Cox,
duction theory, the most popular behavioral theory at Kakolewski, 8c Valenstein, 1969), and other species-
the time. The discovery of self-stimulation and central typical behaviors are elicited from the same brain sites
reinforcement reactivated a number of issues in the as self-stimulation. Not only has this led to an ingen¬
areas of motivation, reinforcement, and learning, and ious theory of reinforcement (Glickman 8c Schiff, 1967)
as we indicate later, the work that followed led to and vigorous investigation of self-stimulation be¬
new conceptions of reinforcement. havior, but it has also helped to promote the discourse
572 CENTRAL REINFORCEMENT

between psychologists and ethologists, a development than 1 sec (.2 and .5 sec being the most popular), al¬
which was long overdue. though in certain situations an operant response may
be reinforced by several trains (e.g., Hawkins 8c Plis-
koff, 1964). The details of the delivery system for the
METHODOLOGICAL CONSIDERATIONS
brain stimulation are dependent on the particular
experimental situation, although typically the brain
Following the initial demonstration of central rein¬
stimulation is delivered through wire leads from a
forcement there was considerable research to delineate
commutator or other device which allows reasonable
the regions of the brain subserving the phenomenon
freedom of movement. When the desired response has
(e.g., J. Olds, 1956; M. E. Olds 8c J. Olds, 1963; Wet¬
occurred the central reinforcement is delivered either
zel, 1968). Although the hypothalamus has been the
automatically or manually.
most popular target, central reinforcing effects are also
Central reinforcement has been demonstrated in
obtained from many other neural structures, through¬
several experimental situations (mazes, obstruction
out the limbic system (Milner, 1970) and the extra-
boxes, runways, shuttleboxes), but it has been studied
pyramidal motor system (e.g., Routtenberg 8c Mals-
almost exclusively with operant methodology. With
bury, 1969). Appendix A indicates the diversity of
operant techniques a high degree of environmental
brain structures shown to be at least mildly reinforc¬
control is possible, and since the operant response
ing and also illustrates the ubiquity of the phenom¬
causes little or no change in the environment the
enon across species.
behavior is very stable (Honig, 1966, p. 4). When the
operant response is maintained with central reinforce¬
Experimental Procedures
ment the motivational state of the animal which re¬
In order to obtain central reinforcement it is neces¬ sults from the brain stimulation is relatively constant
sary, of course, to place a stimulating electrode in or —unaltered by satiation effects, for example—so that
near one of the brain structures referred to in Ap¬ the operant responding is stable for long periods of
pendix A. The operative procedure involved is rela¬ time.
tively simple and straightforward, especially if the
subject is one of the more hardy laboratory animals
Operant Rate as a Measure of
(e.g., rat or gerbil). The animal is first anesthetized
Central Reinforcement
and the top of the head shaved. After the head is
positioned in a stereotaxic instrument an incision is The Skinner box has been the most popular test for
made along the midline to expose the skull. Bone self-stimulation, and as a result rate of response has
sutures are identified as a guide in directing the elec¬ been widely adopted as the measure of central rein¬
trode to subcortical structures. A standard brain atlas forcement. Response rate is not, however, always a re¬
(e.g., Pellegrino 8c Cushman, 1967) is used with refer¬ liable indicator of reinforcement strength. Hodos and
ence to skull bone sutures in order to obtain three- Valenstein (1962) showed in a two-lever preference
dimensional coordinates for placement of the elec¬ test (hypothalamic versus septal stimulation at various
trode in the desired structure. A hole is then drilled current intensities) that animals did not necessarily
through the skull and the electrode is lowered to the select the site of stimulation and current intensity that
desired depth by the micromanipulator of the stereo¬ maintained the highest response rates. For example,
taxic carriage. The electrode is securely attached to rats preferred septal stimulation at a moderate current
the skull and anchoring screws placed around the intensity to hypothalamic stimulation even though
electrode hole by acrylic cement. About one week is the hypothalamic stimulation maintained a higher re¬
usually allowed for the animal to recover from the sponse rate. Similarly, Davis, Davison, and Webster
trauma of surgery before testing is begun; such a (1972) demonstrated in pigeons performing on concur¬
preparation can continue in use for many months. rent variable interval (VI) schedules that the most
Electrical stimulation provided by a stepdown highly preferred brain stimulation maintained low
transformer from a 60-Hz ac source is very effective rates of responding. There is also evidence (Valenstein
for self-stimulation and was used extensively for 8c Beer, 1962) that reinforcement strength as deter¬
several years. Commercial electronic stimulators have mined by competition of central reinforcement with
been used to deliver rectangular pulses which may other reinforcers such as food and shock avoidance is
be varied in pulse duration, frequency, and wave¬ not the same as that indicated by response rate.
form as well as in intensity. The duration of a train Finally, Hawkins and Pliskoff (1964) have made the
of reinforcing stimulation is usually limited to less same point by using a two-member behavioral chain-
Gordon Mogenson and Jan Cioe 573

ing procedure. In this procedure responding on a VI The Advantages of Intermittent Schedules

schedule on a single lever is reinforced by access to a
second retractable lever which delivers central rein¬ It should be noted that many of the criticisms of
forcement on a continuous reinforcement (CRF) the use of response rate as a measure of central rein¬
schedule. The rate of pressing on the first lever con¬ forcement apply mainly when a CRF schedule is used.
tinued to increase at current intensities higher than Intermittent schedules, which are preferable to the
those which maintained peak response rate on the CRF schedule in most operant situations, produce a
second lever, thus indicating that the potency of cen¬ more stable pattern of behavior while still maintain¬
tral reinforcement cannot be assessed adequately by ing a sensitivity to environmental manipulations
self-stimulation rates on a CRF schedule. (Reynolds, 1968). Accordingly, intermittent schedules
Valenstein (1964) has argued on the basis of these may be preferable for a behavioral analysis of self¬
results that the rate of responding on a CRF schedule stimulation. When the trains of pulses are spaced, the
is a poor index of reinforcement strength. He has disruptive side effects of brain stimulation, such as
gone on to point out that there are logical difficulties forced motor responses and carry-over effects, are
in using an average CRF rate since there is the im¬ minimized. Using a two-lever chaining procedure,
plicit assumption that reinforcement strength is Hawkins and Pliskoff (1964) clearly showed that the
homogeneous throughout the test session. Occa¬ response rate on the intermittently reinforced lever
sionally , this assumption is not justified with central paralleled the results obtained with preference tests
reinforcement since some effects of the stimulation when current was manipulated. Moreover, by using
which persist may change the strength of the rein- intermittent schedules one retains the advantages of
forcer after its administration (e.g., seizure activity— using a rate measure as the dependent variable (e.g.,
Mogenson, 1965; Newman & Feldman, 1964). Further¬ Honig, 1966, pp. 6-7; Skinner, 1966b, pp. 15-17).
more, central reinforcement, especially at higher cur¬ There has been, however, some problem in maintain¬
rent intensities, may elicit various respondents m an ing animals on a “reinforcement-poor” intermittent
unconditioned manner (e.g., forced motor responses), schedule using central reinforcement m a single-lever

which can disrupt the response rate even though these situation; the problem (as is discussed more fully in a
high current intensities are preferred over lower in¬ later section) appears at least partially to be one of
tensities which maintain higher response rates (Hodos determining how much central reinforcement is
8c Valenstein, 1962). equivalent tQ the standard amount? of food and water
Although the criticisms of Valenstein against a reinforcement commonly used. The advantages of an
CRF rate of response as an index of central reinforce¬ intermittent schedule also accrue to the use of concur¬
ment appear justified in some situations, such a rent schedules, which appear to be highly sensitive to
measure is meaningful in many others. Clearly, if a changes in the magnitude of reinforcement, as op¬
comparison is being made between diverse sites of posed to single schedules, which are less so (Hollard &
stimulation, or if high intensities are employed, there Davison, 1971). Such sensitivity may be particularly
is a real problem of distortion with a CRF response useful in analyzing the incentive properties of brain
rate measure. It seems equally true, however, that in stimulation.
the majority o£ experiments there is little distortion Another modification of intermittent schedules
as long as moderate current intensities are employed, which has been successfully employed with central
as even Valenstein’s data indicate (Hodos 8c Valenstein, reinforcement is the progressive-ratio procedure de¬
1962). The use of moderate intensities decreases the scribed originally by Hodos (1961) for food reinforce¬
probability of both interfering motor responses and ment. In this procedure the animal is required to emit
seizure activity. This situation is indeed fortunate a progressively increasing number of responses in
since most of the work discussed in the comparison of order to obtain each successive reinforcement. The
central and conventional reinforcers (discussed in a “terminal ratio’' is defined as the highest number of
later section) is based on the CRF response rate responses emitted before a pause of given duration,
measure.1 such as 15 sec, occurs. This is used as the dependent
measure. Hodos (1965) demonstrated that the size of
i It might be useful at this point to establish two conven¬
the terminal ratio is sensitive to long durations of re¬
tions: (1) the species should be understood to be rats unless
otherwise indicated; and (2) the method used in a particular warding brain stimulation. Keesey and Goldstein
study involves a conventional operant response (such as lever (1968) further modified this procedure by defining the
pressing) on a CRF schedule unless a different method is
“terminal ratio” as the highest level of stable ratio
specified. The use of these conventions will help to simplify the
task of describing the many studies of self-stimulation. (FR) responding that a given reinforcement condition
 574
CENTRAL REINFORCEMENT

would maintain, and they were able to demonstrate ities. The measure used is the percentage of time
that the functions relating this terminal ratio to a spent receiving stimulation. Since the positive (brain
wide range of stimulus currents were monotonic at all stimulation) and neutral sides can be randomly inter¬
brain stimulation sites tested. It appears that these changed, this measure is directly amenable to statis¬
more sophisticated behavioral techniques are poten¬ tical evaluation. Some animals, however, self-admin-
tially useful for the study of central reinforcement ister very brief trains of central reinforcement and
and for the analysis of the reinforcement process in may spend less than 50% of the session (chance level)
general. on the positive side, although these same animals re¬
Although there are certain limitations in the use of peatedly cross over and receive the stimulation (Cioe,
a CRF schedule with central reinforcement, a CRF personal observation). Independent measures of rein¬
rate measure is still meaningful in most situations. forcement strength indicate that stimulation will
When a comparison is being made between stimu¬ maintain a lever-press response as well as the operant
lation sites or when high current intensities are used of crossing in the shuttlebox. This effect, however, is
there is the possibility of distortion of a CRF response not reflected in the time score, which limits the useful¬
rate measure. In most experiments there is little dis¬ ness of this measure.
tortion, however, especially if moderate current in¬
tensities are employed.
Stein and Ray (1959) devised a CRF procedure
CENTRAL REINFORCEMENT COMPARED
which allows the animal to self-regulate the current
TO CONVENTIONAL REINFORCEMENT
intensity. Pressing one lever increases the current,
whereas pressing a second lever decreases it. Inter¬
estingly, it was reported that high current levels which In the study of central reinforcement, operant
often produced disruptive motoric responses were methods are both important and useful, as shown in
selected. Modifications of this “titration” method have the previous section; indeed, the area would probably
been used extensively in the analysis of drug effects on not have undergone such rapid and fruitful expansion
self-stimulation behavior (e.g., Stein, 1962). without operant techniques. In this section we would
like to turn the discussion around and demonstrate
Nonrate Measures of Central Reinforcement the contribution that central reinforcement can make
to an analysis of operant behavior. The main issue is
As we have already mentioned, central reinforce¬ this: does self-stimulation differ in any substantial and
ment has been obtained in other experimental situa¬ irreducible way from the more conventional rein¬
tions which do not utilize a rate measure. These pro¬ forcers used before its discovery (e.g., food and water
cedures, however, also have drawbacks which have to a deprived animal, saccharine, etc.)? The initial
limited their usefulness. Hodos and Valenstein (1962) view of this phenomenon was that the neural sub¬
used a preference procedure in which animals choose strate of reinforcement had been discovered, and so
between two conditions available on different levers, it was argued that conventional and central reinforce¬
but this becomes extremely cumbersome when a large ment were essentially identical in nature (J. Olds,
. number of comparisons are to be made. This pro¬ 1956). With further research, however, there emerged
cedure, nonetheless, is quite useful in validating other a growing number of apparent dissimilarities, and
procedures. The obstruction box technique employed elaborate theories were developed to account for these
by J. Olds (1960) is generally not as useful due to the differences. More recently, the similarities of conven¬
inherent problems involved in repeated aversive foot tional and central reinforcement have been stressed
shock as well as the apparent analgesic properties of and the so-called differences attributed to procedural
some brain stimulation (Yunger, Flarvey, 8c Lorens, differences (e.g., Trowill, Panksepp, 8c Gandelman,
1973). A technique which has gained much more 1969) and to the more rapid decay of stimuli that con¬
widespread use than either of those already mentioned trol central reinforcement (Lenzer, 1972). Conven¬
was developed by Valenstein and Meyers (1964) and tional and central reinforcement will be compared in
employs a shuttlebox. They have suggested that a this section under the following headings: acquisition,
shuttlebox situation, in which central reinforcement extinction, secondary reinforcement, priming, inten¬
is continuously delivered as long as the animal re¬ sity and persistence, partial reinforcement, and in¬
mains on one side but not on the other, provides a fluence of drive state on self-stimulation. In each case,
measure of reinforcement value which minimizes the we shall attempt to show that when appropriate com¬
influence of activity level and performance capabil¬ parisons are made and equivalent conditions estab-
Gordon Mogenson and Jan Cioe 575

lished, the two kinds of reinforcers do not differ sub¬ and Keesey (1970), however, tried to control for these
stantially. differences by imposing an arbitrary 1-sec delay of cen¬
tral reinforcement following performance of the oper¬
ant (breaking a photobeam over the food cup). De¬
Acquisition
spite such a control the acquisition rate for central
In the pioneering study of J. Olds and Milner reinforcement (both hypothalamic and septal stimula¬
(1954) it appeared that the rate of acquisition of the tion) was superior as compared with sweetened con¬
operant response for central reinforcement was un¬ densed milk. The study therefore suggests a difference
usually rapid compared to acquisition using conven¬ between central and conventional reinforcers—at least
tional reinforcers. Subsequently this observation was for a discrimination task—not attributable to differ¬
confirmed by other investigators experienced with ences in immediacy of delivery. Linholm and Keesey
shaping animals using conventional reinforcers. A suggest, rather speculatively, that this effect may be
comparison of central and conventional reinforcers is related to differences in the duration of central and
not entirely justified, however, since the temporo- conventional reinforcers. Stimuli associated with in¬
spatial relations between response and reinforcement gestion occur following the presentation of food
are not the same (Gibson, Reid, Sakai, 8c Porter, 1965). reinforcement, and as the food passes through the oral-
For example, when water is used as the reinforcer of esophygeal cavities the possibility of conditioning be¬
an operant response the animal must depress the haviors incompatible with the discriminative response
lever, move to the dipper, and then drink: there is increases, since these stimuli are present for a rela¬
usually a spatial separation between the lever and the tively long period of time. Central reinforcement, m
water which results in a delay of the consummatory contrast, usually involves a relatively short duration
act. With central reinforcement, however, there is no of stimulation with more distinctive onset-offset char¬
such delay in reinforcement. Gibson et al. attempted acteristics. Such an interpretation is not inconsistent
to equate the two situations by having animals that with a view that stresses the similarities of central and
were receiving central reinforcement depress a leyer conventional reinforccrs; central reinforcement is a
which introduced a dry dipper that had to be con¬ case in which the experimenter has greater control
tacted to trigger the stimulation, a situation compar¬ than usual over the duration of the stimuli associated
able to the behavioral chaining procedure of Hawkins with reinforcement, and appropriate manipulations
and Pliskoff (1964); either sugar water or brain stim¬ would result in greater similarity in the effects of cen¬
ulation was delivered immediately when the animal tral and food reinforcement (e g., Linholm & Keesey,
touched the dipper, Panksepp and Trowill (1967a) 1968).
made the two situations even more similar by deliver¬
ing chocolate milk via an intraoral fistula immediately
Extinction
after a lever press. It was found in both studies that
when the conventional reinforcement was immediate, Extinction occurs very rapidly when the operant
acquisition was as rapid as that found with central response is reinforced by brain stimulation (Culber-
reinforcement. This finding is not surprising given ton, Kling, 8c Berkley, 1966; Deutsch 8c Howarth, 1963;
the well-established importance of delay of reinforce¬ J. Olds 8c Milner, 1954; Seward, Uyeda, 8c Olds, 1959),
ment in other situations (Renner, 1964). suggesting a second fundamental difference between
It is not clear what role temporospatial factors play central and conventional reinforcers (Figure 2). How¬
in other test situations. For example, for the acquisi¬ ever, the earlier differences may have been due to
tion of a discrimination task there are conflicting difference in procedure; with conventional reinforcers
reports as to whether or not there are significant differ¬ the animal is usually food- or water-deprived, and
ences in acquisition rate between central and conven¬ there is a temporal delay between operant response
tional reinforcers. Kling and his associates (Kling 8c and reinforcement; whereas with central reinforce¬
Berkley, 1968; Kling 8c Matsumiya, 1962; Terman 8c ment the animal is typically not deprived, and rein¬
Kling, 1968) have failed to find significant differences. forcement is presented immediately.
Sadowsky (1969), however, has demonstrated a faster Extinction occurs less rapidly with central rein¬
rate of acquisition of a multiple schedule (i.e., an forcement when animals are food-deprived (Deutsch
operant discrimination) with central reinforcement as 8c DiCara, 1967). Furthermore, extinction with con¬
compared to food pellets. None of these studies at¬ ventional reinforcement occurs more rapidly when
tempted to control for temporospatial differences be¬ animals are not deprived (Panksepp 8c Trowill,
tween central and conventional reinforcers. Linholm 1967b). When central reinforcement and sugar water
576
CENTRAL REINFORCEMENT

uous reinforcement. Unfortunately, however, neither

of these reports specifies the feeding schedule em¬
ployed. The importance of this variable is indicated
by DiCara (1966) as well as DiCara and Deutsch
(1966). It was found that secondary reinforcement
could be obtained using electrical stimulation of a
brain area sensitive to the level of food deprivation
(see J. Olds, 1958a) if the animals were in fact food-
deprived; if food deprivation did not affect the rate of
self-stimulation, no secondary reinforcement was ob¬
tained. It appears, therefore, that another apparent
difference between central and conventional rein¬
forcers is related to the level of deprivation rather
than to the characteristics of the reinforcers.

Priming

J. Olds and Milner (1954) reported that following

the rapid extinction to central reinforcement it was
necessary to deliver noncontingent brain stimulation
“to show that the current was turned on again” (p.
425) before lever pressing resumed. Furthermore, it
Fig. 2. When an operant response is maintained by central has been reported that it is also necessary to deliver a
reinforcement, extinction occurs very rapidly as shown when few of these “free” stimulations at the start of a session
the stimulation voltage is reduced to zero. (From Olds & Milner, to induce the animal to self-stimulate even though no
1954. © 1954 by the American Psychological Association. Re¬
printed by permission.) formal extinction procedure has been introduced.
This anomaly of central reinforcement has been em¬
phasized by Deutsch and Howarth (1963); they have
were used with comparable testing conditions, extinc¬
suggested that self-stimulation involves the activation
tion curves were similar (Gibson et al., 1965). It may
of a drive system as well as a reinforcement system.
be concluded that extinction with central reinforce¬
Priming (i.e., delivery of “free” stimulation), it is sug¬
ment is comparable to extinction with a high-incen¬
gested, induces the proper drive state so as to motivate
tive conventional reinforcer presented with minimal
the animal to lever-press; without the induction of
delay to an animal sated or following a short depriva¬
this drive state the animal fails to start self-stimulat¬
tion period (Trowill, Panksepp, 8c Gandelman, 1969).
ing. Each stimulation, therefore, sets up the appro¬
priate state in the animal so that the following
Secondary Reinforcement
stimulations are reinforcing and thereby responding is
maintained.
The deprivation state of the animal may also be
the variable which accounts for the mixed success of Although there is a certain appeal to this view,
attempts to demonstrate secondary reinforcement us¬ which has led to some interesting experiments by
ing brain stimulation. Seward, Uyeda, and Olds Deutsch and his colleagues, it has been reported that
(1959), Keys (1964), and Mogenson (1965) were all un¬ many animals do not require priming to initiate self¬
able to establish secondary reinforcement, but these stimulation (Trowill, Panksepp, 8c Gandelman, 1969).
studies were conducted with nondeprived animals. Furthermore, animals can be trained to continue
Stein (1958), using a classical conditioning procedure, lever-pressing after periods of nonreinforcement (Gan¬
paired a tone with hypothalamic or septal stimulation delman, Panksepp, 8c Trowill, 1968; Pliskoff, Wright,
and demonstrated that the tone possessed reinforcing 8c Hawkins, 1965), indicating that central reinforce¬
properties due to its association with central stimula¬ ment “is not totally dependent on the time since the
tion. Similarly, Knott and Clayton (1966) were able to last stimulation” (Trowill et al., 1969, p. 291).
confirm Stein’s results in addition to demonstrating
Intensity and Persistence
that partial reinforcement (i.e., intermittent pairing
of tone and central reinforcement) produces a more One of the more obvious features of brain self-stim¬
durable secondary reinforcement than does contin¬ ulation is the astonishing vigor of the animal’s be-
Gordon Mogenson and Jan Cioe 577

havior, which is frequently reflected in very high rates

of responding. For example, Ray, Hine, and Bivens
(1968) reported an average rate of 130 lever presses per
min on a CRF schedule. It should be noted, of course,
that part of the reason for such high rates with central
reinforcement is that so little time is needed to “con¬
sume” the reinforcement—certainly much less time
than is required to chew pellets or drink water. This
leads to an artifact in the comparison of response rates
based on a CRF schedule and in fact is one reason
why it is more desirable to use an intermittent sched¬
ule for such comparisons. The differences in the time
required to “consume” the reinforcement will only
minimally affect such schedules.
Not only do animals emit relatively high response
rates, but they continue to do so for relatively long
periods of time. J. Olds (1958a) recorded 35 lever
presses per min for 26 hr until the animal was ex¬
hausted and went to sleep (Figure 3)? and Valenstein
Fig 4. Water-deprived ratg press lever for electrical stimulation
and Beer (1964) an average rate of 30 responses per
of the hypothalamus in preference to lever-pressing for water
min for a period of 20 days. Even telencephalic stim¬ as shown for current levels in, IV, and V, When the intensity
ulation, which typically satiates more quickly than of brain Stimulation is 16W they show a preference for water.
diencephalic stimulation, does So only after 4 to 8 hr (Based on Morgan k Mogenson, 1966.)

(J. Olds, 1958b). It is this characteristic—relative in-

Noon
fiwmm/m 4 PM satiability—of central reinforcement which is its most
salient feature as compared to conventional rein¬
a Pm
7//A'/zwwwm 8 PM
forcers and which makes central reinforcement such a
useful laboratory procedure.
8 PM
'///////////////// Midnight
The potency of central reinforcement and the de¬
Midnight
'//////////////// 4 AM
gree to which it may be used to control an animaFs
behavior is most clearly demonstrated in test situa¬
4 AM
////J////////A 8 AM tions involving a choice between conventional rein¬
forcers and reinforcing brain stimulation. Rats show a
8 AM
7/J7/J///////Z Noon preference for the central reinforcement when it is in
competition with food (Routtenberg & Lindy, 1965;
Noon
14-4t ~ 4 PM Spies, 1965) or water (Falk, 1961; Mogenson, 1969b;
Morgan 8c Mogenson, 1966; Phillips, Morgan, 8c
4 PM 8 PM
Mogenson, 1970;Stutz, Rossi, & Bowring, 1971), even
8 PM
when food- or water-deprived for 24 or 48 hr (Figure
Midnight
4). Rats ignored food and water (“self-starved”) but
Midnight 4 AM
lever-pressed for electrical stimulation of the medial
forebrain bundle in the region of the hypothalamus
4 AM 8 AM (Routtenberg 8c Lindy, 1965). Animals also tolerate in¬
tense painful stimuli (Olds, 1960; Valenstein 8c Beer,
8 AM
“ " 'MM Noon 1962) or a cold ambient temperature (Carlisle 8c
Snyder, 1970) in order to self-stimulate the hypo¬
Noon
Vi 4 PM thalamus.
Central reinforcement, although usually stronger
Fig. 3. Operant responding continues for very long periods of than conventional reinforcement in competition tests,
time when central reinforcement is used. This rat pressed the does not produce a rigid, inflexible pattern of re¬
lever for more than 24 hr before going to sleep. (From Olds,
sponding. If the incentive characteristics of the alter¬
1958b. (c) 1958 by the American Psychological Association. Re¬
printed by permission.) native are high, the preference for central reinforce-
578
CENTRAL REINFORCEMENT

ment disappears. When the alternative was a highly forcers. This line of argument was extended by Plis¬
palatable saccharine-glucose solution, lever-press rates koff and colleagues (Hawkins 8c Pliskoff, 1964; Pliskoff,
of more than 100 per min were recorded and rats Wright, 8c Hawkins, 1965; Pliskoff 8c Hawkins, 1967),
showed an equal preference for this solution and elec¬ who suggested that a standard food pellet was equiv¬
trical stimulation of the hypothalamus after being alent to several (5-20) brief trains of brain stimulation
water- and food-deprived for 22 hr (Phillips, Morgan, in terms of reinforcement strength. Using such a mul¬
& Mogenson, 1970). The relative preference for central tiple-stimulation technique it has been possible to
and conventional reinforcement depends on the cur¬ maintain FI 1-min and VI 1-min schedules (Brown 8c
rent intensity (Deutsch, Adams, 8c Metzner, 1964; Falk, Trowill, 1970), and when combined with the two-
1961; Morgan 8c Mogenson, 1966), the length of water member chaining procedure that equates temporo-
deprivation (Deutsch, Adams, 8c Metzner, 1964; Mor¬ spatial factors, performance was maintained on FI 10
gan 8c Mogenson, 1966), the duration of the test ses¬ min, FR 200, and differential reinforcement at low
sion, and the palatability of the conventional rein¬ rates (DRL) 180-sec.
forcer (Phillips, Morgan, 8c Mogenson, 1970). Cantor (1971) has also been able to obtain stable
The results of the studies that have just been re¬ behavior with intermittent schedules (e.g., FR 200,
viewed suggest that when deprivation period, inten¬ VR 30, FI 3-min, VI 2-min, and DRL 20-sec), but us¬
sity, and quality of the reinforcer are manipulated ap¬ ing only single stimulations without an equating of
propriately, central and conventional reinforcers are temporospatial factors (Figure 5). The distinguishing
equally effective in controlling operant responses.

Partial Reinforcement

As mentioned earlier in the discussion of method¬

ological problems, partial-reinforcement schedules
may circumvent some of the difficulties involved in
using response rate on a CRF schedule as a measure
of relative reinforcement value. There are, however,
repeated references in the literature suggesting that
performance for central reinforcement is poorer than
for conventional reinforcers on more complicated
schedules. Sidman, Brady, Boren, Conrad, and Schul-
man (1955) reported obtaining successful performance
on a variable-interval (VI) 16-sec schedule and a fixed-
interval (FI) 7-sec schedule. Brodie, Moreno, Malis,
and Brodie (1960) found that most of their monkeys
would not respond to schedules exceeding fixed ratio
(FR) 20, although one anomalous monkey performed
an FR 150; Culberton, Kling, and Berkley (1966) re¬
ported that it took four times as long to train animals
to respond on an FR 10 using central reinforcement
than with water.
Other investigators (e.g., Brown 8c Trowill, 1970;
Cantor, 1971; Pliskoff, Wright, 8c Hawkins, 1965) have
managed to obtain performance for central reinforce¬
ment similar to that for conventional reinforcers by
slight manipulations of the reinforcement procedure.
Sidman et al. (1955) were the first to point out that
performance on these schedules was related to the in¬
tensity of the brain stimulation; they further sug¬ Fig. 5. Sample cumulative response curves for two rats. Rat
294 received a multiple schedule (FR 100, DRL 20-sec) and an
gested that low-current stimulation was comparable to FR 200; rat 297 is on a VI 2-min schedule. Reinforcement is
small amounts of reinforcement and that to obtain indicated by oblique “pips.” Brain stimulation for rat 297 was
stable performance the “amount” of central reinforce¬ signaled on the record at A, unsignaled at B, and again signaled
at C. (From Cantor, 1971. (c) 1971 by the American Association
ment must be properly equated to conventional rein¬ for the Advancement of Science.)
Gordon Mogenson and Jan Cioe 579

feature of his procedure was that the single stimula¬

tion was made predictable by preceding it with a
brief, exteroceptive warning signal; when the animal
depressed the bar a light came on and continued for
1 sec with the central reinforcement automatically
occurring during the last .5 sec of the light. If this
signal was withdrawn after mastery of the schedule,
performance broke down. These results can also be
viewed as enhancing the reinforcement value of cen¬
tral reinforcement since it has been found that pre¬
dictable central reinforcement (i.e., signaled) is more
reinforcing than unpredictable central reinforcement
(i.e., unsignaled) (Cantor Sc LoLordo, 1970, 1972); the
precise reason for this increase in reinforcement value
is not certain.
It seems, once again, that what was initially con¬
sidered a dramatic difference between central and
more conventional reinforcers (i.e., maintenance of
high intermittent schedules) can be viewed as an arti¬
fact of dissimilar procedures.

Influence of Drive State on Self-Stimulation

The effectiveness of food and water and other con¬

.
Fig. 6 Concurrent self-stimulation and. elicited drinking for a
ventional reinforcers is enhanced when the animal is period of 24 hr. The cumulative lever-press responses are shown
deprived to induce a central motive state. Is central in the top panel, in the low panel cumulative water intake (top
line) and cumulative urine output (bottom line) are plotted.
reinforcement similarly enhanced by a central motive
(From Mogenson Sc Stevenson, 1967.)
state produced either by deprivation of the primary
reinforcer (e.g., food) or by electrical stimulation of
certain areas of the brain—primarily the hypo¬ stimulation of brain sites effective for central rein¬
thalamus? forcement (Figure 6). The rate of selhstimulation of
Early studies of the effects of food and/or water “feeding sites77 was increased during food deprivation
deprivation on self-stimulation rates (e.g., Brady, (Goldstein, Hill, & Tcmplcr, 1970; M&rgvdes 8? Olds,
Boren, Conrad, Sc Sidman, 1957) were not conclusive, 1962) and following the injection of insulin which in-
since the enhancement of rate obtained could haye creases appetite (Hoebel, 1969). On the other hand,
resulted from a general effect of deprivation on activ¬ stomach distention, forced obesity, and injections of
ity rather than a specific interaction between self¬ glucagon, procedures which reduce food intake, re¬
stimulation and the deprivation. Olds (1958a), how¬ duced the rate of self-stimulation of these brain sites,2
ever, was able to demonstrate with castrated male rats Similarly, castration reduced, and the administration
that some animals (medial electrode placements) of androgens increased, the rate of self-stimulation of
showed increased self-stimulation rates when food- hypothalamic sites from which copulatory behavior
deprived whereas other animals (more lateral place¬ could be elicited (Caggiula Sc Hoebel, 1966; Hoebel,
ments) showed an increased rate only when injected 1969).
with androgen (male sex hormone). Not only is this a These studies suggest that central drive or motive
clear demonstration that general activity changes were states influence conventional and central reinforce¬
not responsible for the enhanced response rates, but it ment in a similar manner. According to Deutsch
also suggests that the effect of a particular deprivation (1960) the reinforcement system functions in associ¬
or need state was specific to the site of stimulation. ation with a drive system: during self-stimulation,
In subsequent studies it was observed that feeding both the reinforcement system and the drive system
(Hoebel 8c Teitelbaum, 1962; Margules 8c Olds, 1962),
drinking (Mogenson 8c Stevenson, 1966), copulation 2 Water deprivation was reported not to change the rate of
self-stimulation of hypothalamic “drinking sites” (Mogenson,
(Caggiula 8c Hoebel, 1966; Herberg, 1963), and other
1969a). It is not clear why thirst fails to influence self-stimulation
motivated behaviors could be elicited by electrical whereas hunger and sex drives enhance central reinforcement.
 580 CENTRAL REINFORCEMENT

are activated concurrently. Hoebel (1969, 1971) has perimental analysis of behavior or to the better under¬
explained the results described in the previous para¬ standing of the mechanisms of reinforcement.
graph by assuming several reinforcement systems—a In terms of rate of operant responding and prefer¬
hunger-reinforcement system, a thirst-reinforcement ence behavior, direct stimulation of appropriate brain
system, a sex-reinforcement system, etc., each in asso¬ sites is the most potent reinforcer available. It can be
ciation with a particular drive. used without the undesirable contaminating effects of
conventional reinforcers such as the stress of food
deprivation or aversive peripheral shock and the sati¬
SOME IMPLICATIONS OF COMPARING ating effects of the consummatory behavior. Self¬
CENTRAL AND CONVENTIONAL stimulation may be considered a “pure operant”; it
REINFORCERS can be maintained for relatively long periods of time
without excessive ingestion of food or water or other
From the comparisons that have been made thus consequences which inhibit the operant behavior. It
far, it appears that central and conventional rein¬ is the purest form of incentive whose effects have a
forcers are very similar if allowance is made for rapid onset and rapid offset. Therefore, with the use
methodological and parametric differences in the ex¬ of central reinforcement it is possible to have the
periments on which the comparisons are based. This greatest degree of behavioral control and to maintain
is not to suggest that central and conventional rein¬ this control for long periods of time. This characteris¬
forcers are identical. In fact, there is increasing recog¬ tic of central reinforcement has made it a valuable
nition that conventional, peripheral reinforcers them¬ tool in psychopharmacology and has made possible
selves are not identical or even homogeneous (Bolles, the pioneering studies of biofeedback (see this chap¬
1970; Shettleworth, 1972; Staddon 8c Simmelhag, ter’s ^introduction).
1971). How an animal’s behavior is altered by a rein¬ In the hands of experts trained in operant tech¬
forcer, and what and how it learns, is subject to the nology, the purity, potency, and insatiability of cen¬
constraints of its species-specific behavioral organiza¬ tral reinforcement permit a control of human be¬
tion; as Skinner (1966a) pointed out several years ago, havior which is awesome to contemplate. The decision
behavior is a function of phylogenic contingencies as about the legitimate use of central reinforcement for
well as ontogenic contingencies. behavior modification in the treatment of self-destruc¬
Central reinforcement may also not be homogen¬ tive tendencies and other abnormal behaviors or in
eous. It has been suggested that there are subsystems the training of mentally retarded children will not be
of reinforcement (Gallistel 8c Beagley, 1971), and it has an easy one; Michael Crichton’s recent novel Termi¬
also been suggested that they may be related to feed¬ nal Man (1972) highlights the ethical issues associated
ing, drinking, or other drives (Hoebel, 1969) or to with the use of central reinforcement for such pur¬
various species-typical behaviors (Glickman 8c Schiff, poses. However, there is no question that the neuro-
1967). Even if there is a single central reinforcement behavioral technology is available for an impressive
system (as assumed by Stein, 1969), it may be activated degree of control of man’s behavior.
less directly or less strongly depending on the sites of The other unique feature and advantage of central
stimulation. The electrical stimulation may activate reinforcement is that it permits direct access to the
other neural systems besides central reinforcement, mechanisms of the brain that subserve reinforcement.
and these “side effects” may alter its effectiveness in Although this may not appear important to those in¬
reinforcing operant responses.3 terested in reinforcement exclusively from the view¬
In any case, in view of the similarities of central point of behavioral control, it does excite physiologi¬
and conventional reinforcers it may be asked whether cal psychologists and other neuroscientists concerned
there are any advantages in using central reinforce¬ with the neural substrates of reinforcement, motiva¬
ment which permit a unique contribution to the ex- tion, and learning. Self-stimulation experiments have
implicated a number of brain structures which sub¬
3 Reinforcing electrical stimulation may elicit various sorts of serve reinforcement. Sophisticated experiments which
respondents such as integrated consummatory responses, forced
combine electrophysiological, histological, and neuro¬
motor movements, autonomic changes (e.g., heart rate and blood
pressure), as well as endocrine changes (elevated ACTH), some chemical techniques with operant techniques are be¬
of which may be entirely independent of the reinforcing ginning to elucidate the characteristics of the positive
character of the brain stimulation. Such spurious effects are be¬
reinforcement system and its relationship with mem¬
lieved to result from the simultaneous activation of a number
of closely intermingled neural systems by the relatively indiscrete ory, motor, and perceptual systems (Gallistel, Rolls, 8c
electrical stimulation. Greene, 1969; Rolls, 1972; Smith 8c Coons, 1970). This
Gordon Mogenson and Jan Cioe 581

active and interesting field of research promises to tion theory, Olds was suggesting that central rein¬
reveal the circuitry and the workings of the neural forcement is due to the brain stimulation mimicking
pathways that subserve both central and conventional (or activating neural systems concerned with) the re¬
reinforcers. duction of one of the basic biological drives, which in
At the present time the exact neural basis for cen¬ turn reinforces the operant response. However, a few
tral reinforcement is still uncertain in spite of vig¬ years later observations were made which suggested
orous investigation for nearly two decades. This state just the opposite: feeding (Hoebel 8c Teitelbaum,
of affairs has prompted Crow (1972c) to comment, 1962; Margules 8c Olds, 1962), drinking (Mogenson 8c
“As a bridge between psychological theory and neuro¬ Stevenson, 1966), and copulatory (Caggiula Sc Hoebel,
physiology the value of the discovery [of central rein¬ 1966; Herberg, 1963) behavior were elicited from the
forcement] is limited by the fact that the anatomical same electrode sites that were highly effective for self¬
pathways which must be activated to yield the response stimulation.
have not been identified” (p. 414). However, new de¬ Initially it seemed paradoxical that the animal
velopments in mapping neural pathways using histo- pressed a lever to stimulate its hypothalamus making
chemical techniques (to provide a chemical neuro¬ it seek food or water. Deutsch (1960) proposed that
anatomy of the hypothalamus and limbic system) may there is a distinctive reinforcement system of the brain
lead to the identification of the neural substrates of activated during §elf-stimulation and that it functions
central reinforcement (discussed below), in association with a drive system. He did not see any
paradox, since he assumed that the reinforcement
system and the drive system had to be activated con¬
THE NATURE OF CENTRAL currently. In other words, drive induction, produced
REINFORCEMENT either in a conventional manner (e.g„ water depriva¬
tion) or by electrical stimulation of the brain, is a
An observer who sees an animal self-stimulate its necessary condition for reinforcement. Glickman and
brain for the first time is usually very impressed with Schiff’s (1967) attempt to resolve this paradox in
this fascinating phenomenon, Typically he asks, “Why volved a departure from drive-reduction theory and
does the animal do that?” What is the nature and the a different view of both the nature of reinforcement
mechanism of central reinforcement? and the mechanisms by which feeding and drinking
A number of explanations of self-stimulation have are elicited by electrical stimulation of the brain.
been proposed during the last 20 years. However, in They pointed out that the behaviors elicited from the
spite of intensive and often ingenious research it is not same brain sites as self-stimulation (feeding, drinking,
possible to say with certainty whether any of these gnawing, copulating, etc.) are species-typical responses
interpretations is correct. In this final section we first that contribute to adaptation to the environment and
consider two of the most popular views and then to survival. In the course of biological evolution an¬
present some speculations about the mechanisms of imals endowed with the neurological apparatus to
central reinforcement. make these responses will survive and reproduce* Ac¬
cording to Glickman and Schiff, reinforcement results
from the activation, either by natural stimuli or by
Central Reinforcement from the Viewpoint of Drive
electrical stimulation of the brain, of neural pathways
Theory and Response Reinforcement
that initiate species^typical behaviors. They assumed
When self-stimulation of the brain was discovered, that there is a single system which has conventional
and for several years thereafter, drive-reduction theory reinforcement and drive properties. In some ways,
was the dominant theory of reinforcement and learn¬ their proposal is a new version of Sheffield’s (Sheffield,
ing. It is not surprising that attempts were made to Roby, 8c Campbell, 1954) consummatory response
explain the self-stimulation phenomenon from this theory, and they avoid the paradox referred to above
theoretical point of view (Miller, 1960), and studies of by avoiding the use of terms like hunger drive or
central reinforcement were undertaken with this the¬ drive reduction. Unlike Deutsch, they do not assume a
ory in mind (Deutsch & Howarth, 1963). distinctive drive system.
In one of his earlier papers J. Olds (1956) proposed Although self-stimulation and elicited drinking have
that positively reinforcing brain stimulation “must been observed to occur concurrently (Mogenson,
excite some of the nerve cells that would be excited 1969a), the elicited drinking is not necessary for cen¬
by satisfaction of the basic drives—hunger, sex, thirst, tral reinforcement. The administration of ampheta¬
and so forth” (p. 15). In accordance with drive-reduc¬ mine, which markedly reduced drinking elicited by
582
CENTRAL REINFORCEMENT

hypothalamic stimulation, did not reduce, but rather

increased dramatically, the rate of self-stimulation of
the same hypothalamic site (Mogenson, 1968). Fur¬
thermore, by varying the parameters of stimulation,
the “drive-eliciting” and central reinforcement effects
of hypothalamic stimulation could be dissociated.
Using a train of hypothalamic stimulation .5 sec or
longer both self-stimulation and elicited drinking
were observed, whereas with a train duration of .1—.2
sec self-stimulation occurred in the absence of elicited
drinking (Figure 7). Lesions have also been used to
dissociate the drive-inducing and the central reinforce¬
ment properties of hypothalamic stimulation; it has
been shown that lesions of the substantia nigra selec¬
tively disrupt the elicited carrying of objects that oc¬
curred concomitantly with self-stimulation in a shut-
tlebox but did not interfere with self-stimulation per
se (Phillips, 1973). Finally, it should be noted that the
current intensity threshold to elicit feeding and drink¬
ing is below the threshold for self-stimulation when Duration of stimulation (sec)
stimulus train durations of more than 1 sec are used
Fig. 7. The effect of varying the duration of hypothalamic stim¬
(Coons 8c Cruce, 1968; Huston, 1971, 1972; Miller, ulation on self-stimulation and elicited drinking. Elicited drink¬
1960). ing occurs only when the duration is greater than .4-.5 sec. The
fastest rates of self-stimulation occurred with the shortest dura¬
tion of stimulation. (From Mogenson & Stevenson, 1966.)
Central Reinforcement from the Viewpoint
of Incentive Motivation
zles, maintains that “only in the human brain can
Electrical stimulation of the hypothalamus may thinking activate the limbic pleasure areas” (p. 22).
elicit drinking and feeding not by activating systems It appears that in man and other animals with com¬
for internal deficit signals, as suggested in the previ¬ plex brains, reinforcement and motivated behaviors
ous section, but by mimicking incentive stimuli (or depend on natural processes concerned with higher
“appetite-whetting” stimuli) associated with drinking cognitive functions as well as those that process intero¬
and feeding; it is then not a paradox that animals ceptive, exteroceptive, and proprioceptive inputs.
self-stimulate “feeding sites” and “drinking sites.” Cer¬
tain stimuli, such as sweet and salty solutions, are
Central Reinforcement as the Interaction of Incentive
particularly effective reinforcers (Pfaffman, 1960;
Stimuli and a Central Motive State
Young, 1959), even in the absence of biological deficits
and needs; a sensory stimulus “can function as a rein¬ It is now generally acknowledged that there was
forcer in its own right” (Pfaffman, 1960, p. 255). Pfaff¬ too much emphasis in the past on the role of internal,
man has suggested that central reinforcement is due deficit signals for the initiation of motivated behaviors
to the activation of neural pathways that normally and the so-called drives. A number of investigators
transmit such incentive motivational stimuli. have emphasized the importance of identifying and
The results of experiments by Pfaffman and Young, studying the influence of the various controlling stim¬
as well as those from peripheral self-stimulation experi¬ uli for self-stimulation (Lenzer, 1972) and for other
ments (Campbell, 1971), suggest that brain self-stimula¬ motivated behaviors (Flynn, Vanegas, Foote, 8c Ed¬
tion results from activating pathways of the brain that wards, 1970; Roberts, 1970). There is an increasing
transmit signals from natural reinforcers. These in¬ acceptance of the view that motivated behavior in
clude inputs from smell and taste and other exterocep¬ general and reinforcement in particular depend on
tive stimuli, but also may include proprioceptive and external as well as internal stimuli (Bindra, 1969;
interoceptive inputs. In higher animals “central reward Mogenson 8c Huang, 1973). One of the best examples
pathways” are also activated by cognitive processes. of the importance of this interaction of internal and
Campbell (1971), after noting the pleasure one gets external stimuli is the mislabeling of drive states that
from mathematics, science, chess, or crossword puz- are initiated by internal stimuli in the absence of
Gordon Mogenson and Jan Cioe 5 83

appropriate environmental signals as reported by motivational stimuli when they occur in the presence
Schachter (1967). It appears that the central motive of a central motive state—for example, following a
state that subserves a particular motivated behavior period of water or food deprivation. Stimulation of
depends on both internal, deficit signals and external central reinforcement sites induces a motive state—it
signals. elicits feeding (Hoebel & Teitelbaum, 1962) and drink¬
Support for this view comes from studies of the ing (Mogenson 8c Stevenson, 1966)—so that the animal
effects of sensory stimuli on brain self-stimulation. is more responsive to appropriate external stimuli
Rats self-stimulated faster when induced by the brain (such as the sight of water or smell of food) and its
stimulation to drink water, apparently because of the behavior is appropriate to the situation. The brain
oral stimulation (Figure 8; see also Mogenson & Kap- stimulation also activates neurons concerned with
linsky, 1970), and when the sensory stimulation was the transmission and processing of external, motiva¬
further increased by adding saccharine to the water tional stimuli (Pfaffman, 1960). The rate of self-stimu¬

they self-stimulated at a still faster rate (Phillips 8c lation, and apparently the strength of central rein¬
Mogenson, 1968): external incentive stimuli enhanced forcement, is increased either by the presence of
central reinforcement. Even when the external stimu¬ appropriate external motivational stimuli (see previ¬
lus is not response-contingent, self-stimulation is in¬ ous paragraph) or by increasing the central motive
creased, as demonstrated when rats self-stimulated state (by food deprivation—for example, J, Olds,
with a background odor present (Phillips, 1970). It 1958a). It follows, therefore, that the event designated
appears, therefore, that the external stimulus is not reinforcement is the occurrence of external motiva¬
enhancing response reinforcement but rather is en¬ tional stimuli in the presence of a central motive
hancing or interacting with the central motive state state.4 This combination occurs when an animal en¬
induced by the electrical stimulation. Furthermore, counters food or water in the presence of an appropri-
when the current intensity is below threshold, rats ate drive state, and a similar combination is produced
will not respond unless water is available, so that oral apparently by stimulation of certain brain sites which
stimulation accompanies self-stimulation (Mendelson, yield central reinforcement.
1967; Mogenson, Morgan, Phillips, &* Stevenson, 1968). Rolls (1972) has obtained unit activity data which
External stimuli are frequently most effective as can be interpreted to support the view that the rein¬
forcement of self-stimulation is primarily associated
with incentive, motivational stimuli. Rolls reported
that with self-stimulation from both the lateral hypo¬
thalamus and septum, neurons in the amygdala were
activated. If stimulus-bound feeding and drinking
could be elicited from the hypothalamic placements,
not only was there neural activation of the amygdala
but in addition cells in the midbrain were activated.
This midbram activity did not occur from sites which
maintained self-stimulation only. Rolls has suggested
that these midbrain neurons are associated with in¬
creased “arousal”—or, in our terms, they contribute to
the central motive state. One could speculate further
that activation of the neurons of the amygdala may
be involved primarily with incentive stimuli, given
the amygdala’s involvement with sensory input,
whereas the midbrain structures are more involved
with the induction of a central motive state. It could
be argued, therefore, that since self-stimulation oc¬
curred only when amygdaloid neurons were activated,
central reinforcement depends primarily on activating
Time
neural elements concerned with incentive motivation.
Fig. 8. A rat self-stimulates and is induced by the hypothalamic
stimulation to drink water. At the lower current intensity (15 4 These views are similar to those of Bindra (1968), who has
fiA) the rate of self-stimulation is faster when the animal is in¬ made a comprehensive historical analysis of theories of conven¬
duced to drink. Self-stimulation is facilitated by the oral stimu¬ tional reinforcers and the changing emphasis in the treatment
lation. (From Mogenson & Morgan, 1967.) of motivation.
 584
CENTRAL REINFORCEMENT

It is difficult, however, on the basis of these data to ternal stimuli. External stimuli that are involved in
exclude the possibility that such reinforcing brain incentive motivation have discriminative properties,
stimulation might also influence the central motive but they also, along with internal, deficit stimuli, in¬
state so that the magnitude of the central reinforce¬ duce a central motive state. We are defining reinforce¬
ment increases as the electrical stimulation has a ment, as indicated above, as the occurrence of incen¬
greater effect on the central motive state. The data tive stimuli in the presence of a central motive state.
reviewed earlier which demonstrate increases in self¬ We shall first describe how this model applies to a
stimulation rates with appropriate changes in the conventional reinforcer (i.e., water) and then go on to
central motive state would of course support the in¬ consider central reinforcement.
volvement of the central motive state in central rein¬ When an animal is deprived, water-deficit signals
forcement. initiate behavioral activation and increased explora¬
tion. According to the model, exploratory activity re¬
A Model for Central and Conventional sults from these drive stimuli, increasing arousal
Reinforcement which influences the motor facilitator, and from in¬
creased input to the response switch mechanism, which
If central and conventional reinforcement are sim¬ accounts for the variability of behavioral response. If
ilar, as has been suggested in earlier sections, it should the animal encounters water, which is an appropriate
be possible to deal with both in relation to the same incentive stimulus, and drinks, the response-hold
theoretical model. This will be attempted in this sec¬ mechanism- is activated and it in turn inhibits the
tion. response switch mechanism. The animal will then
The model shown in Figure 9, which we have continue to drink, since this behavior keeps it in the
selected for purposes of illustration, is a slight modifi¬ presence of the incentive stimulus, until the drive
cation of the one proposed by Milner (1970). It as¬ stimuli are reduced by water intake (or inhibited by
sumes, as indicated above, that reinforcement results short-term satiety signals) and the influence of the
from an interaction of appropriate internal and ex- arousal mechanism on the motor facilitator and the
response-hold mechanism disappears.
How is central reinforcement explained by this
model? One suggestion is that central reinforcement
results from the simultaneous activation of pathways
that normally transmit incentive stimuli and drive
stimuli (pathways designated 1 and 2 in Figure 9), a
proposal that is similar to Deutsch’s hypothesis dis¬
cussed earlier. Initially the animal presses the lever by
chance and turns on the brain stimulation; the re¬
sponse-hold mechanism is activated and it quickly
acquires the lever-press response. The response switch
is then inhibited and the self-stimulation behavior
may continue for a very long time. If extinction oc¬
Fig. 9. A model of central and conventional reinforcement. curs rapidly, as is often the case when the brain stimu¬
Reinforcement is assumed to result from the interaction of a
central motive state with appropriate incentive stimuli (see lation ceases, it is because there is no longer input to
text). A central motive state is induced by internal deficit the motor facilitator and the response-hold mech¬
(pathway 2) stimuli and by external incentive (conditioned and anism.
unconditioned, pathway 1) stimuli. Many incentive stimuli have
at least a weak arousing effect (pathway 3) and also influence The strongest central reinforcement is obtained
the central motive state. Central reinforcement according to the with electrical stimulation of sites of the hypothala¬
model is due to the activation of pathway 1 or pathways 1 and mus from which drinking, feeding, and other mo¬
2 concurrently. When the response-hold mechanism is activated
by appropriate inputs along the incentive pathway the response tivated behaviors are elicited. As indicated earlier, the
switch mechanism is inhibited, the response program generator rate of self-stimulation is increased during elicited
sends a fixed command to the motor facilitator, and a stable drinking (Mogenson 8c Morgan, 1967) and is increased
behavior (Rj, such as lever pressing for brain stimulation)
occurs. If this behavior is accompanied by appropriate stimuli further by adding saccharine to the water (Phillips &
(e.g., water or saccharine solution in rats that self-stimulate and Mogenson, 1968). The rate of self-stimulation is also
drink concurrently—Mogenson & Morgan, 1967), the central frequently increased by food deprivation (J. Olds,
reinforcement is facilitated (pathway 5). On the other hand,
drive-reducing stimuli (gastric distention, elevated blood sugar) 1958a). Assuming that pathways 1 and 2 are being
reduce reinforcement (pathway 6). (Adapted from Milner, 1970.) activated during concurrent self-stimulation and drink-
Gordon Mogenson and Jan Cioe 585

ing or feeding, we suggest that the oral stimulation Dopamine

and the deprivation enhance central reinforcement by
increasing the inputs along pathways 1 and 2, re¬
spectively.
Another possibility is that the central reinforce¬
ment is due to activation of only those pathways that
transmit incentive stimuli (pathway 1). Certain con¬
ventional incentive stimuli seem to be effective rein¬
forcers even in the absence of internal, deficit (drive)
stimuli (Pfaffman, 1960). Perhaps this is because they
have at least a weak arousal effect (see postulated path¬
way 3), thereby influencing the motor facilitator as
Noradrenaline
well as the response-hold mechanism.

Central Catecholamine Pathways

and Self-Stimulation

The region of the medial forebrain bundle as it

passes through the lateral hypothalamus (LH) has
been recognized for some time as the “hotspot” for
self-stimulation (J. Olds & M. E. Olds, 1964). For a
time it appeared that this might be because of the
association of this region with drive systems such as
feeding, drinking, etc. (Hoebel, 1969); Mogenson,
1969a). More recent evidence, however, suggests that Tig. 10. Catecholaminergic pathways demonstrated with the
the potent central reinforcement from stimulation of histofluorescence technique which project from the midhrain
and brainstem through the hypothalamus to limbic forebrain
this region results from activating ascending nor¬
structures (tub. olfactorium, bulbus olf., septum, n. accumbens,
adrenergic and dopaminergic neurons which are n. amygdaloid centralis, hippocampus), basal ganglia (n, cauda-
densely concentrated here (German 8c Bowden, 1974). tus), and cerebral cortex. The locus coeruleus is at the site of
the A6 cell bodies as shown in B. The substantia nigra and the
The noradrenergic and dopaminergic pathways are
interpeduncular nucleus are in the region of the A3 and A10
shown in Figure 10. Using histofluorescence tech¬ cell bodies shown in A. (From Ungerstedt, 1971»)
niques a group of Swedish workers demonstrated that
these neural pathways project from the midbrain and
lower in the brainstem through the region of the Experiments involving neuropharmaeologieal ma¬
lateral hypothalamus to the basal ganglia, limbic nipulations of these eateeholaminergic pathways also
forebrain structures, and cerebral cortex (Fuxe, Hok- influence self-stimulation, providing additional evi.
felt, & Ungerstedt, 1970; Ungerstedt, 1971), Self-stimu¬ dence for their role in central reinforcement. Drugs
lation results from stimulation of ascending nor¬ such as alpha-methyl-^ tyrosine, which inhibit cate¬
adrenergic neurons in the medial forebrain bundle cholamine synthesis, and reserpine, which deplete
(Stein, Wise, & Berger, 1972) as well as from stimula¬ catecholamine stores, cause a decrement in self-stimu¬
tion of the locus coeruleus, the site from which fibers lation (Cooper, Black, 8c Paolino, 1971; J. Olds, 1936;
of the dorsal noradrenergic bundle originate (Crow, Poschel & Ninteman, 1966; Stein, 1962). On the other
1972b; Ritter 8c Stein, 1973) and from stimulation of hand, drugs such as monoamine oxidase inhibitors,
the ventral noradrenergic neurons (Ritter 8c Stein, which increase catecholamine levels, and amphetamine
1974). Self-stimulation has also been obtained from a or cocaine, which increase the synaptic release and
number of sites known to contain dopaminergic block the reuptake of catecholamines, facilitate self¬
neurons, such as the substantia nigra (Crow, 1972a; stimulation (Crow, 1970; Domino 8c Olds, 1972; Horo¬
Phillips 8c Fibiger, 1973; Routtenberg 8c Malsbury, witz, Chow, 8c Carlton, 1962; M. E. Olds, 1970; Phillips
1969) and the area adjacent to the intrapeduncular 8c Fibiger, 1973; Poschel, 1969; Stein, 1962, 1964).
nucleus (Dreese, 1966). Furthermore, it has been re¬ Stein and Wise (1969) maintained that noradrenergic
ported that dopamine is released from dopaminergic neurons have an exclusive role in self-stimulation.
terminals during self-stimulation (Arbuthnott, Crow, However, more recent neuropharmaeologieal evidence
Fuxe, Olson, 8c Ungerstedt, 1970). has implicated dopaminergic neurons (Lippa, Antel-
 586
CENTRAL REINFORCEMENT

man, Fisher, 8c Canfield, 1973). Convincing evidence Howard, 8c Leahy, 1971; Phillips, unpublished ob¬
for the role of dopaminergic neurons has been pro¬ servations; Stein 8c Wise, 1971).
vided in experiments utilizing dopamine blockers and The anatomical, neuropharmacological, and lesion¬
agonists. Pimozide and haloperidol, relatively selec¬ ing studies discussed in the previous paragraphs show
tive blockers of dopaminergic receptors when given in that central reinforcement is associated with stimula¬
low doses, cause a marked reduction in self-stimulation tion of brain catecholamine (CA) pathways. We now
(Liebman 8c Butcher, 1973; Wauquier 8c Niemegeer, consider the implications of this relationship for
1972). Rats will lever-press to self-administer, via a understanding the mechanisms of brain self-stimula¬
jugular catheter, a central dopamine-receptor stimu¬ tion.
lant, apomorphine. The self-infusion of apomorphine As indicated above, the most popular explanations
apparently reinforces lever pressing because it activates of brain self-stimulation were from the viewpoint of
dopaminergic synapses of the “central reinforcement drive and incentive theories of motivation. It is not
system” (Baxter, Glickman, Stein, 8c Scerni, 1974). surprising, therefore, that the first attempt to deal
Recently there has been a good deal of interest in with the role of CA pathways in central reinforce¬
the physiological and behavioral effects of lesioning ment was in terms of drive and incentive systems.
the noradrenergic and dopaminergic pathways. Since Crow (1973) suggested that stimulation of the dopa¬
such studies suggest the functional relevance of these minergic (DA) nigrostriatal pathway is reinforcing
pathways and might provide some clues about their because it is involved in the processing of incentive
role in self-stimulation, they will now be reviewed stimuli. Olfactory projections go to the habenular
briefly; for a more complete review of this literature nucleus and then to the interpeduncular nucleus
see Mogenson and Phillips (1975) and Strieker and where, according to Crow, they influence the dopa¬
Zigmond (1975). minergic neurons (A9 cells shown in Fig. 10). Self¬
When the dorsal noradrenergic and the nigrostriatal stimulation of the DA pathway results from the
dopaminergic pathways are damaged or destroyed activation of fibers that transmit olfactory incentive
with electrolytic lesions or more selectively with a stimuli. Crow hypothesized also that the dorsal nor¬
neurotoxin, 6-hydroxydopamine, the tonic alertness adrenergic (NA) bundle subserves central reinforce¬
and phasic behavioral arousal of the animal are dis¬ ment based on drive reduction. Anatomical evidence
rupted (Chu 8c Bloom, 1973; Jones, Bobillier, Pin, 8c is again cited in support of the proposal, particularly
Jouvet, 1973). The animal is drowsy and somnolent, the close relationship between dorsal NA neurons in
suffers from sensory neglect and disturbance of sen¬ the locus coeruleus (A6 cells shown in Fig. 9) and the
sorimotor integration, and has difficulty in initiating nucleus of the tractus solitarius which receives gusta¬
behavior (Fibiger, Zis, 8c McGeer, 1973; Marshall 8c tory input. Since gustatory stimuli are closely associ¬
Teitelbaum, 1973; Strieker 8c Zigmond, 1975; Unger- ated with the termination of gustatory behavior. Crow
stedt, 1971). Included in the behavioral deficits is a suggests that fibers of the dorsal NA pathway are
disturbance in feeding and drinking. The dorsal nor¬ activated by stimuli associated with drive reduction.
adrenergic pathway which projects diffusely to the Crow’s proposal to account for the role of CA path¬
cerebral cortex inhibits cortical inhibitory neurons, ways in self-stimulation emphasized sensory stimuli.
thereby causing cortical activation and behavioral Although olfactory and gustatory signals and path¬
arousal (E. Roberts, 1974). The nigrostriatal dopami¬ ways are stressed, the model might also be extended
nergic pathway projects to extrapyramidal motor to deal with visual, auditory, proprioceptive, intero¬
structures. When these pathways are damaged there ceptive, and other biologically significant stimuli
are deficits in behavioral arousal, extrapyramidal known to be involved in reinforcement (Mogenson &
motor functions, and affect (Strieker 8c Zigmond, 1975). Phillips, 1975). Crow’s formulation will appeal to
If these catecholamine pathways have an essential those who stress the sensory side of the nervous system
role in central reinforcement, as suggested earlier, and for whom the concepts of drive reduction and
damage or destruction of the pathways should cause incentive have special significance. However, it does
a severe decrement in self-stimulation. This has been not deal with the relationship of the CA pathways to
demonstrated using 6-hydroxydopamine, the neuro¬ the neural systems for the initiation and motor con¬
toxin which selectively destroys noradrenergic and trol of behavior, a serious shortcoming for those who
dopaminergic neurons. When central catecholamine favor the views of Glickman and Schiff (1967) or Mil¬
neurons are destroyed by administering 6-hydroxy¬ ner (1970) rather than those of Deutsch and Howarth
dopamine into the ventricles, self-stimulation of the (1963) or Pfaffman (1960). In the final section we con¬
lateral hypothalamus is markedly reduced (Breese, sider the role of the CA pathways in brain self-stimu-
Gordon Mogenson and Jan Cioe 587

lation from the viewpoint of their anatomical and ure 11). Damage to these pathways, as indicated in
physiological relationships with neural systems con¬ the previous section, disrupts feeding, drinking, and
cerned with the motor control of behavior. other goal-directed behaviors; one of the most promi¬
nent deficits is in the initiation of responding. Al¬
though there has been considerable interest in the
Central Reinforcement as the Inhibition by
role of these pathways in feeding and drinking, be¬
Catecholamine Pathways of the Neural
cause of their strategic location at the interface be¬
Systems for the Motor Control of Behavior
tween neural systems concerned with the “intention
The DA nigrostriatal pathway and the dorsal NA to respond” and those concerned with “motor con¬
pathway project to the striatum, the cerebral cortex, trol” it is likely that they contribute to a variety of
and the cerebellum, structures which make important behaviors.
contributions to the motor control of behavior (Fig¬ Motor control of behavior depends on complex in-

Fig. 11. The DA nigrostriatal pathway and the dorsal NA pathxvay project to key struo
tures involved in the initiation and motor control of behavior—the striatum, the cerebral
cortex, and the cerebellum.
(A.) Parallel systems project from the neocortex to the lower motor neurons of the spinal
cord via the striatum and cerebellum. The striatum, which is in a position to sample activ¬
ity in the motor cortex and cerebellum (via projections through the intralaminar nucleus
of the thalamus), the limbic system, and the sensory and association cortex, has an
important role in the translation of the “intention to respond” into the “command sig¬
nals” for motor control. The cerebellum contributes to error detection through pro¬
jections to the motor cortex (via the ventrolateral thalamus) and provides subroutines
for the execution of the intended movement. (Based on Kemp 8c Powell, 1971.)
(B). The DA nigrostriatal pathway and the dorsal NA pathway exert inhibitory effects
on the striatum, cerebral cortex, and cerebellum. According to Milner (1975), these
inhibitory effects are the key to the central reinforcement from direct stimulation of
these pathways, since these effects suppress response-inhibitory systems permitting the
(reinforced) ongoing behavior to be protected and to continue. (From Mogenson &
Phillips, 1975.)
 588
CENTRAL REINFORCEMENT

terrelationships among a number of brain structures tributes this to the activation of a response-inhibitory
(Figure 11). It is not our purpose to consider the system in the cerebral cortex.6 The interruption of
dynamic functioning of these structures and pathways ongoing behavior by sensory stimuli does not occur,
(for further details see Kemp & Powell, 1971; Mogen- however, if the behavior is reinforced either by a con¬
son 8c Phillips, 1975). Instead we are concerned with ventional reinforcer or by central reinforcement. Ac¬
the possibility that the anatomical and functional cording to Milner, activation of the inhibitory cate¬
relationships of the DA and NA pathways with neural cholamine pathways, either by direct stimulation in
systems for motor control may provide some clues the self-stimulation procedure or due to the central
about their role in central and conventional rein¬ effects of conventional reinforcers, suppresses the
forcement. Inhibitory effects are exerted by the DA response-inhibitory system so that the ongoing be¬
nigrostriatal pathway and the dorsal NA pathway on havior is protected and maintained.
the striatum, cerebral cortex, and cerebellum (Conner, In an earlier section we emphasized a number of
1970; Curtis 8c Crawford, 1969; Segal 8c Bloom, 1974). similarities between reinforcement obtained with
This may be surprising, since these pathways make conventional reinforcers and that from direct stimu¬
important contributions to cortical activation and be¬ lation of the brain. Milner’s ingenious hypothesis sug¬
havioral arousal (Jones et al., 1973; Jouvet, 1972). gests a mechanism common to both conventional rein¬
However, the electrophysiological and behavioral data forcement and central reinforcement. Furthermore, it
are easily accounted for by the proposal that DA and provides a role for drive and incentive stimuli as ac¬
NA axon terminals inhibit inhibitory interneurons, tivators of the catecholamine “reward” neurons and
thereby causing the disinhibition of neurons con¬ at the same time links these neurons to neural sys¬
cerned with cortical activation and motor control of tems for motor control. The hypothesis appears to
behavior (E. Roberts, 1974). According to Roberts, the provide a fruitful integration and synthesis of the
central nervous system consists of “genetically pre¬ previous hypotheses of central reinforcement that
programmed circuits which are released for action by stressed sensory input (Deutsch 8c Howarth, 1963;
neurons (command neurons) that are strategically lo¬ Pfaffman, 1960) and response elicitation (Glickman 8c
cated at junctions in neuronal hierarchies dealing Schiff, 1967).
with both sensory input and effector output” (p. 127).
It has been suggested by several investigators (e.g.,
Glickman 8c Schiff, 1967) that such preprogrammed SUMMARY
circuits for biting, chewing, swallowing, and other
components of goal-directed behaviors are represented The discovery of brain self-stimulation in the early
in the brainstem. Roberts suggests that “segmental 1950s aroused a good deal of enthusiasm among be¬
command neurons, like the circuits they control, are havioral and neural scientists interested in the modifi¬
largely inhibited from above, and that a decrease in cation of behavior by experience and the underlying
inhibition allows command neurons to fire, thereby
releasing the preprogrammed circuits over whose ac¬ 6 The increased activity and greater persistence of exploratory
tivity they preside” (p. 128).5 behavior following lesions of certain regions of the cerebral
cortex and hippocampus indicate that these structures are part
Milner (1975) has suggested that the inhibitory
of a response-inhibitory system. Milner (1976) suggests that
effects of the DA and NA neurons, and specifically the “arousal of any cortical activity produces a transient response
disinhibition they exert on neural systems that initiate inhibition” so that “the immediate effect of the presentation
of a new stimulus is to interrupt the on-going response.” Ac¬
and control behavior, are the key to understanding
tivation of the catecholamine pathways protects these responses
self-stimulation and central reinforcement. His pro¬ from being suppressed, however, by inhibiting the response-
posal will now be presented. inhibitory system. “Thus, food, for a hungry animal, not only
elicits approach, chewing, swallowing, and so on, but sends in¬
It is well known that ongoing behavioral responses
hibitory input to [the response-inhibitory system] to ensure that
may be interrupted by a novel stimulus. Milner at- cortical outflow does not interfere with the performance.” Simi¬
larly, direct electrical stimulation of the catecholamine pathways
could prevent cortical outflow from disrupting the responses
5 Roberts (1974), after reviewing the relevant evidence, hy¬ that preceded the central stimulation so that the self-stimulation
pothesizes that fibers of the dorsal NA pathway inhibit behavior continues.
7-aminobutyric acid (GABA) interneurons in the upper layers of Milner’s views could be easily integrated with those of
the cerebral cortex, thereby influencing electroencephalographic Roberts (1974). Although Milner limits his analysis to disinhibi¬
cortical arousal and behavior. He also suggests that inhibitory tion of cortical functions, it could be readily extended to include
GABA neurons in the striatum, which tonically inhibit pre¬ disinhibition of the striatum by the inhibitory effects of DA
programmed neural circuits for patterned postural control and nigrostriatal neurons on striatal inhibitory interneurons. In other
movements, could be inhibited by fibers of the DA nigrostriatal words, the response-inhibitory system postulated by Milner could
pathway. be represented in the striatum as well as the cerebral cortex.
Gordon Mogenson and Jan C/oe 589

modifiability of the nervous system. The potent rein¬ nigrostriatal dopamine pathway transmits incentive
forcement of various operant responses by electrical stimuli, that the dorsal noradrenalin pathway trans¬
stimulation of the hypothalamus, limbic structures, mits drive stimuli, and that these form the basis of
and other regions of the brain seemed to provide a central reinforcement. An alternative hypothesis has
means of studying directly the basic mechanisms of been derived from the observation that these cate¬
reinforcement and learning. A number of the subse¬ cholamine pathways exert inhibitory effects on the
quent studies appeared to demonstrate, however, some cerebral cortex, striatum, and other neural structures
important differences between central and conven¬ concerned with the initiation and motor control of
tional reinforcers—in the rapidity of response acquisi¬ behavior. The hypothesis was considered that stimula¬
tion, in the persistence and relative insatiability of tion of catecholamine pathways is reinforcing because
responding, in extinction, when used in partial rein¬ they exert suppressive effects on response-inhibitory
forcement programs, and when presented with secon¬ systems of the brain. This hypothesis not only ac¬
dary reinforcers. Some of these differences appeared counts for conventional reinforcement but also inte¬
to be anomalies when considered in relation to drive- grates and synthesizes previous hypotheses of central
reduction theory, the most widely accepted and influ¬ reinforcement that emphasized sensory input and
ential theory of motivation and reinforcement at that response elicitation.
time. Also, some of the experimental findings, such as
the elicitation of drinking and feeding from self¬
stimulation sites, seemed paradoxical when considered APPENDIX A
from this theoretical point of view; for example, why
should an animal perform a response which makes it Some examples of studies of self-stimulation from
thirsty when, according to the theory, reinforcement various sites and species
results from drive reduction?
In recent years, there has been less emphasis on Pyriform cortex: Rat—Olds 8c Olds, 196St Cat-O'Donohue

drive theory and increasing interest in incentive moti¬ 8c Hagamen, 1967-

vation. Evidence from self-stimulation experiments Cingulate gyrus: Rat-Olds 8c Olds, 1963: Cat-O’Donohue
has contributed to this trend, which has led to a com¬ it Hagamen, 1967-
pletely different way of thinking about self-stimula¬ Hippocampus: Rat-Olds & Olds, 1963; Ursin, Ursin, 8c Olds.
tion and central reinforcement, as well as about the 1966: Gat-O’Donolme & Hagamen, 1967.
broader field of motivation and conventional rein¬ Fornix; Rat-Olds & Olds, 1963; Gat-O’Dvnvluw k Hsga-
forcement. At the same time, it was demonstrated that men, 1967; Gerbil-Rovtt?nberg & Kramis, 1967= Rabbi t-
the differences between central and conventional rein¬ Brunner, 1966*

forcers were not really of a fundamental nature but Septum: Rat-Olds it Olds, 1963: Cat-O'Donohue it Haga¬
men, 1967; Wilkinson it Peele, 1963: Monkey-Bursten
were due primarily to procedural differences, particu¬
it Delgado, 1968; Lilly, 1967* Man^Heath, 1963; Bishop,
larly in the delay of reinforcement and the depriva Elder, it Heath, 1963: Rabbit-Campbell, 1968.
don state of the animal, This recent concern with the
Olfactory tubercle; Rat-Olds k Olds, 1965; Gat-O*Dunv-
similarities between central and conventional rein¬ hue 8c Hagamen, 1967.
forcement, in contrast to the emphasis on differences
Olfactory bulb* Rat-Roufctenfeerg, 197b Phillips, 1970.
in the past, has revitalized the view that self-stimula¬
Preoptic: Rat-Olds it Olds, 1963; Olds, Travis, 8c Schwing,
tion can provide important insights about reinforce¬
1960: Cat-O’Donohue it Hagamen, 1967; Wilkinson 8c
ment and motivation. Peele, 1963: Monkey-McHugh, Black, & Mason, 1966.
Rate of self-stimulation, a widely used although im¬
Lateral hypothalamus: Rat-Olds 8c Olds, 1963: Gat-
perfect index of central reinforcement, is increased O’Donohue it Hagamen, 1967; Wilkinson it Peele, 1963:
by hunger, the administration of sex hormones, and Monkey-Briese 8c Olds, 1964: Rabbit—Brunner, 1966:
other manipulations that induce or increase a central Goat-Persson, 1962; Squirrel-Wetzel 8c King, 1966; Wet¬
motive state. Self-stimulation is also increased by zel, King, 8c Nowicki, 1967; Dog-Stark 8c Boyd, 1963;
Bacon 8c Wong, 1972: Pigeon-David, Davison, 8c Web¬
taste, smell, and other relevant incentive stimuli.
ster, 1972; MacPhail, 1966; Webster 8c Beale, 1970.
These observations suggest that reinforcement in¬
Ventromedial hypothalamus: Rat-Olds 8c Olds, 1963:
volves an interaction of a central motive state with
Cat-O’Donohue 8c Hagamen, 1967.
incentive stimuli.
Medial forebrain bundle: Rat-Olds 8c Olds, 1963; Olds,
Recently it was shown that central reinforcement
Travis, 8c Schwing, 1960: Cat-O’Donohue 8c Hagamen,
is associated with stimulation of catecholamine path¬ 1967; Schnitzer, Reid, 8c Porter, 1965; Wilkinson 8c Peele,
ways of the brain. It has been proposed that the 1963: Gerbil-Routtenberg 8c Kramis, 1967.
590 CENTRAL REINFORCEMENT

Zona incerta: Rat-Olds 8c Olds, 1963: Monkey-Briese 8c Bacon, W. E., 8c Wong, I. G. Reinforcement value of elec¬
Olds, 1964: Squirrel-Wetzel 8c King, 1966; Wetzel, King, trical brain stimulation in neonatal dogs. Developmental
8c Nowicki, 1967. Psychobiology, 1972, 5, 195-200.
Baxter, L., Gluckman, M. I., Stein, L., 8c Scerni, A. Self¬
Thalamus: Rat-Olds 8c Olds, 1963; Cooper 8c Taylor, 1967:
injection of apomorphine in the rat: Positive reinforce¬
Cat-O’Donohue 8c Hagamen, 1967; Grastyan 8c Angyan,
ment by a dopamine receptor stimulant. Pharmacology,
1967: Man-Heath, 1963; Bishop, Elder, 8c Heath, 1963:
Biochemistry, and Behavior, 1974, 2, 389-391.
Squirrel-Wetzel, King, 8c Nowicki, 1967: Dog-Bacon 8c
Bindra, D. Neuropsychological interpretation of the effects
Wong, 1972.
of drive and incentive-motivation on general activity
Caudate nucleus: Rat-Routtenberg, 1971; Olds, Travis, 8c and instrumental behavior. Psychological Review, 1968,
Schwing, 1960: Cat-O’Donohue 8c Hagamen, 1967; Juste- 75, 1-22.
sen, Sharp, 8c Porter, 1963: Monkey-Briese 8c Olds, 1964; Bindra, D. The interrelated mechanisms of reinforcement
Lilly, 1957: Man-Heath, 1963; Bishop, Elder, 8c Heath, and motivation and the nature of their influence on
1963: Dog-Bacon 8c Wong, 1972: Dolphin-Lilly 8c Miller, response. In Nebraska symposium on motivation. Lin¬
1962. coln: University of Nebraska Press, 1969.
Amygdala: Rat-Olds 8c Olds, 1963: Cat-O’Donohue 8c Bishop, M. P., Elder, S. T., 8c Heath, R. G. Intracranial

Hagamen, 1967: Monkey-Briese 8c Olds, 1964: Man- self-stimulation in man. Science, 1963, 140, 394-396.
Heath, 1963; Bishop, Elder, 8c Heath, 1963. Bolles, R. C. Species-specific defense reactions and avoid¬
ance learning. Psychological Review, 1970, 77, 32-48.
Globus pallidus: Rat-Olds 8c Olds, 1963: Cat-O’Donohue
Boyd, E. S., 8c Gardner, L. C. Positive and negative rein¬
8cHagamen, 1967: Monkey-Routtenberg, Gardner, 8c
forcement from intracranial stimulation of a teleost.
Huang, 1971; Lilly, 1957. Science, 1962, 156, 648-649.
Putamen: Monkey-Lilly, 1957. Brady, J. Vv, Boren, J. J., Conrad, D., 8c Sidman, M. The
effect of food and water deprivation upon intracranial
Central grey: Rat-Cooper 8c Taylor, 1967.
self-stimulation. Journal of Comparative and Physio¬
Ventral tegmentum: Rat-Olds 8c Olds, 1963; Olds 8c logical Psychology, 1957, 50, 134-137.
Peretz, 1960; Routtenberg 8c Malsbury, 1969: Cat- Breese, G. R., Howard, J., 8c Leahy, J. P. Effect of 6-
O’Donohue 8c Hagamen, 1967; Wilkinson 8c Peele, 1963. hydroxydopamine on electrical self-stimulation of the
Substantia nigra: Rat-Crow, 1972a; Routtenberg 8c Mals¬ brain. British Journal of Pharmacology, 1971, 43, 255-
bury, 1969: Cat-O’Donohue 8c Hagamen, 1967: Monkey- 257.
Briese 8c Olds, 1964: Gerbil-Routtenberg 8c Kramis, 1967. Briese, E., 8c Olds, J. Reinforcing brain stimulation and
memory in monkeys. Experimental Neurology, 1964, 10,
Red nucleus: Cat—O’Donohue 8c Hagamen, 1967. 493-508.
Locus coeruleus: Rat-Crow, 1972b. Brodie, D. A., Moreno, O. M., Malis, J. L., 8c Brodie, J. J.
Rewarding properties of intracranial stimulation. Sci¬
Frontal cortex: Rat-Routtenberg, 1971; Routtenberg 8c
ence, 1960, 131, 929-930.
Sloan, 1972: Cat-O’Donohue 8c Hagamen, 1967; Wilkin¬
Brown, S., 8c Trowill, J. A. Lever-pressing performance
son 8c Peele, 1963.
for brain stimulation on F-I and V-I schedules in a
Brachium conjunctivum: Rat-Routtenberg 8c Malsbury, single-lever situation. Psychological Reports, 1970, 26,
1969: Cat-O’Donohue 8c Hagamen, 1967: Monkey-Briese 699-706.
8c Olds, 1964.
Brunner, A. Facilitation of classical conditioning in rabbits
Cerebral peduncle: Rat-Olds 8c Olds, 1963: Squirrel- by reinforcing brain stimulation. Psychonomic Science,
Wetzel, King, 8c Nowicki, 1967. 1966, 6, 211-212.
Bursten, B., 8c Delgado, J. M. R. Positive reinforcement
Paleostriatum: Pigeon-Goodman 8c Brown, 1966; Mac-
Phail, 1966, 1967. induced by intracerebral stimulation in the monkey.
Journal of Comparative and Physiological Psychology,
Neostriatum: Pigeon-MacPhail, 1966, 1967; Webster 8c 1958, 51, 6-10.
Beale, 1970. Caggiula, A. R., 8c Hoebel, B. G. “Copulation-reward”
Hyperstriatum: Pigeon-MacPhail, 1966, 1967. site in the posterior hypothalamus. Science, 1966, 153,
1284-1285.
Ectostriatum: Pigeon-MacPhail, 1966; Hollard 8c Davi¬
Campbell, H. J. Acute effects of pregnene steroids on
son, 1971.
septal self-stimulation in the rabbit. Journal of Physi¬
Pallidum: Goldfish-Boyd 8c Gardner, 1962. ology (London), 1968, 196, 134P-135P.
Campbell, H. J. Pleasure-seeking brains: Artificial tickles,
natural joys of thought. Smithsonian, 1971, 2, 14-23.
Cantor, M. B. Signaled reinforcing brain stimulation facili¬
REFERENCES tates operant behavior under schedules of intermittent
reinforcement. Science, 1971, 174, 610-613.
Cantor, M. B., 8c LoLordo, V. M. Rats prefer signaled
Arbuthnott, G. W., Crow, T. J., Fuxe, K., Olson, L., 8c reinforcing brain stimulation to unsignaled ESB. Journal
Ungerstedt, U. Depletion of catecholamines in vivo in¬ of Comparative and Physiological Psychology, 1970, 71,
duced by electrical stimulation of central monoamine 183-191.
pathways. Brain Research, 1970, 24, 471-483. Cantor, M. B., 8c LoLordo, V. M. Reward value of brain
Gordon Mogenson and Jan Cioe 591

stimulation is inversely related to uncertainty about its DiCara, L. V., 8c Deutsch, J. A. Secondary reinforcement
onset. Journal of Comparative and Physiological Psychol¬ as a function of drive in intracranial self-stimulation.
ogy, 1972, 79, 259-270. Proceedings of the 74th Annual Convention of the
Carlisle, H. J., 8c Snyder, E. The interaction of hypo¬ American Psychological Association, 1966, 105—106.
thalamic self-stimulation and temperature regulation. DiCara, L. V., 8c Miller, N. E. Instrumental learning of
Experientia, 1970, 26, 1092-1093. vasomotor responses by rats: Learning to respond differ¬
Chu, N. D., 8c Bloom, F. E. Norepinephrine containing entially in the two ears. Science, 1968, 159, 1485-1486.
neurones: Changes in spontaneous discharge patterns Domino, E. F., 8c Olds, M. E. Effects of D-amphetamine,
during sleeping and waking. Science, 1973, 179, 908-910. scopolamine, chlordiazepoxide, and diphenylhydantoin
Conner, J. D. Caudate nucleus neurones: Correlation of on self-stimulation behavior and brain acetylcholine.
the effects of substantia nigra stimulation with ionto- Psychopharmacology, 1972, 13, 1-16.
phoretic dopamine. Journal of Physiology (London), Doty, R. W. Electrical stimulation of the brain in be¬
1970, 208, 691-704. havioral context. Annual Review of Psychology, 1969,
Coons, E. F., 8c Cruce, J. A. F. Lateral hypothalamus: 20, 289-320.
Food current intensity in maintaining self-stimulation of Dreese, A. Importance du septeme mesencephalo-telenceph-
hunger. Science, 1968, 159, 1117—11 IQ. alique noradrenergique comme substratum anatom-
Cooper, B. R., Black, W. C., 8c Paolino, R. M. Decreased ique du comportement d’autostimulation. Life Sciences,
septal-forebrain and lateral hypothalamic reward after 1966, 5, 1003-1014.
alpha-methyl-jf?-tyrosine. Physiology and Behavior, 197L Falk, J, L. Septal stimulation as a reinforcer of an alterna¬
6} 425-429. tive to consummatory behavior. Journal of Experimental
Cooper, R. M., 8c Taylor, L. H. Thalamic reticular system Analysis of Behavior, 1961, 4, 213-217.
and central grey: Self-stimulation. Science, 1967, 156, Fibiger, H. C., Zis, A. P., 8c McGeer, E. G. Feeding and
102-103. drinking deficits after 6-hydroxydopamine administra¬
Crichton, M. The terminal man. Toronto: Bantam Books, tion in the rat: Similarities to the lateral hypothalamic
1972. syndrome. Brain Research, 1973, 55, 135-148.
Crow, T. J. Enhancement by cocaine of intracranial self¬ Flynn, J. P„ Vaneqas, H„ Foote, W., & Edwards, S. Neural
stimulation in the rat. Life Sciences, 1970, 9, 375—381. mechanisms involved in a cat’s attack on a rat- In R.
Crow, T. J. A map of the rat mesencephalon for electrical Whalen (Ed.), Neural control of behavior. New York:
self-stimulation. Brain Research, 1972, 56, 265-273. (a) Academic Press, 1970.
Crow, T. J. Intracranial self-stimulation with electrodes in Fuxe, K., Hokfelt, T., & Uncerstedt, U. Morphological
the region of the locus coeruleus. Brain Research, 1972, and functional aspects of central monamine neurons.
56, 275-287. (b) International Review of Neurobiology, 1970, 13, 93-126.
Crow, T. J. Catecholamine-containing neurones and elec¬ Gallistel, G. R., & Beagley, G. Specificity of brain stim¬
trical self-stimulation, 1: A review of some data. Psycho¬ ulation reward in the rat. Journal of Comparative and
logical Medicine, 1972, 2r, 414-421. (c) Physiological Psychology, 1971, 76, l99-£05»
Crow, T. J, Catecholamine-containing neurones and elec¬ Gallistel, C, R,, Rolls, E,, 8c Greene, B, Neuron function
trical self-stimulation, 2; A theoretical interpretation inferred from behavioral and eleetrophysiological esti¬
and some psychiatric implications. Psychological Medi¬ mates of refractory period. Science, 1969, 166, 1028—1030.
cine, 1973, 3, 66-73. Gandelman, R.; Panksepp, J., 8c Trowill, J. A. The effect
Culrerton, J. K., Kling, J. W., 8c Berkley, M. A. Extinc¬ of lever retraction on resistance to the extinction of a
tion responding following ICS and food reinforcement. response rewarded with electrical stimulation of the
Psychonomic Science, 1966, 5, 127—128. brain. Psychonomic Science, 1968, 10, 5-6.
Curtis, D. R., 8c Crawford, J. M. Central synaptic trans¬ German, D. C.; 8c Bowden, D. M. Catecholamine systems as
mission: Microelectrophoretic studies. Annual Review the neural substrate for intracranial self-stimulation:
of Pharmacology, 1969, 9, 209-240. A hypothesis. Brain Research, 1974, 73, 381-419.
Davis, A. H., Davison, M. C., 8c Webster, D. M. Intra¬ Gibson, W. E., Reid, L. D., Sakai, M., 8c Porter, P. B.
cranial reinforcement in pigeons: An analysis using con¬ Intracranial reinforcement compared with sugar-water
current schedules. Physiology and Behavior, 1972, 9, reinforcement. Science, 1965, 148, 1357-1359.
385-390. Glickman, S. E., 8c Schiff, B. B. A biological theory of
Deutsch, J. A. The structural basis of behavior. Chicago: reinforcement. Psychological Review, 1967, 74, 81-109.
University of Chicago Press, 1960. Goldstein, R., Hill, S. Y., 8c Templer, D. I. Effects of food
Deutsch, J. A., Adams, D. W., 8c Metzner, R. J. Choice of deprivation on hypothalamic self-stimulation in stim¬
intracranial stimulation as a function of delay between ulus-bound eaters and noneaters. Physiology and Be¬
stimulations and strength of competing drive. Journal havior, 1970, 5, 915-918.
of Comparative and Physiological Psychology, 1964, 57, Goodman, I. J., 8c Brown, J. L. Stimulation of positively
241-243. and negatively reinforcing sites in the avian brain. Life
Deutsch, J. A., 8c DiCara, K. Hunger and extinction in Sciences, 1966, 5, 693-704.
intracranial self-stimulation. Journal of Comparative Grastyan, E., 8c Angyan, L. The organization of motivation
and Physiological Psychology, 1967, 63, 344-347. at the thalamic level of the cat. Physiology and Behavior,
Deutsch, J. A., 8c Howarth, C. I. Some tests of a theory of 1967, 2, 5-13.
intracranial self-stimulation. Psychological Review, 1963, Hawkins, T. D., 8c Pliskoff, S. S. Brain stimulation inten¬
70, 444-460. sity, rate of self-stimulation, and reinforcing strength: An
DiCara, L. V. Brain stimulation and secondary reinforce¬ analysis through chaining. Journal of Experimental
ment. Dissertation Abstracts, 1966, 27, 2157-B. Analysis of Behavior, 1964, 7, 285-288.
 59 2
CENTRAL REINFORCEMENT

Heath, R. G. Electrical self-stimulation of the brain in and food reinforcement in discrimination and general¬
man. American Journal of Psychiatry, 1963, 120, 571- ization situations. Journal of Comparative and Phys¬
577. iological Psychology, 1968, 65, 507-511.
Hebb, D. O. A textbook of psychology (2nd ed.). Philadel¬ Kling, J. W., 8c Matsumiya, Y. Relative reinforcement
phia: Saunders, 1966. values of food and intracranial stimulation. Science,
Herberg, L. J. Seminal ejaculation following positively 1962, 135, 668-670.
reinforcing electrical stimulation of the rat hypo¬ Knott, P. D., 8c Clayton, K. N. Durable secondary rein¬
thalamus. Journal of Comparative and Physiological forcement using brain stimulation as the primary rein¬
Psychology, 1963, 56, 679-685. forcer. Journal of Comparative and Physiological Psy¬
Hodos, W. Progressive ratio as a measure of reward chology, 1966, 61, 151-153.
strength. Science, 1961, 134, 943-944. Lenzer, I. I. Differences between behavior reinforced by
Hodos, W. Motivational properties of long durations of electrical stimulation of the brain and conventionally
rewarding brain stimulation. Journal of Comparative reinforced behavior: An associative analysis. Psycholog¬
and Physiological Psychology, 1965, 59, 219-224. ical Bulletin, 1972, 78, 103-118.
Hodos, W., 8c Valenstein, E. S. An evaluation of response Liebman, J. M., 8c Butcher, L. L. Effects on self-stimulation
rate as a measure of rewarding intracranial stimulation. behavior of drugs influencing dopaminergic neurotrans¬
Journal of Comparative and Physiological Psychology, mission mechanisms. N aunyn-Schmie deb ergs Archiv fur
1962, 55, 80-84. Pharmakologie und experimentelle Pathologie (Berlin),
Hoebel, B. G. Feeding and self-stimulation. Annals of the 1973, 277, 305-318.
New York Academy of Science, 1969, 157, 758-778. Lilly, J. C. Learning elicited by electrical stimulation of
Hoebel, B. G. Feeding: Neural control of intake. Annual subcortical regions in the unanesthetized monkey.
Review of Physiology, 1971, 33, 533-568. Science, 1957, 125, 748-749.
Hoebel, B. G., & Teitelbaum, P. Hypothalamic control of Lilly, J. C., 8c Miller, A. M. Operant conditioning of the
feeding and self-stimulation. Science, 1962, 135, 375-377. bottlenose dolphin with electrical stimulation of the
Hollard, V., 8c Davison, M. C. Preference for qualitatively brain. Journal of Comparative and Physiological Psy¬
different reinforcers. Journal of Experimental Analysis chology, 1962, 55, 73-79.
of Behavior, 1971,16, 375-380. Linholm, E., 8c Keesey, R. E. Discrimination learning as a
Honig, W. K. Operant behavior: Areas of research and function of the duration of rewarding hypothalamic
application. New York: Appleton-Century-Crofts, 1966. stimulation. Psychonomic Science, 1968, 10, 153-154.
Horowitz, Z. P., Chow, M. I., 8c Carlton, P. L. Self-stim¬ Linholm, E., 8c Keesey, R. E. Faster rates of discrimination
ulation of the brain by cats: Effects of imipramine, learning with centrally elicited reinforcement than with
amphetamine, and chlorpromazine. Psychopharmacol¬ food. Journal of Comparative and Physiological Psy¬
ogy, 1962, 3, 455-462. chology, 1970, 72, 318-327.
Huston, J. P. Relationship between motivating and re¬ Lippa, A. S., Antelman, S. M., Fisher, A. E., 8c Canfield,
warding stimulation of the lateral hypothalamus. Phys¬ D. R. Neurochemical mediation of reward: A significant
iology and Behavior, 1971, 6, 711-716. role for dopamine? Pharmacology, Biochemistry, and
Huston, J. P. Inhibition of hypothalamically motivated Behavior, 1973, 1, 23-28.
eating by rewarding stimulation through the same elec¬ MacPhail, E. M. Self-stimulation in pigeons: The problem
trode. Physiology and Behavior, 1972, 8, 1121-1126. of “priming.” Psychonomic Science, 1966, 5, 7-8.
Jones, B. E., Bobillier, P., Pin, C., 8c Jouvet, M. The
MacPhail, E. M. Positive and negative reinforcement from
effect of lesions of catecholamine-containing neurons intracranial stimulation in pigeons. Nature (London),
upon monamine content of the brain and EEG and be¬ 1967, 213, 947-948.
havioral waking in the cat. Brain Research, 1973, 58,
Margules, D. L., 8c Olds, J. Identical “feeding” and “re¬
157-177.
warding” systems in the lateral hypothalamus of rats.
Jouvet, M. Some monaminergic mechanisms controlling
Science, 1962, 135, 374-375.
sleep and waking. In A. G. Karczmar 8c J. C. Eccles
Marshall, J. F., 8c Teitelbaum, P. A comparison of the
(Eds.), Brain and human behavior. Berlin: Springer-Ver-
lag, 1972. eating in response to hypothermic and glucoprivic chal¬
Justesen, R., Sharp, J. C., 8c Porter, P. B. Self-stimulation lenges after nigral 6-hydroxydopamine and lateral hypo¬
of the caudate nucleus by instrumentally naive cats. thalamic electrolytic lesions in rats. Brain Research,
Journal of Comparative and Physiological Psychology, 1973, 55, 229-233.
1963, 56, 371 374. McClelland, D. C. Some social consequences of achieve¬
Keesey, R. E., 8c Goldstein, M. D. Use of progressive fixed- ment motivation. In Nebraska symposium on motiva¬
ratio procedures in the assessment of intracranial rein¬ tion. Lincoln: University of Nebraska Press, 1955.
forcement. Journal of Experimental Analysis of Be¬ McHugh, P. R., Black, W. C., 8c Mason, J. W. Some hor¬
havior, 1968, 11, 293-301. monal responses to electrical self-stimulation in the
Kemp, J. M., 8c Powell, T. P. S. The connexions of the Macaca mulatta. American Journal of Physiology, 1966,
striatum and globus pallidus: Synthesis and speculation. 210, 109-113.
Philosophical Transactions of the Royal Society of Lon¬ Mendelson, J. Lateral hypothalamic stimulation in satiated
don, 1971, 262, 441-457. rats: The rewarding effects of self-induced drinking.
Keys, N. W. Secondary reinforcement and reinforcing Science, 1967, 157, 1077-1079.
intracranial stimulation. Dissertation Abstracts, 1964, Miller, N. E. Some motivational effects of brain stimula¬
28, 3436. tion and drugs. Federation Proceedings, 1960, 19, 846-
Kling, J. W., 8c Berkley, M. A. Electrical brain stimulation 854.
Gordon Mogenson and Jan C/oe 593

Milner, P. M. Physiological psychology. New York: Holt, Journal of Comparative and Physiological Psychology,
Rinehart 8c Winston, 1970. 1958, 51, 675-678. (b)
Milner, P. M. Theories of reinforcement, drive and moti¬ Olds, J. Differentiation of reward systems in the brain by
vation. In L. L. Iverson, S. D. Iverson, 8c S. H. Snyder self-stimulation techniques. In E. R. Ramey 8c D. S.
(Eds.), Handbook of psychopharmacology. (Vol. 7) New O’Doherty (Eds.), Electrical studies in the unanesthetized
York: Plenum Press, 1976. brain. New York: Hoeber, 1960.
Mogenson, G. J. An attempt to establish secondary rein¬ Olds, J., 8c Milner, P. Positive reinforcement produced by
forcement with rewarding brain stimulation. Psycho¬ electrical stimulation of septal area and other regions
logical Reports, 1965, 16, 163-167. of the rat brain. Journal of Comparative and Physio¬
Mogenson, G. J. Effects of amphetamine on self-stimulation logical Psychology, 1954, 47, 419-427.
and induced drinking. Physiology and Behavior, 1968, Olds, J., 8c Olds, M. E. The mechanism of voluntary be¬
3, 133-136. havior. In R. Heath (Ed.), The role of pleasure in be¬
Mogenson, G. J. General and specific reinforcement systems havior. New York: Harper, 1964.
for drinking behavior. Annals of the New York Acad¬ Olds, J. & Peretz, B. A motivational analysis of the reticu¬
emy of Sciences, 1969, 157, 779-797. (a) lar activating system. Electroencephalography and Clini¬
Mogenson, G. J. Effects of drugs on the preference between cal Neurophysiology j 1960, 12, 445—454.
electrical stimulation on the lateral hypothalamus and Olds, J., Travis, R. P., & Schwing, R. C. Topographic
water. Psychonomic Science, 1969, 77, 13-14. (b) organization of hypothalamic self-stimulation functions.
Mogenson, G. J., 8c Huang, Y. H. The neurobiology of Journal of Comparative and Physiological Psychology,
motivated behavior. In G. A. Kerkut 8c J. W. Phillis 1960, 55, 23-32.
(Eds.), Progress in neurobiology (Vol, 1). Oxford: Perga- Olds, M. E. Comparative effects of amphetamine, scopola¬
mon Press, 1973. mine, chlordiazopoxide, and diphenylhydantom on oper¬
Mogenson, G. J., 8c Kaplinsky, M. Brain self-stimulation ant and extinction behavior with brain stimulation and
and mechanisms of reinforcement. Learning and Moti¬ food reward. Neuropharmacology, 1970, 9, 519-532.
vation, 1970, 1, 186-198. Olds, M. E„ & Olds, J. Approach-avoidance analysis of rat
Mogenson, G. J., 8c Morgan, C. Effects of induced drinking diencephalon. Journal of Comparative Neurology, 1963,
on self-stimulation of the lateral hypothalamus. Experi¬ 120, 259-295,
mental Brain Research, 1967, 3, 111-116. Panksepp, J,, 8c Trowxll, J. A. Intraoral self-injection, I;
Mogenson, G. J., Morgan, G. W., Phillips, A. G., 8c Effects of delay of reinforcement on resistance to extinc¬
Stevenson, J. A. F. Studies of self-stimulation and drink¬ tion and implication for self-stimulation. Psychonomic
ing. Paper presented at 24th International Congress of Science, 1967, 9, 405-406. (a)
Physiology, Washington, D.C., Jidy 1968. Panksepp, J., 8c Trowill, J. A. Intraoral self injection, II:
Mogenson, G. J., & Phillips, A. G. Motivation: A psycho¬ The simulation of self-stimulation phenomenon with a
logical construct in search of a physiological substrate. conventional reward. Psychonomic Science, 1967, 9, 407-
In A. N. Epstein 8c J. M. Sprague (Eds.), Progress in 408. (b)
psychobiology and physiological psychology (Vol. 6). Pellegrino, L. J., & Cushman, A. J. A stereotaxic atlas of
New York: Academic Press, 1976. the rat brain. New York: Appleton-Century-Crofts, 1967.
Mogenson, G. J., & Stevenson, J. A. E. Drinking and self¬ Persson, N. Selhstimulation in the goat. Acta Physiologica
stimulation with electrical stimulation of the lateral Scandinavica, 1962, 55, 276-285.
hypothalamus. Physiology and Behavior, 1966, 1, 251- Pfaffman, G. The pleasures of sensation. Psychological
254. Review, 1960, 67, 253-268.
Mogenson, G. J., 8c Stevenson, J. A. F. Drinking induced Phillips* A. G. Enhancement and inhibition of olfactory
by electrical stimulation of the lateral hypothalamus.
Experimental Neurology, (New York: Academic Press,
Inc., 1967), 17, 119-127.
1970, 5,
bulb self-stimulation by odours* Physiology and Behavior,
1127-1131.
Phillips, A, G. The role of the substantia nigra in behavior

Morgan, C. W., 8c Mogenson, G. J. Preference of water- elicited by electrical stimulation of the brain, Paper
deprived rats for stimulation of the lateral hypothalamus presented at the meeting of the Canadian Psychological
rather than water. Psychonomic Science, 1966, 6, 337— Association, Victoria, B.C., June 1973.
Phillips, A. G., Cox, V. C., Kakolewski, J. W., 8c Valen-
338.
stein, E. S. Object-carrying by rats: An approach to the
Newman, B. L., 8c Feldman, S. M. Electrophysiological
behavior produced by brain stimulation. Science, 1969,
activity accompanying self-stimulation. Journal of Com¬
166, 903-905.
parative and Physiological Psychology, 1964, 57, 244-247.
Phillips, A. G., 8c Fibiger, H. C. Substantia nigra: Self¬
O’Donohue, N. E., 8c Hagamen, W. D. A map of the cat
stimulation and poststimulation feeding. Physiological
brain for regions producing self-stimulation and uni¬ Psychology, 1973, 1, 233-236.
lateral inattention. Brain Research, 1967, 5, 289-305. Phillips, A. G., 8c Mogenson, G. J. Effects of taste on self¬
Olds, J. Runway and maze behavior controlled by baso- stimulation and induced drinking. Journal of Compara¬
medial forebrain stimulation in the rat. Journal of tive and Physiological Psychology, 1968, 66, 654-660.
Comparative and Physiological Psychology, 1956, 49, Phillips, A. G., Morgan, C. W., 8c Mogenson, G. J. Changes
507-512. in self-stimulation preference as a function of incentive
Olds, J. Effects of hunger and male sex hormone on self¬ of alternative rewards. Canadian Journal of Psychology,
stimulation of the brain. Journal of Comparative and 1970, 24, 289-297.
Physiological Psychology, 1958, 51, 320-324. (a) Pliskoff, S. S., 8c Hawkins, T. D. A method for increasing
Olds, J. Satiation effects in self-stimulation of the brain. the reinforcement magnitude of intracranial stimulation.
594 CENTRAL REINFORCEMENT

Journal of Experimental Analysis of Behavior, 1967, 10, Schnitzer, s. B., Reid, L. D., 8c Porter, P. B. Electrical
281-289. intracranial stimulation as a primary reinforcer for cats.
Pliskoff, S. S., Wright, J. E., 8c Hawkins, T. D. Brain Psychological Reports, 1965, 16, 335-338.
stimulation as a reinforcer: Intermittent schedules. Jour¬ Segal, M., 8c Bloom, F. E. The action of norepinephrine
nal of Experimental Analysis of Behavior, 1965, 8, 75-88. in the rat hippocampus, II: Activation of the input
Poschel, B. P. H. Mapping of rat brain for self-stimulation pathway. Brain Research, 1974, 72, 99-114.
under monoamine oxidase blockade. Physiology and Seward, J. P., Uyeda, A., 8c Olds, J. Resistance to extinc¬
Behavior, 1969, 4, 325-331. tion following cranial self-stimulation. Journal of Com¬
Poschel, B. P. H., 8c Ninteman, F. W. Hypothalamic self¬ parative and Physiological Psychology, 1959, 52, 294-299.
stimulation: Its suppression by blockade of norepineph¬ Sheffield, F. D., Roby, T. B., 8c Campbell, B. A. Drive
rine biosynthesis and reinstatement with methamphe- reduction versus consummatory behavior as determinants
tamine. Life Sciences, 1966, 5, 11-16. of reinforcement. Journal of Comparative and Physio¬
Ray, O. S., Hine, B., 8c Bivens, L. W. Stability of self¬ logical Psychology, 1954, 47, 349-354.
stimulation responding during long test sessions. Physi¬ Shettleworth, S. J. Constraints on learning. In D. S.
ology and Behavior, 1968, 3, 161-164. Lehrman, R. A. Hinde, 8c E. Shaw (Eds.), Advances in
Renner, K. E. Delay of reinforcement: A historical review the study of behavior (Vol. 4). New York: Academic
Psychological Bulletin, 1964, 61, 341-361. Press, 1971.
Reynolds, G. S. A primer of operant conditioning. Glen¬ Sidman, M., Brady, J. V., Boren, J. J., Conrad, D. G., 8c
view, Ill.: Scott, Foresman, 1968. Schulman, A. Reward schedules and behavior main¬
Ritter, S., 8c Stein, L. Self-stimulation of noradrenergic cell
tained by intracranial self-stimulation. Science, 1955,
group (A6) in locus coeruleus of rats. Journal of Com¬ 122, 830-831.
parative and Physiological Psychology, 1973, 85, 443-452. Skinner, B. F. The phylogeny and ontogeny of behavior:
Ritter, S., 8c Stein, L. Self-stimulation in the mesencephalic
Contingencies of reinforcement throw light on contin¬
trajectory of the ventral noradrenergic bundle. Brain gencies of survival in the evolution of behavior. Science,
Research, 1974, 81, 145-157. 1966, 153, 1205-1213. (a)
Roberts, E. Disinhibition as an organizing principle in the
Skinner, B. F. Operant behavior. In W. K. Honig (Ed.),
nervous system: The role of gamma-aminobutyric acid. Operant behavior: Areas of research and application.
Advances in Neurology. Raven Press, 1974, 5, 127-143. New York: Appleton-Century-Crofts, 1966. (b)
Roberts, W. W. Hypothalamic mechanisms for motiva¬ Smith, N. S., 8c Coons, E. E. Temporal summation and
tional and species-typical behavior. In R. E. Whalen refractoriness in hypothalamic reward neurons as mea¬
(Ed.), The neural control of behavior. New York: Aca¬ sured by self-stimulation behavior. Science, 1970, 169,
demic Press, 1970. 782-785.
Rolls, E. T. Activation of amygdaloid neurones in reward, Spies, G. Food vs. intracranial self-stimulation reinforce¬
eating, and drinking elicited by electrical stimulation ment in food-deprived rats. Journal of Comparative and
of the brain. Brain Research, 1972, 45, 365-381. Physiological Psychology, 1965, 60, 153-157.
Routtenberg, A. Forebrain pathways of reward in Rattus
Staddon, J. E. R., 8c Simmelhag, V. L. The “superstition”
norwegicus. Journal of Comparative and Physiological experiment: A re-examination of its implications for the
Psychology, 1971, 75, 269-276. principles of adaptive behavior. Psychological Review,
Routtenberg, A., Gardner, E. L., 8c Huang, Y. H. Self¬ 1971, 78, 3-43.
stimulation pathways in the monkey, Macaca mulatta. Stark, P., 8c Boyd, E. S. Effects of cholinergic drugs on
Experimental Neurology, 1971, 33, 213-224. hypothalamic self-stimulation response rates in dogs.
Routtenberg, A., 8c Kramis, R. C. “Foot-stomping” in the American Journal of Physiology, 1963, 205, 745-748.
gerbil: Rewarding brain stimulation, sexual behavior, Stein, L. Secondary reinforcement established with sub¬
and foot shock. Nature, 1967, 214, 173-174. cortical stimulation. Science, 1958, 127, 466-467.
Routtenberg, A., 8c Lindy, J. Effects of the availability of Stein, L. Effects and interactions of imipramine, chlorpro-
rewarding septal and hypothalamic stimulation on bar mazine, reserpine, and amphetamine on self-stimulation:
pressing for food under conditions of deprivation. Jour¬ Possible neurophysiological basis of depression. Recent
nal of Comparative and Physiological Psychology, 1965, Advances in Biological Psychiatry, 1962, 4, 288-308.
60, 158-161. Stein, L. Chemistry of proposive behavior. In J. T. Tapp
Routtenberg, A., 8c Malsbury, C. Brainstem pathways of (Ed.), Reinforcement and behavior. New York: Acade¬
reward. Journal of Comparative and Physiological Psy¬ mic Press, 1969.
chology, 1969, 68, 22-30. Stein, L., 8c Ray, O. S. Self-regulation of brain stimulating
Routtenberg, A., 8c Sloan, M. Self-stimulation in the current intensity in the rat. Science, 1959, 130, 570-572.
frontal cortex of Rattus norwegicus. Behavioral Biology, Stein, L., 8c Wise, C. D. Release of norepinephrine from
1972, 7, 567-572. hypothalamus and amygdala by rewarding medial fore¬
Sadowsky, S. Discriminative responding on associated brain bundle stimulation and amphetamine. Journal of
mixed and multiple schedules as a function of food and Comparative and Physiological Psychology, 1969, 67,
ICS reinforcement. Journal of Experimental Analysis of 189-198.
Behavior, 1969, 12, 933-945. Stein, L., 8c Wise, C. D. Possible etiology of schizophrenia:
Schachter, Cognitive effects on bodily functioning. In
S. Progressive damage to the noradrenergic reward system
D. C. Glass (Ed.), N euro physiology and emotion. New by 6-hydroxydopamine. Science, 1971, 171, 1032-1036.
York: Rockefeller University Press and Russell Sage Stein, L., Wise, C. D., 8c Berger, B. D. Noradrenergic
Foundation, 1967. reward mechanisms, recovery of function, and schizo-
Gordon Mogenson and Jan C/oe 595

phrenia. In J. K. McGaugh (Ed.), The chemistry of reinforcing brain stimulation. Journal of Experimental
mood, motivation, and memory. New York: Plenum Analysis of Behavior, 1964, 7, 183-184.
Press, 1972. Valenstein, E. S., 8c Meyers, W. J. Rate-independent test
Stricker, E. M., 8c Zigmond, M. J. Recovery of function of reinforcing consequences of brain stimulation. Jour¬
following damage to central catecholamine-containing nal of Comparative and Physiological Psychology, 1964,
neurons: A neurochemical model for die lateral hypo¬ 57, 52-60.
thalamic syndrome. In J. M. Sprague 8c A. N. Epstein Wauquier, A., 8c Niemegeer, C. J. Intracranial self-stimula¬
(Eds.), Progress in psychobiology and physiological psy¬ tion in rats as a function of various stimulus parameters,
chology (Vol. 6). New York: Academic Press, 1976. II: Influence of haloperidol, pimozide, and pipamper-
Stutz, R. M., Rossi, R. R., 8c Bowring, A. M. Competition one on medial forebrain bundle stimulation with mono-
between food and rewarding brain shock. Physiology and polar electrodes. Psychopharmacology, 1972, 27, 191-202.
Behavior, 1971, 7, 753-757. Webster, D. M„ 8c Beale, I. L. Intracranial self-stimulation
Terman, M., 8c Kling, J. W. Discrimination of brightness in the pigeon: The effects of current intensity. Psycho-
differences by rats with food or brain-stimulation rein¬ nomic Science, 1970, 20, 15-17.
forcement. Journal of Experimental Analysis of Be¬ Wetzel, M. C. Self-stimulation anatomy: Data needs. Brain
havior*, 1968, IT 29-37. Research, 1968, 10, 287-296.
Trowill, J. A., Panksepf, J., 8c Gandelman, R. An incen¬ Wetzel, M. G., 8c King, J. E. Self-stimulation with mono-
tive model of rewarding brain stimulation. Psychological phasic current in the rock squirrel and rat, Psychonomic
Review, 1969, 76, 264-281. Science, 1966, 6, 7—8.
Ungerstedt, U. Stereotaxic mapping of the monoamine Wetzel, M. C., King, J. E., 8c Nowicki, L. E. Some mono-
pathways of the rat brain. Acta Physiologica Scandi- phasic self-stimulation loci in the rock squirrel and rat.
navica, 1971, Suppl. 367, 1-48. Psychonomic Science, 1967, 9, 35-36.
Ursin, R., Ursin, H., 8c Olds, J. Self-stimulation of hippo¬ Wilkinson, H. A., 8c Peele, T. L. Intracranial self-stimula¬
campus in rats. Journal of Comparative and Physiologi¬ tion in cats. Journal of Comparative Neurology, 1963,
cal Psychology, 1966, 61, 353-359. 121, 425-440.
Valenstein, E. S. Problems of measurement and interpreta¬ Young, P, T, The role of affective processes in learning
tion with reinforcing brain stimulation. Psychological and motivation. Psychological Review, 1959, 66, 1Q4—
Review, 1964, 71, 415-437. 125.
Valenstein, E. S., 8c Beer, B. Reinforcing brain stimulation Yuncer, L. M., Harvey, J. A., 8c LorEnS, S. A. Dissociation
in competition with water reward and shock avoidance. of the analgesic and rewarding effects of brain stimula¬
Science, 1962, 137, 1052-1054. tion in the rat. Physiology and Behavior, 1973, 10, 909-
Valenstein, E. S., & Beer, B. Continuous opportunity for 913.
 20

The Experimental Production

of Altered Physiological States
concurrent and contingent
behavioral models*

Joseph Brady
and
Alan Harris

INTRODUCTION duced by experimentally-programmed environmental

consequences (e.g., food delivery and/or shock avoid¬
The development of laboratory behavioral ap¬ ance) which follow such autonomic changes. Over the
proaches to the production of altered physiological decade since the topic of behaviorally-induced physio¬
states reflects the emergence of two general models for logical alterations was reviewed for the original vol¬
their experimental analysis. The more traditional con¬ ume on Operant Behavior (Brady, 1966), a substantial
current model emphasizes the effects of prior-occurring literature has emerged on instrumental visceral-
or accompanying environmental-behavioral interac¬ autonomic conditioning. Both the laboratory and
tions on physiological processes. The early work of clinical descriptions of such phenomena provide con¬
Pavlov and Cannon relating autonomic changes to vincing evidence of the extensive behavioral-environ¬
environmental antecedents provides classical examples mental influences available for experimental analysis
of such laboratory studies. Current applications of within the framework of this contingent psychophysi-
this model have extended the analysis of both respon¬ ological model.
dent and operant conditioning effects upon a broad
range of physiological processes (e.g., hormonal secre¬
tions). The more recent contingent model, in con¬ CONCURRENT MODELS

trast, focuses on environment-behavioral interactions

which follow physiological change and provide con¬ Early Studies
trolling consequences for instrumental conditioning
effects involving visceral-autonomic responses. Appli¬ Research in this area has traditionally emphasized
cations of this model have shown, for example, that Pavlovian (i.e., respondent) conditioning processes
both increases and decreases in heart rate can be pro- concerned primarily with adjustments of the orga¬
nism's “internal economy" (e.g.. Cannon, 1915; Gantt,
* Supported by NHLI Grants HL-06945, HL-17958, HL-17970, 1960; Pavlov, 1879), and an active experimental inter¬
and HL-17680. est in such classical psychophysiology continues to be

596
Joseph Brady and Alan Harris 597

reflected in contemporary research literature (Black, ical effects associated with such instrumental condition¬
1971; Figar, 1965; Harris 8c Brady, 1974). By the mid¬ ing procedures also appeared somewhat less transient
dle of the present century, however, reliable methods than similarly induced cardiorespiratory alterations.
were developing for direct observation and measure¬ The systematic analysis of such psychoendocrinologi-
ment of a wide range of physiological processes in cal relations was extensively described in a series
situations involving both classical respondents and in¬ of studies providing at least a partial basis for the
strumental or operant performances. In the middle psychophysiological approach which has continued to
1960s, a review of this area (Brady, 1966) surveyed characterize many aspects of the more recent develop¬
three major groups of laboratory operant conditioning ments in this research field (Mason 8c Brady, 1956;
studies related to the experimental production of Mason, Brady, Polish, Bauer, Robinson, Rose, 8c Tay¬
altered physiological states. lor, 1961; Mason, Brady, Robinson, Taylor, Tolson,
;
8c Mougey, 1961 Mason, Brady, 8c Sidman, 1957 ;
Mason, Mangan, Brady, Conrad, 8c Rioch, 1961;
Transient Physiological Changes
Mason, Nauta, Brady, Robinson, 8c Sachar, 1961; Sid¬
The first group of reports described relatively tran¬ man, Mason, Brady, 8c Thach, 1962).
sient cardiorespiratory and neurophysiological changes
which were for the most part confined to experimen¬
Chronic Somatic Effects
tal periods during which the animal subjects were
engaged in some required operant performance (e.g., The third and final group of studies surveyed in
Sidman shock-avoidance), and appeared to represent the 1966 review dealt with the role of operant be¬
physiological responses related to stimuli produced by havior in the production of chronic somatic changes
the instrumental behavior. Cardiorespiratory effects of the type associated with gastric ulcers and systemic
were found to accompany both appetitively and aver- infections. The relatively few studies which had ap¬
sively maintained instrumental performances, but peared in this area prior to the first Operant Be¬
they seldom if ever endured beyond the limits of the havior volume testified to the many difficulties pre¬
specific experimental condition which provided the sented by an experimental analysis of the behavioral
occasion for their appearance (Berlanger 8c Feldman, factors involved. The observed somatic changes were
1962; Eldridge, 1954; Hahn, Stern, & McDonald, 1962; generally irreversible. This imposed severe restrictions
Malmo, 1961, Perez-Cruet, Black, & Brady, 1963; Perez- upon replications within individual animals, a pro¬
Cruet, Tolliver, Dunn, Marvin, & Brady, 1963; Sha¬ cedure which has typically characterized operant re¬
piro 8c Horn, 1955; Wenzel, 1961). Central nervous search. In 1956, for example, Sawrey and Weisz first
system effects described in electrophysiological studies reported the production of gastric ulcers in rats ex¬
involving instrumental conditioning procedures were posed to an instrumental “conflict” procedure, though
found to be characterized by a similar transience (An- subsequent studies (Conger, Sawrey, k Turrell, 1958 ;
liker, 1959; Hearst, Beer, Sheatz, 8c Galambos, 1960; Sawrey, Conger, 8c Turrell, 1956; Weisz, 1957) clearly
John 8c Killam, 1959; Porter, Conrad, 8c Brady, 1959; suggested that multiple interacting factors, including
Ross, Hodos, & Brady, 1962) though the findings were food deprivation, “fear,” shock* weight loss, and even
suggestive of relationships involving behaviorally- social experience could not be readily teased apart in
induced autonomic and somatic changes. such “conflict''-produced alterations, A subsequent
series of studies at the Walter Reed Medical Center
in Washington was concerned with the production of
Durable Physiological Alterations
peptic ulcers in rhesus monkeys who were exposed to
The second group of studies called attention to recurrent instrumental avoidance performance re¬
more durable endocrinological and visceral-alimentary quirements (Brady, 1958; Brady, 1963; Brady 8c Polish,
changes. The production of marked obesity in normal 1960; Brady, Porter, Conrad, 8c Mason, 1958; Polish,
rats by controlling drinking behavior as a shock avoid¬ Brady, Mason, Thach, 8c Neimeck, 1962; Porter,
ance response (Williams 8c Teitelbaum, 1956), and the Brady, Conrad, Mason, Galambos, 8c Rioch, 1958) and
experimental elevation of alcohol ingestion levels in these experiments confirmed the complexity of such
rhesus monkeys during and for prolonged periods fol¬ effects. Two additional studies, however, one with
lowing exposures to shock-avoidance conditioning humans (Davis 8c Berry, 1963) and one with animals
(Clark 8c Polish, 1960) were represented as two of the (Rice, 1963), were able to provide further support for
more dramatic examples of relatively durable be- the indicated relationship between instrumental
haviorally-induced alimentary changes. Endocrinolog¬ avoidance performances and gastro-intestinal changes.
598 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

And at least two other reports (Rasmussen, Marsh, Sc in the lever pressing rate during tone presentations
Brill, 1957; Simson, 1958) had appeared by the mid (i.e., conditioned suppression) over the course of the 7
1960’s relating the incidence of infectious and other CER acquisition trials were accompanied by progres¬
disease processes to instrumental performance require¬ sive increases in 17-OH-CS levels (as compared to
ments involving escape and avoidance in laboratory lever-pressing control sessions with no tone-shock
animals. pairings). Essentially the same CER procedure has
also been studied in chronically catheterized rhesus
monkeys monitored for heart rate and blood pressure
RECENT DEVELOPMENTS changes during both acquisition and long-term main¬
tenance of conditioned suppression using a 3-min
Over the past decade, the endocrine and cardio¬ clicker presentation as the CS (Brady, Kelly, Sc Plum¬
vascular systems have continued to provide a major lee, 1969). Over the course of the first 8 to 10 clicker-
focus for laboratory behavioral studies within the shock pairings, all 5 animals in the experiment
framework of traditional concurrent models for the showred consistent and systematic decreases in both
production of altered physiological states. Some atten¬ heart rate and blood pressure during the clicker. How¬
tion has also been directed to gastro-intestinal ever, continued daily pairings of clicker and shock,
processes, central nervous system effects, and related superimposed upon the lever-pressing performance,
physiological states. Significant methodological ad¬ produced abrupt and sustained reversals in both the
vances related to the continuous measurement and direction and the magnitude of the cardiovascular re¬
recording of blood pressure (Perez-Cruet, Plumlee, Sc sponse, usually commencing at about trial 9 or 10 as
Newton, 1966; Swinnen, 1968; Werdegar, Johnson, Sc shown in Figure 1. Significantly, these changes were
Mason, 1964) have contributed in an important way observed to persist as large magnitude increases in
to the progressive development of better models. In¬ heart rate and both systolic and diastolic blood pres¬
creasing use of long-term preparations (Brady, 1965; sure in response to the behaviorally suppressing clicker
Findley, Brady, Robinson, Sc Gilliam, 1971; Forsyth, presentations for from 50 to 100 daily conditioning
1969; Herd, Morse, Kelleher, Sc Jones, 1969) has pro¬ trials.
vided for more extended observation and experi¬ The results of these more extended studies empha¬
mental manipulation. Behavioral procedures of estab¬ size the differential temporal course of the skeletal and
lished effectiveness, including conditioned suppression autonomic components of the CER development. The
(Estes Sc Skinner, 1941) and free-operant avoidance cardiovascular changes (i.e., heart rate and blood pres¬
(Sidman, 1953) have received increasing attention in sure decreases) accompanying the initial conditioning
psychophysiology. Progressively more precise analysis trials probably reflected the reduction in motor activ¬
of the observed interactions highlights the develop¬ ity (i.e., suppression of lever-pressing rate) which
ments to be reviewed in the remainder of this section. developed in response to the clicker over the first 8 to
10 trials. The later-appearing “conditioned cardiac
Endocrine and Cardiovascular Changes
respondent” could be considered to have developed
only after this initial “suppression” stage and to have
Systematic increases in plasma 17-hydroxy cortico¬ been maintained in the form of sustained cardiovas¬
steroid (17-OH-CS) levels during acquisition of con¬ cular activation (i.e., increased heart rate and blood
ditioned suppression (i.e., “conditioned emotional re¬ pressure) during clicker presentations over the ex¬
sponse” or CER) have now been described, for tended course of the experiment. Such an account
example, in the rhesus monkey (Mason, Brady, Sc could, to some extent at least, reconcile recurrent con¬
Tolson, 1966). Presentations of a 5-min tone were flicting reports regarding the direction of heart rate
terminated contiguously with brief foot shock (5 ma changes in response to the CS in CER studies (De-
for 0.2 sec); these were superimposed upon a lever¬ Toledo, 1971; DeToledo Sc Black, 1966; DeVietti Sc
pressing performance maintained by food on an inter¬ Porter, 1970; Nathan Sc Smith, 1968; Parrish, 1967;
mittent reinforcement schedule (VI 60"). Only one Smith Sc Nathan, 1967, Smith Sc Stebbins, 1965; Steb-
tone-shock pairing was programmed during each of bins Sc Smith, 1964; Sutterer, Howard, Loth, 8c Obrist,
seven separate experimental sessions. Each CER “ac¬ 1970; Williams, 1969). In addition to the obvious
quisition” trial was programmed 15 min after the start problems associated with species differences (i.e.,
of the animal’s daily lever-pressing session and blood studies with monkey subjects generally report heart
samples were drawn immediately before and immed¬ rate increases while rat studies usually describe heart
iately after each 30-min session. Progressive decreases rate decreases during conditioned suppression), these
Joseph Brady and Alan Harris 599

the “emotion” process or the primacy of either one

(Brady, 1975).
The extended analysis of behaviorally induced en¬
docrine and cardiovascular changes over the past
decade has focused even more intensively upon condi¬
tioned avoidance models, predominantly of the free-
operant or Sidman variety using well-standardized 20-
sec RS and 2-sec SS intervals. There have been several
reconfirmations (Brady, 1965, 1967) of the two-fold to
four-fold elevations in 17-OH-CS levels associated with
even relatively brief, shock-free experimental ex¬
posures to such avoidance contingencies. Furthermore,
marked differences in the hormone response were ob¬
served (Mason, Brady, & Tolson, 1966) when the free-
operant avoidance procedure included a discriminable
exteroceptive warning signal or when “free” shocks
were superimposed upon the performance baseline.
Significantly, the corticosteroid response was consis¬
tently reduced during “discriminated’’ avoidance ses¬
sions including an exteroceptive auditory stimulus
presented 5 sec before shock whenever 15 sec had
elapsed since a previous response, though removal of
the “warning signal” resulted in the immediate reap¬
pearance of the steroid elevations. Conversely, super¬
imposing “free” or unavoidable shocks (at the rate of
approximately 2 or 3 per hour) upon a well-estab¬
lished avoidance performance without a “warning
signaf” wag observed to produce more than a 100%
increase over the elevated corticosteroid levels evident
during the regular nondiscriminated Sidman avoid¬
Fig. 1. Minute-by-minute changes in blood pressure, heart rate, ance procedure.
and leyer-pressing response rate for Monkey A on successive Significant advances have also been made during
three min, clicker-shock trials during acquisition of the condi¬
tioned emotional response. The zero points represent control
the past decade in determining the endocrine and
values calculated from the three-min interval immediately pre¬ cardiovascular consequences of frec operant avoidance
ceding the clicker. (From Brady et al., 1969.) performance requirements during long-term studies of
months and even years. A report by Brady (1965), for
example, describes tbe dfect§ of repeated exposure to
more recent findings, involving biphase cardiovas¬ continuous 72-hour avoidance over periods up to and,
cular changes in the course of CER development and in some cases, exceeding one year, upon patterns of
maintenance, strongly suggest that the temporal thyroid, gonadal, and adrenal hormone secretion in a
course of such experimental observations may be an series of five chair-restrained rhesus monkeys. Two of
important source of variability. More importantly, the the five monkeys participated in the 72-hour avoid¬
results of these studies would seem to support the ance experiment on six separate occasions over a 6-
view that significant aspects of the behavioral and month period with an interval of approximately 4
autonomic effects described are not causally depen¬ weeks intervening between each exposure. The re¬
dent. Rather, it would appear that the cardiovascular maining three animals performed on a schedule which
and skeletal changes are more accurately represented repeatedly programmed 72-hour avoidance cycles fol¬
as independently conditioned effects of the same ex¬ lowed by 96-hour non-avoidance or “rest” cycles (3
perimental procedures. This characterization of emo¬ days “on” and 4 days “off”) for periods up to and ex¬
tional conditioning (i.e., CER) would seem to have ceeding one year.
important implications for the validity of theoretical The two animals exposed to repeated 72-hour avoid¬
formulations which emphasize either the causal inter¬ ance at monthly intervals for 6 months showed a pro¬
dependence of behavioral and physiological events in gressively increasing lever-pressing response rate with
600 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

each of the six successive 72-hour avoidance sessions, all hormone levels, with recovery cycles extending in
illustrated in Figure 2. During the initial 72-hour some instances (thyroid) for 3 weeks following the 72-
avoidance experiment with these two animals, response hour avoidance period. A detailed experimental and
rates averaged 16 and 18 per minute, respectively. Re¬ interpretive analysis of such multiple hormone
sponse-rate values for these same monkeys during the changes induced by exposure to the 72-hour Sidman
sixth 72-hour avoidance experiment averaged 28 and procedure has been provided in an exhaustive multi-
27 resp/min, respectively. In contrast, shock frequen¬ authored monograph (Mason et al., 1968) describing
cies over this same period showed a sharp decline this most systematic series of laboratory studies yet to
within the first two 72-hour avoidance cycles. Hor¬ appear in the psychoendocrine literature.
mone changes related to the repeated 72-hour avoid¬ The three remaining monkeys described in the
ance cycles showed consistent and replicable patterns Brady (1965) report as performing on the 3 days “on,”
over the 6-month experimental period for both 4 days “off” avoidance schedule showed an initial in¬
animals. During the initial experimental sessions, as crease in lever-pressing response rates for approxi¬
shown in Figure 2, both monkeys showed approx¬ mately the first 10 avoidance sessions similar to that
imately three-fold elevations in 17-OH-CS levels dur¬ seen with the two animals described above. By ap¬
ing 72-hour avoidance and returned to near baseline proximately the 20th weekly session with these
levels about 6 days afterwards. The remaining four animals, however, lever-pressing response rates during
monthly experiments were characterized by substan¬ the 72-hour avoidance period had decreased to a value
tial, though diminished steroid responses (approxi¬ well below that observed during the initial avoidance
mately two-fold elevations in 17-OH-CS levels) during sessions, and the performance tended to stablize at
avoidance, with essentially the same 6-day period re¬ this new level for the ensuing weeks of the experi¬
quired for recovery of basal levels. Significant changes ment. In contrast, shock frequencies for all animals
related to the extended avoidance performance were quickly approximated a stable low level within the
also observed in catecholamine, gonadal, and thyroid first two or three exposures to the avoidance schedule

Fig. 2. Steroid levels, avoidance

response rates, and shock fre¬
quencies for animals M-736 and
M-77 during 6 monthly 72-hour
avoidance sessions. (From Brady,
1965.)
Joseph Brady and Alan Harris 601

and seldom exceeded a rate of 2 shocks per hour for ule programmed in this experiment may produce a
the remainder of the experiment. Food and water in¬ dissociation between the avoidance performance and
take, however, remained relatively stable throughout the 17-OH-CS response. Although a definitive analysis
the entire course of the study. The typical pattern, of such relationships is not possible on the basis of
exemplified by monkey M-157, is illustrated in Figure these data alone, a critical role of the temporal
3. The initial 72-hour avoidance sessions were charac¬ parameters (work-rest cycles) is clearly indicated. Cer¬
terized by progressive increases in lever-pressing and tainly, related findings (Mason et al., 1968) on the
elevations in 17-OH-CS levels. In the succeeding course of recovery for a broad range of hormone
weeks, 17-OH-CS levels gradually declined but rose measures provide additional support for this focus
again by the 30th week. The general pattern obtained upon temporal factors in the experimental analysis of
with M-157 was replicated with only minor variations behaviorally induced physiological states.
in the two additional animals on this same experi¬ A trend toward more extended periods of experi¬
mental program. Perhaps the most consistent and mental observation and measurement has also been
striking observation in all three monkeys was the apparent in concurrent avoidance studies focusing
change in responsivity of the pituitary-adrenal system upon cardiovascular changes, particularly in primates.
to the avoidance stress with continued exposure to Both rhesus (Forsyth, 1969) and squirrel monkeys
this procedure over extended time periods. These (Herd, Morse, Kelleher, & Jones, 1969) have been re¬
findings are somewhat at variance with the repeated ported to develop hypertensive blood pressure levels
observations in many previous acute studies of a close with recurrent exposure to free-operant avoidance
positive relationship between steroid elevations and requirements for periods up to and exceeding 12
avoidance performance. These more extended studies, months. Chair-restrained baboons (Findley, Robinson,
in contrast, suggest that continued exposure to re¬ &* Gilliam, 1971) performing on a discrete-trial fixed
peated performance requirements on the time sched¬ ratio instrumental escape-avoidance procedure) how-

Fig. 3. Steroid levels, avoidance

response rates, shock frequen¬
cies, and food and water intake
levels for animal M-157 through¬
out 65 weekly 72-hour avoidance
weeks sessions. (From Brady 1965.)
602 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

ever, were not found to maintain elevated blood pres¬ for the baboons studied by Findley, Brady, Robinson,
sure levels over the year or more during which they 8c Gilliam (1971).
participated in the study (Findley, Brady, Robinson, A recent series of studies at the Johns Hopkins
& Gilliam, 1971). During two six-hour periods each University School of Medicine on cardiovascular
day (separated by rest, feeding, and sleep intervals), changes associated with operant avoidance procedures
the animals were required to respond on an FR 100 (Anderson 8c Brady, 1971, 1972, 1973a, 1973b; Ander¬
schedule to terminate a red light presented intermit¬ son, Daley, Findley, 8c Brady, 1970; Anderson 8c
tently (average interval 5 min) and associated with Tosheff, 1973) provides further evidence which is at
occasional unavoidable shocks. Indeed, the baboons in least consistent with the relationship between muscle
this extended study did show substantial pressure in¬ activity and the dynamic interplay of cardiac output
creases during the actual escape-avoidance perfor¬ and peripheral resistance suggested by Findley et al.
mance intervals within the daily experimental sessions, (1971). The focus of Anderson’s studies with dogs has
and there were some periods during the first several been upon continuous monitoring of blood pressure
months on the program characterized by general eleva¬ and heart rate during free-operant (panel press) shock
tions in both blood pressure and heart rate as illu¬ avoidance, and, significantly, during a pre-avoidance
strated in Figure 4. But a significant differentiating period of fixed duration systematically programmed to
feature of the schedule requirements in these latter precede the required avoidance performance. Under
studies generated substantial ratio performances on a these conditions, a unique divergence between heart
rather heavy Lindsley manipulandum and the per¬ rate and blood pressure changes was observed during
sistent cardiac output (i.e., heart rate) elevations at¬ pre-avoidance intervals up to 15 hours in length, with
tendant upon the recurrent 24-hour exposure to this virtually all animals showing a characteristic systolic
high “work-activity” level do not appear to have been and diastolic pressure increase and either a decrease
a prominent feature of the studies by Forsyth (1969) or no change in heart rate. Comparisons involving
and Herd et al., (1969) involving more conventional similar performance requirements on a variable inter¬
(i.e., non-fixed ratio) free-operant avoidance proce¬ val food reinforcement schedule revealed a markedly
dures. Indeed, this characteristic of the behavioral- different pre-performance cardiovascular pattern char¬
cardiovascular interaction pattern may well have acterized by systematic increases in both heart rate
played a critical role in the long-term return to and blood pressure. And this differential “prepar¬
normotensive pressure levels illustrated in Figure 4 atory” pattern has now been confirmed both between

ESCAPE | •
FOOD 2 * * *
SLEEP A-*"*
FULL PROGRAM ESCAPE REMOVED FULL PROGRAM

Fig. 4. Mean systolic blood

pressure (top three lines in up¬
per section), mean diastolic
blood pressure (bottom three
lines in upper section), and
mean heart rate (bottom sec¬
tion) for baboon Sport plotted
at approximately weekly inter¬
vals during the course of the
experiment for each of the
three major activity cycles.
(From Brady et al., 1971.)
Joseph Brady and Alan Harris 603

individual animals maintained separately on each of ing the same experimental procedure confirm the role
the procedures, and “within” the same animal alter¬ of sympathetic arousal in the sustained pressure eleva¬
nately performing on the avoidance and food rein¬ tions during avoidance but suggest that factors other
forcement schedule, as shown in Figure 5. The bar than sympathetic mediation may be involved in the
graph to the left shows the average blood pressure, progressive pre-avoidance pressure increases (Ander¬
heart rate, and panel response rate for dog Simon son & Brady, 1973b).
during consecutive 10-min intervals for 10 “avoid¬ The results of these experiments establish firm re¬
ance” sessions and illustrates the divergent change in lationships between a broad range of endocrine and
heart rate and blood pressure which occurs during the cardiovascular response processes and free-operant be¬
one-hour pre-avoidance period. The middle graph havioral performances. Both general and specific sup¬
shows the same measures for the same dog taken dur¬ port for these findings has now been provided by
ing 10 subsequent “food” sessions illustrating the numerous published reports with rodents, carnivores,
characteristic concordance between heart rate and and primates (Banks, Miller, & Ogawa, 1966; Black,
blood pressure increases in the course of the preper¬ 1959; Black 8c Dalton, 1965; Brady, 1967, 1969, 1970a,
formance hour. And finally, the bar graph to the right 1970b, 1971, 1972, 1974; Brady, Anderson, Harris, &
shows Simon’s recovery of the pre-avoidance pattern Stephens, 1973; Brady, Findley, 8c Harris, 1971; Brady,
during a single “avoidance” session following ex¬ Harris, & Anderson, 1972; Brady & Nauta, 1972;
posure to the 10 “food” sessions shown in the middle Brown, Schalch, 8c Reichlm, 1971; Brush 8c Levine,
graph. 1966; Coover, Goldman, 8c Levine, 1971; Forsyth,
Direct measurements of cardiac output in such dogs 1968, 1971, 1972; Forsyth 8c Harris, 1970; Forsyth,
prepared with aortic flow probes during exposure to Hoff brand, 8c Melmon, 1971; Frazier, Weil-Melherbe,
the avoidance performance have indicated that the & Lipscomb, 1969; Granger, 1970; Graham, Cohen, &
pre-avoidance pressure changes are attributable to in¬ Shmavonian, 1967; Higgins, 1971; Hokanson, De-
creased peripheral resistance while the pressure in Good, Forrest, 8c Brittain, 1971) Jennings, Averill,
creases during the avoidance performance per se occur Opton, 8c Lazarus, 1970; Jolley, 1970; Kelleher, Morse,
under conditions of increased cardiac output and de¬ & Herd, 1972; Krahenbuhl, 1971; Laforge, 1971; Law¬
creased peripheral resistance. Additional beta adren¬ ler, Meyers, 8c Obrist, 1972; Levine, Gordon, Peterson,
ergic blockade studies with the drug propranolol dur- & Rose, 1970; Malcuit, Ducharme, & Berlanger, 1968;

Simon

SYSTOLIC

Blood Pressure
tmiii Hf 1

DIASTOLIC

Heart Rate
(beats min.)
li.

Key Presses
(reap./min.) tend
Lft.. Hour-1 1-Tw. H—r.-

PRE—AVOIDANCE PRE-FOOD FOOD PERFORMANCE PRE-AVOIDANCE AVOIDANCE

Fig. 5. Average blood pressure, heart rate, and panel response rate during consecutive
10-min pre-performance and performance intervals for 10 “avoidance” sessions (left
panel), 10 “food” sessions (middle panel), and one additional “avoidance” session (right
panel) following exposure to the 10 “food” sessions (middle panel) with the same
dog (Simon). (From Brady et al., 1973.)
604 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

Miller, Banks, 8c Caul, 1967; Miyata 8c Soltysik, 1968; 1962; Sawrey 8c Sawrey, 1963, 1964a, 1964b, 1966), but
Morse, Herd, Kelleher, 8c Gross, 1971; Rose, Mason, 8c replication and confirmation of the reported relation¬
Brady, 1969; Soltysik 8c Kowalski, 1960; Stern 8c Word, ships continue to present problems (Ader, Beels, 8c
1962; Stoyva, Forsyth, 8c Kamiya, 1968; Swadlow, Tatum, 1960; Ader, Tatum, 8c Beels, 1960; Pare,
Hosking, 8c Schneiderman, 1971; Vanderwolf 8c Van- 1964). Similarly, recurrent descriptions of avoidance
derwart, 1970; Weiss, Stone, 8c Harrel, 1970). In many performance effects upon the gastrointestinal system
respects, the changes in absolute levels of selected have characteristically presented something less than a
hormones and autonomic activity can be viewed as consistent picture with regard specifically to the con¬
reflecting relatively undifferentiated consequences of ditions under which ulcers are most likely to occur.
arousal states associated with behavioral responses The reported incidence of peptic ulcers in rhesus
under aversive conditions. The reliable temporal monkeys intermittently exposed to free-operant shock-
course of visceral and steroid changes under such avoidance requirements (Brady, Porter, Conrad, 8c
conditions and the quantitative relations between de¬ Mason, 1958) has proven difficult to repeat under
gree of behavioral involvement and at least short-term some laboratory conditions (Folz 8c Miller, 1964) in¬
physiological response levels has been well docu¬ cluding those under which the study originated
mented. In addition, the organism’s behavioral history (Brady, 1964). Additionally, several investigations
would appear to be critical in determining the nature with laboratory rats in escape-avoidance situations
and extent of autonomic-endocrine response to such have failed to find an incidence of gastric lesions in
performance situations. Clearly however, the most experimental animals which exceeded that of controls,
meaningful level of analysis for such hormone and and in some instances, yoked control animals receiv¬
visceral response processes in relationship to more ing unavoidable shocks showed a greater degree of
chronic emotional interactions would appear to be the ulceration than their avoiding partners (Moot,
broader patterning or balance of secretory and visceral Cebulla, 8c Crabtree, 1970; Pare, 1971; Weiss, 1971a).
change in many interdependent autonomic and en¬ To some extent, a clarification and at least partial
docrine systems which in concert regulate metabolic reconciliation of these apparently conflicting develop¬
events. Indeed, it would appear that such autonomic- ments in the delineation of behavioral effects upon
endocrine response patterns can be usefully differen¬ the gastrointestinal system has been suggested by
tiated in relationship to the historical and situational Weiss in a systematic series of published experimental
aspects of behavioral events. This differential analysis reports from the Rockefeller University in New York
may well provide a first step in the direction of iden¬ (Weiss, 1970, 1971a, 1971b, 1971c). These studies
tifying distinguishable intraorganismic consequences started with the observation that laboratory rats which
which are associated with both episodic and persistent received intermittent tail shock following presentation
behavioral interactions. of a 10-second beeping tone developed significantly
less gastric ulceration than animals receiving the same
Gastrointestinal Effects shock without the “predictability” provided by the
pre-aversive “warning” stimulus. Weiss then examined
While the obvious constraints imposed upon lab¬ the effects of adding an operant escape-avoidance
oratory studies of behaviorally induced somatic path¬ (“coping”) panel-press to the procedure. Under these
ology have continued to limit the range of experi¬ conditions, markedly fewer gastric lesions were found
mental activity in this general area over the past in the experimental animals when compared with
decade, the recent research literature does reflect an “helpless” controls similarly exposed to warning sig¬
abiding concern with the effects of environmental nals and shocks (1 per min for 21 hours) but without
interactions upon the gastrointestinal system (Ader, escape-avoidance “coping.” The interactions between
1971; Smith 8c Hain, 1970). Of particular interest warning signals and the escape-avoidance responses
would seem to be the rather extended analysis of fac¬ were tested in a subsequent experiment in which rats
tors which influence the incidence of peptic ulcers in received electric shock that was preceded by either a
rodents and primates under aversive behavioral con¬ warning signal, a series of signals providing an “ex¬
trol. Some further support for the efficacy of “conflict” ternal clock,” or no signal at all. Under all three con¬
and related procedures in the production of gastric ditions, animals which could avoid and/or escape
lesions in laboratory rats has been provided by studies shock developed less ulceration than did yoked “help¬
focusing upon approach-avoidance methods and com¬ less” animals. In addition, there was a clear difference
parisons involving individual and group “stress” ex¬ in favor of the warning signal condition reducing ul¬
posure (Lower, 1967; Sawrey, 1964; Sawrey 8c Long, ceration as compared to the no-signal controls regard-
Joseph Brady and Alan Harris 605

less of whether they were “helpless” or could escape/ trical stimulation did not disrupt the concurrent
avoid the shock. avoidance behavior, indicating clearly that the corre¬
On the basis of this rather mammoth 180-rat ex¬ lated hippocampal theta activity was not essential to
periment, Weiss theorized that the incidence of peptic the performance. Food-maintained instrumental pedal¬
ulcers may be a function of the interaction between pushing in the dog has also been found to produce an
strength of the escape-avoidance performance (i.e., increase in hippocampal theta rhythms accompanied
the frequency of “coping” responses) and the prob¬ by a highly correlated acceleration in heart rate
ability of discriminable response-contingent signals (Konorski, Santibanez-H, 8c Beck, 1968). Large ampli¬
associated with the absence of aversive stimuli (i.e., tude spindle electrocorticogram activity topograph¬
“feedback” about shock-free conditions). In these ically restricted to the parieto-occipital region has as
terms, the incidence of peptic ulcers in monkeys per¬ well been reported in the cat during lever pressing on
forming on free-operant avoidance is accounted for by a variable interval schedule for milk reward (Marczyn-
the fact that they respond frequently in the absence ski, Rosen, 8c Hackett, 1968), and systematic changes in
of warning stimuli, and that the “safe” signals pro¬ the cortical evoked activity (auditory and visual cor¬
duced by these responses are not readily discriminable tex) of the cat have been related to the rate of dis¬
as “feedback” stimuli. The yoked-control monkeys, in criminative avoidance acquisition (Saunders, 1971).
contrast, characteristically emitted “avoidance” re¬ Several studies involving depth recording of both
sponses only infrequently, received only a few shocks single and multiple neural unit activity during oper¬
well-distributed in time (due to the high performance ant performances have been reported in the brain
rates of the experimental animals), and were found to physiology literature over the past several years. Single
be free of gastrointestinal pathology. Some further neurons in the midbrain tegmentum q£ rat§ have been
confirmation of this formulation has been provided by shown to respond discriminatiyely (i.e., increased fir¬
Weiss in a subsequent series of experiments which ing rates) to tones following a lever press, signaling
showed that the frequency of ulcers was increased in food, water, or no reinforcement under differential
avoidance rats punished with shock for responding deprivation conditions (Phillips 8c Olds, 1969). EEG
(i.e., “coping” response in high strength plus weak potential and amplitude changes in the reticular
“feedback” about shock-free conditions), and decreased formation and amygdala of Wistar rats have also been
in animals producing a brief tone with each shock- shown to differ as a function of food reward and non¬
postponing panel press (i.e., strong “feedback” about reward during a discriminated lever pressing perfor¬
shock-free conditions). mance using auditory signals (Norton, 1970). Multiple
It is perhaps worth noting (though a bit out of unit activity recorded from implanted monopolar
place in the “physiological systems” organization o£ maeroelectrodes in the reticular formation, thalamus,
this chapter) that Weiss and his colleagues at the cochlear nucleus, inferior colliculus, medial genic¬
Rockefeller University have also found hormonal and ulate, and auditory cortex of the cat during discrim¬
body weight changes which reflect the interactions inative instrumental avoidance training with tone as
suggested by the ulcer studies* and that levels of brain a warning signal has also been shown to reflect the
norepinephrine are increased in escape-avoidance sequential development of neuronal conditioning
animals and decreased in non-performing (“helpless”) (Halas, Beardsley, Sandlie, 1970). The process was
shocked rats (Weiss, 1972; Weiss, Stone, & Harrel, observed to start with the reticular formation &nd
1970). progress upward from the cochlear nucleus to the
auditory cortex. While one would be bard put to sup¬
port an interpretation of these findings in terms of
Central Nervous System Interactions
direct operant control of such neurophysiological
Electrophysiological changes continue "to provide a events, some evidence has been produced for dis¬
hazy focus for a handful of studies over the past tinguishable neural activity patterns during classical,
decade involving the use of operant methodology in instrumental, and discrimination learning using the
the analysis of behaviorally-induced central nervous same multiple unit model (Beardsley, 1969). In this
system alterations. Sidman avoidance performance in regard, for example, a recent study of EEG discrimi¬
the rat, for example, has been reported (Bremner, nators of delayed matching to sample behavior in
1964) to change irregular hippocampal EEG activity Macaca nemestrina has shown that coherence values
such that regular 5-7 cps theta activity appears just associated with correct responses were generally higher
prior to and during lever pressing. Significantly, how¬ than those for incorrect responses in the frequency
ever, alteration of this theta pattern by direct elec¬ bands 1-3, 3-4, 5-7, and 8-13 Hz (Campeau, Adey, Dur-
606 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

ham, Tolliver, Ringler, 8c Kanner, 1971). The authors rats recently described by Delini-Stula (1970) as en¬
suggest that such elevated electrophysiological coher¬ during over an extended series of extinction trials may
ences may be a condition for optimal performance be related directly or indirectly to such changes in
under these operant circumstances. brain chemistry.
The relationship between electrical signs of “expec¬
tancy” in the brain as reflected in the EEG contingent
negative variation (CNV) wave form (Walter, 1966) CONTINGENT MODELS
and operant performance requirements has received
increasing experimental attention in several recently Technological and Methodological Developments
reported studies (Delse, Marsh, 8c Thompson, 1972;
Doncliin, Gerbrandt, Leifer, 8c Tucker, 1972; Peters, Research over the past decade concerned with the
Knott, Miller, VanVeen, 8c Cohen, 1970; Rebert, 1972; effects of conditioning procedures on physiological re¬
Tecce, 1972). Without belaboring the details of the sponses has focused prominently upon contingency
several stimulus and response variable analyses (e.g., relationships between antecedent visceral and glan¬
task difficulty, pitch discrimination accuracy, reaction dular changes on the one hand, and experimentally
time) which have been described, the general conclu¬ programmed environmental consequences (e.g., food
sion that a widespread and protracted negative poten¬ delivery and/or shock avoidance), on the other. Stud¬
tial in the frontal cortex can be associated with dis¬ ies within the framework of this instrumental condi¬
criminative stimulus control of minimal instrumental tioning paradigm have clearly emphasized some

response tendencies (Hefferline, 1958) seems now to physiological response systems (e.g., cardiovascular)
be well established both clinically and experimentally. more than others. This uneven distribution of
Quantitative relations between the CNV measure as a measures reflects the technological and methodolog¬
predictor and the properties of the operant response ical developments which have paced the emergence of
(he., accuracy, latency, etc.) however, remain to be an operational laboratory psychophysiology. It is ap¬
worked out in more precise detail. The range and propriate to acknowledge at least some of the major
variability of such performance-related brain electrical innovations and refinements upon which this bur¬
activity changes must also be extended to include the geoning research domain depends. Particularly note¬
“contingent positive variation” wave form (large 180- worthy have been the technical advances in the re¬
300 microvolt positive steady potential shift associated cording and measurement of heart rate (Blizard 8c
with high-voltage alpha activity over the posterior Welty, 1970; Brener, 1965; DeToledo 8c Black, 1965;
Ferraro, Silver, 8c Snapper, 1965; Fitzgerald, Var-
marginal gyrus) reported to be associated with re¬
daris, 8c Teyler, 1968; Krausman, 1970; Pare, Isom, 8c
sponse-produced food reinforcement in deprived cats
(Marczynski, York, 8c Hackett, 1969). Reus, 1970; Perez-Cruet, Tolliver, Dunn, Marvin, 8c
A few reports over the past several years have sug¬ Brady, 1963; Ramsay, Pomerleau, 8c Snapper, 1968),
gested that behaviorally induced changes in the chem¬ blood pressure (DiCara, Pappas, 8c Pointer, 1969; For¬
syth 8c Rosenblum, 1964; Herd 8c Barger, 1964; Kraus¬
ical constitution of the brain may be systematically
man, 1969; Krausman, Ehrlich, 8c Brady, 1972; Perez-
related to operant performance requirements under at
Cruet, Plumlee, 8c Newton, 1966; Swinnen, 1968;
least some aversive control conditions. The work of
Werdegar, Johnson, 8c Mason, 1964), hormone levels
Hyden as reviewed recently by Deguchi (1969) in
relationship to biochemical research on learning gen¬ (Mason et al., 1968), and electromyographic activity
erally supports the finding that RNA in Dieter’s (Dixon, DeToledo, 8c Black, 1969; Hefferline, Keenan,
nucleus of the rat is both increased and changed in Harford, 8c Birch, 1960) in both restrained and free-
composition as a function of wire-climbing, and that moving animals. As these and other psychophysiolog-
similar changes are observed in cortical cells when ical developments (Brown, 1967) have increased the
ease and accessibility of visceral and autonomic
transfers in “handedness” training are required. In¬
strumental escape and avoidance training have also measurement techniques, an ever-broadening range of
biological events has been exposed to experimental
been reported to increase the incorporation of uridine
scrutiny in relationship to behavioral conditioning
into polyribosomes of the mouse brain and produce
procedures.
higher poly- and monosome ratios as compared to
both yoked (receiving shocks only) and non-yoked con¬
Early History
trol mice (Upliouse, Maclnnes, 8c Schlesinger, 1972a,
1972b). It also seems possible that the shock-avoid- Studies concerned with the analysis of instrumental
ance-induced hyperthermia in instrumentally trained visceral-autonomic conditioning represent a relatively
Joseph Brady and Alan Harris 607

recent development in the experimental production of inative control of the specific stimulus complex which
altered physiological states with laboratory roots orig¬ provided the occasion for reinforcement of the re¬
inating in the work of Neal Miller and his students sponse either of raising or lowering the heart rate. In
at Yale in the mid-1960s, (e.g., Miller, 1969). Indeed, a further confirmation of this instrumental heart rate
several earlier reports with human subjects (Crider, conditioning effect, Hothersall and Brener (1969), us¬
Shapiro, 8c Tursky, 1966; Fowler 8c Kimmel, 1962; ing curarized rats and electrical brain stimulation re¬
Johnson, 1963; Kimmel 8c Hill, 1960; Kimmel 8c Kim¬ ward, incorporated a feedback light whenever the
mel, 1963; Lisina, 1958; Razran, 1961; Shapiro, prescribed criterion was met, and extended their in¬
Crider, 8c Tursky, 1964; Shearn, 1962) had foretold of vestigation to include a demonstration of operant
such “operant” learning effects involving visceral extinction when the instrumentally conditioned heart
and autonomic processes, and an extensive literature rate response was no longer reinforced with brain
on “voluntary” physiological control by Yoga medita¬ stimulation.
tion and breathing techniques (Anand & Chhina,
1961; Anand, Chhina, 8c Singh, 1961; Bagchi 8c Wen¬ Mediat!ona! Events
ger, 1959; Wenger & Bagchi, 1961; Wenger, Bagchi, 8c
Anand, 1961) has long been available. But the experi¬ Persistent concern with the degree of skeletal in¬
mental analysis of such instrumental autonomic condi¬ volvement in such instrumental autonomic condition¬
tioning effects in the animal laboratory has clearly ing was reflected in a series of experiments by Black
activated a new research area in the investigation and (1967a) with dogs initially trained on a lever-pressing
application of such “visceral learning” phenomena. shock-avoidance task and subsequently curarized for
operant conditioning of either electromyographic or
heart rate responses. The level of curarization insured
Recent Past Reports
little or no overt movement but did not completely
The earliest reported animal learning experiments eliminate the EMG response. The results showed that
on instrumental autonomic conditioning involved an the instrumentally conditioned heart rate changes
attempt by Miller and Carmona (1967) to change the were closely associated with the conditioned EMC
rate of salivation in a water-deprived dog by reinforc¬ changes which readily transferred to affect the per¬
ing both increases and decreases in this antecedent formance in the non-curarized state. In a later report,
autonomic response with a contingent environmental however, Black (1968) concluded that the heart rate
consequence (i.e., water reward). Although the results response could be conditioned independently of overt
of this study showed clearly that such autonomic re¬ movement, without conceding that the operant
sponses could be controlled by operant procedures, autonomic changes occurred m the absence of some
attention was focused upon the possible role of central event related to the initiation and perfor¬
skeletal muscle activity as a “mediator” of the ob¬ mance of skeletal motor responses. Miller and DiCara
served visceral changes. Since the curarization tech¬ (1967) had in fact hypothesized that such central
nique used to control, such skeletal muscle mediation activity (e.g., motor cortex impulses), classically con¬
produced direct effects upon salivation, an experiment ditioned to elicit heart rate changes, might account
by Trowill (1967) explored the operant control of for the demonstrated autonomic effects. A control pro¬
heart rate in curarized laboratory rats using rewarding cedure, however, involving strong tail shocks (3 ma)
electrical brain stimulation (medial forebram bundle produced smaller heart rate increases (10%) than did
at the level of the posterior hypothalamus) as a rein¬ instrumental conditioning (20%), suggesting that such
forcing consequence. Although the actual changes indirect skeletal mediation was unlikely to account for
were small, both increases and decreases in heart rate the observed heart rate changes, They further showed
were successfully conditioned. A subsequent study by (DiCara & Miller, 1968a) that the completely curarized
Miller and DiCara (1967) showed that the magnitude rat (i.e., no EMG responses from the gastrocnemius
of the instrumentally conditioned heart rate response muscle) could learn to both increase and decrease
could be influenced dramatically (producing changes heart rate as an operant shock-avoidance response,
approximating 20% of the basal values) by a “shap¬ thus establishing that the instrumental autonomic
ing” procedure which required the animals to meet a conditioning effect was not an artifact of the electrical
progressively more difficult criterion in order to ob¬ brain stimulation reinforcer. The results of this study
tain the rewarding brain stimulation. In addition, this also showed that discriminative control over the con¬
experiment also demonstrated that such operant ditioned heart rate change could be developed and
autonomic changes could be brought under discrim¬ maintained by a stimulus which always preceded
608 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

shock presentation. Such instrumentally learned heart ant methods to decrease heart rate have, for example,
rate changes have also been shown to persist in the been shown to subsequently acquire (in the non¬
absence of reinforced practice trials over extended curarized state) shuttle-box escape-avoidance behavior
periods (e.g., 3-month retention tests), and they can be more readily than rats similarly pretrained to increase
relearned after extinction (DiCara 8c Miller, 1968d). their heart rate (DiCara 8c Weiss, 1969). It has also
been reported (Engel 8c Gottlieb, 1970) that blood
pressure changes were significantly positively corre¬
Response Specificity
lated with heart rate decreases instrumentally condi¬
Transfer of learning effects have also been demon¬ tioned as an avoidance response in rhesus monkeys,
strated in a series of studies by DiCara and Miller but such blood pressure effects were uncorrelated with
(1969a, 1969b) in which heart rate changes instru¬ instrumentally conditioned heart rate increases in
mentally conditioned under curare were subsequently these same animals. Differences in the opposite direc¬
(one week later) observed in free-moving rats, and tion have been reported with respect to epinephrine
additional training in the noncurarized state was and norepinephrine by DiCara and Stone (1970) who
shown to produce changes of even greater magnitude. found higher endogenous cardiac and brainstem
Similar transfer was also demonstrated from the non¬ catecholamine levels in rats instrumentally trained to
curarized to the curarized state, and the differences in increase heart rate as compared to rats instrumentally
respiration and gross movement were found to de¬ conditioned to decrease heart rate. Cardiac H3-norep-
crease as the differences in heart rate increased. This inephrine retention studies by these same authors,
emergent response specificity has also been docu¬ however, suggested that rats trained to decrease heart
mented in an experiment by Miller and Banuazizi rate under curare were subjected to greater stress than
(1968) in which independent operant control of both rats trained to increase heart rate. Of additional inter¬
heart rate changes and intestinal contractions was est in this regard is the finding by DiCara, Braun, 8c
demonstrated. Fields (1970) subsequently demon¬ Pappas (1970) that an intact neocortex is essential for
strated the remarkable specificity of such conditioning instrumental autonomic conditioning though this ap¬
effects by producing instrumentally learned increases pears not to be the case with respect to the classical
or decreases in the P-R interval of the EKG inde¬ conditioning of the same heart rate and gastrointesti¬
pendently of changes in the P-P interval. The issues nal responses.
related to a linkage between somato-motor and cardio¬ The instrumental conditioning of blood pressure
vascular activities in the instrumental autonomic con¬ in the curarized rat was convincingly demonstrated
ditioning process have not been definitely settled, by DiCara and Miller (1968b) using a shock avoidance
however. This is seen from a recent report by procedure to reinforce both increases and decreases in
Goesling and Brener (1972) which showed that two systolic pressure levels independently of changes in
different training procedures (i.e., immobility train¬ heart rate and rectal temperature. And in a subse¬
ing versus activity training), given prior to instru¬ quent study, these same investigators (1968c) using
mental heart rate conditioning under curare, can have electrical brain stimulation as a reinforcer with the
a greater effect upon heart rate changes in subse¬ curarized rat dramatically confirmed the specificity of
quently curarized rats than the reinforcement con¬ such instrumental autonomic learning by selectively
tingencies per se. Other types of training (e.g., lever conditioning vasomotor tone increases in one ear and
pressing) given prior to the instrumental conditioning vasomotor tone decreases in the other ear of the same
of heart rate in rats have appeared related to subse¬ animal. Significantly, these conditioned blood flow
quent performance in heart rate conditioning studies changes were not correlated with heart rate, rectal
(Miller 8c DiCara, 1967), but negative findings (i.e., no temperature, or vasomotor tone in the tail, suggesting
relationship between prior bar press training and sub¬ a remarkable and previously unrecognized localization
sequent heart rate conditioning) have also been re¬ of sympathetic action. Following a replication of these
ported (Slaughter, Hahn, 8c Rinaldi, 1970). findings, Pappas, DiCara, and Miller (1970) further
demonstrated that instrumentally conditioned systolic
blood pressure increases and decreases in non-cur-
Response Interactions
arized rats did not transfer to the curarized state, but
Interaction effects involving operant heart rate con¬ that retraining the same animals after curarization
ditioning and other related psychophysiological produced even larger magnitude pressure changes than
processes have been investigated in a number of in the noncurarized state. Similar observations with
animal studies. Curarized rats pretrained using oper¬ respect to the specificity of instrumentally conditioned
Joseph Brady and Alan Harris 609

cardiovascular changes previously reported by DiCara could be established and maintained in baboons by
and Miller (1968a, 1968b) received additional support an operant conditioning procedure which provided
from the finding of Pappas et al. (1970) that the in- for food delivery and shock avoidance programmed
strumentally conditioned blood pressure effects were as environmental consequences contingent upon pre¬
independent of heart rate and gross skeletal activity. scribed increases in diastolic pressure levels. The in¬
strumentally conditioned blood pressure changes were
sustained for intervals up to and exceeding 5 min and
Response Magnitude and Duration
appeared to bear somewhat systematic but complex
Large magnitude diastolic blood pressure elevations temporal relationships to variation in heart rate.
(50-60 mm Hg) conditioned instrumentally as a shock More recently, these same authors (Harris, Gilliam,
avoidance response in the rhesus monkey were first Findley, & Brady, 1973) have extended this basic in¬
reported by Plumlee (1969), though the relatively strumental autonomic conditioning procedure with
short duration of the changes and the observed the baboon to produce more sustained and clinically
postural effects suggested mediation by a Valsalva relevant increases (30-40 mm Hg) in both systolic and
maneuver (i.e., alteration of intrathoracic pressure by diastolic blood pressure throughout daily 12-hour ex¬
abdominal muscle contraction). Somewhat more perimental sessions, as illustrated m Figure 6. Signif¬
modest elevations in mean arterial pressure (e.g., 25 icantly, the maintained instrumentally conditioned
mm Hg) were maintained in squirrel monkeys by Ben¬ blood pressure increases can be seen to be accom¬
son, Herd, Morse, and Kelleher (1969) for periods of panied by elevated but progressively decreasing heart
20 min or longer as a result of an operant reinforce¬ rate levels.
ment contingency arrangement which required the
indicated pressure change as a shock avoidance re¬
Other Response 5ystems
sponse. And Harris, Findley, and Brady (1971) have
also shown that substantial elevations in both systolic Instrumentally conditioned glandular response
and diastolic blood pressure (e.g., 50-60 mm Hg) changes have also been demonstrated in an experi-

Fig. 6. Average blood pressure

and heart rate values for four
baboons over consecutive 40-
min intervals during 16 pre-
experimental baseline determi¬
nations (left panel) compared
with 16 12-hour “conditioning-
on”, 12-hour “conditioning-off”
sessions (right panel). (From
12 4 8 12 4 8 Harris et al., 1973. © 1973 by
a.m. p.m. the American Association for
Time Time the Advancement of Science.)
610 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

ment with curarized rats rewarded by electrical brain reinforcers bridging the temporal gap between the
stimulation for both increases and decreases in the visceral response and its maintaining environmental
rate of urine formation by the kidney (Miller 8c Di- consequences. Such stimulus feedback applications
Cara, 1968). Using insulin-14C and tritiated p- have been convincingly demonstrated in the instru¬
aminohippuric acid (PAH), it was also determined mental heart rate conditioning studies of Hothersall
that both glomerular filtration rate and renal blood and Brener (1969) with curarized rats, and the oper¬
flow were systematically altered by the operant condi¬ ant blood pressure conditioning experiments of Harris
tioning procedure, though heart rate, blood pressure, et al. (1971) with laboratory baboons.
and peripheral blood flow were not, confirming the
high degree of specificity and localization of action
emphasized in previous reports (DiCara 8c Miller,
Interpretive and Theoretical Issues
1968b, 1968c; Fields, 1970; Miller 8c Banuazizi, 1968).
More recently, Banuazizi (1972) has extended the ex¬ Despite this emergent operational orientation,
perimental analysis of selectively conditioned intesti¬ interpretive and theoretical accounts of instrumental
nal contractions in the curarized rat by demonstrat¬ autonomic conditioning procedures and results con¬
ing discriminative stimulus control of both short- and tinue to focus upon “mediational” issues as these have
long-duration intestinal responses reinforced by shock been reviewed and discussed at length in several re¬
avoidance. Significantly, the study also included a cent reports (Grider, Schwartz, 8c Schnidman, 1969;
control for the unconditioned effects of the electric Katkin 8c Murray, 1968; Katkin, Murray, 8c Lachman,
shock as a possible “state dependent” influence upon 1969; Kimmel, 1967; Schoenfeld, 1970, 1971). The
the contractile response, and confirmed the instru¬ controversies surrounding experimental attempts to
mental intestinal conditioning under even these more control such “voluntary mediators” in instrumental
stringent requirements. autonomic conditioning studies have at least empha¬
sized the need to reexamine some basic formulations
regarding conventional distinctions between the two
types of learning (Schoenfeld, 1966, 1972). With re¬
Feedback Factors
spect to more focused concern involving the interrela¬
The role of feedback stimulation in the establish¬ tionship between autonomic-visceral and somato¬
ment and maintenance of instrumentally conditioned motor activity, two more or less distinguishable points
visceral and autonomic response changes has been of view can be identified. Miller, DiCara, and their
emphasized in several recent studies (Harris, Findley, associates (DiCara, 1970; DiCara 8c Miller, 1968a,
8c Brady, 1971; Harris, Gilliam, Findley, 8c Brady, 1968b, 1968c; Miller 8c Banuazizi, 1968; Pappas, Di¬
1973; Hothersall 8c Brener, 1969; Lang, 1970) despite Cara, 8c Miller, 1970; Trowill, 1967) on the one hand,
the somewhat equivocal status of the “interoceptive have appeared to take the position that evidence from
discrimination” issue as reflected in the literature of their own experiments and those of others (Schwartz,
the past decade (Kadden, Snapper, Schoenfeld 8c Kop, Shapiro, 8c Tursky, 1971; Shapiro, Tursky, 8c Schwartz,
1970; Kadden, Schoenfeld, 8c Snapper, 1970; Mandler 1970) supports the independence of somato-motor and
8c Kahn, 1960; Slucki, Adam, 8c Porter, 1965; Slucki, autonomic-visceral control. Black (1967a, 1967b, 1968),
McCoy, 8c Porter, 1969). The early and more recent Brener and Goesling (1968), and Obrist, Webb, Sut-
studies by Miller and his students as reviewed above terer, and Howard (1970), on the other hand, prefer
involved reinforcing stimulus changes (e.g., electric to represent autonomic-visceral and somato-motor
brain stimulation) which provided immediate feed¬ activities as two components of a more general, cen¬
back following visceral or autonomic response varia¬ trally controlled response process. That the dividing
tions. As more extended duration instrumental auto¬ line between the two formulations may not be too
nomic conditioning effects have been investigated, firmly drawn, however, would seem to be indicated by
however, exteroceptive stimuli (e.g., lights, tones) the fact that virtually all the adherents to the latter
linked with the internal environment by advanced school of thought (Black, 1971; Goesling 8c Brener,
electrophysiological recording and amplification tech¬ 1972; Obrist et al., 1970) appear willing to concede
niques (Brown 8c Thorne, 1964; Budzynski 8c Stoyva, that the postulated “normal” linkage between the two
1969; Hefferline 8c Keenan, 1963; Krausman, 1972) systems may be modified in a variety of ways. Indeed,
have provided both digital and analogue presenta¬ a more moderate “separable but interacting” formula¬
tions of critical interoceptive events and processes. In tion (Brady, 1972) of the observed psychophysiological
addition, such feedback stimuli serve as conditioned relationships may better serve the purposes of both
Joseph Brady and Alan Harris 611

clinical and experimental investigators concerned REFERENCES

with the conditions under which dissociation or de¬
coupling of the two systems can and do occur in the Ader, Robert. Experimentally induced gastric lesions. Ad¬
course of ongoing behavioral transactions between vances in Psychosomatic Medicine, 1971, 6, 1-39 (Karger,
Basel).
organism and environment.
Ader, R., Beels, C. C., 8c Tatum, R. Social factors affecting
It is probably noteworthy that amidst this flurry emotionality and resistance of disease in animals. II.
of interpretive and theoretical discourse, recent re¬ Susceptibility to gastric ulceration as a function of inter¬
ports by Miller and Dworkin (1974), DiCara (1974), ruptions in social interactions and the time at which
and others (Brener, Eissenberg, 8c Middaugh, 1974) they occur. Journal of Comparative & Physiological Psy-
chology, 1960, 53, 455-458.
have raised searching questions about the replicability
Ader, R., Tatum, R., 8c Beels, C. Social factors affecting
of earlier findings related to instrumentally condi¬ emotionality and resistance of disease in animals. I. Age
tioned heart rate changes. Specifically, there has been of separation from the mother and susceptibility to gas¬
a progressive decline in the magnitude of learned tric ulcers in the rat. Journal of Comparative & Physio¬
changes in heart rate from the first experiment con¬ logical Psychology, 1960, 53, 446-454.
Anand, B„ 8c Ghhina, G. Investigations on yogis claiming to
ducted in 1966 through those of the 1970’s (Miller 8c
stop their heart beats. Indian Journal of Medical Re¬
Dworkin, 1974). A number of factors are being in¬ search (Bombay) 1961, 49, 90—94.
vestigated in order to provide an account of these Anand, B., Chhina, G. S., It Singh, B. Some aspects of
“extraordinarily perplexing and vexing phenomena/7 electro-encephalographic studies of Yoga. Electroenceph■
as Miller terms it, but no proposed explanation for alography and Clinical Neurophysiology, 1961, 13, 452-
456.
the discrepancy has yet been confirmed. Much of the
Anderson, D, E,, 8c Brady, J. V, Pre-avpidangc blood pres¬
controversy focuses upon technological and methodo¬ sure elevations accompanied by heart rate decreases in
logical details related to the proper use of curare and the dog. Science, 1971, 112, 595-597.
artificial respiration procedures. In this regard, Anderson, D. E., & Brady, J. V. Differential preparatory

Howard, Galosy, Gaebelein, and Obrist (1974) have cardiovascular responses to aversive and appetitive be¬
havioral conditioning. Conditional Reflex, 1972, 7, (2).
enumerated several of the problems associated with
82-96.
curarization, including dosage, criteria for muscular Anderson, D, E., 8c Brady, J, V, Prolonged proaToidance
blockade and artificial respiration, the side effects of effects upon blood pressure and heart rate in the dog.
ganglionic blockade, histamine release, and the alter¬ Psychosomatic Medicine, 1973, IS, 4-12. (a)
Anderson, D. E., & Brabv, J. V. Effects af beta blockade on
ation of sensory processes.
cardiovascular responses to avoidance performance in
Controversies regarding mechanism and methodol¬
dogs. Psychosomatic Medicine, 1973, 35, §4. (b)
ogy notwithstanding, the evidence that operant condi Anderson, d. E., Daley, l. A., Findley, j. d., & Brady,
tioning procedures, whether centrally or peripherally J. V. A restraint system for the psychophysiological study
mediated, exert orderly and specific effects upon the of dogs. Behavioral Research Instrumentation and Meth~
functional properties of physiological systems seems odology, 19?0, 2,191-194,
Anderson, J), £,, & Tqsheff, j, Cardiac output and total
incontrovertible. Clearly, systematic analysis of such
peripheral resistance changes during pro-ay^idan^e pe¬
noninvasive, nonpharmacologic influences upon so¬ riods in the dog. Journal of Applied Physiology, 1973,
matic processes holds considerable promise for en¬ 34, 650-654.
riching both clinical and experimental approaches to Anliker, J- » ram waves and operant behavior. Journal of
the Experimental Analysis of Behavior, 1959, 2, 252.
the physiopathology of health disorders. It is at least
Bagchi, B. K., & Wenger, M. A. Electrophyslologieal cor¬
equally important that this work clearly confirms the
relates of some Yoga exercises. Electroencephalography
importance of the contributions made by basic be¬ and Clinical Neurophysiology, 1959, Suppl. #7, 132-149,
havioral science to a comprehensive physiology of the Banks, J. H., Miller, R. E., 8c Ogawa, N. The development
intact, unanesthetized, conscious organism. Indeed, of discriminated autonomic and instrumental responses
increasing emphasis upon the application of biotele¬ during avoidance conditioning in the rhesus monkey.
Journal of Genetic Psychology, 1966, 108, 199-211.
metric techniques (McCutcheon, 1973) in psycho-
Banuazizi, A. Discriminative shock avoidance learning of
physiological investigations provides tangible recog¬ an autonomic response during curare. Journal of Com¬
nition of this developing frontier. Certainly, the parative & Physiological Psychology, 1972, 80, 236-246.
identification and operational definition of such “in¬ Beardsley, J. V. A comparison of multiple unit activity
ternal state” variables and their functional interac¬ during classical, instrumental, and discrimination learn¬
ing using a noxious unconditioned stimulus. Disserta¬
tions is of central importance in delineating a research
tion Abstracts, 1969, 29 (11-B), 4391-4392.
domain which emphasizes the critical role of environ¬ Benson, H., Herd, J. A., Morse, W. H., 8c Kelleher, R. T.
mental-behavioral influences in the production of Behavioral inductions of arterial hypertension and its re¬
physiological alterations. versal. American Journal of Physiology, 1969, 217, 30-34.
 612 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

Berlanger, D., 8c Feldman, S. Effects of water deprivation Brady, J. V. Conditioning and emotion. In L. Levi (Ed.),
upon heart rate and instrumental activity in the rat. Emotions: Their Parameters and Measurements. New
Journal of Comparative & Physiological Psychology, York: Raven Press. 1975.
1962, 55, 220-225. Brady, J. V., Anderson, D. E., Harris, A. H., 8c Stephens,
Black, A. H. Heart rate changes during avoidance learning J. H. The effects of classical and instrumental condi¬
in dogs. Canadian Journal of Psychology, 1959, 13, 229- tioning upon blood pressure and heart rate in mongrel,
242. monkey and man. Conditional Reflex, 1973, 8 (3), 174-
Black, A. H. Operant conditioning in curarized dogs. Con¬ 175.
ditional Reflex, 1967, 2 (2), 158. (a) Brady, J. V., Findley, J. D., 8c Harris, A. H. Experimental
Black, A. H. Transfer following operant conditioning in psychopathology and the psychophysiology of emotion.
the curarized dog. Science, 1967, 155, 201-203. (b) In H. D. Kimmel (Ed.), Experimental Psychopathology.
Black, A. H. Operant conditioning of autonomic responses. New York: Academic Press, 1971.
Conditional Reflex, 1968, 3 (2), 130. Brady, J. V., Harris, A. H., 8c Anderson, D. E. Behavior
Black, A. H. Autonomic aversive conditioning in infra¬ and the cardiovascular system in experimental animals.
human subjects. In F. R. Brush (Ed.), Aversive Condi¬ In A. Zanchetti (Ed.), Neural and Psychological Mecha¬
tioning and Learning. New York: Academic Press, 1971. nisms in Cardiovascular Disease. Milan: Casa Editrice
Black, A. H., 8c Dalton, A. J. The relationship between (11 Ponte), 1972.
the avoidance response and subsequent changes in heart Brady, J. V., Kelly, D., 8c Plumlee, L. Autonomic and
rate. Acta Biologiae Experimentalis (Warsaw), 1965, 25, behavioral responses of the rhesus monkey to emotional
107-119. conditioning. Annals of the New York Academy of Sci¬
Blizard, D., 8c Welty, R. A technique for monitoring the ence, 1969, 159, 959-975.
heart rate of mice. Psychophysiology, 1970, 1, 143-144. Brady, J. V. 8c Nauta, W. J. H. Principles, Practices and
Brady, J. V. Ulcers in “executive” monkeys. Scientific Positions in Neuropsychiatric Research. Oxford: Perga-
American, 1958, 199, 95-100. mon Press, 1972.
Brady, J. V. Further comments on the gastrointestinal sys¬ Brady, J. V., Porter, R., Conrad, D., 8c Mason, J. Avoid¬
tem and avoidance behavior. Psychological Reports, ance behavior and the development of gastroduodenal
1963, 12, 742. ulcers. Journal of the Experimental Analysis of Be¬
Brady, J. V. Behavioral stress and physiological change: A havior, 1958, 1, 69-72.
comparative approach to the experimental analysis of Brady, J. V., 8c Polish, E. Performance changes during pro¬
some psychosomatic problems. Transactions of the New longed avoidance. Psychological Reports, 1960, 7, 554.
York Academy of Science, 1964, 26, 483-496. Bremner, F. J. Hippocampal activity during avoidance be¬
Brady, J. V. Experimental studies of psychophysiological havior in the rat. Journal of Comparative & Physiolog¬
responses to stressful situations. Symposium on Medical ical Psychology, 1964, 58, 16-22.
Aspects of Stress in the Military Climate. Walter Reed Brener, J. M. The measurement of heart rate. In P. H.
Army Insitute of Research, Government Printing Office, Venables 8c I. Martin (Eds.), Manual of Psychophysio¬
Washington, D.C., 1965. logical Methods. Amsterdam: North-Holland Publishing
Brady, J. V. Operant methodology and the production of Co., 1965.
altered physiological states. In W. Honig (Ed.), Operant Brener, J. M., Eissenberg, E., 8c Middaugh, S. Respiratory
Behavior: Areas of Research and Application. Engle¬ and somato-motor factors associated with operant condi¬
wood Cliffs, N.J.: Prentice-Hall, Inc., 1966. tioning of cardiovascular responses in curarized rats. In
Brady, J. V. Emotion and the sensitivity of the psychoendo- P. A. Obrist, A. H. Black, J. Brener, 8c L. V. DiCara
crine system. In D. Glass (Ed.), N euro physiology and (Eds.), Current Issues in Response Mechanisms, Biofeed-
Emotion. New York: The Rockefeller Univ. Press, 1967. hack and Methodology. Chicago: Aldine Publishing Co.,
Brady, J. V. Recent developments in the measurement of 1974.
stress. In B. P. Rourke (Ed.), Explorations in the Psy¬ Brener, J. M., 8c Goesling, W. J. Heart rate and condi¬
chology of Stress and Anxiety. New York: Longmans, tioned activity. Presented at Meetings of the Society for
1969. Psychophysiological Research. Washington, D.C., 1968.
Brady, J. V. Emotion: Some conceptual problems and psy¬ Brown, C. C. Methods in Psychophysiology. Baltimore:
chophysiological experiments. In M. Arnold (Ed.), Feel¬ Williams and Wilkins, 1967.
ings and Emotions. New York: Academic Press, 1970. (a) Brown, C., 8c Thorne, P. An instrument for signalling heart
Brady, J. V. Endocrine and autonomic correlates of emo¬ rate to unrestrained human subjects. Psychophysiology,
tional behavior. In P. Black (Ed.), Physiological Corre¬ 1964, 1 (2), 192-194.
lates of Emotion. New York: Academic Press, 1970. (b) Brown, G. M., Schalch, D. S., 8c Reichlin, S. Patterns of
Brady, J. V. Emotion revisited. Journal of Psychiatric Re¬ growth hormone and cortisol responses to psychological
search, 1971, 8, 343-384. stress in the squirrel monkey. Endocrinology, 1971, 88,
Brady, J. V. Emotion: Some conceptual problems and psy¬ 956-963.
chophysiological experiments. In D. Singh 8c C. T. Brush, F. R., 8c Levine, S. Adrenocortical activity and avoid¬
Morgan (Eds.), Current Status of Physiologic Psychol¬ ance learning as a function of time after fear condition¬
ogy: A Book of Readings. Monterey, Calif.: Brooks, Cole ing. Physiology and Behavior, 1966, 1, 309-311.
Publishing Co., 1972. Budzynski, T. H., 8c Stoyva, J. M. An instrument for pro¬
Brady, J. V. Psychophysiological syndromes resulting from ducing deep muscle relaxation by means of analog infor¬
overly-rigid environmental control: Concurrent and con¬ mation feedback. Journal of Applied Behavior Analysis,
tingent animal models. In J. H. Cullen (Ed.), Experi¬ 1969, 2, 231-237.
mental Behavior. Dublin, Ireland: Irish Univ. Press, 1974. Campeau, E., Adey, W. R., Durham, R. M., Tolliver, J.
Joseph Brady and Alan Harris 613

D., Ringler, R., 8c Kanner, R. M. EEG discriminators of Journal of Comparative irPhysiological Psychology, 1968,
delayed matching to sample performance in Macaca 65, 1-7. (a)
nemestrina. Physiology and Behavior, 1971, 6 (4), 413- DiCara, L. V., 8c Miller, N. E. Instrumental learning of
418. systolic blood pressure responses in curarized rats: Dis¬
Cannon, W. B. Bodily Changes in Pain, Hunger, Fear and sociation of cardiac and vascular changes. Psychosomatic
Rage. New York: Appleton, 1915. Medicine, 1968, 5, 489-494. (b)
Clark, R., 8c Polish, E. Avoidance conditioning and alcohol DiCara, L. V., 8c Miller, N. E. Instrumental learning of
consumption in rhesus monkeys. Science', 1960, 132, 223- vasomotor responses by rats: learning to respond differ¬
224. entially in the two ears. Science, 1968, 129, 1485-1486. (c)
Conger, J. J., Sawrey, W. L., 8c Turrell, E. S. The role DiCara, L. V., 8c Miller, N. E. Long-term retention of
of social experience in the production of gastric ulcers instrumentally learned heart rate changes in the curar¬
in hooded rats placed in a conflict situation. Journal of ized rat. Communications in Behavioral Biology, 1968,
Comparative & Physiological Psychology, 1958, 31, 214- 2 (pt. A), 19-23. (d)
22°. DiCara, L, V., 8c Miller, N, E. Heart rate learning in the
Coover, G. D., Goldman, L., 8c Levine, S. Plasma cortico¬ non-curarized state, transfer to the curarized state. Physi¬
sterone increases produced by extinction of operant ology and Behavior, 1969, 4, 621-624. (a)
behavior in rats. Physiology and Behavior, 1971, 6 (3), DiCara, L. V., & Miller, N. E. Transfer of instrumentally
261-263. learned heart rate changes from curarized to noncurar*
Grider, A., Schwartz, G. E., & Shnidman, S. On the criteria ized states: Implications for a mediational hypothesis.
for instrumental autonomic conditioning; A reply to Journal of Comparative & Physiological Psychology,
Katkin and Murray. Psychological Bulletin, 1969, 77(6), 1969, 62 (2, pt 1), 159-162. (b)
455-461. DiCara, L., Pappas, B. A., 8c Pointer, F. A technique for
Crider, A., Shapiro, D., 8c Tursky, B. Reinforcement of chronic reading of systemic arterial blood pressure in
spontaneous electrodermal activity. Journal of Compara¬ the unrestrained rat. Behavior Research Methods and
tive & Physiological Psychology, 1966, 61, 20-27. Instrumentation, 1969, 1 (6), 221-223.
Davis, R. C., 8c Berry, F. Gastrointestinal reactions during a DiCara, L., 8c Stone, E. A. Effect of instrumental rate
noise avoidance task. Psychological Reports, 1963, 12, training on rat cardiac and brain catecholamines. Psy¬
135-137. chosomatic Medicine, 1970, 32 (4), 359-368.
Deguchi, T. On the biochemical theory of learning. Jap¬ DiCara, L., & Weiss, J. M. Effect of heart rate learning
anese Psychological Review, 1969, 12 (1), 18-29. under curare on subsequent non-curarized avoidance
Delini-Stula, A. The development and the extinction of learning. Journal of Comparative t Physiological Psy¬
hyperthermia induced by conditioned avoidance behav¬ chology, 1969, 69, 368-374.
ior in rats. Journal of the Experimental Analysis of Dixon, G., DeToledo, L., & Black, A. H. A technique for
Behavior, 1970,11 (2), 213-218. recording electromyographic activity in free-moving rats
Delse, F. G., Marsh, G. R., 8c Thompson, L. W. GNV cor¬ using an all-purpose slip ring commutator. Journal of
relates of task difficulty and accuracy of pitch discrimina¬ the Experimental Analysis of Behavior, 1969,11 (3), 507-
tion. Psychophysiology, 1972, 9, 53-62. 509.
DeToledo, L, Changes in heart rate during conditioned Donchin* E., Gerbrandt, L. A.. Leifer, L„ fc Tucker, l.
suppression in rats as a function of US intensity and type Is the contingent negative variation contingent on a
of OS. Journal of Comparative ir Physiological Psy¬ motor response? Psychophysiology, 1972, 9, 178-188.
chology, 1971, 77, 528-338. Eldridqe, L. Respiration rate change and its relation to
DeToledo, L., 8c Black, A, H, A technique for recording avoidance behavior. Unpublished Doctoral Dissertation,
heart rate in moving rat§. Journal of the Experimental Columbia Unry£f§ltyj 1954i
Analysis of Behavior} 1965, 8 (8), 181-182. Encw? B. T.? & Gottuebj S. H. Th fferentigl Qpef^nj £§Ildl-
DeToLEdo, L.? 8c Black, A. II. Heart rate: Changes during tioftmg of heart rate m the unrestrained monkey. Jour¬
conditioned suppression in rats. Science, 1966, 132, 1404— nal of Comparative J?* Physiological Psychology j 1970,
1406. 73 (2), 217-225.
DeVietti, T. L., & Porter, P. B. Heart rate response during Estes, EE. K., Skinner, B. F. Sdiwc quantitative properties
aversive conditioning. Psychological Reports, 1970, 27 (2), of anxiety. Journal of Experimental Psychology, 1941,
651-658. 29, 390-400.
DiCara, L. V. Learning in the autonomic nervous system. Ferraro, D. P., Silver, M. P., 8c Snapper7 A, O- A method
Scientific American, 1970, 222 (1), 30-39. for cardiac recording from surface electrodes in the rat
DiCara, L. V. Some critical methodological variables in¬ during free-operant procedures, Journal of the Experi¬
volved in visceral learning. In P. A. Obrist, A. H. Black, mental Analysis of Behavior, 1965, 8 (1), 17-18.
J. Brener, 8c L. V. DiCara (Eds.), Current Issues in Re¬ Fields, C. Instrumental conditioning of the rat cardiac
sponse* Mechanisms, Biofeedback and Methodology. control systems. Proceedings of the National Academy of
Chicago: Aldine Publishing Company, 1974. Science, 1970, 69 (2), 293-299.
DiCara, L. V., Braun, J. J., 8c Pappas, B. A. Classical con¬ Figar, S. Conditioned circulatory responses in men and ani¬
ditioning and instrumental learning of cardiac and gas¬ mals. In F. Dow (Ed.) Handbook of Physiology (Sec. 2,
trointestinal responses following removal of neocortex Vol. 3). Baltimore: Williams and Wilkins, 1965.
in the rat. Journal of Comparative ir Physiological Psy¬ Findley, J. D., Brady, J. V., Robinson, W. W., & Gilliam,
chology, 1970, 73 (2), 208-216. W. J. Continuous cardiovascular monitoring in the
DiCara, L., 8c Miller, N. E. Changes in heart rate instru- baboon during long-term behavioral performances. Com¬
mentally learned by curarized rats as avoidance responses. munications in Behavioral Biology, 1971, 6, 49-58.
614 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

Findley, J. D., Robinson, W. W., 8c Gilliam, W. J. A Harris, A. H., Findley, J. D., 8c Brady, J. V. Instrumental
restraint system for chronic study of the baboon. Journal conditioning of blood pressure elevations in the baboon.
of the Experimental Analysis of Behavior, 1971, 15, Conditional Reflex, 1971, 6 (4), 215-226.
69-71. Harris, A. H., Gilliam, W., Findley, J. D., 8c Brady, J. V.
Fitzgerald, R. D., Vardaris, R. M., 8c Teyler, T. J. An Instrumental conditioning of large-magnitude daily 12-
on-line method for measuring heart rate in conditioning hour blood pressure elevations in the baboon. Science,
experiments. Psychophysiology, 1968, 4 (3), 352-353. 1973, 182, 175-177.
Folz, E. L., 8c Miller, F. E., Jr. Experimental psychosomatic Hearst, E., Beer, B., Sheatz, G., 8c Galambos, R. Some
disease states in monkeys. I. Peptic ulcer “executive electrophysiological correlates of conditioning in the
monkeys”. Journal of Surgical Research, 1964, 4, 445- monkey. Electroencephalography and Clinical Neuro¬
453. physiology, 1960, 12, 137-162.
Forsyth, R. P. Blood pressure and avoidance conditioning. Hefferline, R. F» The role of proprioception in the con¬
Psychosomatic Medicine, 1968, 30 (1), 125-135. trol of behavior. Transactions of the New York Academy
Forsyth, R. P. Blood pressure responses to long-term of Science, 1958, 20 (8), 739-764.
avoidance schedules in the unrestrained rhesus monkey. Hefferline, R. F., 8c Keenan, B. Amplitude-induction gradi¬
Psychosomatic Medicine, 1969, 31, 300-309. ent of a small-scale (covert) operant. Journal of the
Forsyth, R. P. Regional blood-flow changes during 72-hour Experimental Analysis of Behavior, 1963, 6, 307-315.
avoidance schedules in the monkey. Science, 1971, 173, Hefferline, R. F., Keenan, B., Harford, R. A., 8c Birch, J.
546-548. Electronics in psychology. Columbia Engineering Quar¬
Forsyth, R. P. Sympathetic nervous system control of dis¬ terly, 1960, 13, 10-15.
tribution of cardiac output in unanesthetized monkeys. Herd, J. A., 8c Barger, A. C. Simplified technique for
Federation Proceedings, 1972, 31 (4), 1240-1244. chronic catheterization of blood vessels. Journal of
Forsyth, R. P., 8c Harris, R. E. Circulatory changes during Applied Physiology, 1964, 19, 791.
stressful stimuli in rhesus moneys. Circulation Research, Herd, J. A., Morse, W. H., Kelleher, R. T., 8c Jones, L. G.
1970, 27, Suppl.: 1-13. Arterial hypertension in the squirrel monkey during be¬
Forsyth, R. P., Hoffbrand, B. I., 8c Melmon, K. L. Hemo¬ havioral experiments. American Journal of Physiology,
dynamic effects of angiotensin in the normal and en¬ 1969, 217, 24-29.
vironmentally stressed monkeys. Circulation, 1971, 44, Higgins, D. J. Set and uncertainty as factors influencing
119-129. anticipatory cardiovascular responding in humans. Jour¬
Forsyth, R. P., 8c Rosenblum, M. A. A restraining device nal of Comparative & Physiological Psychology, 1971,
and procedure for continuous blood pressure recordings 74 (2), 272-283.
in monkeys. Journal of the Experimental Analysis of Hokanson, J. E., DeGood, D. E., Forrest, M. S., 8c Brit¬
Behavior, 1964, 7 (5), 367-368. tain, T. M. Availability of avoidance behaviors in modu¬
Fowler, R. L., 8c Kimmel, H. D. Operant conditioning of lating vascular-stress responses. Journal of Personality
the GSR. Journal of Experimental Psychology, 1962, 63, and Social Psychology, 1971, 19 (1), 60-68.
563-567. Hothersall, D., 8c Brener, J. Operant conditioning of
Frazier, T. W., Weil-Melherbe, H., 8c Lipscomb, H. S. changes in heart rate in curarized rats. Journal of Com¬
Psychophysiology of conditioned emotional disturbances parative & Physiological Psychology, 1969, 68 (3), 338—
in humans. Psychophysiology, 1969, 5 (5), 478-503. 342.
Gantt, W. H. Cardiovascular component of the conditional Howard, J. L., Galosy, R. A., Gaebelein, C. J., 8c Obrist,
reflex to pain, food, and other stimuli. Physiological P. A. Some problems in the use of neuromuscular block¬
Review, 1960, 40, 266-291. ade. In P. A. Obrist, A. H. Black, J. Brener, 8c L. V.
Goesling, W. J., 8c Brener, J. Effects of activity and im¬ DiCara (Eds.), Current Issues in Response Mechanisms,
mobility conditioning upon subsequent heart rate con¬ Biofeedback and Methodology. Chicago: Aldine Publish¬
ditioning in curarized rats. Journal of Comparative & ing Co., 1974.
Physiological Psychology, 1972, 81, 311-317. Jennings, J. R., Averill, J. R., Opton, E. M., 8c Lazarus,
Graham, L. A., Cohen, S. I., 8c Shmavonian, B. M. Urinary R. S. Some parameters of heart rate change: Perceptual
catecholamine excretion during instrumental condition¬ versus motor task requirements, noxiousness, and un¬
ing. Psychosomatic Medicine, 1967, 29 (2), 134-143. certainty. Psychophysiology, 1970, 7 (2), 194-212.
Granger, L. Variation of heart rate in different types of John, E. R., 8c Killam, K. F. Electrophysiological correlates
visual attention situations. Canadian Journal of Psy¬ of avoidance conditioning in the cat. Journal of Pharma¬
chology, 1970, 24 (5), 370-379. cology and Experimental Therapeutics, 1959, 125, 252-
Hahn, W. W., Stern, J. A., 8c McDonald, D. G. Effects of 274.
water deprivation and bar pressing activity on heart Johnson, R. J. Operant reinforcement of an autonomic
rate of the male albino rat. Journal of Comparative & response. Dissertation Abstracts, 1963, 24, 1255-1256.
Physiological Psychology, 1962, 55, 786-790. Jolley, A. Effects of food deprivation and consummatory
Halas, E. S., Beardsley, J. V., 8c Sandlie, M. E. Condi¬ activity on the heart rate of the rat. Animal Behavior,
tioned neuronal responses at various levels of condition¬ 1970, 18 (1), 92-95.
ing paradigms. Electroencephalography and Clinical Kadden, R. M., Schoenfeld, W. N., 8c Snapper, A. G.
Neurophysiology, 1970, 28 (5), 468-477. Cardiac pacing in the rhesus monkey: A technique and
Harris, A. H., 8c Brady, J. V. Animal learning: visceral and some behavioral data. Proceedings of 78th Annual Con¬
autonomic conditioning. Annual Review of Psychology, vention of the American Psychological Association, 1970.
1974, 25, 107-133. Kadden, R. M., Snapper, A. G., Schoenfeld, W. N., 8c Kop,
Joseph Brady and Alan Harris 615

D. Intravenous intracardiac pacing in the rhesus mon¬ Malcuit, G., Ducharme, R., 8c Berlanger, D. Cardiac

key. Communications in Behavioral Biology, 1970, 5, activity in rats during bar-press avoidance and “freezing”
255-258. responses. Psychological Reports, 1968, 23 (1), 11-18.
Katkin, E. S., & Murray, E. N. Instrumental conditioning Malmo, R. B. Slowing of heart rate after septal self-stimula¬

of autonomically mediated behavior: Theoretical and tion in rats. Science, 1961,133, 1129.
methodological issues. Psychological Bulletin, 1968, 70 Mandler, G., 8c Kahn, M. Discrimination of changes in
(1), 52-68. heart rate: Two unsuccessful attempts. Journal of the
Katkin, E. S., Murray, E. N., & Lachman, R. Concerning Experimental Analysis of Behavior, 1960, 3 (1), 21—26.
instrumental autonomic conditioning: a rejoinder. Psy¬ Marczynski, T. J., Rosen, A. H., 8c Hackett, J. T. Post¬

chological Bulletin, 1969, 71 (6), 462-466. reinforcement electrocortical synchronization and facili¬
Kelleher, R., Morse, W., & Herd, J. A. Effects of pro¬ tation of cortical auditory evoked potential in appetitive
pranolol, phentolamine, and methylatropine on cardio¬ instrumental conditioning. Electroencephalography and
vascular function in the squirrel monkey during be¬ Clinical Neuro physiology, 1968, 27 (3), 227-241.
havioral experiments. Journal of Pharmacology and Ex¬ Marczynski, T. J., York, J, L„ 8c Hackett, J, T, Steady
perimental Therapeutics, 1972, 182, 204. potential correlates of positive reinforcement: Reward
Kimmel, E. & Kimmel, H. D. Replication of operant condi¬ contingent positive variation. Science, 1969, 163, 301-
tioning of the GSR. Journal of Experimental Psychology, 304.
1963, 65, 212-213. Mason, J. W., k Brady, J. V. Plasma 174iydroxycortico¬
Kimmel, H. D. Instrumental conditioning of autonomically steroid changes related to reserpine effects on emotional
mediated behavior, Psychological Bulletin, 1967, 67 (5), behavior. Science, 1956,161, 983-984.
337-345. Mason, J, W., Brady, J. V., Polish, E„ Bauer, J. A.,
Kimmel, H. D., 8c Hill, F, A. Operant conditioning of the Robinson, J, A., Rose, R. M,, 8c Taylqr, E, D, Patterns
GSR. Psychological Reports, 1960, 7, 555—562. of corticosteroid and pepsinogen change related to emo¬
KoNOrski, J., SanTIbAnez H, G., 8c Beck, J. Electrical hippo tional stress in the monkey. Science, 1961, 133, 1596-
campal activity and heart rate in classical and instru¬ 1598.
mental conditioning. Acta Biologiae Experimentalis Mason, J. W., Brady, J. V., Robinson, J. A., Taylor, E. D.,
(Warsaw), 1968, 28 (3), 169-185. Tolson, W. W., & Moucey, E. H. Patterns of thyroid,
Krahenbuhl, G. S. Stress reactivity in tennis players. gonadal and adrenal hormone secretions related to psy¬
Research Quarterly, 1971,42 (1), 42-46. chological stress in the monkey. Psychosomatic Medicine,
Krausman, D. Automatic on-line print-out of peak systolic 1961, 23, 446.
and diastolic blood pressures. Psychophysiology, 1969, Mason, J. W., Brady, J. V., 8* Sidman, M. Plasma 17

5 (3), 337-341. hydroxycorticosteroid levels and conditioned behavior in

Krausman, D. Heart rate derived from blood pressure and the rhesus monkey. Endocrinology, 1957, 60, 741-752.
other physiological signals. Psychophysiology, 1970, 7, Mason, J. W., Brady, J. V., Tolliver, G. A., Tolson, W.,

503-507. Gilmore, L., Mougey, E. H., Taylor, E., Robinson, J.,

Krausman, D. A system for providing an on line analogous JOHNSON, T. A., Kenyon, C., Collins, D., Jones, J.,
display of beat-by-beat cardiac output. Medical and Driver, G.; Beer, B., Ricketts, P., Wherry, F,, Pen-

Biological Engineering, 1972, 10, 81-88. NINOTON, L., Goodman, A., 8c Wool, M. Organization of
Krausman, D., Ehrlich, W., 8c Brady, J. V. Preprocessor psychoendocnne mechanisms. Psychosomatic Medicine,
adapts cardiovascular signals to a minicomputer. Psy¬ 1968, 39, 565-808.
chophysiology, 1972, 9 (5), 554-563. MASON, J. W., Brady, J. V., 8c Tolson, W. WJ. Behavioral
LafoRGE, H. The influence of random reinforcement on adaptations and endocrine activity. In R. Levine (Ed.),

heart rate during extinction. Journal of Psychology, Proceedings of the Association for Research m El ervons
1971, 77 (1), 89-99. and Mental Diseases. Baltimore: Williams and Wilkins,
Lang, P. J. Autonomic control or learning to play the in¬ 1966.
ternal organs. Psychology Today, 1970, 3. Mason, J. W., Mangan, G., Bradv, J. V., Conrad, D., &
Lawler, J. E., Meyers, K. A., 8c Obrist, P. A. Cardiovascular Rioch, D. Concurrent plasma epinephrine, norepineph¬
integration during performance on a Sidman avoidance rine, and 17-hydro:x.y£ortieost£roid levels during ee>n.-
schedule. Presented at Annual Meeting of Society for ditioned emotional disturbances in monkeys. Psycho¬
Psyehophysiological Research, Boston, 1972* somatic Medicine, 1961, 23, 344-353.
Levine, M. D„ Gordon, T. P,, Peterson, R, H,, 8c Rose, Mason, J. W., Nauta, W. J. H„ Brady, J. V., Robinson,
R. M. Urinary 17-OH-CS response of high- and low- J. A., 8c Sachar, E. J. The role of limbic system struc¬
aggressive rhesus monkeys to shock avoidance. Physiology tures in regulation of ACTH secretion. Acta Neuro-
and Behavior, 1970, 5 (8), 919-924. vegetativa, 1961, 23, 4-14.
Lisina, M*. I. The role of orientation in the transformation McCutcheon, E. P. (Ed.), Chronically Implanted Cardio¬
of involuntary reactions into voluntary ones. In L. G. vascular Instrumentation. New York: Academic Press,
Voronin, A. N. Leontiev, A. R. Luris, E. N. Sokolov, 8c 1973.
O. S. Vinogradova (Eds.), Orienting Reflex and Explora¬ Miller, N. E. Learning of visceral and glandular responses.
tory Behavior. Washington: American Institute of Bio¬ Science, 1969, 163, 434-445.
logical Sciences, 1958. Miller, N. E., Banks, J. H., 8c Caul, W. F. Cardiac condi¬

Lower, J. S. Approach-avoidance conflict as a determinant tioned responses in avoidance and yoked-control rats.
of peptic ulceration in the rat. Ph.D. Dissertation, Wes¬ Psychonomic Science, 1967, 9, 581-582.
tern Reserve University, 1967. Miller,N. E., 8c Banuazizi, A. Instrumental learning by
616 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

curarized rats of a specific visceral response, intestinal Perez-Cruet, J., Black, W. C., 8c Brady, J. V. Heart rate:
or cardiac. Journal of Comparative fa Physiological Psy¬ differential effects of hypothalamic and septal self-stimu¬
chology, 1968, 65, 1-7. lation. Science, 1963, 140, 1235-1236.
Miller, N. E., 8c Carmona, A. Modification of a visceral Perez-Cruet, J., Plumlee, L., 8c Newton, J. E. O. Chronic
response, salivation in thirsty dogs, by instrumental basal blood pressure in unanesthetized dogs using the
training with water reward. Journal of Comparative fa ring-catheter technique. Proceedings of the Symposium
Physiological Psychology, 1967, 63, 1-6. on Biomedical Engineering, 1966, 1, 383-386.
Miller, N. E., 8c DiCara, L. Instrumental learning of heart Perez-Cruet, J., Tolliver, G., Dunn, G., Marvin, S., 8c
rate changes in curarized rats: Shaping and specificity Brady, J. V. Concurrent measurement of heart rate and
to a discriminative stimulus. Journal of Comparative fa instrumental avoidance behavior in the rhesus monkey.
Physiological Psychology, 1967, 63, 12-19. Journal of the Experimental Analysis of Behavior, 1963,
Miller, N. E., 8c DiCara, L. Instrumental learning of urine 6, 61-64.
formation by rats: changes in renal blood flow. American Peters, J. V., Knott, J. K., Miller, L. H., VanVeen, W. J.,
Journal of Physiology, 1968, 215, 677-683. 8c Cohen, S. J. Response variables and magnitude of the
Miller, N. E., 8c Dworkin, B. R. Visceral learning: Recent contingent negative variation. Electroencephalography
difficulties with curarized rats and significant problems and Clinical Neurophysiology, 1970, 29, 608-611.
for human research. In P. A. Obrist, A. H. Black, J. Phillips, M. I., 8c Olds, J. Unit activity: motivation-depend¬
Brener, 8c L. V. DiCara (Eds.), Current Issues in Re¬ ent responses from midbrain neurons. Science, 1969,
sponse Mechanisms, Biofeedback and Methodology. 165, 1260-1271.
Chicago: Aldine Publishing Co., 1974. Plumlee, L. A. Operant conditioning of increases in blood
Miyata, Y., 8c Soltysik, A. The relations between salivary, pressure. Psychophysiology, 1969, 6, 283-290.
cardiac and motor responses during instrumental per¬ Polish, E., Brady, J. V., Mason, J. W., Thach, J. S., 8c
formance. Acta Biologiae Experimentalis (Warsaw), 1968, Niemeck, W. Gastric contents and the occurrence of
28 (4), 345-361. duodenal lesions in the rhesus monkey during avoidance
Moot, S. A., Cebulla, R. P., 8c Crabtree, J. M. Instru¬ behavior. Gastroenterology, 1962, 43, 193-201.
mental control and ulceration in rats. Journal of Com¬ Porter, R. W., Brady, J. V., Conrad, D., Mason, J. W.,
parative fa Physiological Psychology, 1970, 71 (3), 405- Galambos, R., 8c Rioch, D. Some experimental observa¬
410. tions on gastrointestinal lesions in behaviorally-condi-
Morse, W. H., Herd, J. A., Kelleher, R. T., 8c Gross, S. A. tioned monkeys. Psychosomatic Medicine, 1958, 20, 379-
Schedule-controlled modulation of arterial blood pres¬ 394.
sure in the squirrel monkey. In H. Kimmel (Ed.), Exper¬ Porter, R. W., Conrad, D. G., 8c Brady, J. V. Some neural
imental Psychopathology: Recent Research and Theory. and behavioral correlates of electrical self-stimulation of
New York: Academic Press, 1971. the limbic system. Journal of the Experimental Analysis
Nathan, M. S., 8c Smith, O. A. Jr. Differential conditional of Behavior, 1959, 2, 43-55.
emotional and cardiovascular responses—A training tech¬ Ramsay, D. A., Pomerleau, O. F., 8c Snapper, A. G. Two
nique for monkeys. Journal of the Experimental Analysis methods of obtaining electrocardiograms of chair-re¬
of Behavior, 1968, 11 (1), 77-82. strained monkeys. Conditional Reflex, 1968, 3 (3), 200-
Norton, P. R. Differences in the electrophysiological cor¬ 204.
relates upon receipt of food reward and nonreward in Rasmussen, A. J., Marsh, J. T., 8c Brill, N. Q. Increased
the rat. Brain Research, 1970, 24 (1), 134-138. susceptibility to herpes simplex in mice subjected to
Obrist, P. A., Webb, R. A., Sutterer, J. R., 8c Howard, avoidance-learning stress or restraint. Proceedings of the
J. L. The cardiac-somatic relationship: some formula¬ Society for Experimental Biology and Medicine, 1957,
tions. Psychophysiology, 1970, 6 (5), 569. 96, 183-189.
Pappas, B. A., DiCara, L. V., 8c Miller, N. E. Learning Razran, G. The observable unconscious and the inferable
of blood pressure responses in the noncurarized rat: conscious in current Soviety psychophysiology: Intero¬
Transfer to the curarized state. Physiology and Behavior, ceptive conditioning, semantic conditioning, and the
1970, 5 (9), 1029-1032. orienting reflex. Psychological Review, 1961, 68, 81-147.
Pare, W. The effect of chronic environmental stress on Rebert, C. S. The effect of reaction time feedback on reac¬
stomach ulceration, adrenal function, and consummatory tion time and contingent negative variation. Psycho¬
behavior in the rat. Journal of Psychology, 1964, 57, 143- physiology, 1972, 9, 334-339.
151. Rice, H. K. The responding-rest ratio in the production of
Pare, W. Six-hour escape-avoidance work shift and pro¬ gastric ulcers in the rat. Psychological Reports, 1963, 13,
duction of stomach ulcers. Journal of Comparative fa 11-14.
Physiological Psychology, 1971, 77 (3), 459-466. Rose, R. M., Mason, J. W., 8c Brady, J. V. Adrenal re¬
Pare, W., Isom, K. E., 8c Reus, J. F. Recording the electro¬ sponses to maternal separation and chair adaptation in
cardiogram from the squirrel monkey, Saimira sciureus. experimentally-raised rhesus monkeys (Macaca mulatto).
Physiology and Behavior, 1970, 5, 819-821. Proceedings of Second International Congress on Prima-
Parrish, J. Classical discrimination conditioning of heart tology, Atlanta, Ga., New York: Karger-Basel, 1969.
rate and bar-press suppression in the rat. Psychonomic Ross, G. S., Hodos, W., 8c Brady, J. V. Electronencephalo-
Science, 1967, 9, 267-268. graphic correlates of temporally spaced responding and
Pavlov, I. Uber die normalen Blutdruk-scliwantugen
P. avoidance behavior. Journal of the Experimental Analy¬
bein hunde. Archiv fur Gesamte Physiologie, 1879, 20, sis of Behavior, 1962, 5, 467-472.
215. Saunders, J. C. Selective facilitation and inhibition of
Joseph Brady and Alan Harris 617

auditory and visual evoked responses during avoidance of Science Laboratories, Denison University, 1958, 44,
conditioning in cats. Journal of Comparative & Physio¬ 135-150.
logical Psychology, 1971, 76 (1), 15-25. Slaughter, J., Hahn, W., 8c Rinaldi, P. Instrumental con¬
Sawrey, J. M., 8c Sawrey, W. L. Ulcer production with ditioning of heart rate in the curarized rat with varied
reserpine and conflict. Journal of Comparative & Physio¬ amounts of pretraining. Journal of Comparative & Phy¬
logical Psychology, 1964, 57, 307-309. siological Psychology, 1970, 72 (3), 356-369.
Sawrey, J. M., 8c Sawrey, W. L. Age, weight, and social Slucki, H., Adam, G., 8c Porter, R. W. Operant discrimina¬
effects on ulceration rate in rats. Journal of Comparative tion of an interoceptive stimulus in rhesus monkeys.
& Physiological Psychology, 1966, 61, 464-466. Journal of the Experimental Analysis of Behavior, 1965,
Sawrey, W. L. Conditioned responses of fear in relation¬ 8 (6), 405-414.
ship to ulceration. Journal of Comparative & Physio¬ Slucki, H., McCoy, F. B., 8c Porter, R. W. Interoceptive
logical Psychology, 1961, 54, 347-348. SD of the large intestine established by mechanical
Sawrey, W. L., Conger, J. j., 8c Turrell, E. S. An experi¬ stimulation. Psychological Reports, 1969, 24 (1), 35—42.
mental investigation of the role of psychological factors Smith, O. A., Jr., 8c Nathan, M. A. The development of
in the production of gastric ulcers in rats. Journal of cardiac and blood flow conditional responses during the
Comparative & Physiological Psychology, 1956, 49, 457- acquisition of a differentiated “conditioned emotional
461. response” in monkeys. Conditional Reflex, 1067, 2 (2),
Sawrey, W. L., 8c Long, D. H. Strain and sex differences in 155-i 56.
ulceration in the rat. Journal of Comparative £r Physio¬ Smith, O. A., Jr., 8c Stebbins, W. C. Conditioned blood flow
logical Psychology, 1962, 603-605- and heart rate in monkeys. Journal of Comparative fo
Sawrey, W. L., 8c Sawrey, J. M. Fear conditioning and Physiological Psychology, 1965, 59, 432-436.
resistance to ulceration. Journal of Comparative la3 Physi¬ Smith, R. M., & Hain, J. D. Relationship between somatiza¬
ological Psychology, 1963, 57, 150-151. tion and effects of stress on electrogastric waveforms in
Sawrey, W. L., 8c Sawrey, J. M. Conditioned fear and humans. Psychological Reports, 1970, 27 (3), 755-765.
restraint in ulceration. Journal of Comparative & Physio¬ Soltysik, 5., & Kowalski, M. Studies on the avoidance
logical Psychology, 1964, 56, 821-823. (a) conditioning; Relations between cardiac (Type I) and
Sawrey, W. L., 8c Weisz, J. D. An experimental method of motor (Type II) effects in the avoidance reflex. Acta
producing gastric ulcers. Journal of Comparative Biologiae Expcrimentahs (Warsaw), 1960, 22? 157—170.
Physiological Psychology, 1956, 49, 269-270. Stebbins, W. C., 8c Smith, O. A., Jr. Cardiovascular con¬
Schoenfeld, W. N. Some old work for modern condition comitants of the conditioned emotional response in the
ing theory. Conditional Reflex, 1966, 1, 219-223. monkey. Science, 1964, 144, 881-883.
Schoenfeld, W, N. Oyepk on mediating mechanisms of the Stern, J. A., & Word, T. J. Heart rate changes during avoid¬
conditioned reflex. Conditional Reflex, 1970, 5, 165-170. ance conditioning in the male albino rat. Journal of
Schoenfeld, W. N. Conditioning the whole organism. Con- Psychosomatic Research, 1962, 6, 167-175.
ditional Reflex, 1971, 6, 125-128. Stoyva, J., Forsyth, R. P., & Kamiya, J. Blood pressure
Schoenfeld, W. N. Problems of modern behavior therapy. during sleep in the rhesus monkey before and after stress.
Conditional Reflex, 1972, 7, 33-65. American Journal of Physiology, 1968, 214 (5), 1122—
Schwartz, G, E», Shapiro, D,, k Tursky, B. Learned con¬ 1125.
trol of cardiovascular integration in man through oper¬ Sutterer, J, R.? flowARD? J. L., Loth, E., & Obrist, P. A.
ant conditioning. Psychosomatic Medicine, 1971, 79 (4), Alterations of heart rate and general activity during
57-62- aversive conditioning procedures. Psychophysiology, 1970,
Shapiro, A. P., & Horn, P. W. Blood pressure, plasma 6, 635-636.
pepsinogen and behavior in cats subjected to experi Swablow, H. A., Hoskinc, K E., k Scmneiberman, N-
mental production of anxiety. Journal of Nervous and Differential heart rate conditioning and lever lift sup¬
Mental Disease, 1955, 122, 222-23L pression in restrained rabbits. Physiology and Behavior*
Shapiro, D., Crider, A. B., 8c Tursky, B. Differentiation of 1971, 7 (2), 257-260.
an autonomic response through operant conditioning. Swinnen, M. E. T; Blood pressure digitizer, Proceeding* of
Psychonomic Science, 1964, 1, 147-148. the Annual Conference on Engineering in Mediei™e and
Shapiro, D., Tursky, B., 8c Schwartz, G. E. Differentiation Biology j 1968,10, 18.
of heart rate and systolic blood pressure in man by Tegge, J. J. Contingent negative variation. Archives Gen¬
operant conditioning. Psychosomatic Medicine, 1970, 32 eral Psychiatry, 1972, 24, 1M6.
(4), 417-423. Trowill, J. A. Instrumental conditioning of the heart rate
Shearn, D. W. Operant conditioning of heart rate. Science, in the curarized rat. Journal of Comparative <£r Physio¬
1962, 137, 530-531. logical Psychology, 1967, 63, 7—\\.
Sidman, M. Avoidance conditioning with brief shock and Ufhouse, L. L., MagInnes, J. W., 8c Sghlesinger, K. Effects
no exteroceptive warning signal. Science, 1953, US, 157— of conditioned avoidance training on polyribosomes of
158. mouse brain. Physiology and Behavior, 1972, 8 (6), 1013—
Sidman, M., Mason, J. W., Brady, J. V., 8c Thach, J. 1018. (a)
Quantitative relations between avoidance behavior and Uphouse, L. L.; MacInnes, J. W., 8c Schlesinger, K.
pituitary-adrenal cortical activity. Journal of the Experi¬ Uridine incorporation into polyribosomes of mouse
mental Analysis of Behavior, 1962, 5, 353-362. brain after escape training in an electrified T-maze.
Simson, L. R. Some physiological correlates of psychological Physiology and Behavior, 1972, 8 (6), 1019-1023. (b)
stress in the adrenal organs of the white mouse. Journal Vanderwolf, C. H., 8c Vanderwart, M. L. Relations of
618 THE EXPERIMENTAL PRODUCTION OF ALTERED PHYSIOLOGICAL STATES

heart rate to motor activity and arousal in the rat. Comparative & Physiological Psychology, 1970, 12, 153-
Canadian Journal of Psychology, 1970, 24 (6), 434-441. 160.
Walter, W. G. The relations between electrical signs of Weisz, J. D. The etiology of experimental gastric ulcera¬
expectancy in the human brain and autonomic function tion. Psychosomatic Medicine, 1957, 19, 61-73.
during operant and classical conditioning. Abhund- Wenger, M. A., 8c Bagchi, K. Studies of autonomic func¬
lungen der Deutschen Akademie der Wissenschaften zu tions in practitioners of Yoga in India. Behavioral Sci¬
Berlin, 1966, No. 2, pp. 47-55. ence, 1961, 6, 312-323.
Weiss, J. M. Somatic effects of predictable and unpredict¬ Wenger, M. A., Bagchi, K., 8c Anand, B. K. Experiments
able shock. Psychosomatic Medicine, 1970, 32 (4), 397- in India on “voluntary” control of the heart and pulse.
408. Circulation, 1961, 24, 1319-1325.
Weiss, J. M. Effects of coping behavior in different warning Wenzel, B. M. Changes in heart rate associated with
signal conditions on stress pathology in rats. Journal responses based on positive and negative reinforcement.
of Comparative & Physiological Psychology, 1971, 77, 1- Journal of Comparative & Physiological Psychology,
13. (a) 1961, 42, 638-644.
Weiss, J. M. Effects of punishing the coping response (con- * Werdegar, D., Johnson, D. G., 8c Mason, J. W. A technique
flict) on stress pathology in rats. Journal of Comparative • for continuous measurement of arterial blood pressure
fc Physiological Psychology, 1971, 77, 14-21 (b) in unanesthetized monkeys. Journal of Applied Physi¬
Weiss, J. M. Effects of coping behavior with and without a ology, 1964, 19, 519-521.
feedback signal on stress pathology in rats. Journal of Williams, D. R., 8c Teitelbaum, P. Control of drinking
Comparative & Physiological Psychology, 1971 77 22- behavior by means of an operant-conditioning tech¬
30. (c) nique. Science, 1956, 124, 1294-1296.
Weiss, J. M. Psychological factors in stress and disease. Williams, J. L. Response contingency and effects of pun¬
Scientific American, 1972, 6, 104-113. ishment: changes in autonomic and skeletal responses.
Weiss, J. M., Stone, E. A., 8c Harrel, N. Coping behavior Journal of Comparative & Physiological Psychology,
and brain norepinephrine level in rats. Journal of 1969, 68, 118-125.
 21

Procedures
for the Acquisition
of Syntax

George Robinson

A spoken word is a complex sound that can be objects, abstractions, relations, attributes, events, etc.
produced by the vocal apparatus. As an auditory stim¬ This is just another way of stating the law of gen¬
ulus or as a piece of speech behavior, a word can be eralization. When more than one organism shares the
studied in the laboratory in ways that are similar to same word-concept association, a number of linguistic
the investigation of other stimuli and behaviors. For functions can be performed. A speaker can, by emit¬

example, reinforcing the production of a particular ting the word, induce a change in the behavior of the
word in the presence of a particular stimulus object hearer which is appropriate to the concept the word
can increase the probability that the word will be stands for. For instance, uttering the word blimp on a

produced in the presence of the same object or similar crowded streetcorner is usually followed by most hear¬
objects in the future. Employing a word as a dis¬ ers looking upward.
criminative stimulus in a particular context links it to Uttering the word which stands for a particular
the presence of a set of reinforcement contingencies. concept can serve other functions besides alerting the
Skinner (1957) refers to these two situations as, re¬ hearer to the presence of an instantiation of that con¬
spectively, “controlling” the word and “being con¬ cept. Using the same word in the presence of different
trolled” by the word. Outside the behavior laboratory, things may be used to indicate a class resemblance;
we say that the word means something; the control¬ i.e., the shared word facilitates generalization. Utter¬
ling stimulus or controlled behavior involves or is ing the word which stands for one concept in the
related to the word’s referent, the concept it stands presence of a thing which instantiates a clearly differ¬
for. Individual words associated with “things” in the ent concept may induce the hearer to discover a con¬
world become associated with the concepts which the nection between the concepts. Even a metaphoric
things instantiate. Concepts usually are instantiated connection might be communicated in which the
by many things that fall under them (e.g., the concept speaker indicates that two different things are the
“dog”); proper names normally have only one instanti¬ same in some special sense. With information derived
ation as a concept (e.g., the concept “Victor Borge”). from appropriate nonlinguistic context, uttering a
The term things is used in a general sense to include word can serve as a request for the thing which in-

619
 620 PROCEDURES FOR THE ACQUISITION OF SYNTAX

stantiates the word’s concept or for something con¬ can observe a small portion of a moving conveyer belt
nected with it, as an offer to produce the thing, or as which passes by the windows of each of them. On the
a proposal that the speaker and hearer become in¬ belt at irregular intervals appear objects of a limited
volved with or enact the thing. number of types. Presupposing some survival value in
A set of words each associated with a concept is a being able to anticipate the appearance of particular
lexicon. With a lexicon, speakers can name, mention, kinds of objects, imagine that the creatures have a
or raise the topic of the concept(s) which each in¬ lexicon which enables them to name each of the ob¬
dividual word stands for. The acquisition of a lexicon, jects on the conveyer belt. By calling out these names,
the production of and response to individual words, a creature can apprise her downstream neighbor of
does not seem to be an impossible extrapolation from what will appear in the window and when. The only
laboratory studies of operants and discriminative stim¬ data which the language must encode are the type of
uli. Critics (e.g., Chomsky, 1959) of the learning the¬ each object and its temporal appearance. The tem¬
orist’s approach to language do not direct their main poral order and proximity of the objects are directly
criticisms against explanations of the acquisition of a mapped by the temporal order and proximity of the
lexicon in and of itself. Nor is the learning of lexicons uttered names. The relationship in time among ob¬
considered to be unique to the human species (see jects on the conveyer belt is exactly the same as the
Segal, chapter 22 in this volume). relationship in time among the words which name
However, even with the varied use of contexts the the objects. The “syntax” of this language maps the
uttering of and the response to an individual word temporal order and spacing of objects on the con¬
associated with a concept is only a small part of lin¬ veyer belt as the temporal order and spacing of the
guistic behavior. The speaker of a natural language words which stand for these objects. Such a mapping
not only mentions topics but can say something about is called “iconic” (Peirce, 1931). It hardly presents a
them. To say something about a concept or its in¬ problem for the learning theorist; the system requires
stantiation usually requires more than a single word no learning beyond the lexicon.
—e.g., a word for an object and a word for its at¬ The difficulties arise when we leave the simple
tribute. Not only must the words for the object and world of the conveyer belt creatures. In natural lan¬
attribute be present in an utterance, but there must guage, the word symbols of speech still appear one
also be a way to inform the hearer that the object and after the other in a linear temporal sequence, but the
attribute named by the words in an utterance are relationships among concepts and things communi¬
related. This information about relatedness makes the cated by natural-language speakers are much more
difference between a grammatical string and a list of varied and complicated. Somehow, the linear se¬
words. A grammatical string, therefore, uses two quence must contain sufficient information to enable
information-carrying systems: the hearer to determine how the individual words
work together to encode the relationships among con¬
1. Lexical—the individual words (more properly, cepts and/or things. Encoding these relationships by
lexemes) encode, stand for, signify the things and/ means of a linear sequence of individual symbols re¬
or concepts the message is about; and quires that there be a pattern to the appearance of
2. Syntactic—a system which encodes, maps the rela¬ the symbols. In the pattern must be the syntactic in¬
tionships among, the things and/or concepts as formation from which the hearer infers the way in
relationships among the words in the string.
which the concepts and/or things named by the in¬
dividual words are related.
In English, conceptual relationships are encoded by The simplest kind of syntactic information signals
the order of the words in a string and the use of in¬ the hearer that a sequence of words should be treated
flections (additions to and changes in the forms of as a linguistic unit. Even without any other syntactic
words).
information, this can be done by, for example, tem¬
A learning theoretic account of language must show porally grouping a sequence of words by placing a
how such a system of encoding relationships is ac¬ relatively long pause at its beginning and end. The
quired. This is not a trivial problem. The difficulty information that the individual words in a sequence
can be more easily seen by comparing the real-world, are to be taken as a linguistic unit is adequate for the
natural-language situation to an artificial, very simpli¬ complete understanding of many strings so long as
fied world and language. Imagine a population of or¬ the hearer has learned the meanings of the individual
ganisms that are rooted in place, each one in front of words and has adequate nonlinguistic experience. For
a narrow window. Through the window, a creature example, no additional syntactic information is neces-
George Robinson 621

sary to understand the string “Pregnant deer nibble autoclitic pattern of grammatical categories upon
tree roots.” Even if the word order is scrambled, hearing the following sentence: “With a beat beat the
thereby destroying the syntactic information, knowing beat beat with a beet” (Rhythmically strike the ex¬
that the words function together as a linguistic unit is hausted hippie with a red vegetable). Since a pro¬
enough. There is only one state in the mundane cedure for assigning grammatical categories to each of
world whose description can be derived by consider¬ the words could not succeed by treating each word in
ing these words as a linguistic unit. In a science- isolation, the language user must consider the rela¬
fiction tale of carnivorous trees in ecological competi¬ tionship each word has with other words—i.e., what it
tion with Cervidae, two interpretations are possible. does with and to other words. That is, the language
Even without such bizarre situations, the syntactic user must perform some kind of analysis on the sen¬
datum that the words of a string “go together’1 is tence; syntactic information is not available as an
often, perhaps usually, inadequate by itself to under¬ autoclitic cue by taking the sentence as an unanalyzed
stand a sentence unambiguously. More information whole.
from the pattern of the word sequence is necessary. There is another argument, based on what linguists
Consider the simple sentence “The fast girl pushes call the generative property of language, that the
the fat boy.” Unambiguous interpretation could not grammatical category sequence of a sentence taken as
be made without using the syntactic information a whole could not always be available as an auto-
which is derived from the order of the individual clitic! language users can process (i.e., construct and
words in the sentence. If the order is scrambled, this interpret appropriately) sentences whose grammatical
interpretation would be lost: "The fast boy pushes sequences (or similar ones) they have not been ex¬
the fat girl”; “The girl pushes the fat boy fast”; etc. posed to before. Considering the sequence of gram¬
As we will see later, the most essential syntactic in¬ matical categories of the words of the entire sentence
formation is that which enables the hearer to deter¬ as an autoclitic cue, generalization from previously
mine which words form a subunit which in turn experienced sentences is usually inadequate to explain
works together with other subunits in the string. the processing of novel sentences. The dimensions of
Skinner (1957) suggested that the language user can similarity along which generalization occurs are im¬
learn to discriminate the grammatical categories of possible to specify for unanalyzed whole sentences.
individual words. The series of grammatical cate¬ But the capacity to process novel sentences can be ex¬
gories of the words in a sentence form a pattern which plained if we assume that the language user is able
serves as a complex discriminative stimulus control¬ to combine and permute components of grammatical
ling the response to the set of words in the sentence. sequences.
Skinner called these patterns of grammatical categories The advantages of a generative, combining system
“autoclitics.” In the presence of (i.e., appearing in the can be appreciated by considering the problem of
format of) a particular autoclitic, the individual words training an organism to emit only a restricted subset

are taken to be related in a particular sort of way. For of a set of possible sequential behavior patterns. Im¬
example, in “Alice gives Susan the tereneh” the &UtO- agine a row of six levers each of which ean be pushed
clitic “proper noun-verb-proper noun-artide-noun” up or down from the neutral position, Wo arrange th?
signals that the first proper noun is the agent, the system so that the set of lever patterns leading to reins
verb is the activity, etc. Along wi th intonation forcement (call it “set A”) is defined, for example, as
cues or punctuation, the pattern also indicates the follows; the first lever must be up and somewhere m
mode of the sentence, whether declarative, interroga¬ the sequence an adjacent pair of levers must be up.
tive, etc. There are different patterns m sOt A Out Of 5. total
Although the parallel with laboratory paradigms is of 64 possible patterns altogether. To encourage the
suggestive, Skinner was not clear about how the gram¬ organism to learn the entire set, we arrange the rein¬
matical patterns and their significance could be forcement schedule so that any particular pattern is
learned. There are complicating factors which make reinforced only once every 24 reinforced trials. The
this approach unwieldy. For example, identifying the experiment may be terminated after 24 reinforced
pattern of grammatical categories of a string of words trials.
cannot be done by treating the words one at a time. A subject treating the lever sequence as a set of
The category to which a word belongs often depends whole patterns, even one with a perfect memory (e.g.,
on its relationship to other words in the sentence, a scratchpad), would need to produce all 64 patterns
including those which are not adjacent to the word in order to be sure of having determined all the ones
in question. Consider the difficulty in identifying the belonging to set A. A picture of the whole-pattern
 622 PROCEDURES FOR THE ACQUISITION OF SYNTAX

learner’s information would be a list of 24 sextuples: There are at present no general learning
UUDDUD, UDDUUD, UUUDDU, etc. If you try theories that can account for [linguistic struc¬
this experiment as a paper-and-pencil task, you will ture]. It is hard to imagine what any such
find that very few subjects require experience with all theories could be like. Therefore, it is plausible
64 patterns to be able to produce all and only those of to assume that there can be no such theories.
the reinforced set. In fact, most subjects will be able But the argument is fallacious: Nothing follows
from a lack of imagination.
to list every one of the members of set A before hav¬
ing reinforcement experience with all of them.
What the subjects learn is something about the The set of grammatically correct sentences of a
way in which components of the lever position se¬ natural language is somewhat analogous to set A in
quences go together. It is something which is not itself the lever sequence experiment. The language user
one of, or all of, the members of set A. What they must be capable of generating and recognizing se¬
learn might be pictured something like this: quences of words that are well-formed sentences in
the language. Because the language user, analogous to
U
the lever sequence subject, can generate and identify
more sentences than have previously been experi¬
enced, some sort of underlying information structure
—a* generating algorithm—must be involved. As men¬
tioned earlier, the pattern in which the words of a
sentence are arranged encodes information about the
It is an algorithm, or generating principle, for all and 'ways in which the things and/or concepts indicated
only the sequences of set A. Not only can such an by the words are said to be related. This encoding
algorithm be learned without the subject having ex¬ system is syntax. Since language users employ the syn¬
perience with every member of set A, but the al¬ tax encoding system to process sentences they have not
gorithm is very much easier to remember than the previously experienced, syntax must, like generative
list of 24 sextuples. capacity per se, involve the underlying information
Of subjects who are able to produce nonrandomly structure.
the remaining members of set A without having prior We can describe the lever sequence generating
reinforcement experience with them, we must infer structure as being used by the subject to parse the
that they are using a generating algorithm. Many sub¬ reinforcement contingencies into those of set A and
jects, in fact, are able to describe such an algorithm. those not of set A. Natural language syntax performs
A traditional account involving generalization from an analogous task.
previously learned to novel lever sequences requires The following section demonstrates that the learn¬
the dimensions of similarity between new and old ing and use of the underlying linguistic information
sequences to be specified. A description of the simi¬ structure need not be considered a mysterious process.
larity turns out to be a restatement of the algorithm. A very simple learning mechanism, controlled by lin¬
The algorithm can be called an “underlying struc¬ guistic input and reinforcement, shows how the un¬
ture.” Skinner and other behaviorists have been reluc¬ derlying structure could be acquired and used. The
tant to allow explanations involving underlying struc¬ discussion is based on the syntax crystal model (Block,
tures which include more than input and output Moulton, 8c Robinson, 1974, 1975; Robinson 8c Moul¬
surface data. On the other hand, linguists and some ton, 1972).
psychologists have readily adopted underlying struc¬ The task is threefold. First, the syntactic model must
tures as formal description of language data while show how relationships among concepts can be
disregarding the question of how such structures, gen¬ mapped onto relationships among words—i.e., order
erating algorithms, might be acquired by the princi¬ and inflections. Such a mapping must enable the lan¬
ples of learning. In fact, many linguists, because they guage user to generate a sentence from which another
believe such structures are innate and could not pos¬ language user can determine the way in which the
sibly be learned, assume that the learning theorist concepts named by the words go together. How the
should not have an interest in underlying language concepts go together is what is meant by the seman¬
structure. Lakoff (1973) critically summarizes the ar¬ tics of the string. The model must show how and
guments by some of his fellow linguists (Chomsky in under what conditions an organism can learn to per¬
particular) that underlying linguistic structure should form this reversible mapping operation. Finally, the
not be in the domain of the learning theorist: psychological operations required should be plausible
George Robinson 623

and clear. To the extent that these demands are satis¬ must constrain the correlation of word order and con¬
fied, we shall have demonstrated that syntactic struc¬ ceptual relations to those of the set of well-formed
ture is accessible to the learning theoretic approach. English sentences.
Consider the set of concepts, discussed earlier, The syntax crystal is described in terms of a two-
named by the following words: girl, boy, fast, fat, dimensional crystal built up of rectangular units
push(es). These concepts can combine with each other which join together under the control of “connection
in several ontologically coherent ways. For example, if codes” on their edges. Two rectangles can join to¬
our language user observes the intramural sprint gether, like dominoes, if their apposing edges bear the
champion of Wellesley giving her overweight brother same code. The rectangular “cards” are originally
a swing ride, the concepts are connected as in Figure blank and completely undifferentiated. Words and
1A. connection codes are entered, reinforced, and extin¬
To apprise another of the essentials of the scene, guished during the language-acquisition process. Upon
the language user must transmit the names of the completion, the units of the crystal are so different^
concepts and the set of relationships portrayed by ated that they can only be assembled in structures
connections 1—4. This particular set of relationships is which correctly correlate the word order and con¬
transmitted by ordering the words which name the ceptual relationships of well-formed English sentences,
concepts as follows: “Fast girl push(es) fat boy ” The The body of any completed crystal actually takes the
syntactic model must map the set of brackets into the shape of the correlation map of the sentence (like that
proper sequence of words. Further, the hearer must shown in Figure IB).
be able to determine the correct set of conceptual re¬ We assume that syntax acquisition requires a lin¬
lationships from the order of the words. The reversi¬ guistically competent speaker (Parent) who talks with
ble mapping is portrayed in Figure IB. The connec¬ the language learner (Child) about aspects of the
tions are numbered to show their correspondence to world which the Child can comprehend. We further
those in Figure 1A. We think erf the upper diagram assume that the Child has learned the meanings of
as an orrery (planetarium mobile), and the job of some individual words. To begin the process, the
syntax is revolving the units around the swivel joints Parent might stride about the room and say to the
(the dots) until they are in the proper left-right rela¬ Child: “Parent walksAll the Child need do is recog»
tion to one another. In essence, the syntactic model nize that there is some relationship in the world be¬
tween what it knows to be the referents of “parent**
and “walks." It doesn’t need to distinguish this rela¬
tionship as, for instance, “actor-activity,” It only needs
to detect that a relationship, any relationship, exists.
In this cas£? a description of th6 data used to ^detect
the presence of a relationship might be that at the
occurrence of the utterance “Parent walks," the things
named by the two words are simultaneously very
salient. The Gestalt ‘ law of common fate” could be
invoked here. Whenever the Child detects that two
things spoken of are related, It forms a connection
between the words of the accompanying utterance
which name those things, The conceptual connection,
derived from the Child’s observation of the environ¬
ment, is combined with the order of the words derived
from the utterance (see Figure 2A). The syntax crystal
models this as shown in Figure 2B. The words are en¬
tered on the bottom of adjacent blank cards and the
connection pathway is formed by entering the (arbi¬
trary) codes R, R, A, A, S, S, on apposing edges of the
word cards and two additional blank cards as shown.
Fig. 1. Conceptual structure underlying a sentence portrayed If separated, these four cards can combine with one
by the orrery model. A: Unordered concepts are connected another in only one way. Each pair of identical let¬
according to their scope and dependency relations. B: Using
ters on apposing edges of two cards symbolizes a
syntactic information, the same connections are rotated to yield
the word order of the corresponding sentence. learned connection. Since the word cards are not con-
 624 PROCEDURES FOR THE ACQUISITION OF SYNTAX

that such a connection is “predicate plus qualifier.” It

just needs to recognize that there is some state of the
R
(S) world that obtains when “walks” and “fast” are used
PARENT B
A (
PARENT
1
WALKS
s
s
B T
T
together which does not obtain when they are used
separately. To connect the “walks” card to the new
-,wa - fast “fast” card, the Child constructs the set of cards
shown in Figure 2E and joins it to the previously
created structure as shown in Figure 2F. “Parent” is
A A
R s connected to the entire phrase “walk(s) fast” and not
R S just to the verb “walk(s).”
PARENT WALKS
Placing parentheses around a connection code in¬
dicates that the connection is optional. An optional
code is one which may be left unconnected in a com¬
R A A s pleted crystal. This permits “Walk fast” to be con¬
c S structed using existing cards as a result of the Parent
WALKS saying “Walk fast” to the Child.
BABY
Two or more connected words which are in turn
connected to another part of the crystal structure by
a single card (e.g., the card in Figure 2G) form a “con¬
R
D stituent.” (A single word may also be a constituent.)
BABY
If a word or phrase can be substituted for a particular
constituent and the result is a well-formed utterance,
the new unit is connected by the same code as the
constituent it replaces. For example, the set of cards
s B
B shown in Figure 2H could be substituted for that in
T
S T Figure 21; so it, too, is given the (S) top code.
WALKS FAST Feedback as to whether the substitution of a con¬
stituent results in a well-formed utterance comes from
Fig. 2. The first of a set of syntax crystal cards generated the response of the Parent. In most cases the response
according to the learning algorithm. Conceptual relations and
word order determine the construction of underlying syntactic informs the Child whether its trial utterance is ac¬
structure. See text for details. ceptable. Usually such feedback confounds syntactic,
semantic, and possibly stylistic acceptability; parents
tend to withhold positive responses when their chil¬
nected directly to one another, the connection be¬ dren’s utterances are ungrammatical, nonsensical, im¬
tween word cards is a mediated one. The mediation is polite, or any combination of these faults. In addition,
abstract and internal—the information on the two criteria of acceptability vary widely across parents and
upper cards has no relationship to the linguistic or the age of the child. The important point is that the
nonlinguistic input other than representing the lan¬ linguistic knowledge symbolized by connections in the
guage learner’s detection of a conceptual connection syntax crystal is shaped by parental responses: a coded
between “parent” and “walks” connection is maintained when its use results in an
Now, perhaps the Parent, observing the Child acceptable utterance; a connection is eliminated when
walking, says: “Baby walks ” The Child, noticing that its use results in an unacceptable utterance.
the concepts named by the two words are related, This “on-or-off” connection code is not a necessary
forms a connection between “baby” and “walks.” To feature and is used here for simplicity. Alternatively,
rely as much as possible on past learning, the Child we can suppose that the “strength” of each code varies
uses as many of the previously established cards as as a function of how frequently its use results in an
possible—namely, the three attached cards shown in acceptable utterance. Then if the use of a new code
Figure 2C—and adds the connection code “R” onto results in unacceptable utterances, extinction will
the “baby” card, as shown in Figure 2D. lower the strength of the new code below threshold
For more complicated strings, “Parent walks fast” while the stronger, more frequently used codes re¬
and “Baby walks fast” the Child must realize that main active.
there is some connection between the concepts ex¬ We can now outline in brief a “strict learning pro¬
pressed by “walks” and “fast.” It doesn’t have to know cedure” (as distinct from the “strict training proce-
George Robinson 62 5

dure” of Premack, 1970) which constructs a set of able utterances. In these cases, the old con¬
syntax crystal cards representing underlying linguistic nection codes which are shared by other
structure. constitutions are replaced by new, unique
codes, as in I above.
I. Basic Connection. B. Whenever a unique side code (on a short
A. For a two-word string or a longer string in edge of a card) is assigned, the bottom code
which the Child only knows the meanings of that card must also be changed in order
of two words: to make it unique.
1. Place two blank cards with short edges III. Optionally Connectable Codes.
adjacent.
Whenever an acceptable string can result from
2. Enter the first word on the left card, the leaving a coded edge unconnected, the code is
second word on the right card. made optional by placing parentheses around
3. Place two blank cards above the word it.
cards and assign them arbitrary but dis¬
tinct matching codes that connect them Here follows an illustration of these procedures to
to one another and to the word cards. generate a syntax crystal. For heuristic purposes, the
B. For sentences where three or more words connection code letters suggest the conventional gram¬
are known to name things which are con¬ matical categories: N for noun, etc. Finer distinctions
ceptually connected: Connect two adjacent within a category are made by subscripting the letters
words and then connect the block of four
N1? N2, etc.
cards so formed to the third word and then
Starting with the string "Big parent ” the learning
that block to the fourth word and so on.
procedure generates four cards (reading the card edges
Which two words are first connected to¬
gether as a subunit is determined by the clockwise):
way in which the concepts named by the
words are related. The most important Card Bottom Left Top Right
feature of the relationship is the scope of
1. big — —
each word—i.e., which other words name h
concepts that are operated on by the con¬ 2- Ji — - Ji
cept named by the word in question. An 3. N, — —
ji
example shows what is involved: “Big 4. parent — N, —
parent walks ” The scope of ubig” is “par¬
ent/’ while the scope of “walks” is “big The codes on cards 1 and 4 must be different from
parent,” Therefore, the syntax crystal con¬ each other by rule IIA because “big big” and “patent
nects "big" to "parent” and then connects
parent” are unacceptable strings.
that pair as a unit to “walks.” There are
A second input string, “Parent walks ” results in:
many details about scope which are omitted
from this discussion. Well-formed sentences
can still be processed by a syntax crystal 5. Ni - 5,
without these details, but some power to 6. V, Sj
unambiguously encode complex conceptual 7. walk(s) — Vi —
relations may be lost. The interested reader
should consult the references. and card 4 will have the code N added to the top as
II. Substitution of Constituents. required by rule IIB:
A. If replacing a previously connected constit¬
uent by a new word or string results in an 4. parent — Nx 2 -
acceptable sentence, the new “candidate
constituent” receives at its attachment edge A novel string can be produced by adding (N2) to the
the same code as the one for which it is top of card 3:
substitutable. Existing connection codes
should be tried first. If assigning existing
3. Nj Ji (N2)
codes results in the production of unaccept¬
able sentences, the candidate constituent
As described earlier, the connections are made with
is connected with a new code. As the need
for finer syntactic distinctions develops, ubig parent” and “walks” as constituents which would
some previously acceptable substitutions eventually predominate over “big” and “parent walks”
may result in the production of unaccept¬ The string “Walk slowly” results in:
626 PROCEDURES FOR THE ACQUISITION OF SYNTAX

8. slowly — Bx — The string “Parent can push” results in:

9. Bi B1 - -
10. V2 - 17. can — Mi —
18. Mi — Vi Mx
and the addition of V2 to the top of card 7: 19. v4 - —

7. walk(s) — V12 — and card 11 will have V,t added to its top:

“Parent walks slowly” can be produced by adding 11. push(es) — Vj-4 -

(V]^) to the top of card 10:
Only 10 input strings were provided. With the sub-
10. V2 - (Vi) B1 stitution tests and modification on the basis of paren-
tal reinforcement, the following 19 cards of a syntax
“Parent pushes” results in: crystal result:

11. pushes — V12 _ Card Bottomi Left Top Right

1. big — ji —
after producing and receiving positive feedback for
2. Ji — —
“Parent pushes slowly.” “Pushes” is at this time syn¬ Ji
3. Ni Ji (N2) -
tactically equivalent to “walks.”
4. parent - Nj-s —
This equivalence breaks down with the string “Par¬
ent pushes baby.” At first, “baby” appears to be sub¬ 5. n2 — — Si
stitutable for “slowly” in the previous string. But sub¬ 6. V, Sx — —
stitution for “slowly” in “Parent walks slowly” fails 7. walk(s) — Vlt2,4 —
(assuming the less usual transitive meaning of “walk” 8. slowly — »i —
has not yet been learned). Therefore, “baby” receives 9. Bx — —
a unique top code: 10. v2 — (Vi,2,4) Bx
11. push(es) — V]-4 —
12. baby - N3 - 12. baby — Nls —
13. N3 N3 13. Ns n3 — —
14. V3 — (Vli2,4) n3
Also required is: 15. toward — —
Pi
16. Pi — Bj N3
14. V3 - V, N3 17. can — —
M,
18. Mx — Vi Mi
and the modification of card 11:
19. v4 — —

11. pushes — V123 —

With these 8 word cards and 11 “structure” cards,
over 1,000 well-formed strings of nine words or less
The string “Push parent” allows the Vx code of
can be generated. Examples are: “Big parent can
card 14 to be made optional:
push slowly ” “Baby walks slowly toward parent.”
Connecting word cards hierarchically through
14. V3 - (Vi) N3 structure cards is not an arbitrary procedure. Struc¬
ture cards can be considered to arise from a kind of
and requires that N3 be added to the top of card 4: natural selection process. Think of the word cards to
be connected as surrounded by a matrix of blank
4. parent — N1>2>3 — cards. Initially the language learner generates a (per¬
haps random) variety of paired codes on surrounding
The string “Walk toward baby” results in: cards, connecting the two word cards through a num¬
ber of different pathways, direct word-word connec¬
15. toward — J>1 — tions, and mediated hierarchical connections. Through
16. Px - Bx N3 substitutivity of words and larger constituents, the
George Robinson 627

hierarchical pathways are more general and used more nected produces a structure (like those illustrated in
often compared with the direct word-word pathways skeleton in Figure 3) which shows how the various
which are specific to particular word sequences. Let constituent units go together. This information, the
the connections be strengthened as a function of use, parse, enables the hearer to infer the conceptual rela¬
perhaps supplemented by a “housecleaning” opera¬ tionships encoded by the syntax of the sentence. The
tion which weakens or eliminates the infrequent con¬ reader interested in further details of the model
nections. This does not deny that some word-word should consult the references.
connections remain—they are such stuff as cliches are To sum up, this discussion tries to describe the un¬
made on. derlying structure of natural-language syntax in a
The syntax crystal is thus a system for acquiring manner congenial to the operant approach. Structure
and using underlying linguistic information. It repre¬ is crucially involved in understanding and generating
sents coordinately the relations among concepts and sentences. Many linguists and psychologists view lan¬
the sequence of words which encodes these relations. guage structure as beyond the reach of learning theory
Linguists often test the adequacy of such representa¬ and consider language acquisition to be a SpeciCS-
tions by applying them to ambiguous sentences to specific, innately controlled process. This chapter uses
determine whether the representation is able to por¬ the recently developed syntax crystal model to show
tray the difference between the multiple meanings. A that the hierarchical structure which appears to un¬
famous stumbling block is a sentence like “Visiting derlie language Can be acquired through learning. An
professors can be dull” It may be taken to suggest the iterative mediated association principle, controlled by
possibility either that paying social calls on academics reinforcement, can produce such structure without
is dull or that academics from other universities are assuming innate linguistic organization.
dull. These two interpretations can be “read off” the The model m its present stage of development con¬
two different crystal structures which generate the stitutes a logical argument that structure can develop
sentence. They are shown in Figure 3 in skeleton by learning principles; we have yet to demonstrate
form. The second half of the sentence, “can be dull ” that children do operate this way. There is much to
is the same for both versions, but in the crystal in be done. I hope that researchers in the field of learn¬
Figure 3A it is connected directly to “visiting” with ing will be encouraged to consider the structure of
“professors” connected less directly as a qualifier. In language as falling within their domain.
the crystal in Figure 313 “can be dull” is connected
directly to "professors ” with “visiting” connected less
directly as a qualifier. REFERENCES
Given any well-formed string, the syntax crystal
can be used to determine the conceptual structure(s) Block, h. d., Moulton, j. M., 8c Robinson, G M. A robot
underlying the sentence. Building the crystal up from to learn natural languages. In Proceedings of the 1314
Conference on Biologically Motivated Automata Theory.
the words until no more coded edges remain uncon-
New York; Institute of Electrical and Electronic Engn
r "i n and MIT Research Corporation* 1971.
Block, IT D., Myymw, J, M„ & Robinson, 6. Mi Natural
language acquisition by a robot, International Journal
of Man-AIachme Studies , 1975) f (ip
Chomsky, N. Review of Skinner's Verbal behavior, Lan¬
VISIT ING PROFESSORS CAN BE DULL
guage, 1959, 91, g6-58.
Lakovf, G. Deep language; A letter to the editors. New
York Review of Boohs, 1973, 20 (1), 34.
Peirce, G. The collected papers of Charles Sanders Peirce
(Vol. 1) (C. Hartshorne & P. Weiss, Eds.). Cambridge:
Belknap Press of Harvard University Press, 193E
Premack, D. A. A functional analysis of language. Journal

of the Experimental Analysts of Behavior, 1970, 14, 107—

125.
Fig. 3. Simplified skeleton crystal structures for two interpreta¬ Robinson, G. M., 8c Moulton, J. M. Three games for psy¬
tions of an ambiguous sentence. A: The second half is con¬ chologists to play with. Paper presented to the Duke
nected directly to “visiting,” and less directly to “Professors” University Psychology Colloquium, Spring 1972.
to provide one meaning. B: The reverse arrangement provides Skinner, B. F. Verbal behavior. New York: Appleton-Cen-
a different meaning. tury-Crofts, 1957.
 22

Toward
a Coherent Psychology
of Language*

Evalyn Segal

TOWARD A COHERENT PSYCHOLOGY search on language and verbal behavior that psycho¬
OF LANGUAGE linguists and some behaviorists have been doing, but
perhaps they will find that a wish to learn more has
In the vintage year 1957 two important theoretical been awakened and that they are open to what each
treatises on language appeared, Chomsky’s Syntactic group has to offer. Throughout, I shall speak inter¬
Structures and Skinner’s Verbal Behavior. In the changeably of psycholinguists, generative grammari¬
years since, two rival camps have grown up in psychol¬ ans, and cognitive theorists, although the terms are
ogy, one following Chomsky and the other Skinner. not exactly coextensive. As I see it, the two broad
The rivalry seems vain, however, for grammatical divisions that cut across several disciplines are cogni¬
theory, especially in its more recent versions, rather tive theory and functional theory. Among cognitive
complements the behavioral view of language than theorists I count generative grammarians, psycholin¬
clashes with it. The plan of this chapter is to sketch guists, mentalists, nativists, Gestaltists, information
one version of generative grammatical theory, the theorists, memory theorists, and so on. Among func¬
version of Chomsky’s (1965) Aspects of the Theory of tional theorists I count the many varieties of behav¬
Syntax, which is often called the “standard” generative- iorists and learning theorists as well as students of
transformational theory of syntax. Then I will sketch verbal learning in the traditional functional camp
Skinner’s (1957) theory of verbal behavior. Then I (Hilgard 8c Bower, 1975).
will try to show, in a general way, how the theories
complement one another.
COMPETENCE:PERFORMANCE::STRUCTURE:
Readers will learn little about the interesting re-
FUNCTION

* I thank Derek Hendry, for his sympathetic criticism of an At places in his writings Chomsky claims to be aim¬
earlier version of the manuscript, and Suzette Elgin, whose ex¬
pert criticism rid the manuscript of a few (surely not all) of its
ing only for an economical but comprehensive formal
more glaring linguistic solecisms. (structural) description of the sentences a language

628
 Evalyn Segal
6 29

may contain. This, when reached, would be a theory large number of structures. This system of rules
of “competence.” Ideally, it would be general enough can be analyzed into the three major compo¬
to apply to all natural (human) languages, and not nents of a generative grammar: the syntactic,
just to one language. At other places Chomsky sug¬ phonological, and semantic components.
gests a special relevance of grammatical theory for The syntactic component specifies an infinite
psychology—that is, that grammatical theory shows set of abstract formal objects, each of which
the way to a theory of “performance.” The least rel¬ incorporates all the information relevant to a
single interpretation of a particular sentence.
evance a competence theory would have is that it
Since I shall be concerned here only with the
would prescribe part of the scope that an adequate
syntactic component, I shall use the term “sen¬
functional theory must have, the grammatical facts
tence” to refer to strings of formatives rather
that psychological mechanisms must explain (Catania, than to strings of phones. ... A String of for¬
1972, 1973). Beyond this, the formal theory of gen¬ matives specifies a string of phones uniquely
erative grammar could offer more; its proposals about • • . but not conversely.
deep and surface structures and the formal rules that The phonological component . . . determines
(deductively) generate each of these may (not “must”) the phonetic form of a sentence generated by the
be adopted as hypotheses about psychological reality, syntactic rules. , . . The semantic component

about the psychological machinery that (functionally) determines the semantic interpretation of a sen¬
tence. . . . The syntactic component of a gram¬
generates utterances. Psycholinguists have treated
mar must specify, for each sentence, a deep
grammatical theory in this second way; they have
structure that determines its semantic interpre¬
adopted one or another current version of generative
tation and a surface structure that determines its
theory as a theory of verbal performance (productive phonetic interpretation. . . .
and receptive). The central idea of transformational gram¬
The value of the theory as a linguistic formalism mar is that [deep and surface structures] are, in
(as a theory of competence) of course does not depend general, distinct and that the surface structure
on its utility as a theory of verbal behavior (a theory is determined by repeated application of certain
of performance). Whether generative grammar turns formal operations called “grammatical transfor¬
out to be a useful psychological theory will be an¬ mations” to objects of a more elementary sort.
• • * The syntactic component must generate
swered in the course of time. I will not evaluate that
deep and surface structures, for each sentence,
question, nor will I survey the various competing
and must interrelate them. . . .
variants of generative grammar that are the daily con¬ The base of the syntactic component is a sys¬
cern of linguists. The question I address is this: as a tem of rules that generate a highly restricted
theory of performance, what relation does the theory (perhaps finite) set of basic strings, each with an
of generative grammar (as exemplified in Chomsky’s associated structural description called a base
1965 theory) bear to a behavioral, functional theory Phrase-marker. These base Fhrase=markers are
(as exemplified in Skinner’s 1957 theory)? The sur¬ the elementary units of which deep structures
prising conclusion I have come to is. that the theories are constituted. . . . Underlying each sentence
are in part complementary, in the sense that they of the language there is a sequence of base
deal with different but not conflicting problems, and Phrase-markers, each generated by the base of
the syntactic component, I shall refer to this
in part isomorphic, in the sense that when they do
sequence as the basis of the sentence that it un¬
address the same problems, they propose (roughly) the
derlies.
same answers, d his broad compatibility (neglecting In addition to its base, the syntactic compo¬
differences in detail) has gone unnoticed because of nent of a generative grammar contains a trans¬
the different vocabularies in which the theories are formational subcomponent. This is concerned
couched. As it turns out, mentalists and behaviorists with generating a sentence, with its surface struc¬
are talking about much the same things. ture, from its basis. . . .
Since the base generates only a restricted set
of base Phrase-markers, most sentences will have
CHOMSKY'S STANDARD THEORY OF a sequence of such objects as an underlying
basis. Among the sentences with a single base
TRANSFORMATIONAL-GENERATIVE
Phrase-marker as basis, we can delimit a proper
GRAMMAR
subset called “kernel sentences.” These are sen¬
tences of a particularly simple sort that involve
A generative grammar must be a system of a minimum of transformational apparatus in
rules that can iterate to generate an indefinitely their generation. . . . One must be careful not
630 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

to confuse kernel sentences with the basic strings In each case, the term to the left of the arrow can
that underlie them. be rewritten as the terms to the right of the arrow.
A grammar that generates simple Phrase- Brackets indicate that the term on the left can be
markers . . . may be based on a vocabulary of rewritten as any one of the sets of terms within the
symbols that includes both formatives (the, boy, brackets on the right, and parentheses indicate that
etc.) and category symbols (S, NP, V, etc.). The the parenthetical term is optional. When these rewrit¬
formatives, furthermore, can be subdivided into ing rules are interpreted graphically, the result is a
lexical items (sincerity, boy) and grammatical “tree diagram” that shows diagramatically the hier¬
items (Perfect, Possessive, etc. ;...). (Chomsky,
archical structure of a base phrase marker. For exam¬
1965, pp. 15-18, 65—italics his)
ple, the rules given above would generate, among oth¬
In summary, the “base” in Chomsky’s theory is a ers, this tree diagram:
set of “phrase structure” or “rewriting” rules that
generate “base phrase markers.” (I will give an exam¬
ple in a moment.) The set of all base phrase markers
underlying a sentence are together called the “deep
structure” of that sentence. The base phrase markers PRE-ART OF ART POST-ART A A TENSE V MANNER
of a language are “highly restricted (perhaps finite).” I
However, transformational rules applied to single PAST

base phrase markers or to sets of base phrase markers

allow for the generation of an infinite variety of ob¬ The diagram shows that a sentence (S) is constituted
served surface structures in a language. This is an of two main parts (“immediate constituents”): noun
important point, for Chomsky has often argued (e.g., phrase (NP) and predicate phrase. These two main
Chomsky, 1959) that a theory of verbal performance, constituents are themselves constituted of immediate
just as a theory of verbal competence, must account constituents: for the noun phrase, Det (determiner),
for the infinite number of sentences that can occur in N (noun). S' (embedded sentence), and for the predi¬
a language, and that associationistic stimulus-response cate phrase, Aux (auxiliary) and VP (verb phrase). At
theories, in principle, cannot do so, for they lack the the next level (“up” or “down” makes no difference)
requisite transformational rules that permit a speaker in the phrase structure hierarchy, the determiner is
to generate a variety of surface sentences from a sin¬ shown to consist of the immediate constituents pre¬
gle basis. (But see Robinson, chapter 21 in this vol¬ article of, article, and postarticle; the embedded sen¬
ume and later sections of this chapter.) tence (S') is subdivided into two “dummy” parts to
show that it, like all sentences, consists at the least of
a noun phrase and a predicate phrase; the auxiliary
Deep Structures
in this case is rewritten as its single immediate con¬
The phrase structure rewriting rules that generate stituent, tense; and the verb phrase is subdivided into
the restricted set of base phrase markers incorporate its immediate constituents (V: verb; Manner: adverb
basic grammatical relations like subject-predicate, of manner). Except for the “lexical formative” of and
verb-object, and so on and describe the hierarchical the “grammatical formatives” Past and S', the tree
arrangement of these grammatical categories. Here is diagram shows no “terminal symbols” but only “pre¬
an example, from Chomsky (1965, pp. 106-107), of terminal category symbols,” and it is therefore un¬
some phrase structure rewriting rules: finished. Further rewriting rules would rewrite all the

i. S —> NP Predicate phrase

ii. Predicate phrase —» Aux VP (Place) (Time)
iii. yp rCopula Predicate
V f (NP) (Prep-Phrase) (Prep-Phrase) (Manner)
< js'
[Predicate
iv. Predicate —> rAdjective T
{ (like) Predicate-Nominal/
vii. NP -> (Det) N (S')
xvi. Aux-* Tense (M) (Aspect)
xvii. Det —> (pre-Article of) Article (post-Article)
Evalyn Segal 631

preterminal symbols into terminal formatives, yield¬ markers. [For example, simple transformational rules
ing such basic strings as: would transform “(Past) learn,” “(Past) sign,” “(Past)
lie” and so on into “learned,” “signed,” “lied,” and so
la. Many of the young chimpanzees (S') (Past) learn on.] If the phrase markers associated with set 1 did
quickly. not contain embedded S's, and if the phrase markers
lb. One of the clever chimpanzees (S') (Past) sign associated with set 2 did not contain symbols [such as
fluently. “(That)”] marking their status as embedded, then
lc. Several of the verbal chimpanzees (S') (Past) lie these transformations on individual base phrase mark¬
shamelessly. ers could complete the generation of six simple kernel
sentences:
Surface Structures
3 a. Many of the young chimpanzees learned quickly.
One must distinguish between basic strings (such 3b. One of the clever chimpanzees signed fluently.
as the strings of formatives la, lb, lc) and their as¬ 3 c. Several of the verbal chimpanzees lied shamelessly.
sociated structural descriptions or base phrase markers.
3d. Rumbaugh trained chimpanzees.
The unfinished tree diagram given above shows part
3 e. The Gardners raised chimpanzees.
of the structural description of each of the basic
3f. Chimpanzees talked to Premack.
strings in diagramatic form. (An alternative method
of displaying these phrase markers is “labeled bracket¬
(I am simplifying these examples. In standard genera¬
ing" of the string.) The distinction is important be¬
tive theory, the adjectives in pronominal position in
cause transformations apply to the base phrase mark¬
the basic strings of 1 might actually indicate other
ers and not directly to the basic strings. The basic
embedded phrase markers, associated with the basic
strings, as such, do not show the “derivational history"
strings:
of the strings, but the hierarchical base phrase mark¬
ers do show exactly the derivational history by which
4a. The chimpanzees were young,
the strings were generated by phrase structure rules
4b. The chimpanzees were clever.
from the initial symbol S. This derivational history
(or hierarchical structure of the string) determines the 4c. The chimpanzees were verbal,

transformations that will apply to the deep structure

Then a transformation would permute the adjectives
to generate a surface structure.
from predicate adjective to pronominal position,)
The basic strings of set 1 still contain the symbol
However, the base phrase markers of set 1 do con¬
S', standing for an embedded sentence. That means
tain embedded S' symbols, so further transformational
that the analyses of these deep structures are still in
rules must be applied, to the sets of base phj-a§e mark¬
complete. Another base phrase marker would need
ers associated with (la, 2a), (lb, 2b) and (lc, 2c) to
to be generated by phrase structure rules to yield the
yield the following complex surface sentences (with
structural description of each of these embedded sen¬
associated surface structures):
tences. Suppose the additional base pbrase markers
were associated with these basic strings of terminal
5 a. Many of the young chimpanzees that Rumbaugh
formatives:
trained learned quickly.
5b. One of the clever chimpanzees that the Gardners
2a. (That) Rumbaugh (Past) train chimpanzees.
raised signed fluently.
2b. (That) The Gardners (Past) raise chimpanzees. 5 c. Several of the verbal chimpanzees who talked to
2 c. (That) Chimpanzees (Past) talk to Premack. Premack lied shamelessly.

Then the two base phrase markers associated with All the sentences in set 1 are called “matrix’7 sen¬
strings la and 2a would together make up the deep tences, and all the sentences in set 2 are called “con¬
structure of surface string 3a (see below); base phrase stituent” sentences. One of the axioms of generative
markers associated with strings lb and 2b would theory is that any base phrase marker can contain
make up the deep structure of surface string 3b; and the symbol S'. In other words, a constituent phrase
so on. marker can be at the same time a matrix phrase
Given the deep structures, transformational rules marker, incorporating another embedded sentence
would now be applied to generate surface structures. within it. Transformational-generative syntax is thus
Some transformational rules apply to single base phrase said to be “recursive,” infinitely recursive, and this
 632 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

is one of the reasons that the grammatically ac¬ In fact, any performance model that supposes that
ceptable sentences in any language are infinite in surface structures are generated by transformations
number and, theoretically, infinite in length as well. from deep structures has the problem of accounting
In performance theory, however, there is realistically for deep structures themselves—that is, the problem
a stopping point determined by the speaker’s inability of what sets the phrase structure rewriting rules into
to generate endlessly long sentences and the listener’s operation. In a rival theory of generative grammar
inability to understand them (Gleitman 8c Gleitman, called “generative semantics,” “deep structures and
1970). Such items as “The rat the cat my mother semantic representations are identical” (Fodor et al.,
bought bit died” (Roeper, 1973) occur with a saving 1974, p. 388), but in the standard theory of transfor¬
infrequency outside linguistic discussions.1 mational-generative grammar this is not so, and then
performance theory requires a prior step in “the trans¬
lation function from mentalese to English” that maps
Semantics and Phonology
“from formulas in mentalese to deep structures”
Deep structures “determine semantic interpreta¬ (Fodor et al., p. 389). We shall return to the problem
tion” and surface structures “determine phonetic in¬ of “translating from mentalese to English” at the end
terpretation.” This seems odd at first, because if gen¬ of the chapter.
erative theory is to account for the production of
utterances, one might have supposed that semantics
(what the speaker “means to say”) would determine THE PSYCHOLOGICAL REALITY OF

deep structure, rather than the other way around. On TRANSFORMATIONAL-GENERATIVE

the other hand, if generative theory is to account GRAMMAR

only for the reception (understanding) of utterances,

one might have supposed that the heard utterance As a general approach to formulating the prob¬
(the phonetic event) comes first, and then the listener lems of syntax, the division between deep structures
infers an appropriate surface structure for what was (or their relatively unmodified manifestations as
heard, as in the traditional problem of speech per¬ kernel sentences) and surface structures (generated by
ception. Fodor, Bever, 8c Garrett (1974, p. 389) resolve more elaborate transformations from deep structures)
the puzzle: seems plausible. There is evidence (e.g., Brown, 1973)
that children’s first sentencelike utterances (i.e., ut¬
Direction of information flow makes no differ¬ terances longer than a unit verbal response) have the
ence in a grammar. According to Chomsky, simple and straightforward syntactic structure implied
. . . “the standard theory generates quadruples by the notion of kernel sentence. Only later do utter¬
[of phonetic representations, surface structures, ances appear that have more complex syntactic struc¬
deep structures, and semantic representations]. ture. The point transformational grammarians make
It is meaningless to ask whether it does so by
is that these more complex utterances could be gen¬
‘first’ generating [a deep structure then map¬
erated from one or more kernel sentences (with equiv¬
ping it onto a semantic representation (on one
alent meaning) by application of transformational
side) and a phonetic representation on the
other].” (material in brackets is theirs) rules. It does not seem implausible that complex sen¬
tences are so generated, that kernel sentences are psy¬
They go on to remark: chologically more fundamental, that speakers learn to
generate them before they learn to transform them,
In the case of performance models, however, the and that underlying any complex utterance is one or
situation is quite different. ... A performance more kernel sentences (or base phrase markers) that,
model attempts to specify the actual sequence of in some sense, are prior to it and give rise to it. The
computations which underlies the speaker- idea is not unlike Skinner’s theoretical distinction be¬
hearer’s production and recognition of sentences. tween “primary” verbal behavior and autoclitically
For such models, the direction of information modified verbal behavior.
flow is critical, (p. 390) These issues, however they are formulated, are
within the scope of a functional analysis of verbal be¬
i W. K. Honig pointed out to me that such sentences are not havior, for which the central problem is to determine
infrequent in German. German’s case inflections, very likely, the behavioral laws that account for the performances
help the reader to make sense of multiply embedded sentences,
but English, almost devoid of case inflections, offers no such of speakers and listeners. If, as seems possible, the
help. formal distinction between deep structure and sur-
Evalyn Segal 633

face structure reflects some aspect of psychological Mands

reality, and if, as seems certain, the formal structure
A mand is a verbal response controlled by an ante¬
of utterances is hierarchical, then functional theory
cedent motivational state. Formally, the motivational
must deal with the questions: What are the behav¬
state occupies the position in the threee-term contin¬
ioral variables that determine (or that instantiate)
gency more commonly occupied by a discriminative
deep structures and surface structures? and What
stimulus. It sets the occasion for reinforcement of a
“meanings” are carried by the internal, hierarchical
specific class of response topographies. For example,
structure of utterances that are not carried by the in¬
food deprivation sets the occasion when the mand
dividual response terms viewed as an unstructured
“Food” or the mand “Gimme eats” can be reinforced
string? (See Robinson, chapter 21 in this volume.)
by a listener’s supplying food. Because food is only a
Psycholinguistics, l°ng m the thrall of transforma¬
reinforcer when an organism is food-deprived, depriva¬
tional-generative syntax, seems lately to be shifting
tion sets the occasion for reinforcement of the class of
to a greater concern with semantic problems (i.e.,
food mands. There is a peculiarly close fit between the
problems of stimulus control) and pragmatic problems
motivational condition setting the occasion for rein¬
(i.e., problems of the reinforcement contingencies that
forcement of a mand and the particular form of rein-
shape and maintain verbal behavior) (e.g., Brown,
forcer appropriate to that mand, which Skinner
1973; Carter, 1975; Farwell, 1975; McNeill 1974;
summed up by saying that the mand “specifies its
Moore, 1973; Slobin, 1975; Thompson & Chapman,
reinforcement.” We can avoid the teleological seduc¬
1975). This may reflect psycholinguists’ despair at
tions of this expression, however, by focusing on the
solving the problems of syntax set by transformational-
antecedent motivational control of the mand. (Note,
generative grammarians, but it may alternatively re¬
however, that from the listener’s viewpoint the mand
flect their growing sense that the solution to these
does “specify” very explicitly what behavior the lis¬
syntactic problems will come from a fuller analysis of
tener should engage in.) In Verbal Behavior, Skinner
semantic (stimulus control) and pragmatic (reinforce¬
almost invariably speaks of the antecedent conditions
ment) variables.
controlling emission of the mand as “states of depriva¬
tion or aversive stimulation,” but this seems to give
an unwanted drive-reduction flavor to the mand that,
SKINNER'S FUNCTIONAL THEORY
as MacCorquodale (1970) pointed out, was never in¬
OF VERBAL BEHAVIOR
tended, Among “states of deprivation” Skinner in¬
cluded such items as needing a pencil to complete a
Skinner’s theory, as MacCorquodale (1969, 1970)
drawing, but completing a drawing is not your usual
emphasized, is a plausible extrapolation of the laws of
primary reinforcer, and needing a pencil is not your
animal operant behavior to human verbal behavior,
usual primary state of deprivation. MacCorquodale
embodying the hypothesis that the facts of verbal
(1970) suggests that mands for such conditioned rein¬
behavior can be accounted for by application of the
forcers as pencils to complete drawings are best un¬
familiar “three-term contingency” of operant analysis.
derstood as under the control of some other, IIIOIC
In this case, the three-term contingency must explicate
primary deprivation correlated with some other, more
two functional relations: (1) the function between dis¬
primary reinforcer with which pencils and completed
criminative stimuli (or a motivational state) and the
drawings have been paired in the speaker’s past his¬
response they control, and (2) the function between
tory. I wonder if a simpler solution would not be to
the operant unit constituted of this antecedent vari¬
abandon the primary-conditioned reinforccr distinc¬
able-response relation and the reinforcing conse**
tion, as Premack (1959) did, and simply to regard rein¬
quences shaping the operant (S—R) unit,
forcers and their correlated motivational states as in
Skinner defined the domain of verbal behavior as
various ways situationally determined. The neo-
operant behavior whose third term (reinforcement) is
Hullian concept of incentive motivation seems a
mediated by other organisms. The definition was in¬
suitable way to talk about the antecedent motivational
tended to capture, in a nutshell, the communicative
state controlling a mand. (I shall return to this
function of verbal behavior, the fact that it constitutes
shortly.)
a social transaction between a speaker and a listener
and that the transaction has utility for both partici¬
Echoics
pants. Having defined the domain, Skinner proceeded
to subdivide it by classifying verbal operants in terms An echoic is a verbal response controlled by an
of their antecedent controlling variables. antecedent verbal stimulus where, in addition, rein-
634 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

forcement depends on a strict one-to-one correspond¬ “definitive” for that “word” (the community imposes
ence between the sound pattern of the discriminative differential reinforcement contingencies to bring this
stimulus and the auditory product of the response. about), then, for Skinner, we have a case of generic
The stimulus and the auditory product of the re¬ extension of the tact and, for cognitive theorists, a case
sponse must match, within limits set by the reinforc¬ of concept learning. (See Robinson, Chapter 21 in this
ing community. For example, if a parent says “Dino¬ volume.) Despite the vocabulary difference, every¬
saur,” and a child repeats, “Dinosaur,” the parent’s one is talking about the same phenomenon. For ex¬
utterance serves as a verbal stimulus for the child’s ample, when a child who has learned to tact an ac¬
echoic response. Fragmentary echoics also occur. In tivity performed by his parent as “walking” extends
this case some part of the response matches the sound the response to other instances of the activity, such
pattern of the prior stimulus, but the match is not as his own walking or the dog’s walking, the child has
complete. Alliteration is an example of fragmentary acquired the generic tact “walking,” or alternatively,
echoic verbal behavior. If someone says, “Sing,” and the concept of walking.
a speaker responds, “A silly song,” the repeated s When the basis of transfer of the tact is accidental
sounds suggest fragmentary echoic control. Rhymes stimulus properties that just happen to be shared by
are other examples of fragmentary echoics. Whenever two environmental events (one of them, of course, the
the sound match is not complete, then other variables event in whose presence the tact was originally rein¬
must be supposed to have determined other aspects forced), Skinner speaks of metaphoric extension of
of the speaker’s response. the tact, and cognitive theorists of overextension of a
word or concept. Again, despite the vocabulary differ¬
ence, everyone is talking about exactly the same phe¬
Intraverbals
nomenon. The literature on children’s language
An intraverbal is a verbal response controlled by acquisition abounds with amusing examples of meta¬
an antecedent verbal stimulus where, in addition, th£ phoric extension. For example, a child who first
sound match is lacking. It is the verbal unit par excel¬ learned to utter “moon” in the presence of the
lence of conventional paired-associates verbal learning. moon later extended the response to round postmarks
If a parent says, “One, two,” and a child responds, and the letter O. (This is so common a metaphoric
“Button your shoe,” the response as a whole is most extension, apparently, that we have an expression
likely a rote-learned intraverbal. (The sound match “moon-faced,” meaning round-faced.) Another child
between the last part of “two” and the last part of who learned the response “fly” in the presence of the
“shoe” suggests fragmentary echoic control as well. In appropriate little insect later extended the response to
a given instance fragmentary echoic control may not his own little toes (Clark, 1973, pp. 80-81).
occur: the child’s “shoe” may simply be part of the Finally, when the basis of transfer of the tact is not
rote-learned unit phrase “Button your shoe.” Still, the shared properties of two environmental events, but
historical origin of the rhyme certainly owed some¬ rather some fortuitous association of events, then
thing to fragmentary echoic control.) More “signifi¬ Skinner speaks of metonymic extension and everyone
cant” examples of intraverbals are easy to find. A lot else speaks of plain “associative learning” or, to revive
of the educated speaker’s repertoire consists of “book- an apt phrase of Thorndike’s (1911), “associative shift¬
learned” or otherwise rote-learned intraverbals. ing.” According to Ervin-Tripp (1973, p. 267), meto¬
nymic extension may be the origin of the syntax of
Tacts
the possessive in children’s speech. It is common, for
example, to find a young child saying, “Mommy,”
A tact is a verbal response controlled by a non¬ when pointing to Mommy’s shoe. (Later on the child
verbal discriminative stimulus. It is the unit most will learn to tact the shoe as a generic, tact Mommy
susceptible to “semantic” generalization, what Skin¬ as a metonymic, order them and affix -'5 to the
ner and psycholinguists (e.g., Moore, 1973) have both metonymic, and utter “Mommy’s shoe.”)
called “extension.” When a tact, initially learned to An example of metonymic extension offered by
the particular package of stimulus properties making Skinner (1957) and discussed by MacCorquodale (1969)
up a specific, dated environmental event, later trans¬ is uttering the response “orange” in the presence of
fers to other packages of stimulus properties making a fruit bowl or a breakfast table that sometimes con¬
up other specific, dated environmental events, and tains an orange but does not in this instance. The
when, in addition, the basis of transfer is exactly those sophisticated speaker will usually qualify the response
stimulus properties the verbal community regards as with other responses, such as “That fruit bowl makes
Evalyn Segal 635

me think of oranges,” or “Strange, there’s no orange clitic having clear response dimensions and is trig¬
on the breakfast table this morning.” Only “orange” gered by the prior strengthening of “oranges,” “fruit
is a “tact in metonymic extension.” The qualifying bowl,” and something peculiar about the stimulating
terms (“no,” “makes me think of”) are autoclitics. conditions evoking “oranges,” the temporal arrange¬
ment of the terms into a grammatical sentence re¬
quires the notion of a (dimensionless) autoclitic frame.
Autoclitics
(The fixed-interval scallop is, in this sense, a “dimen¬
An autoclitic, finally, was defined by Skinner as a sionless frame” into which concrete responses such as
verbal response controlled by stimulation (demonstra¬ bar presses or key pecks “fit.” “Temporal frame” is
ble or merely hypothesized) arising from the speaker’s perhaps a more apt expression.)
immediately prior (covert, usually) verbal behavior. The difficulty with the concept of an autoclitic
Autoclitics are first cousins of intraverbals, in the frame is the same difficulty posed by deep structures,
sense that their controlling stimuli are said to be or Robinson’s (this volume) cards with side codes but
preceding (albeit covert) verbal stimuli (auto = “self,” no words written on them. Robinson’s card metaphor
clitic — “leaning on”). The autoclitic depends for its Suggests how the autoclitic frames that “mediate” the
occurrence on a self-generated verbal stimulus or, syntactic connections between words might be learned
really, some more abstract and hypothetical preverbal within a reinforcement theory framework, but none of
(possibly even central) event. Autoclitics can have the the present notions (autoclitic frames, deep structures,
clear topographical dimensions of an uttered word, or coded cards) helps one to imagine what the physical
they can be, so to speak, dimensionless, being de¬ embodiment of syntactic structure might be. All of
tectable only by their effect of ordering these more these notions imply some mysterious preverbal
familiar terms syntactically. processes within the organism, to which Fodor et al.
(1974) give the name “mentalese.” Autoclitic frames,
Some examples of autoclitics have just been men¬ deep structures, and coded cards equally represent
tioned. In “That fruit bowl reminds me of oranges” abstract theories about syntactic behavior, and as
speakers, so to say, find themselves about to utter theoretical concepts they can, of course, function with¬
“oranges” but, noticing that the response is not at out reference to physical events within the organism.
peak strength, or perhaps vaguely sensing that the To the extent that one takes them as references to real
circumstances are not quite right for saying biological events, they must be understood as meta¬
“oranges,” qualifies the response in the process of phors. (More on this, too, later.)
uttering it. The qualifying autoclitic, having the
dimension of a “word,” is “reminds.” The sentence
illustrates the difference between “primary” verbal COMMENTS ON SKINNER'S
responses and autoclitic responses. “Oranges” is a FUNCTIONAL THEORY
metonymic tact controlled by the fruit bowl as a dis¬
criminative stimulus, “Fruit bowl” is a generic tact This brief review has laid out the bare elements of
controlled by the same discriminative stimulus. “Me” Skinner’s theory of verbal behavior. For fuller exposi¬
is perhaps a generic tact of a complex private event, tions, the interested reader should consult MacGor-
or perhaps should be regarded as another autoclitic; quodale (1969, 1970), Segal (1975a), or best of all,
a response to the speaker s privately sensed disposition Skinner himself (1957). Ag MacCorquodale (1969)
to speak. The distinction between primary verbal be¬ noted,
havior (the immediate responses to the stimulating
environment) and autoclitic behavior (complex re¬ Everything considered, the basic explanatory
sponses to the disposition to speak) is reminiscent of apparatus seems very meager, while verbal be¬
the distinction between deep and surface structure, havior is very complex, [But] the power of a
but the correspondence is not exact. (More on this single variable is seen to multiply when we take
later.) into account its multiplicity of effects. . . .
First, a single variable controls many responses,
Skinner suggested that the syntactic structure of
giving a speaker a great deal to say even in a
utterances is determined, at least in part, by autoclitic
static environment. . . . [Second,] if several
frames. These would correspond, roughly, to such
responses are concurrently strong, additional
syntactic sequences as noun phrase-predicate phrase, variables . . . control the order of their emis¬
or determiner-noun. Thus, although “reminds” in sion and whatever response selection, and rejec¬
“That fruit bowl reminds me of oranges” is an auto¬ tion, occurs, (pp. 838-839)
636 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

Skinner’s theory remains in its pristine state. It has forced (that, as I take it, is the definition of incentive).
been too little modified since its publication in 1957, (See Ayllon 8c Azrin, 1968, for a demonstration of
first because behaviorists have apparently not been human incentive motivation at work.)
interested in subjecting it to the experimental test and Given a novel motivational state induced by a
revision appropriate to theories, and second, because novel incentive and an environmental setting which,
psycholinguists, who are acquainted with real data in concert with incentive motivation, has already
germane to the theory, have not been at pains to im¬ come to control mands of the form “Buy me
prove a theory they scorn. Even so, the theory is _,” the mand frame is emitted. The “gen¬
powerful, it is in touch with much of the data that erative” appearance of the mand frame on this novel
psycholinguists have uncovered, and it is available as occasion, then, is understandable if all induced incen¬
a means of systematizing and making sense of what tive motivations, novel or not, have enough in com¬
psycholinguists are finding out, especially now that mon, and if all stores (places where incentives are
their interests have begun to shift to “semantics” and bought) have enough in common, so that these two
“pragmatics.” In the interests of advancing under¬ variables together are able to support generative ap¬
standing as rapidly as may be, one hopes that psycho¬ pearance of the response form “Buy me
linguists will not insist on working out all over again
The appearance of the response “a doodler” in the
the integrative insights into semantic and pragmatic
blank in the mand frame is still to be explained.
issues that Skinner’s book makes available, free for
Other things being equal, a response that tacts an
the asking.
attractive incentive should have a high probability of
emission (as a tact, mind you, not yet as a mand).
Comments on the Mand There must have been occasions in the past when the
child “merely tacted” an incentive but was under¬
What is one to say about the reinforcers, and the
stood by his parents to be manding, and so (unex¬
associated controlling motivational states, for novel,
pectedly) the child found the incentive object handed
generative mands? Suppose, to take an example of
over to him. Supposing that the incentive had aroused
Skinner’s (1957), a child in a toy store sees an attrac¬
an incentive motivation, obtaining the incentive
tive new toy whose name he doesn’t know. On being
should have reinforced the tact response, which from
told by his parents that it’s a “doodler,” he immed¬
then on could be evoked, as mand, by the motiva¬
iately says, “Buy me a doodler!” As Skinner notes,
tional state, as well as evoked, as tact, by the object
“He has never been reinforced for this response in the
itself. After a few such experiences, all responses that
manner required to construct a mand” (1957, p. 188).
were previously acquired as tacts (when the child was
If the response has never been reinforced, it could
not motivated to have the object) might be available
never have come under the control of the child’s new
also as potential mands when the child was in the
motivational state. What accounts for its emission,
then? Skinner suggests, “It is possible that all mands appropriate incentive motivational state. In short, a
which are reinforced by the production of objects or new set of conditional discriminations, with genera¬
other states of affairs may be interpreted as manding tive effects, would have been learned. Given an incen¬
the behavior of the listener and tacting the object or tive, and given availability of a tact of the incentive,
state of affairs to be produced” (1957, p. 180). Let us and given a correlated incentive motivation, the child
consider this further. would emit the tact. The specified reinforcement
The doodler is an incentive, automatically induc¬ would then follow this emission of the tact, which
ing a novel motivational state. Piaget (1952) gives would then come under independent control of the
many examples of novel objects apparently function¬ motivational state and would no longer depend, as
ing automatically as incentives—automatically, that is, mand, on the presence of the incentive to evoke it.
presupposing certain prior sensorimotor or “cogni¬ (Notice one new factor here. We presuppose that,
tive” learning. Novel incentives of this kind do not given a novel incentive motivation, the speaker can
clearly sort out as primary or conditioned reinforcers. discriminate the incentive object that roused the
Perhaps they are best regarded as induced (Segal, motive and will not be likely to tact irrelevant ob¬
1972) incentives, their incentive value arising in some jects.) The analysis suggests why speakers are prone
yet-to-be-explicated way from prior learning. Being to tact their motivational states by reference to ex¬
incentives, they induce a correlated motivational state pected reinforcers, giving to statements of intention
(incentive motivation) and so insure that any behavior their characteristically teleological flavor.
that results in obtaining the incentive will be rein¬ This account is given at length to suggest what
Evalyn Segal 637

must be a common and powerful basis for generative learning laboratory, as well as in the natural environ¬
(creative, productive) manding: tacting incentives. We ment. (Possibly the relatively low energy requirements
shall find problems enough later on when we come to involved in vocal responding make it, like perceptual
consider generative syntax, but generative echoic re¬ responding, especially available to a simple contiguity¬
sponding, generative tacting, and generative manding learning process.)
do not seem to present serious difficulties for Skinner’s
functional theory of verbal behavior. Working
The Stimulus Control of Verbal Behavior
through the functional analysis in individual cases
and testing it against experimental or naturalistic When we leave the mand, we leave a dominating
data are not necessarily easy, but they can be done. concern with reinforcement and correlated motiva¬
The general strategy for attacking the semantic and tional states. From here on, the focus of the analysis
pragmatic problems of language seems clear. is on stimulus control. Echoic control leads eventually
to the emergence of the “minimal (phonemic) echoic
repertoire/' enabling speakers to match, phoneme by
The Reinforcement of Verbal Behavior
phoneme, verbal stimuli they hear, even for the
Once Skinner leaves the discussion of the maild, the first time. Generative eehoic responding raises no ex¬
third term in the three-term contingency (the reinforc¬ planatory problem? JU§t because the echoic stimulus
ing consequences of verbal responses) is taken pretty provides the speaker with such detailed and explicit
much for granted—not because reinforcement is unim¬ instructions on what to do. Also arising out of the
portant in the acquisition and maintenance of verbal minimal echoic repertoire are formal classes of re¬
behavior, but rather because the reinforcements flow sponses controlled by isolated phonetic features of
so naturally from the fact that the speaker is commu¬ verbal stimuli. In the dominant vernacular of the
nicating with a responsive listener. Except for a limited day, responses in the “memory store" can be ad¬
class of mands that specify primary commodity rein¬ dressed by such “retrieval cues” as “Give me some
forcers the listener is to supply, the reinforcers for words that begin with p” or “Give me some words
verbal behavior mostly reside in the social response that rhyme with luck" or “Give me some words in
of the listener. Furthermore, they may be self-supplied spondaic meter.”
by the speaker, in his or her role as a listener. Much Among the by-products of the development of
of a mature speaker s verbal behavior is addressed to intraverbals and tacts are the appearance of thematic
oneself, as self-instructions and the like. Whatever classes of responses controlled by events (whence
reinforcers ultimately flow from the speaker-listener’s “episodic memory”) and by words (whence “semantic
reactions to their own self-instructions provide the memory”). The organization of verbal memory is cur¬
necessary conditions for the maintenance of self-ad¬ rently an active topic of research, Skinner argued that
dressed verbal behavior. all the organization to be found in the speaker's inter¬
Finally, intermittent reinforcement must play at related tact and intraverbal repertoires (i.ev in his or
least as important a role in the acquisition and main¬ her thematic classes) were the result of sheer contiguity,
tenance of verbal behavior as it plays in the control but contemporary memory researchers, reviving some
of other operant behavior. It would not Violate the of the perceptual insights of Gestalt psychM$gy? have
spirit of a behavioral analysis if it turned out, even, insisted that other factors besides sheer contiguity are
that new verbal operants (new stimulus-response important. Hilgard and Bower (1975) review some of
units) are often learned without any evident rein¬ the argument? in their chapter on Ge?talt psychology.
forcement. (MacCorquodale, 1970, makes the same To summarize their arguments, responses tacting
point.) A kind of secondary contiguity principle may events that occur together in the speaker’s perceptual
operate, such that the direct reinforcement of some of experience are much more likely to become inter¬
the speaker’s verbal acquisitions generates a strong related in memory (such that the perceptual cues con¬
and persisting tendency to acquire new verbal oper¬ trolling each tact as a generic will also have a high
ants even when there is no possibility of their imme¬ probability of evoking the other tacts as metonymies)
diate reinforcement. Experimental analyses of gener¬ if the original events were so structured as to be per¬
ative imitation—for example, Sherman (1971)—seem ceived as related to one another in some intrinsic way.
to demonstrate the operation of such a secondary For example, two stimulus features structured so as to
contiguity principle. Such a principle also seems the be attributes of a coherent figure are more likely to
most parsimonious way to account for much of the evoke one another’s generic tacts as metonymies than
paired-associates learning that goes on in the verbal are two stimulus features presented with identical
 638
TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

spatial contiguity but so structured that one is per¬ an explicit feature of the physical environment? What
ceived as a figural attribute and the other as an inde¬ are the physical dimensions of the stimulating en¬
pendent attribute of the background. (See Hilgard vironment that control relational terms such as
and Bower, 1975, Figure 8.6, p. 265, for several familiar,” “similar,” “Mozart” (said of a piece of
examples taken from experiments by Asch, 1969.) The music), “Dutch” (said of a painting), and so on? Cog¬
arguments against sheer contiguity seem, on balance, nitive theorists (e.g., Chomsky, 1959) object to Skin¬
to be persuasive. It seems clear now that there are ner’s treating the stimuli controlling such tacts as
much more interesting things to say, and to discover, purely objective, physical events. They would argue,
about the organization of verbal memory than simply for example, that the relation of similarity is not in
that it derives from contiguity of perceptual experi¬ the physical stimuli, but in the way organisms (are
ences.
built to) react to (“process”) them. Skinner would
Despite the fall of sheer contiguity as the exclusive agree. The issue, of course, is not whether organisms
organizing principle of thematic response classes, Skin¬ can respond in such a way as to exemplify the rule
ner’s insight into the organization of the verbal reper¬ “Pick the comparison stimulus that is similar to the
toire in terms of controlling stimuli remains valid. sample stimulus.” The literature on matching to
Some of the phenomena that memory researchers sample clearly demonstrates that they can. The issue is
puzzle over seem not quite so puzzling, nor their con¬ whether it is necessary to posit a “cognitive process”
temporary explanations quite so original, viewed in or a ‘.‘comparator device” within the organism, over
this light. “Encoding specificity” (Tulving & Thom¬ and above what is in the external physical world, to
son, 1973) is the current expression for the inde¬ account for the organism’s ability to match to sample.
pendence of verbal operants controlled by different I think a comparator device is called for, and be¬
antecedent stimuli. Some of the phenomena of en¬ havior theory will have to make room for it. In gen¬
coding specificity, such as the greater ease of “recall” eral, relations are not describable solely in physical
over “recognition” in some circumstances (Tulving, terms. Behaviorists can, of course, give an operational
1974; Tulving & Thomson, 1973), or the appearance definition of what it means for an organism’s behavior
of formal (“auditory”) “errors” in immediate free re¬ to be controlled by relations between physical stimuli,
call (“short-term memory”), replaced by thematic but the fact that such operational definitions require
(“semantic”) “errors” in delayed free recall (“long¬ mention of an organism’s behavior makes such defini¬
term memory”) (Kausler, 1974), could have been pre¬ tions question begging, so far as cognitive theorists
dicted from Skinner’s analysis. It must be said, how¬ are concerned.
ever, that they were not predicted. Alas, no original A point that is rarely mentioned, and little under¬
research on verbal memory flowed from Skinner’s stood, is that it does no violence to functional be¬
theory, because no one conversant with it was inter¬ haviorism to concede that the stimulus relations
ested in doing the research, and so equivalent theo¬ which control behavior may not literally be in the
retical explanations have had to be reinvented with¬ environment but in the way the organism reacts to the
out help from Skinner, and behaviorists can take no environment. Functional behaviorists take for granted
credit for the interesting results that have been emerg¬ that the organism brings something of its own to the
ing in research on verbal memory. behavior-environment interaction that defines the
operant. Perhaps this should be mentioned more
often. MacCorquodale (1970) spent a good deal of
Control by Relations Between Stimuli space on this matter, but it was almost as an aside that
Skinner himself remarked, in Verbal Behavior, “All
This is not to say there are no difficulties in Skinner’s behavior, verbal or otherwise, is subject to Kantian a
account of the stimulus control of verbal responses. priori’s in the sense that man as a behaving system has
There are basic problems in the definition and speci¬ inescapable characteristics and limitations” (1957, p.
fication of controlling stimuli. For example, Skinner 451). Among the “inescapable characteristics” that
suggested that the regular past tense inflection on organisms bring to an operant transaction with the
English verbs, -ed, is a tact controlled by “that subtle environment is their disposition (call it “cognitive” or
feature of the environment called action-in-the-past.” not) to perceive (even impose) relations between phys¬
But it is not clear that action in the past is an ex¬ ical events. The relations do not inhere in the physical
plicit feature of the physical environment. Other rela¬ events themselves; Skinner pointed out, for example,
tional terms present the same problem. Is plurality, that it is not literally a temporal relation between
which is said to control the tact inflection -s on nouns. earlier and later occurrences of a stimulus that marks
Evalyn Segal 639

the later-occurring stimulus as “familiar/’ but some The Functional Approach to Syntax
change in the way the organism responds to earlier
and later occurrences. Still, perceived relations among The difference between functional and cognitive
physical events are describable, even if it takes an approaches to verbal behavior can be illustrated in the
organism (such as the experimenter) to describe them, approach to problems of syntax. Schumaker and Sher¬
and this is all that a functional analysis requires. Only man (1970) applied differential reinforcement con¬
if the experimental subject were able to perceive a tingencies that succeeded in getting retardates to utter
relation that the experimenter could not would func¬ the verb inflections -cd and -ing in appropriate intra¬
tional analysis be in trouble, for then the experi¬ verbal contexts—for example, in such sentences as
menter would be unable to formulate the deter¬ “Now the man is painting. Yesterday he . . .
minants of the subject’s behavior, What divides paintedn and “Yesterday the man painted. Now he is

behaviorists and cognitive theorists on this issue is , , . paintingEchoic prompts were initially used to
whether to focus on analyzing the structure of the induce the inflectional responses on sample verb stems
environment or the structure of the organism. Clearly, such as paint, but eventually the subjects added the
both participate in determining the functional rela¬ inflections to novel untrained verb stems such as skate
tions between organism and environment embodied m under appropriate intraverbal control of the phrases

operant behavior. The behaviors focus on the en¬ “Now he is . . or “Yesterday he . , • •” This quali¬
vironment, regarding the internal structure of the fies as generative (productive) syntactic behavior be¬
organism as beyond reach of current experimental cause (1) from an unfamiliar stimulus word such as
technique. The cognitive psychologists focus on the skating subjects extracted the verb stem (skate)', (2)

organism, even if they must invent the internal struc¬ they added the alternate ending to yield skated; and
ture they seek to understand. (3) they uttered the two response elements in gram¬
matically correct order (skated, not edskate).
In several experiments (Garcia, Guess7 8c Byrnes,
1973; Guess, 1969; Guess 8c Baer, 1973; Guess, Sailor,
Transfer of Verbal Resp©n§e§ t© New Stimuli
Rutherford, Baer, 1968; Sailor, 1971), experimenters
Although the problem of similarity does raise ques¬ trained retardates to utter the plural noun inflection
tions about the necessity of “filtering” the description -x under appropriate nonverbal stimulus control—that
of the stimulus relations controlling the tact “similar’? is, to tact plurality. Again this qualifies ag generative
through another perceiving organism, this problem is syntactic behavior because (1) subjects added -j to
not necessarily involved in generative (productive) novel noun forms on which they had not had echoic
extensions of verbal responses. Skinner’s treatment of training in the plural; (2) they uttered the plural
extensions of stimulus control is a “common elements forms in response to newly paired objects (familiar
theory of transfer,” (Prokasy and Hall 1963). Perhaps previously only as singletons)^ and (3) they uttered

that is why Skinner avoided the expression “stimulus the response elements in grammatically correct order
generalization” in Verbal Behavior} an avoidance (e.g.? rotkfj ndt smek). Again, the experimental
MacCorquodale (3 969) found puzzling. Still, a too- methods employed to bring about produ^iiys syntac-
casual use of the term stimulus causes confusion. If a tive behavior involved differential reinforcement of
tact were extended from one “stimulus5’ to anythsc* cchoically prompted r£sp5n§£§ ad gradual shifting Of
nonidentical “stimulus/’ the extension would have to stimulus control from echoic prompts to nonverbal
be on the basis of some troublesome “similarity.” But stimuli (single objects or pairs of objects). Lutzlter and
if a tact is extended from one collection of stimulus Sherman (1974) employed comparable pr<5££dures to
elements or properties or features, each of which can produce generative utterance of the verb forms is and
be separately specified, to another collection of stim¬ are under appropriate nonverbal stimulus control of
ulus elements or properties or features, on the basis of pictures of single or several actors engaging in some
shared, common, identical elements or properties or action.
features, then a “comparator device” that evaluates Whitehurst (1971, 1972) taught 2-year-old children
degree of similarity is not required. For cognitive a miniature artificial language consisting of some
theorists the problem of “recognition” of identical nonsense syllables that functioned as color “adjec¬
stimuli then replaces the problem of evaluating the tives” and some nonsense syllables that functioned as
degree of similarity of nonidentical stimuli. Behavior¬ “noun” labels of geometric forms. A procedure in¬
ists can shed no light on the question of stimulus volving differential reinforcement, echoic prompting,
recognition. They simply take it for granted. and shifting of control from echoic to nonverbal
 640
TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

(color and form) stimuli created appropriate func¬ ing that some psycholinguists (e.g., Brown, 1973) en¬
tional response classes (a class of color tacts and a class tertain the possibility that the environmental relations
of form tacts) and appropriate syntactic ordering of a young child tacts in his first multiunit utterances
response terms in two-word utterances. The prescribed are exactly those relations he learned to respond to
order was color tact (adjective) followed by form tact nonverbally in the immediately prior sensorimotor
(noun), as in standard (surface) English. By these pro¬ stage of development (Piaget, 1952). In any case, it is
cedures Whitehurst succeeded in producing generative certainly true that behaviorists have shown no inter¬
syntactic behavior: given novel combinations of famil¬ est in the question of what internal cognitive machin¬
iar color-plus-form stimuli, the children correctly ery makes possible the perception of relations, al¬
tacted the two properties in the prescribed adjective- though this kind of question is the meat-and-potatoes
noun order.
of cognitive theorists. Premack is correct: a functional
Premack (1970) has also reported getting his chim¬ analysis is not a cognitive theory (although Skinner’s
panzee subject, Sarah, to emit generative tact se¬ hypothesis of the autoclitic probably qualifies as a
quences in syntactically prescribed orders; Gardner cognitive theory).
and Gardner (1969, 1971, 1975) have reported gen¬
erative syntactic response sequences in their chimpan¬
zee subject, Washoe; and Rumbaugh, von Glaserfeld, FUNCTIONALISM VS. MENTALISM
Warner, Pisani, and Gill (1974) obtained generative
syntactic mand sequences in their chimpanzee subiect,
Lana. Theories

These are only an exemplary few of the spate of The question of whether to have or not to have
research reports in recent years dealing with a func¬ theories (alternatively, what constitutes a proper ex¬
tional analysis of simple syntactic behavior. In all this planatory theory) is at the heart of the so-called men-
work, explicit training procedures involving differ¬ talist-behavioral controversy. Cognitive theorists often
ential reinforcement, prompting, and stimulus fading employ the vocabulary of mentalism, but a look at the
(associative shifting) were employed to bring about footnotes (e.g., Chomsky, 1965, pp. 193-194) reveals
the productive syntactive behavior. Premack, however, that all they mean by mind is a theory about the
has concisely stated the demurrer that divides be¬ internal machinery of the organism. Sometimes such
havioral and cognitive theorists in dealing with prob¬ theories are couched in the metaphors of neurophys¬
lems of verbal behavior: “A strict training procedure iology, sometimes in the metaphors of computer and
is not an explanation of how, as a result of carrying information science, sometimes in the metaphors of
out the prescribed steps, the organism accomplished plant genetics (unfolding of phenotypic characters in
the function in question. A recipe is a method, not a a favorable soil, and so on). But behaviorists should
theory” (1970, p. 107). Functional behaviorists are take note that the temptation to theorize in metaphors
content to devise “strict training procedures” that suc¬ appropriate to another domain is so great, up against
ceed in producing generative syntactic behavior in the mysteries of syntax, that even Skinner succumbed.
experimental subjects. In the course of devising effec¬ The concept of the autoclitic is a groping step toward
tive procedures, such research unavoidably identifies a theory of syntactic behavior, and, like the theories
critical functional variables controlling syntactic be¬ Skinner elsewhere (1950) enjoined, the theory of the
havior, insofar as these reside in environmental con¬ autoclitic is metaphoric. (MacCorquodale, 1969, dis¬
tingencies among the antecedent variables, the be¬ agrees.) It proposes that the unobserved processes that
havior, and the reinforcing consequences. A func¬ must be supposed to underlie the observed (surface)
tional analysis, however, does not attempt to specify syntactic structure of verbal behavior are in some
what, apart from its ontogenetic history, the organism sense operant, albeit covert, even “preperipheral.” It
brings to the learning task. Premack (1970) suggested remains to be seen if Skinner’s choice of metaphor for
that strict training procedures do no more than teach the unseen processes underlying syntax was apt. The
the experimental subject to “map concepts” it already internal parts of machines rarely resemble their ex¬
possesses—that is, to tact the environmental events and ternal parts, and so other cognitive theorists (for in
relations it already perceives. (Also see Robinson, this matter Skinner himself must be classed as a cog¬
Chapter 21 in this volume.) Premack doubted the nitive theorist) have opted for other metaphors. Note
possibility of teaching subjects to perceive relations that “deep structure” and “surface structure” and the
they do not perceive spontaneously. This is an open whole transformational-generative apparatus make up
question for the functional behaviorist. It is interest- an extended metaphor, too, insofar as they are
Evalyn Segal 641

claimed to describe psychological processes and not perceptual mechanisms and that of determining the
simply to stand as an abstract, formal analysis. behavioral direction of learning, corresponds approx¬
imately to Premack’s (1970) distinction between the
organism’s perceiving environmental events and rela¬
Nativism vs. Environmentalism
tions and its learning to map its perceptions verbally.
The so-called nativist-environmentalist controversy The search for “linguistic universals" through com¬
between behaviorists and cognitive theorists is another parison of culturally given “pairing[s] of linguistic
expression of the different emphases they give to the data and grammars" then becomes, not a search for
functional analysis of environmental contingencies of the unknowable noumena behind verbal behavior,
reinforcement versus the invention of models of the but a search for functional correlations between en¬
organism s internal machinery. The following passage vironmental contingencies and syntactic behavior—
briefly summarizes Chomsky's supposedly nativist view that is, a search for the general characteristics of
of language learning: verbal behavior “ultimately determined by the genetic
and ecological universals of a species” (Bram£? 1971,

Consider an acquisition model . . . that uses p. 185). The search for linguistic universals, in short,
linguistic data to discover the grammar of the is part of the search for general laws of behavior, and
language. ... Just how the device * . . selects surely the physiological constitution of organisms
a grammar will be determined by its internal helps to determine the form of such laws. Psycho-
structure, by the methods of analysis available
linguistic theories of linguistic performance modeled
to it, and by the initial constraints that it im¬
after abstract theories of linguistic competence antic¬
poses on any possible grammar. If we are given
ipate physiological knowledge by proposing models of
information about the pairing of linguistic data
neurological functioning in advance of empirical
and grammars, we may try to determine the
nature of the device. (1972, p. 119) neurological findings. As Salzinger (1975) has noted,
behaviorists regard such anticipatory theories as no
This passage appears to rest all the weight of syntactic more necessary than physiological theories of learning
development on genetically given characteristics of the in general, but there is no attempt to proscribe
organism. Yet in a footnote, Chomsky (1965, p. 202) physiologizing by those so inclined.
acknowledged that the language-acquisition device's
internal structure 'might possibly be developed on Lashley^s Criticjue
the basis of deeper innate structure, in ways that de¬
pend in part on primary linguistic data and the order Lashley's (1951) classic paper on the problem of
and manner in which they are presented." This serial order in behavior is often cited by cognitive
sounds like learning, Finally, Chomsky drew a distinc¬ theorists as posing an unanswerable challenge to a be¬
tion between “two functions of external data—the havioral analysis of language, but in fact Lashley's
function of initiating or facilitating the operation of paper (which is perhaps more widely cited than read)
innate mechanisms and the function of determining indicts, not behavioral analysis, bnt simplistic
in part the direction that learning will take" (1965, phybiologizing in of ^concepts of the reflex arcj
or of associated chains of neurons” (p. 526 in the
P- 34).
The influence of the organism's “deeper" internal Beach et al.? 1960? reprinting). Skinner has consistently
structure on the grammar it “selects" is a matter of refrained from such simplistic physiologizingi even in
Kantian a prioris, the organism’s “inescapable charac¬ his most theoretical discussions of syntax. To quote
teristics and limitations.” Again it should be said that Lashley further:

behaviorists (until quite recently) have been amiss in

too seldom acknowledging genetic constraints and Language presents in a most striking form the
thus allowing the misconception to be broadcast that integrative functions that are characteristic of
the cerebral cortex. ... In spite of the ubiquity
behaviorism denies their very existence. On the other
of the problem [of serial order] there have been
hand, Chomskians have been derelict in too seldom
almost no attempts to develop physiological
mentioning Chomsky’s admission that the language-
theories to meet it. ... I have chosen to discuss
acquisition device's internal structure and functioning the problem of temporal integration here, not
might very well evolve under the influence of environ¬ with the expectation of offering a satisfactory
mental learning contingencies. physiological theory to account for it, but be¬
Chomsky’s distinction between two functions of ex¬ cause it seems to me to be both the most im¬
ternal data, that of engaging internal learning and portant and also the most neglected problem of
642 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

cerebral physiology, (in Beach et al., pp. 507-508; sumed its final form. The subject is a difficult
italics mine) one because it has all the disadvantages of
private stimulation, (p. 371; italics mine)
In the remainder of the paper Lashley proceeds to
describe the problem of syntactic behavior with a per¬ Now back to Lashley:
ceptiveness befitting a great psychologist. So far from
conflicting with Skinner’s theory of autoclitic be¬ There are at least three sets of events to be
havior, Lashley’s description closely parallels it. (It is accounted for. First, the activation of the expres¬
sive elements (the individual words . . .) which
not clear which came first, inasmuch as Skinner’s Wil¬
do not contain the temporal relations. Second,
liam James Lectures on verbal behavior were given at
the determining tendency, the set, or idea. . . .
Harvard University in 1947 and widely circulated in
Third, the syntax of the act, which can be de¬
mimeographed form thereafter. Questions of priority scribed as an habitual order or mode of relating
are hardly relevant in any case.) Continuing from the expressive elements; a generalized pattern or
Lashley: schema of integration which may be imposed
upon a wide range and a wide variety of specific
There are indications that, prior to the internal acts.. This is the essential problem of serial
or overt enunciation of the sentence, an aggre¬ order: the existence of generalized schemata of
gate of word units is partially activated or action which determine the sequence of specific
readied, (p. 512) acts (p. 515).

Compare Skinner: These passages from Skinner and Lashley pose the
problems of syntactic behavior almost identically,
The important properties of verbal behavior except for vocabulary differences. It is unclear, there¬
which remain to be studied concern special ar¬ fore, why cognitive theorists approve Lashley’s for¬
rangements of responses. Part of the behavior of mulation and reject Skinner’s. Skinner carefully re¬
an organism becomes in turn one of the variables frained from speculating about the internal locus and
controlling another part. . . . The events avail¬ dimensions of autoclitic processes, regarding the
able to him as stimuli consist of the products of
organism’s physiological machinery as outside his pur¬
his own behavior as speaker. He may hear him¬
view. The whole extent of his “physiologizing,” if one
self or react to private stimuli associated with
should call it that, was to assign the status of covert
vocal behavior, possibly of a covert or even in¬
cipient form. . . . The term “autoclitic” is in¬ “responses” and “response”-produced “stimulation” to
tended to suggest behavior which is based upon the hypothetical controlling variables immediately
or depends upon other verbal behavior. (1957, responsible for the autoclitic processes that order
pp. 313-315; italics mine) verbal behavior into surface syntactic structures. Lash¬
ley, with his excellent physiological credentials, went
Also from Skinner: much farther and translated the behavioral problems
of serial order into problems and hypotheses for
The manipulation of verbal behavior, particu¬ neurophysiology. Translations of problems from the
larly the grouping and ordering of responses, is vocabulary of one science into the vocabulary of
. . . autoclitic. Responses cannot be grouped or another do not, of course, constitute explanations. So
ordered until they have occurred or at least are
far as I know, the physiological problems Lashley set
about to occur, (p. 332; italics mine)
in 1951 have not yet found solutions.

Once more from Skinner:

Much of the self-stimulation required in the THE COMPLEMENTARITY OF FUNCTIONAL

autoclitic description and composition of verbal AND COGNITIVE THEORY
behavior seems to occur prior to even subaudible
emission. In both written and vocal behavior
The abiding problem is structure. As Lashley
changes are made on the spur of the moment
noted, problems of temporal structure are not con¬
and so rapidly that we cannot reasonably attrib¬
ute them to an actual review of covert forms. fined to language, but exist everywhere in behavior.
. . . Evidently stimulation associated with the Research on operant behavior divides into two broad
production of verbal behavior is sufficient to en¬ categories: research on the exteroceptive discrimina¬
able one to reject a response before it has as¬ tive stimulus control of operants (which has broad
Evalyn Segal 643

though so far mostly unexploited relevance to the two-morpheme) utterances first appear, tend to utter
semantics of verbal behavior), and research on the an “uninflected” unit response interpretable as “sub¬
determinants of temporal structure in schedule-gen¬ ject” followed by an uninflected unit response inter¬
erated performances. Ferster and Skinner (1957) de¬ pretable as “predicate.” (To be sure, the interpreta¬
voted a large book to the hypothesis that the temporal tions are generous, but they seem to correspond to the
structure of schedule-generated performances is at way parents respond to their children’s utterances.)
least partially to be explained in terms of stimuluslike But the subject “noun phrase” can itself be parsed
processes generated by the organism’s own behavior. into an (optional) “determiner” (the, a, and so on)
As they put it, “The primary purpose of the present and an (obligatory) “noun” or “pronoun.” In kernel
book is to present a series of experiments designed to sentences of English, the responses constituting the
evaluate the extent to which the organism's own be¬ determiner (if there is one) are uttered first within the
havior enters into the determination of its subsequent subject phrase, and responses constituting the noun
behavior” (p. 13). Although they did not say so, it arc uttered second. Speakers must learn these two
seems possible that Skinner’s hypothesis of autoclitic temporal positions within the subject constituent; that
processes in syntax motivated Ferster and Skinner’s is, they must learn the “internal structure” of the sub¬
choice of a guiding hypothesis for their extended ject phrase.
experimental analyses of schedules of reinforcement, The predicate “verb phrase,” too, can be parsed
Research on the temporal structure of operant be¬ into an “auxiliary” (at the least, a “tense marker”)
havior continues. Recently, Hawkes and Shimp (1975) and a “verb.” In simple kernel sentences of English,
succeeded in demonstrating that differential contin¬ the responses constituting the verb are uttered first
gencies of reinforcement imposed directly on temporal within the predicate phrase, and the responses con¬
structure were effective in generating the prescribed stituting tense are uttered as a suffix inflection on the
structure in the key pecking of pigeons. They shaped verb. Speakers must learn these two temporal posi¬
an “ideal” fixed-interval-like scallop by delivering a tions within the predicate phrase; that is, they must
reinforcer at the end of a 5-sec trial only if the pattern learn the “internal structure” of the verb phrase, (In
of responding during the trial approximated constant standard transformational-generative grammar, the
acceleration. Research on higher-order schedules of tense marker—and auxiliaries in general—appears be¬
reinforcement (e.g., Findley, 1962; Kelleher, 1966) has fore the verb in deep structure. It is moved to verb
demonstrated that hierarchically organized behavioral suffix position in surface structure by a transforma¬
structures, too, are within the operant purview. It is tion,)
not too much to hope that further research on higher-
order schedules will illuminate not only how the
Braine's Functional Experiments on Structure;
structured nonverbal performances generated by first-
Temporal Position as a Controlling Variable
order schedules become organized into higher-order
(hierarchical) structures, but also how the hierarchical The hierarchical structure so far described can be
structure of verbal behavior comes about. generated by a “binary fractionation” model of pl\r&§£
structure (Braiiie, 1963b), First* a “sentence” is
divided into two “fractions,” subject and predicate,
The Problem of Verbal Structure Stated
and then each of these two fractions is diyij^4 into
in Functional Terms
two fractions, subject becoming determiner plus noun,
Let us review the problem. The string of verbal re= predicate becoming verb plus auxiliary. Braine pre¬
spouses that makes up the surface of a kernel sentence sents some sketchy evidence suggesting that speakers
is hierarchically structured. Kernel sentences can be may learn syntactic structure of natural languages by
parsed into two broad constituents, a “subject” and a iterative binary fractionations rather than by fraction¬
“predicate.” In the kernel sentences of English, the ations into, say, thirds of a phrase. That is, his expert
responses constituting the subject constituent are ments (to be reviewed shortly) suggest that speakers
uttered first and responses constituting the predicate find it easier to learn the “absolute” temporal posi¬
constituent second. Speakers must learn these two tions of first vs. last than “relative” temporal positions
temporal positions within the sentence. There is evi¬ such as “middle” or “after the first but before the
dence (Brown, 1973) that children just entering the last.” Moreover, Braine suggests that the learning of
mysteries of syntax have in fact learned them (or are temporal positions in a sentence may be a species of
in process of learning them). That is, children in auditory perceptual learning, speakers learning which
“stage 1” of syntactic development, when two-word (or verbal responses “sound right” in first position and
644 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

which “sound right” in second position. Having year-old children learned the same artificial language
learned which responses sound right in first (subject) in the same way, except that before making their final
position and which sound right in last (predicate) completion choices for each sentence frame the chil¬
position, speakers then proceed to learn which re¬ dren were required to say aloud both sentence com¬
sponses sound right in first vs. last position within the pletions, the incorrect one produced by putting an
subject phrase and which sound right in first vs. last A-word in a blank P-position or a P-word in a blank
position within the predicate phrase. Thus the learn¬ A-position, and the correct one produced by putting
ing of hierarchical structure might unfold a binary an A-word in A-position or a P-word in P-position.
fractionation at a time. Then the children were permitted to make their own
Braine’s (1963a) ingenious experiments involved completion choices, and they were told whether their
the echoic or textual teaching of miniature artificial completions were correct or not, but they were not
languages (devoid of “semantic content”) to children permitted to read their final completions aloud. By
between the ages of 4 and 11 years. In Experiment I this procedure Braine attempted to equate auditory
(A&P language with word constituents), the experi¬ exposure (actually, echoic practice) with correct and
menter presented “sentence frames” with either the incorrect sentences. This experiment did not include
anterior (A) or posterior (P) position already filled by a a test of generative syntax. Nevertheless, the results
nonsense-syllable word from the A-class or the P-class, from the learning trials themselves were instructive.
respectively. The experimenter also presented two Whereas all children in the standard Experiments I
choice words, one from the A-class and one from the and V learned to complete the training sentences cor¬
P-class, from which the children had to choose one rectly within a median of 10.5 or 13 trials, respec¬
word to fill in the blank position in the sentence tively, and while making a median of only 4 or 4.5
frame. Correct completions were reinforced with errors, respectively, some children in Experiment IV
poker chips backed up with candy; incorrect comple¬ did not meet the learning criterion within the allotted
tions were simply corrected by the experimenter. In number of trials (50), the median trials to criterion
either case, a learning trial terminated with the sub¬ was 32, and the median errors to criterion was 9.
ject, and then the experimenter, reading aloud the Braine concluded that in Experiments I and V “the
correctly completed two-word sentence. In this way, relevant cue was the temporal position in the spoken
the children learned what words constituted the A- sentence, and as learning progressed words . . . came
class (what words were permissible completions of a to sound familiar in the positions in which they re¬
blank in first position) and what words constituted the curred. The subjects [of Experiments I and V] were
P-class (what words were permissible completions of a then able to respond correctly in generalization prob¬
blank second position). As a test of generative syntax, lems by picking the alternative which made the sen¬
the experimenter then presented sentence frames in tence [literally] ‘sound right’ ” (1963a, p. 335).
which a novel A- or P-word occupied first or last posi¬ Another experiment (Experiment II: A&P language
tion, respectively, and the child had to complete the with phrase constituents) investigated whether learn¬
frames by choosing the correct one of two offered ing would be impaired if the response elements con¬
words, one a familiar A-word and the other a familiar stituting A- and P-classes were sometimes two-word
P-word. Because the supplied words in the test sen¬ phrases instead of single words. With one group of 9-
tences were novel, they provided no intraverbal cues and 10-year-old children, a whole (one- or two-word)
to correct completions. The only available cues, then, phrase was left blank in the sentence frame, and chil¬
were the positions of the blanks in the test sentence dren completed the frame by choosing between a one-
frames. Subjects 9 and 10 years old completed 78% of or two-word A-phrase and a one- or two-word P-
the test problems correctly, showing that they had phrase. (No cues indicated whether the frame called
learned the A- and P-classes on the basis of temporal for a one- or two-word completion.) With a second
position within the sentence and that temporal posi¬ group of subjects, only the second word of a two-word
tion was a sufficient intraverbal stimulus controlling A-phrase was missing in the sentence frame, or only
productive choice of an A- or P-word. In a replication the first word of a two-word P-phrase was missing.
of the experiment adapted for 4-year-old children Thus the completed sentences were two, three, or four
(Experiment V), 75% of the test sentence frames were words long, and the blanks were in first, second, or
completed correctly, showing that the intraverbal third position in the frame. In this group, then,
learning called for was within the reach of younger temporal position controlling the formation of A-
children. classes and P-classes and cuing correct sentence com¬
In another experiment (Experiment IV), 9- and 10- pletions was not “absolute” first vs. last position (with
Evalyn Segal 645

phrase constituents), but “relative” position (a P-word Then the children were given a “within-phrase” gen¬
was called for after an A-word or phrase and before eralization test. Sentence frames were presented con¬
the last word of the P-phrase, or an A-word was called taining a familiar A-phrase and a novel p- or q-word.
for before a P-word or phrase and after the first word The frames thus lacked only one word (in p- or q-
of the A-phrase; the missing constituents were words, position). Subjects were given three, instead of two,
not phrases). Subjects in the first group, learning familiar alternative words to choose from (“because
whole-phrase constituents under control of absolute there were three parts of speech”—Braine, 1963a, p.
first or last position in the sentence, performed as well 331), a familiar A-word (or phrase), a familiar p-word,
as subjects in the standard Experiments I and V, who and a familiar q-word.
learned single-word constituents in absolute first or All subjects in Experiment III reached the initial
last position. However, subjects in the second group, learning criterion in a number of trials comparable to,
learning part-phrase (word) constituents under control and with fewer errors than, the subjects in Experi¬
of relative temporal position in the sentence, showed ments I, II, and V. In the between-phrase generaliza¬
markedly inferior performances. The results of this tion test they completed 74% of the novel test frames
experiment led Braine to conclude that absolute first correctly, and in the within-phrase generalization test
vs. last position is more easily learned than relative they completed 68% of the novel test frames correctly,
position, and therefore that the learning of syntactic percentages comparable to those obtained in the gen¬
structure should proceed more easily by iterative eralization tests of Experiment L Taken together,
binary building up of a sentence into its hierarchical these results showed that children learned concur-
structure. rently both the A- and PQ-phrase structure of the
In the remaining experiment of this series (Experi¬ sentences (the “higher-order” structure corresponding
ment III: A&PQ and AB&P languages), Braine (1963a) to subject-predicate) and the internal structure of the
investigated whether comparably simple procedures PQ-phrases (the “lower-order” structure corresponding
would enable 9- to 11-year-old subjects to learn the to the internal structure of a verb phrase, say). Both
“internal structure” of phrases. Half the subjects words within phrases and phrases within sentences
learned the A&PQ language, and half learned the “tend to become associated with the sentence positions
AB&P language, its mirror image. Only the A8cPQ in which they recur, . . . [and] within fairly wide
language will be described. The A-class in this lan¬ limits, the constitution of the elements in first and
guage consisted of one- or two-word phrases, but each last position is not an important variable for either
phrase was invariant in composition, so that terms in learning or generalization” (Braine, 1963a, p. 333).
the A-class had no internal structure. The PQ-class Braine summed up the results of these five experi¬
consisted of two subclasses, the p-class and the q-class. ments as follows:
Any word from the p-class could be combined with
any word from the q-class, yielding internal structure (a) ”What is learned” are the locations of expres¬
Pi^i- A sentence consisted of the following sequence; sions in utterances, (b) Units (i.e.? expressions
(phrase from the A-class) [(word from the p-class) whose position is learned) can form a hierarchy
(word from the q-class)]. During learning, children in which longer units contain shorter units as
completed sentence frames by filling in a whole A- parts, the location that is learned being the loca¬

phrase or a whole PQ-phrase (of varying internal tion of a unit within the next-larger containing
structure). Next, a “between-phrase” generalization unit, up to the sentence, (c) The learning is a
case of perceptual learning—a process of becom¬
test was given. Sentence frames were presented con¬
ing familiar with the sounds of expressions in
taining novel A-phrases (with the PQ-position blank)
the positions in which they recur. (1963a, p. 337)
or novel PQ-phrases (with the A-position blank), and
children were offered a choice between a familiar A-
The learning of the temporal positions of words and
phrase and a familiar PQ-phrase to complete the
phrases, moreover, appears to be
frame. Again, the novelty of the supplied A- and PQ-
phrases precluded the children’s completing the be-
tween-phrase generalization sentences on the basis of a process of auditory differentiation. . . . Per¬
ceptual learning is usually assumed to be a
simple intraverbal phrase cues. Only the position of
rather primitive process and there is therefore
the blank in the sentence frame indicated which
no reason to suppose that it demands much in
choice was correct. After the between-phrase general¬ the way of intellectual capacity in the learner.
ization test children practiced again completing famil¬ Learning of this sort would therefore satisfy at
iar sentence frames with familiar A- and PQ-phrases. least one requirement of any process postulated
646 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

to be involved in first language learning, namely, the intraverbal control of syntactic structure in more
that it not require intellectual capacities obvi¬ elaborate artificial languages (which were still, it
ously beyond the reach of the 2-year-old. (1963a, should be noted, devoid of “semantic content” or, in
p. 326) other words, contributions from the nonverbal stim¬
ulus control that defines the tact). He instructed adult
“Primitive process” or not, the temporal control of
subjects to echo to spoken strings of nonsense words
behavior is familiar territory to operant behaviorists
having the structures p(Af)(Bg) and (Bg)q(Af)r, where
(Catania, 1970; Dews, 1970; Jenkins, 1970; Killeen,
p marked a sentence type (analogous, say, to a declara¬
1975; Morse, 1966; Staddon, 1972; Weiss, 1970). What
tive sentence marker), and q r marked a different
is perhaps less familiar is Braine’s important insight
sentence type (analogous, say, to an interrogative
that temporal position in a sentence constitutes a
sentence marker-“Wh ” words, for example). (Af)
strong intraverbal variable controlling syntactic word
and (Bg) were sentence constituents with internal
order. In a later experiment Braine (1965a) showed
structure, f and g being fixed elements (analogous to
that middle temporal position in a spoken sentence
Braine’s earlier—e.g., 1963b—notion of a “pivot”
string (aXb, pXq) also serves as an intraverbal vari¬
class) and A and B being word classes with
able controlling the formation of a response class (of
several members each (analogous to Braine’s 1963b
X-words). In this case, however, temporal position was
notion of an “open” class). In this more elaborate
not the only controlling intraverbal stimulus, for
language, the internal structure of elements within Af
X-words generalized to novel a_b and p_q contexts,
and Bg phrases was presumably controlled by
but not (to any considerable extent) to “anomalous”
temporal position within the phrases, but the larger
a_q and p_b contexts. In other words, the intra-
structure of constituents within the sentence was pre¬
verbally linked classes a__b and p_q combined with
sumably controlled jointly by temporal position with¬
middle position to determine the generative appear¬
in the sentence and by the more familiar kind of
ance of X-words. This kind of combined control is
intraverbal stimulus, the “word” elements p and q_r.
what Skinner (1957) termed “multiple causation” and
(Note that q_r forms a “discontinuous” intraverbal
which is otherwise familiar to behaviorists as condi¬ stimulus—Braine, 1965b.)
tional discrimination.2
An interesting feature of Braine’s (1971) experiment
In still other experiments, Braine (1971) pursued with the more elaborate language is that he induced
one group of subjects to echo not only “well-formed”
21 think all of Skinner’s (1957) discussion of the multiple strings but also “anomalous” strings. One type of anom¬
causation of verbal behavior can be most easily understood in
alous string consisted of “first-order approximations” to
terms of conditional discriminations. Conditional discrimina¬
tions, reflecting the concerted influence of at least two dis¬ the grammatical language (random orders of up to six
criminative stimuli, have the general form: Given stimulus A, elements, where, over the set of anomalous strings, the
response 1 will be reinforced conditionally on the presence of
frequency of each element was proportional to its fre¬
stimulus B, but response 2 will be reinforced conditionally on
the presence of stimulus C. Then, knowing that stimulus A was quency in the well-formed corpus). A second type of
present, one could predict that either response 1 or response 2 anomalous string consisted of “third-order approxi¬
would be emitted, but not which one. Knowing, additionally,
mations” to the language (strings of 3-11 elements
that stimulus B was present, one could more confidently predict
response 1, while knowing that stimulus C was present (rather consisting of “running triads” from the well-formed
than B), one could more confidently predict response 2. For sentences of the language). Together these two types
example, if stimulus A were “Give me a word beginning with
of anomalous strings made up 7% of all the strings to
m,” then the response “Marie” would be likely in the presence
of the additional instruction, “which is the first name of a which the experimental subjects were exposed (and
scientist who received Nobel Prizes in both physics and which they were asked to echo). Control subjects were
chemistry”; but the response “mother” would be likely in the
exposed to (and asked to echo) only well-formed
presence of the additional instruction, “which rhymes with
brother” strings. Recognition tests, sentence-completion tests,
The point about multiple causation is that any single con¬ and “word-association” (“lexical class”) tests (asking
trolling variable typically raises the probability of a whole class
the subject to pick out from a written vocabulary list
of verbal responses, as “Give me a word beginning with m” by
itself raises the probability of “Marie,” “mother,” and all other all the words that “went with” f and all words that
verbal responses, beginning with m, that happen to be current “went with” g) indicated that experimental subjects
in the speaker’s repertoire. Lacking other information, it would
had learned the syntactic structure of the language as
be impossible to predict which response from the class a speaker
would utter. But if one knew all the variables controlling each well as had control subjects, in spite of the diversion¬
response in a speaker’s verbal repertoire, it would be theoretically ary anomalous strings which experimentals had been
possible to predict exactly what the speaker would say next on
any occasion, as “Give me a word beginning with m—and
required to echo. The immunity of the experimental
rhyming with brother” zeroes in on just a single item. subjects to deleterious effects of echoing a corpus that
Evalyn Segal 647

included 7% anomalous strings led Braine to suggest verbal responses to determine a public verbal re¬
that the “degenerate” or ill-formed sentences and sponse. (Strictly speaking, only the private variables
sentence fragments to which children may be exposed are the autoclitic contribution to syntactic verbal be¬
in the course of learning their natural languages need havior, but the full power of the autoclitic concept
not be thought necessarily to interfere with the depends on multiple causation—the combined action
development of control over children’s syntactic be¬ of many determinants at once, only one of which is
havior by the regular intraverbal variables in the privately generated.)
linguistic corpora to which they are exposed (and Segal (1975b) pointed out the approximate corre¬
which they probably echo, at least in part and spondence between the grammatical account of syntax
covertly). He suggests, quite plausibly, that the intra¬ in terms of deep structure vs. surface structure and
verbal regularities (both those of temporal position the behavioral account in terms of verbal primitives
and those of intraverbal word linkages) are more than vs. autoclitics. According to transformational-genera¬
enough to override the distraction of “degenerate” tive theory, the terms m deep structure are not un¬
speech samples. ordered strings but rather are structured in ways that
reflect basic grammatical relations (subject-predicate
and so on), basic semantic relations (actor-action and
Intraverbal Contributions to Syntax
so on), and complex interrelations among these (e.g.,
Brained (1971) paper is worth close study for the semantic-syntactic constraints on “animate” grammat¬
further plausible suggestions he makes concerning the ical-semantic subjects with "inanimate” grammatical-
course of development of intraverbal syntactic control, semantic objects, such as those proscribing the string
including the control of inflections. For example, he “The boy may frighten sincerity”). According to auto-
notes that some intraverbal regularities may be more clitic theory, also (Skinner, 1957, pp. 332-333), there is
frequent, more consistent, and simpler, and hence some basic order among the verbal primitives. This
may more quickly acquire a controlling role in the “primitive order” reflects variously (1) phonological
child’s syntactic behavior than do other intraverbal processes responsible for meaningful sequences of
variables. If that is so, the aspects of syntactic behavior phonemes; (2) intraverbal order (“A 'train of thought'
controlled by these variables should appear earlier in in free association follows the order in which verbal
children’s speech, resulting in what Whitehurst and stimuli evoke other verbal responses”—Skinner, 1957,
Vasta (1975) have called “selective imitation/7 More¬ p. 333); (3) the overall, long-standing probability of
over, these intraverbal variables might provide a sort verbal responses in the speaker’s repertoire (perhaps,
of “seed” around which more complex intraverbal but not necessarily, mirroring the relative frequencies
control could “grow,” rather like a crystal (see Robin¬ of “words” in the “language”); (4) the momentary
son, Chapter 21 in this volume). That is, once the probability of responses reflecting (a) the momentary
children were responding reliably to the simpler intra¬ salience of particular environmental variables, (b) the
verbal variables, they might begin to come under the temporal order of the environmental variables (what
control of further intraverbal regularities between Robinson, in his chapter in this volume, calls “iconic”
them and more subtle intraverbal stimuli. structure), and (c) the momentary (contingent) ar¬
rangement of environmental variables into specific
combinations and relations (other than sheerly
Autoclitic Contributions to Syntax
temporal arrangements),
Intraverbal control cannot provide a complete ac¬ This first-order structure, supplied by the overall
count of syntactic structure, however powerful it organization of the speaker’s verbal repertoire (items
proves to be as an explanatory principle. As Braine 1, 2, and 3, above) and the structure of the environ¬
(1963a) noted, for example, it will not account for ment itself (“semantic” items 4a, b, and c, above), is
“contrastive” word order. For example, there is still “agrammatical.” The grammatical structuring
nothing in the intraverbal variables to account for the occurs when the primitively structured responses call
contrastive word orders of “Boy bites dog” vs. “Dog into play autoclitic processes. Skinner would say the
bites boy.” The nonverbal variables controlling the verbal primitives function as verbal stimuli evoking
tact are importantly involved here, and it is here that intraverbal (autoclitic) responses. Consider “Boy bites
the autoclitic hypothesis becomes most useful. By dog”:
autoclitic hypothesis I mean the hypothesis that ex¬
ternal nonverbal and verbal stimuli combine with 1. A semantic relation among the events in the en¬
private “stimuli” generated by covert or incipient vironment (the boy is doing something), function-
648 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

ing as a second-order tact variable, in combination terms in deep structure are not covert verbal re¬
with the covert availability of the verbal primitive sponses. They are dummy terms representing semantic
(tact) “boy,” functioning as one kind of intraverbal and syntactic “features.” For cognitive theorists, these
variable, and position within the utterance, func¬ features are simple “ideas” in the modality (whatever
tioning as another kind of intraverbal variable,
it may be) of mentalese. The particular collection of
together determine the ordering of the response
features that makes up a single dummy term (or “com¬
“boy” in first (subject) position in “deep structure.”
plex symbol” in Chomsky, 1965) represents a first-
2. Two additional semantic relations among the
order complex “idea,” and the organization of dummy
events in the environment (the actor of the action
terms that makes up a single deep structure repre¬
is singular, the action is occurring now), function
sents, so to say, a higher-order complex “idea.” Func¬
as second-order tact variables jointly evoking -s.
tionally these features can be construed as the inde¬
3. The covert availability of two verbal (tact) primi¬
pendent variables (nonverbal and verbal) that deter¬
tives bite and -s, functioning as intraverbal vari¬
mine the tacts and intraverbals of the autoclitic
ables, in combination with the semantic relations
analysis, and in the autoclitic analysis these variables
just mentioned (the actor is singular, the action is
ongoing), functioning as tact variables, determine directly determine verbal responses (although the
the autoclitic ordering of the primitives as bites. determination * is complex). But in grammatical
(This step is separated from step 2 because, one theory, even if the uttered sentence is a kernel sen¬
must suppose, -s cannot be ordered until it is avail¬ tence, certain basic transformations are called for be¬
able. Nevertheless, the semantic relations men¬ fore the dummy terms (ideas) in deep structure be¬
tioned in step 2 as controlling the availability of -5 come realized as words. The speakers must dip into
must be mentioned again in step 3 to account for their “lexicon” to find “formatives” that match exactly
the fact that -5 gets suffixed to bite and not to boy the particular collections of semantic and syntactic
or dog.) features assigned to the dummy terms in deep struc¬
4. The covert availability of the autoclitically ordered ture. Then the speakers must dip into their “morpho-
“response pair” bites, functioning as an intraverbal phonemic rule book” to transform the formatives into
variable, in combination with the semantic relation their realized sound sequences, words.
between the boy and the biting, functioning as a Despite the different ways behaviorists and gram¬
second-order tact variable, and with temporal posi¬
marians talk about these matters, it seems that they
tion in the utterance, functioning as another sort of
equally recognize the same complex set of variables,
intraverbal variable, together determine the order¬
and equally recognize the necessity for a complex
ing of the response pair bites in second (predicate
verb) position in the utterance. analysis of how the variables combine to determine
utterances. I do not think it matters a great deal
5. The semantic relation between the biting and the
whether one employs the terms “deep structure” and
dog, functioning as a second-order tact variable,
together with the covert availability of the tact “surface structure,” or the terms “primitive verbal
primitive dog, functioning as an intraverbal vari¬ responses” and “autoclitically modified verbal re¬
able, and temporal position within the utterance, sponses” to describe an important distinction between
functioning as another intraverbal variable, to¬ controlling variables in the external environment and
gether determine the ordering of the response dog in controlling variables generated by covert processes
third (predicate object) position in the utterance. within the speaker. I do not think it matters a great
The final result is “Boy bites dog.” (I have not deal whether one assigns “abstract semantic and syn¬
included determiners in this analysis. For a percep¬ tactic features” to “dummy terms in deep structure”
tive discussion of the complex combination of in¬
(which must then be transformed through “lexical
traverbal and tact variables controlling the English
and morphophonemic rules” into “words”), or
articles the and a, see Brown, 1973).
whether one regards the “nonverbal and intraverbal
variables” as determining the final, uttered string of
The Isomorphism of Autoclitic and
“verbal responses” through one continuous (but com¬
Transformational Theories
plex) “autoclitic behavioral process” occurring in real
time. Both the behaviorisms and the grammarian’s ac¬
At what point does deep structure turn into surface counts of utterances are hypothetical (and meta¬
structure? For the functional behaviorist, the answer phoric), and yet nothing less than their intricate, but
is simply when the responses are uttered publicly. isomorphic, hypothetical accounts seems to account
Transformational-generative grammarians cannot ac¬ for the complexities of syntax.
cept such an answer, however, because for them the MacCorquodale (1969) briefly summarized the
Evalyn Segal 649

theoretical assumptions underlying autoclitic theory. (rightly, I think, despite MacCorquodale’s disclaimer)
As he put it, the plausibility of the theory sensed its cognitive tendencies. Cognitive theorists
have not wanted to claim it because of its behavioristic
depends upon one’s being able to accept the no¬ parentage—and for another reason, it must be said:
tion that a speaker can respond discriminatively they have a richer, more fully developed version of the
to (1) what he is about to say; ... (2) why he is same idea in the theory of transformational-generative
about to say it; and (3) how strong the [covert grammar. The autoclitic was a serious attempt to
or incipient] operant is. (p. 840; italics his) grapple with the difficult problems of syntactic struc¬
ture and the evident need to distinguish between
MacCorquodale went on to argue:
something akin to deep structure and something akin
to surface structure, while remaining as close as the
The discriminations concern complex relations
problem permitted to the concepts and language of
between speech and its causes, and they are very
behaviorism. Verbal behavior is operant behavior, a
rapid. In this respect it is important not to re¬
lapse into conceiving of discrimination as a product of the three-term contingent relation among
separate prebehavioral act. Ordering is discrimi¬ behavior and antecedent and consequent environmen¬
native behavior, not the result of it, so that the tal events. It turns out, though, that a full description,
complex discriminations in autoclitic behavior functional Q1T cognitive^ of syntactic verbal behavior
need not be allotted prebehavioral time. , . , requires the postulation of hypothetical processes
The situation that strengthens the tacts the, boy within the organism, mediating between environmen¬
and rum also contains the relation that deter¬ tal “input” and response “output.” There is simply
mines the order of their emission as the boy rum. no gainsaying this.
If I am correct in this, autoclitic behavior is not,
Perhaps it is time for the hypothesis of the auto¬
strictly speaking, controlled by other behavior,
clitic to give way to more sophisticated analyses of
but by other operants. There is a difference, (p.
840; italics his) syntactic processes. Nevertheless, its place in the his¬
tory of the psychology of language is an honorable

In the second passage, as I understand it, Mac¬ one. Its lasting contribution is the insistence that syn¬
Corquodale is suggesting that we regard as merely tax is the result of a complex blend of variables, at
metaphoric Skinner’s references to incipient responses least some of which (the variables determining tact
and stimulation arising from incipient responses. and intraverbal responses) have familiar or conceiv¬
These metaphors simply represent the combined effects able dimensions. It may be that current attempts
of multiple tact and intraverbal variables controlling within linguistics (associated with the term generative
syntactic behavior. semantics) to incorporate semantic variables within
“Prebehavioral time” may or may not reflect the determinants of deep structure may profit from a
psychological reality, but it is not an issue distinguish¬ study of Skinner’s perceptive theorizing in the auto
ing an autoclitic account of syntactic behavior from clitic framework. “The speaker i§ the organism which
any other account of the complex three-term con¬ engages in or executes verbal behavior. He is also a
tingencies determining operant behavior. There need locus—a place in which a number of variables come
be no more (or less) “prebehavioral time” involved in together in a unique confluence to yield an equally
the temporal structuring of syntactic behavior than in unique achievement” (Skinner, 1957, p. 313).
the temporal structuring of, say, the fixed-interval
scallop. Nevertheless, in both syntax and the fixed-
interval scallop, the organism’s earlier behavior is said PARAPHRASE, THE PROBLEMATIC

by Skinner to function as a controlling variable con¬ LISTENER, AND MENTALESE

tributing to the determination of succeeding behavior.

If intraverbal variables, and not only tact variables, Skinner’s functional account of verbal behavior was
control autoclitics, then, metaphoric or not, incipient deficient in its neglect of the listener. Verbal Be¬
verbal responses seem to play an important role in the havior contains few serious references to the problems
theory of the autoclitic. of verbal comprehension, and what references there
are are mostly unsatisfactory. To a large extent Skin¬
ner shrugged off problems of accounting for how
An Appraisal of the Autoclitic
listeners learn to understand verbal stimuli as ordi¬
The autoclitic is an orphan nobody wants. Be- nary problems of discrimination, amenable to simple
haviorists have not wanted to claim it because they operant analysis and not requiring the special treat-
650 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

ment he devoted to the behavior of the speaker. verbal behavior in one language may give rise to
“Much of the behavior of the listener has no resem¬ private events within the individual which he
blance to the behavior of the speaker and is not ver¬ may then describe in another language. ... In
bal according to our definition. . . . The behavior giving the gist of what one has read in a book or
of a person as listener is not to be distinguished from heard someone else describe, in the same or a
other forms of his behavior” (Skinner, 1957, pp. different language, the speaker is often con¬
cerned with generating behavior having the
33-34).
same effect upon himself.
If the verbal stimuli to which the listener responds
are extremely simple, if their temporal arrangement
[The listener-speaker] tries out a [paraphrase],
is “iconic” (Robinson, chapter 21 in this volume), or
comparing the effects of the two versions upon
if the listener’s only role is as an assiduous supplier himself and changing the [paraphrase] until the
of goods and services for the speaker’s pleasure (his or effects are roughly the same. But this does not
her role as reinforcement mediator), this account is un¬ account for the behavior which he thus com¬
objectionable. Problems arise, however, when the pares. (1957, pp. 198, 78).
listener is called upon to paraphrase, to translate
freely from one language into another, or to under¬ The “behavior which he thus compares” is the
stand “a fairly difficult paper ... in the field of listener’s reaction to another’s utterance and the
scientific and philosophic discourse” (Skinner, 1957, listener’s reaction to the self-generated paraphrase, but
p. 278). Here the listener becomes, for Skinner, an¬ the nature of this listener reaction is never specified,
other speaker. The listener can properly say he or she beyond the suggestion that it may consist of “private
understands a difficult verbal passage, having complex events.” In other words, listeners infer the speakers’
syntactic structure, “only when he can emit corre¬ “semantic intentions” or “deep structure” or the
sponding behavior such as might occur ... in re¬ “variables controlling the speaker’s utterance” by
sponse to nonverbal or intraverbal stimuli'' (1957, pp. consulting their own private responses to the utterance.
277-278; italics mine). The problem of paraphrase is From these, somehow, the listener constructs a para¬
central, then. And Skinner does not handle the prob¬ phrase having the same private effects on themselves.
lem of paraphrase very satisfactorily, it seems to me. To the extent that the listeners succeed in this, they
The listener can paraphrase may be said, “elliptically,” to have inferred the vari¬
ables that controlled the speakers’ utterances.
only after he has identified the variables which Consider a concrete example. Premack (1970) de¬
were mainly effective [in evoking the original scribed a match-to-sample experiment with his chim¬
speaker’s utterance]. . . . panzee, Sarah, in which she was required to give a
It is . . . difficult to say what happens when “features analysis” of an apple. With the apple present
[a person paraphrases or] listens to a passage in as sample, she had to choose, between pairs of com¬
one language and restates it in another. The case parison stimuli, those comparison stimuli that roughly
is often offered as showing the need for some
matched the visual features of the apple. The pairs of
such concept as “idea” or “proposition,” since
comparison stimuli were: “a red plaque vs a green
something common to two or more languages
one; a square plaque vs a round one; a square plaque
[or, in the case of paraphrase, two or more utter¬
ances in the same language] appears to account with a stem-like protuberance vs a plain square one;
for their interchangeability. ... To say that and a square plaque with protuberance vs a plain
[the listener] emits behavior wThich is controlled round one” (p. 123). After Sarah had given her fea¬
by the variables which he infers to have been tures analysis of the apple, she was required to do an¬
responsible for [the original speaker’s verbal other features analysis, but this time with the word
behavior] . . . is . . . elliptical. (1957, pp. 280, apple rather than a real apple as sample stimulus.
78; italics mine) The two features analyses were identical (Premack,
1970, Table 1, p. 124). With the word as sample stim¬
It is indeed elliptical. Skinner wanted to avoid ulus, she chose between pairs of comparison stimuli
positing what Pylyshyn (1973) has called “abstract those features that match the visual features of an
propositional knowledge” and Fodor et al. (1974) have apple, not the visual features of the piece of blue
called “formulas in mentalese.” This is what “lies be¬ plastic that functioned in Sarah’s language as the
hind” and “motivates” deep structure in cognitive word “apple.” One might say Sarah “paraphrased”
grammatical theory. Rather evasively, Skinner sug¬ the word “apple” by indicating some of the physical
gests that features of an apple.
Evalyn Segal 651

The only account I can offer of Sarah’s success at are we not begging the question of how a speaker
this task is to suppose that the apple and the word chooses to utter a linguistic form if we say that
apple evoked similar private “listener reactions” in the choice is contingent upon an (unexplained)
previous choice of a message to communicate?
Sarah. Perhaps they both evoked a private visual im¬
. . . Nevertheless, it seems to us that there is
age of an apple, or perhaps both activated some more
much to be said for the old-fashioned view that
“abstract propositional” form in which her “knowl¬
speech expresses thought. . . .
edge” about apples was “stored.” Many cognitive It seems reasonably clear that there are cases
theorists (e.g., Fodor et ah, 1974; Pylyshyn, 1973) sug¬ in which . . . thinking consists in merely saying
gest that central “storage” of “abstract propositional to oneself bits of natural language which one
knowledge”—perhaps more or less directly in a form might equally well have said aloud. The re¬
representing a features analysis—is more plausible hearsal which often goes on in short-term memory
than to suppose that all such instances of “para¬ tasks ... is a persuasive example, . . . and it
phrase” are based on perceptual imagery in some is not implausible that some of the thinking
that goes on in problem solving might consist
covert peripheral modality.
in saying to oneself sentences or sentence frag¬
Experiments such as Premack’s (1970) seem to offer
ments in one’s language. . . .
strong justification for regarding the terms underlying
But ... it seems quite clear that underlying
deep structure as having some more abstract status many mental capacities, there must be computa¬
than covert verbal responses. Psycholinguistic research tional processes which are carried out in codes
on paraphrase and on verbal comprehension generally other than natural languages. The computations
(e.g., Gleitman & Gleitman, 1970; Johnson-Laird, underlying problem solving and the integration
1974) suggests that listeners understand utterances, as of percepts and motor gestures in nonverbal
they appear in surface structures, by inferring their organisms must be of this kind. . . .
underlying deep structures or the variables underly¬ We are, in effect, commending a view of the

ing deep structures. These represent, equivalently, ab¬ cognitive organization of organisms which bor¬
rows heavily from the actual organization of
stract propositional knowledge, semantic intentions,
multipurpose computers. Such devices typically
private listener responses, or the antecedent variables
perform their computations in an “internal”
(motivational, verbal, nonverbal) controlling verbal language which may be quite different from the
behavior. This, in a roundabout way, seems to be languages m which they accept their inputs and
what Skinner intended in his discussions of para¬ encode their outputs, (Fodor, Beyer, & Garrett,
phrase, translation, and understanding. 1974, pp. 374-377)

Questions about the character of mentalese,

MORE ON THE COMPLEMENTARITY OF however hopelessly metaphysical they may at
FUNCTIONAL AND COGNITIVE THEORIES first appear, are not entirely beyond the reach
of the combined methodologies of psychology
and linguistics: we can imagine data which xvouid
We come full circle back to problems qf accounting
bear directly on such questions. . . . (Fodor et
for the behavior of the speaker, problems of speech al., 1974, p. 383)
production, For both cognitive theorists and Skinner,
problems of speech production and speech reception
These passages illustrate q point Itl?vdc earlier m
are inextricably intertwined because, in any but the
this chapter, that cognitive theorists focus on the hy=
most simple comprehension problems, the listener
pothetical “computing” processes that must be occur¬
must function simultaneously as a speaker in order to
ring within the organism whenever complex environ¬
function at all. The theoretical problem is to identify
mental “input’ ’ variables lead to complex response
the determinants behind deep structure (behind prim¬
“outputs.” Research on artificial intelligence and com¬
itive verbal behavior), for these determine both what
puter simulation of complex “cognitive behavior”
a speaker says and how a listener understands a speak¬
seems to hold great promise as a means of illuminat¬
er’s utterances. Here are cognitive statements of the
ing the character of (or at least delimiting the possi¬
problem:
bilities for) such hypothetical processes. (For an ex¬
cellent and persuasive argument, see Turner, 1971.)
Common sense invites the view that what hap¬
pens in speech production is this: a speaker Behavior theorists interested in language and other
starts with a message he wants to communicate. complex behavior would profit from a greater famili¬
. . . But what sort of thing is a “message”? And arity with this information-theoretic approach.
652 TOWARD A COHERENT PSYCHOLOGY OF LANGUAGE

On the other hand, cognitive theorists would profit Catania, A. C. Reinforcement schedules and psychophysical
judgments: A study of some temporal properties of
from a greater familiarity with behavioral research
behavior. In W. N. Schoenfeld (Ed.), The theory of
on the role of complex environmental variables in reinforcement schedules. Englewood Cliffs, N.J.: Pren¬
operant behavior. As behaviorists have been amiss in tice-Hall, Inc., 1970.
ignoring the contributions of information theorists, Catania, A. C. Chomsky’s formal analysis of natural lan¬
so cognitive theorists have been amiss in minimizing guages: A behavioral translation. Behaviorism, 1972, 1,
the role of environmental factors in complex behav¬ 1-15.
Catania, A. C. The psychologies of structure, function,
ior. Not all the determinants of cognitive behavior are and development. American Psychologist, 1973, 28, 434-
within the organism, out of direct reach of experimen¬ 443.
tal analysis. Psycholinguists seem not to be aware of Chomsky, N. Syntactic structures. The Hague: Mouton,
the rich literature on stimulus control of operant be¬ 1957.
Chomsky, N. Verbal behavior by B. F. Skinner. Language,
havior and the effects of complex reinforcement con¬
1959, 35, 26-58.
tingencies, a literature which would go a long way CUomsky, N. Aspects of the theory of syntax. Cambridge:
toward explaining the origins of the semantic and MIT Press, 1965.
pragmatic aspects of verbal behavior, and which un¬ Clark, E. V. What’s in a word? On the child’s acquisition
doubtedly has relevance as well to the syntax of ver¬ of semantics in his first language. In T. E. Moore (Ed.),
Cognitive development and the acquisition of language.
bal behavior.
New York: Academic Press, 1973.
Psychology seems to be maturing, at last, into a Dews, P. B. The theory of fixed-interval responding. In
science. The balkanization of psychology into doc¬ W. N. Schoenfeld (Ed.), The theory of reinforcement
trinaire schools, each with its separate language spoken schedules. Englewood Cliffs, N.J.: Prentice-Hall, Inc.,
only by initiates, is giving way to a unified conception 1970.
Ervin-Tripp, S. Some strategies for the first two years. In
of problems, methods, and theories. If this chapter
T. E. Moore (Ed.), Cognitive development and the
persuades a few cognitive theorists and behaviorists to acquisition of language. New York: Academic Press,
venture out from their partisan positions and join 1973.
forces in grappling with the difficult problems of lan¬ Farwell, C. Aspects of early verb semantics; Precausitive
guage, it will have succeeded in its aim. development. Paper presented at the Stanford Child
Language Research Forum, April 1975.
Ferster, C. B., 8c Skinner, B. F. Schedules of reinforcement.
New York: Appleton-Century-Crofts, 1957.
Findley, J. D. An experimental outline for building and
REFERENCES
exploring multioperant behavior repertoires. Journal of
the Experimental Analysis of Behavior, 1962, 5, supple¬
Asch, S. E. Reformulation of the problem of association. ment, 113-166.
American Psychologist, 1969, 24, 92-102. Fodor, J. A., Bever, T. G., 8c Garrett, M. F. The psychol-
Ayllon, T., & Azrin, N. H. Reinforcer sampling: A tech¬ ogy of language. New York: McGraw-Hill, 1974.
nique for increasing the behavior of mental patients. Garcia, E., Guess, D., 8c Byrnes, J. Development of syntax
Journal of Applied Behavior Analysis, 1968, 1, 13-20. in a retarded girl using procedures of imitation, rein¬
Beach, F. A., Hebb, D. O., Morgan, C. T., 8c Nissen, H. W. forcement, and modelling. Journal of Applied Behavior
(Eds.). The neuropsychology of Lashley. New York: Mc¬ Analysis, 1973, 6, 299-310.
Graw-Hill, 1960. Gardner, B. T., 8c Gardner, R. A. Two-way communica¬
Braine, M. D. S. On learning the grammatical order of tion with an infant chimpanzee. In A. Schrier (Ed.),
words. Psychological Review, 1963, 70, 323-348. (a) Behavior of non-human primates (Vol. 4). New York:
Braine, M. D. S. The ontogeny of English phrase structure: Academic Press, 1971.
The first phase. Language, 1963, 39, 1-13. (b) Gardner, R. A., 8c Gardner, B. T. Teaching sign language
Braine, M. D. S. The insufficiency of a finite state model to a chimpanzee. Science, 1969, 165, 664-672.
for verbal reconstructive memory. Psychonomic Science, Gardner, R. A., 8c Gardner, B. T. Early signs of language
1965, 2, 291-292. (a) in child and chimpanzee. Science, 1975, 187, 752-753.
Braine, M. D. S. On the basis of phrase structure: A reply
Gleitman, L. R., 8c Gleitman, H. Phrase and paraphrase.
to Bever, Fodor, and Weksel. Psychological Review, 1965,
New York: Norton, 1970.
72, 483-492. (b)
Guess, D. A functional analysis of receptive language and
Braine, M. D. S. On two types of models of the internali¬
productive speech: Acquisition of the plural morpheme.
zation of grammars. In D. I. Slobin (Ed.), The onto¬
genesis of grammar. New York: Academic Press, 1971. Journal of Applied Behavior Analysis, 1969, 2, 55-64.
Brown, R. A first language: The early stages. Cambridge: Guess, D., 8c Baer, D. M. An analysis of individual differ¬
Harvard University Press, 1973. ences in generalization between receptive and productive
Carter, A. L. The transformation of sensorimotor mor¬ language in retarded children. Journal of Applied Be¬
phemes into words: A case study of the development of havior Analysis, 1973, 6, 311-329.
“here” and “there.” Paper presented at the Stanford Guess, D., Sailor, W., Rutherford, G., 8c Baer, D. M. An
Child Language Research Forum, April 1975. experimental analysis of linguistic development: The
Evalyn Segal 653

productive use of the plural morpheme. Journal of Ap¬ language: A progress report. Brain and Language, 1974,
plied Behavior Analysis, 1968, 1, 297-306. 1, 205-212.
Hawkes, L., 8c Shimp, C. P. Reinforcement of behavioral Sailor, W. Reinforcement and generalization of productive
patterns: Shaping a scallop. Journal of the Experimental plural allomorphs in two retarded children. Journal of
Analysis of Behavior, 1975, 23, 3-16. Applied Behavior Analysis, 1971, 4, 305-310.
Hilgard, E. R., 8c Rower, G. H. Theories of learning. Salzinger, K. Are theories of competence necessary? Annals
Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1975. of the New York Academy of Sciences, 1975, 263, 178-
Jenkins, H. M. Sequential organization in schedules of 196.
reinforcement. In W. N. Schoenfeld (Ed.), The theory of Schumaker, J., 8c Sherman, J. A. Training generative verb
reinforcement schedules. Englewood Cliffs, N.J.; Pren¬ usage by imitation and reinforcement procedures.
tice-Hall, Inc., 1970. Journal of Applied Behavior Analysis, 1970, 3, 273—287.
Johnson-Laird, P. Experimental psycholinguistics. Annual Segal, E. F. Induction and the provenance of operants. In
Review of Psychology, 1974, 25, 135-160. R. M. Gilbert & J. R. Millenson (Eds.), Reinforcement:
Kausler, D. H. Psychology of verbal learning and memory. Behavioral analyses. New York: Academic Press, 1972.
New York: Academic Press, 1974. Segal, E. F. Language: A behavioral perspective, San
Kelleher, R. T. Chaining and conditioned reinforcement. Diego; San Diego State University Press, 1975. (a)
In W. K, Honig (Ed.), Operant behavior: Areas of re¬ Segal, E. F. Psycholinguistics discovers the operant: A re¬
search and application. Englewood Cliffs, N.J.: Prentice- view of Roger Brown’s A first language: The early stages.
Hall, Inc., 1966. Journal of the Experimental Analysis of Behavior, 1975,
Killeen, P. On the temporal control of behavior. Psycho¬ 23, 149-158. (b)
logical Review, 1975, 82, 89-115. Sherman, J. A. Imitation and language development. In
Lashley, K. S. The problem of serial order in behavior. Advances in child development and behavior (Yol. 6).
In L. A. Jeffress (Ed.), Cerebral mechanisms in behavior. New York; Academic Press, 1971.
New York; Wiley, 1951. (Reprinted in Beach, Hebb, Skinner, B. F. Are theories of learning necessary? Psycho¬
Morgan, Nissen, I960,) logical Review j 1950, 57; 193—216.
Lutzker, J. R.f 8c Sherman, J. A. Producing generative Skinner, B. F. Verbal behavior. New York: Appleton-Cen-
sentence usage by imitation and reinforcement pro¬ tury-Crofts, 1957.
cedures. Journal of Applied Behavior Analysis, 1974, 7, Slorin, D. The more it changes . . . : On understanding
447-460. language by watching it move through time. Paper pre¬
MacCorquodale, K. B. F. Skinner’s Verbal behavior: A sented at the Stanford Child Language Research Forum,
retrospective appreciation. Journal of the Experimental April 1975.
Analysis of Behavior, 1969,12, 831-841. St addon, J. E. R. Temporal control and the theory of rein¬
MacCorquodale, K. On Chomsky’s review of Skinner’s forcement schedules. In R, M» Gilbert 8c j, R, Millenson
Verbal behavior. Journal of the Experimental Analysis (Eds.), Reinforcement: Behavioral analyses, New York.
of Behavior, 1970, 13, 83-99. Academic Press, 1972.
McNeill, D. Semiotic extension. Paper presented at the Thompson, J., 8c Chapman, R. S. Semantic development,
Loyola Symposium on Cognition, Chicago, April 1974. Paper presented at the Stanford Child Language Re¬
Moore, T. E. (Ed.). Cognitive development and the acquisi¬ search Forum, April 1975.
tion of language. New York: Academic Press, 1973. Thorndike, E. L. Animal Intelligence. New York; Mac¬
Morse, W. H. Intermittent reinforcement. In W. K. Honig millan, 1911.
(Ed.), Operant behavior; Areas of research and applica¬ Tulying, E. Gue-dependent forgetting. American Scientist*
tion. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1966. 1974, <52, 74-82.
Piaget, J. The origins of intelligence in children. (2nd ed.) Tulving, E., fc Thomson, D. M. Encoding specificity and
New York: International Universities Press, 1952. (1st retrieval processes in episodic memory, Psychological
ed., 1936) Review, 1973. SO, 352-373,
Premack. D. Toward empirical behavior laws, I: Positive Turner M, B, Realism and the explanation of behavior>
reinforcement. Psychological Review, 1959, 66, 219-233. Englewood Cliffs, N.J.; Prentice-Hall, Inc,, 1971*
Premack, D. A functional analysis of language. Journal of Weiss, B, The fine structure of operant behavior during
the Experimental Analysis of Behavior, 1970, 14, 107- transition states. In W. N. Schoenfeld (Ed,), The theory
125. of reinforcement schedules, Englewood Cliffs, N,J,; Pren¬
Prokasy, W. F., 8c Hall, J. F. Primary stimulus generaliza¬ tice-Hall, Inc., 1970.
tion. Psychological Review, 1963, 70, 310-322. Whitehurst, G. J. Generalized labeling on the basis of
Pylyshyn, Z. W. What the mind’s eye tells the mind's structural response classes by two young children,
brain: A critique of mental imagery. Psychological Bul¬ Journal of Experimental Child Psychology, 1971, 12,
letin, 1973, 80, 1-24. 59-71.
Roeper, T. Connecting children’s language and linguistic Whitehurst, G. J. Production o£ novel and grammatical
theory. In T. E. Moore (Ed.), Cognitive development utterances by young children. Journal of Experimental
and the acquisition of language. New York: Academic Child Psychology, 1972, 13, 502-515.
Press, 1973. Whitehurst, G. J., 8c Vasta, R. Is language acquired
Rumbaugh, D. M., von Glasersfeld, E., Warner, H., through imitation? Journal of Psycholinguistic Research,
Pisani, P., 8c Gill, T. V. Lana (chimpanzee) learning 1975, 4, 37-59.
 133, 134, 135, 137, 139, 142, Besley, S., 85, 439, 444, 446, Braun, J. J., 608

Author 143, 147

Azrin, N. H., 23, 89, 116, 129,
136, 139, 177, 182, 183, 184,
470, 471
Bever, T. G., 632, 651
Bigelow, G., 563
Breese, G. R., 168
Breland, K., 3, 7, 22, 273, 406
Breland, M., 3, 7, 22, 273, 406
185, 194, 253, 309, 350, 357, Bilbrey, J., 56, 127 Bremner, F. J., 408, 409

Index 358, 359, 384, 385, 386, 394,

409, 417, 418, 420, 422, 426,
429, 468, 469, 561, 563, 636
Bindra, D., 62, 128, 144, 407,
582, 583
Birch, D., 144, 146
Brener, J. M., 606, 607, 608,
610,611
Brengelmann, G. L., 160
Azzi, R., 219 Birch, J., 606 Bresnahan, E. L., 501, 507
Bishop, M. P., 589, 590 Brethower, 73, 77
NOTE: This index provides Brezenoff, H. E., 168
Bacon, F., 13 Bivens, L. W., 577
information on the citation of Bacon, W. E., 589, 590 Black, A. H., 118, 119, 120, Bridgman, P. W., 30
names in the regular text, but Briese, E., 589, 590
Badia, P., 397, 398, 405 340, 352, 393, 397, 597, 598,
not in the reference lists Baenninger, R., 116 603, 606, 607, 610 Brigham, T. A., 1
occurring at the end of each Black, W. C„ 585, 589, 006 Brill, N. Q,, 598
Baer, D. M., 639
chapter, Each author of a Bagchi, B. K., 607 Blackman, D. E., 340-363, 5, Brimblecombe, R. IV., 168
chapter in this book is indi¬ 305, 341, 345, 346, 347, 348, Brimer, G. J., 116, 393
Baker, R. A., 32, 33
cated by a reference to the 350, 354, 356, 358, 359, 517 Brissey, C., 515
Balagura, S., 32, 33, 41
inclusive page numbers of his Blanchard, R., 3g0, 3g4, 326 Brittain, R. T., 167
Balboni, 249
or her chapter set in bold£ace? Blandau, 21 Brittain, T. M., 603
Baldock, M. D., 57, 58, 66, 78,
immediately following his or Blaustein, J., g01 Brobeck, J. R., 31, 154
her name. 127
Baldwin, B. A., 153 Blizard, D., 606 Brodie, D. A., 578
Balliet, R. F., 519 Block, EE D., 6g2 Brodie, J. ]., 578
Balster, R. L., 556 Bloom, F. E., 586, 588 Brody, 5., 32
Bandler, R. J., 15 Bloom, W., 136 Brogden, W. J., 421, 426
Abelson, R. M., 261, 282 Banks, J. H., 603, 604 Bloomfield, T. M., 71, 75, 76, Brower, L. P., 484
Abrams, R. M., 153, 157, 158 Banuazizi, A,, 608, 610 85, 267, 273, 318, $19, 324, Brown, A. C., 160
Aceto, M. D. G.? 356 Barash, D. P., 46 325, 326 Brown, G., 610
Ackil, J. E., 58, 60, 81, 120 Barger, A. G., 606 Blough, X>, $„ $14-539, 5, 144, Brown, G. G., 606
Adair, E., 40, 153, 154, 155, 164 Barker, E. G., 438, 483 177, 208, 225, 238, 246, 264, Brown, C. EE, 446, 465, 467,
Adams, D. W., 578 Baron, A., 291 434, 435, 437, 438, 448, 449, 4^1, Vft
Adams, H,, 65 Baron, M. R., 482, 494 451, 452, 453, 463, 464, 495, Brown, G. M., 603
Ader, R., 604 Barrera, F. J., 13, 65 504, 521, 522, 525, 530, 535, Brown, J. L,, 590
Adey, W. R., 605 Barrett, J,, 357 536, 537 Brown, J. S., 433
Adler, N. T„ 21, 59 Barry, H., 555, 561 Blough, P. M., 514-539, 5, 225, Brown, P. L., 2, 7, 54, 55, 60,
Adolph, E. F., 17, 32, 33, 35, 36 Bauer, I. A., 597 434, 517, 518, 521, 528, 524, 62, 119, 120, 127, 139, 252,
Ahlskog, J. E., 85, 447, 469 Baum, W. M., 80, 111, 127, 525, 530 272, 359, 406
Aiken, L. C., 532 128, 235, 236, 238, 239, 240, Boakes, R. A., 61, 76, 77, 80, Brown, R,, OdZ, Odd, 640, 643,
Akerman, B„ 139 241, 242, 243, 246, 248, 254, 84, 85, 87, 90, 318, 320, 325, 648
Alferink, L. A., 211 264, 267, 272, 277, 297, 298, 326, 532 Brown, R. T., 509
Allaway, T. A., 57 381, 386, 387, 388, 410 Bobillier, P., 586 Brown, S,, 570, 590
Alleman, H. D„ 224 Baxter, L., 586 Bogert, C. M., 157 Brown, T. G., 139
Allen, J. D., 74, 75, 135, 138 Beach, F. A., 14, 20, 21, 642 Boice, B , 44 Brown, W., 307
Allison] J., 29, 106, 107 Beach, H. D., 557 Boling, J. L., 21 Browne, M,, 65, 138, 314, 318,
Allison, T. S., 243, 244 Beagley, G., 580 Bolles, R. e., 14, 33, 45, 90, 320
Amsel, A., 76, 226, 509 Beale, I.,L., 435, 447, 464, 528, 113, 116, 117, 120, 138, 165, Brownstein, A. J., 127, 233,
Anand, B., 607 589 252, 204, 304, 372, 393, 397, 288, 242, §43, 246, §48
Anchel, H., 407 Beard, R. R., 562 398, 406, 407, 408, 409, 410, Bruch, H., 7
Andelman, L., 507 Beardsley, J. V., 605 580 Bruinvels, J„ 167, 169
Anderson, D. E., 602, 603 Beck, J., 005 Bonbright, J. C„ 384, 521 Brunner, A., 589
Anderson, R. C., 7, 57 Becker, D. I., 554 Bond, N. W„ 350 Brush, F. R., 603
Andersson, B., 14, 139 Beckman, A. L., 168, 169 Boneau, C. A., 77, 78, 445, 496, Bryant, R. G., 356
Andry, D. K„ 489 Bedart E, W„ 516 497, 534 Buckley, J, P,, 356, 561
Anger, D., 143, 223, 224, 225, Beds, C. C., 604 Booth, D.A., 32, 33, 35 Bndgell, P., 153
263, 264, 333, 393, 400, 426 Beer, B, 178, 572, 577, 597 Bootzin, R. R„ 7 Budnik, J, i„
Angyan, L., 590 Beier, E. M.? 258, 278 Boren, J, J„ 540-569, 5, 225, Btidiysuki, T. II., 610
Anliker, J., 597 Belaiehe, R , 163 355, 369, 372, 378, 388, 394, Burks, G. Dj; 130
Annan, Z„ HZ, 357 Bdl, R. Hr V-, 33, 36 SOSj 396j 46J, 540, 544, 578, Burr, B. E. fi., 408, 501, SG2
Anokhin, P. K.} 144 Belleville, R. E., 367 579 Burstcin. K. R., 445
Bursten, B., 589
Antelman, 5. M., 585 Belluseio, P. R., 502 Borcns M, G« P„ SOS, S92. 301
Appel, J. B.} 139? 214f 356, 381f Benefidd, R L., 48§; 490 Boudin, IE M., 563 Butcher, L. I ., 586
382, 554 Beuinger, R. J., 267 Bouzas, 274, 275 Butcher; R: Lu 19S
Arazie, R., 135 Bowden, D, M., 571, 585 Byrd, L. B., 193, 205, 206, 304,
Benson, IE, 609
Arbuthnott, G. W., 585 Bowen, C., 132 314, 429
Berger, B. D., 57R 565
Armstrong, R. P., 562 Berkley, M. A., 520, 575, 578 Bower, G. H., 80, 261, 281, 324, Byrnes, J„ 039
Arnett, F. B., 78 Berlanger, D., 597, 603
336, 628, 637, 638
Asch, S. E., 638 Bowring, A. M., 577 Cabanac, M., 153, 157, 158,
Berlyne, D. E., 318, 324, 326
Askew, H. R., 85, 447, 469 Boyd, E. S., 589, 590 162, 163, 164, 165, 166
Berman, A. J., 222 Caggiula, A. R., 579, 581
Atkinson, J. W., 144, 146 Brady, J. V., 596-618, 5, 38, 89,
Bernal, J. D., 549 Caldwell, F. T., 153, 157, 158,
Auge, R. J., 320, 322, 325 141, 299, 302, 308, 342, 344,
Bernard, C., 10 347, 349, 351, 352, 353, 354, 166
Autor, S. M., 251, 297, 313, 326,
327, 329, 330, 332, 333 Bernheim, H. A„ 166 355, 356, 357, 359, 366, 396, Callard, I. P., 162
Averill, J. R., 603 Bernheim, J. W., 75, 78, 85, 540, 556, 578, 579, 596, 597, Campbell, B. A., 35, 40, 105,
Avery, D. D., 168, 169 268, 471 598, 599, 600, 601, 602, 603, 261, 263, 281, 421, 581
Avin, E., 524 Bernstein, D. J., 107, 108 604, 609, 610 Campbell, H. J., 582, 589
Axelrod, S., 77, 78 Berry, F., 597 Brahlek, J. A., 89, 359 Campbell, R. L., 183, 380
Ayllon, T., 309, 384, 385, 386, Berryman, R., 58, 128, 141, 213, Braine, M. D. S., 641, 643, 644, Campeau, E., 605
429, 636 375, 525 645, 646, 647 Cane, V., 142
Ayres, S. L., 126, 130, 131, 132, Bersh, P. J., 381, 382, 405 Branch, M. N., 325 Canfield, D. R., 586

655
 656
Author Index

Cannon, W. B., 31, 596 Conner, J. D., 588 De Lorge, J., 303, 304, 317 Eisenberger, R., 103, 105, 106,
Cantor, A., 167, 168 Conrad, D. G., 185, 260, 280, Delse, F. C., 606 107, 108
Cantor, M. B., 578, 579 358, 578, 579, 597, 604 Dement, W. C., 22 Eisenman, J. S., 154
Capaldi, E. J., 509 Cook, L., 74, 143, 185, 189, 191, Deneau, G. A., 192 Eissenberg, E., 611
Capehart, S., 75 212, 226, 358, 427, 540, 555, Denny, M. R., 222, 226 Elder, S. T., 589, 590
Caplan, H. J., 446, 465, 467, 557, 559 Denny-Brown, D., 19 Eldridge, L., 597
468, 469, 471, 472 Coons, E. E., 14, 580, 582 De Ruiter, L., 31 Ellen, P., 388
Carlisle, H. J., 129, 138, 139, Cooper, B. R., 585 Descartes, R., 8, 29 Ellis, B. B., 136
153, 154, 164, 168, 169, 577 Cooper, K. E., 167 De Toledo, L, E., 352, 598, Ellis, W. R., 456
Carlton, P. L., 153, 344, 345, Cooper, R. M., 590 606
585 Elsmore, T. F., 534, 535
Coover, G. D., 603 Deutsch, J. A., 471, 472, 575,
Carmona, A., 607 Emley, G. S., 418, 420, 422, 423
Coppock, H. W., 557 576, 577, 579, 581, 584, 586, Engel, B. T., 608
Carroll, M. E., 14 Corballis, M. C., 528 588
Carter, A. L., 633 Engen, T., 533
Corbit, J. D., 154, 158, 164 De Valois, R. L., 518
Catania, A. C., 1, 71, 73, 75, Epstein, A. N., 17, 20, 153, 156
Coscina, D. V., 32, 33, 40 De Vietti, T. L., 598 Emits, T., 164
76, 78, 79, 84, 140, 143, 176, Costa, L. D., 532 de Villiers, P. A., 233-287, 4, Ernst, A. J., 489
189, 191, 213, 214, 215, 216, Coulson, G., 379, 380 78, 86, 129, 132, 140, 244,
217, 218, 220, 224, 233, 234, Ervey, D. H., 113, 116
Coulson, V., 379 248, 249, 250, 253, 254, 258,
235, 237, 240, 241, 243, 246, Ervin, F. R., 484, 485
Coulter, G., 407 260, 261, 267, 270, 271, 348, Ervin-Tripp, S., 634
248, 249, 254, 257, 258, 259, Cowles, R. B., 156, 157 352, 354, 355, 371, 372, 373,
263, 264, 265, 266, 292, 307, Estes, R. D., 33, 47, 88
Cox, V. C., 15, 571 374, 375, 382, 383, 386, 387,
321, 328, 375, 398, 410, 435, Estes, W. K., 185, 253, 341, 347,
Crabtree, J. M., 604 402
452, 463, 464, 557, 629, 646 353, 357, 393, 409, 598
Craig, W.; 58 Devos, M., 32, 33, 44
Caul, W. F., 604 Evans, G., 447
Cranston, W. I., 167 De Weese, J., 294, 316 Everett, P. B., 130
Cave, C., 143 Crawford, J. M., 588 Dewey, W. L„ 553
Cebulla, R. P., 604 Crawshaw, L. I., 158, 168, 169 Dews, P. B., 188, 202, 208, 209, Faber, B., 35
Champlin, G., 72 Crespi, L. P., 86, 258, 259, 278 214, 218, 219, 220, 222, 223, Fabricius, E., 139
Chan, S. W. C., 162 Crichton, M., 571, 580 228, 229, 309, 333, 348, 540, Faillace, L. A., 563
Chapman, R. S., 633 Crider, A., 606, 607, 610 542, 551, 552, 554, 556, 559, Falk, J. L., 23, 32, 128, 129,
Chase, S., 499, 524, 536 Crissman, J. K., 162 560, 646 130, 132, 136, 137, 138, 577,
Cheek, M. S., 135 Crites, R, J., 437 DiCara, K., 571, 575, 576 578
Cheng, M. F., 18 Crossman, E. K., 211, 225 DiCara, L. V., 606, 607, 608, Fallon, D., 131
Cherek, D. R., 136, 139, 552, Crow, T. J., 581, 585, 586, 590 609, 610, 611
562 Fantino, E., 313-338, 4, 61, 158,
Crowder, W. F., 554 Dickinson, A., 74
Chhina, G., 607 182, 184, 237, 240, 241, 242,
Cruce, J. A. F., 582 Dickson, J. F., 460, 462
Chipman, G. D., 157 244, 246, 248, 252, 277, 298,
Cruze, W. W., 69 Didamo, P., 344, 345
Cho, C., 116 300, 303, 305, 316, 324, 326,
Culberton, J. K., 575, 578 Dillow, P. V., 143, 296
Chomsky, N., 5, 620, 622, 628, 327, 328, 329, 330, 331, 332,
Culler, E., 421, 426 Di Lollo, V., 258, 278 333, 334, 335, 336, 346, 355,
629, 630, 632, 638, 640, 641, Cumming, W. W., 208, 213,
648 Dinsmoor, J. A., 261, 282, 296, 397
224, 525 297, 314, 318, 319, 320, 321, Fantz, R. C., 69
Chorny, H., 426 Cunningham, D. J., 162 322, 324, 325, 326, 368, 384, Farmer, J., 222
Chorover, S. L., 14 Cunningham, S., 7
Chow, M. I., 585 385, 391, 392, 398, 400, 401, Farrar, C. H., 406, 407
Curtis, D. R., 588 407, 408, 427 Farrell, L., 57, 61, 127
Christoph, G., 90 Curto, K., 455 Dixon, C., 606
Chu, N. D„ 586 Farris, H. F., 59
Cushman, A. J., 572 Dobrzecka, C., 483, 484, 485 Farthing, G. W., 85, 359, 439,
Chung, Shin-Ho, 141, 233, 251, Cutts, D., 234, 235, 243 Domino, E., 585
252, 258, 266, 301, 304, 308, 444, 446, 452, 454, 470, 471,
Donchin, E., 606 482, 492, 496, 499
316 Daley, L. A., 602 Doty, R. W., 570 Far well, C., 633
Church, R. M., 165, 365, 375, Dalland, J. I., 519, 520, 521,
398, 400 Downs, D. A., 553 Fearing, F., 29
523 Drabman, R. S., 309 Fekety, F. R., 166
Cioe, J., 570-595, 5, 574 Dalrymple, S. D., 428, 429 Dragoin, W. B., 14, 485 Feldberg, W., 167, 168, 169
Clack, T. D., 516, 522, 523 Dalton, A. J., 603 Dreese, A., 585 Feldman, R. S., 408, 409
Clark, E. V., 634 Davenport, D. G., 378, 379, 380 Drexler, 446 Feldman, S. M., 573, 597
Clark, F. C., 137, 139, 143, 192, Davenport, J. W., 258, 278 Driscoll, J., 323 Felton, M,, 209, 210, 213, 217,
367, 370, 371, 373, 374, 377 Davidson, A. B., 74, 555
Clark, H. B., 307 Driver, G., 600, 601 346
Davidson, E. H., 282 Ducharme, R., 603 Ferguson, J., 22
Clark, R. L., 132, 134, 142, 597 Davidson, R. S„ 153 Duclaux, R., 153, 158, 166 Ferrari, E. A., 120, 406, 408
Clark, W. G., 153, 166 Davis, A. H., 572, 589 Dukhayyil, A., 454 Ferraro, D. P., 606
Clayton, K. N., 576 Davis, D. D., 154, 156 Duncan, A. R., 35 Ferster, C. B., 7, 38, 54, 140,
Clements, R. O., 78, 127, 319, Davis, D. J., 555
322, 323, 324, 325 Duncan, B., 252, 298, 335 176, 201, 204, 206, 207, 209,
Davis, E., 201, 216 Duncan, I. H. H., 35, 535
Clopton, B. M., 524 210, 212, 213, 217, 219, 222,
Davis, H., 341, 342, 356, 360 Dunham, P. J., 98-124, 3, 4, 223, 224, 225, 228, 234, 290,
Coates, J., 73 Davis, J. M., 139, 446, 447
Coger, R. W., 378, 379 65, 86, 99, 100, 104, 107, 108, 291, 293, 295, 296, 298, 345,
Davis, R. C., 597 643
Cohen, H., 22 117, 129
Davison, M. G, 240, 241, 248, Fibiger, H. C., 585, 586
Cohen, M., 563 Dunn, G., 597, 606
252, 255, 275, 276, 277, 298, Field, G. E., 394
Cohen, P. S., 136 Durham, R. M., 605
302, 324, 332, 336, 572, 573, Fields, C., 608, 610
Cohen, S. I., 603, 606 589, 590 Dworkin, B. R., 611
Cohen, S. L., 304, 305, 308, 316 Dykstra, L. A., 554 Figar, S., 597
Davitt, G. A., 507
Cohn, M., 462 Dysart, J., 249, 460, 461 Findley, J. D., 38, 78, 226, 234,
Dawes, G. S„ 158
Coker, C., 397 235, 249, 291, 292, 295, 299,
Day, R. B., 226
Cole, B. K., 201, 376 Eck, K. O., 498, 501, 502 302, 308, 366, 396, 598, 601,
Deadwyler, S. A., 130 602, 603, 609, 610, 643
Cole, J. L., 140, 534 De Casper, A. J., 224, 227 Eckerman, D. A., 322, 323
Coleman, W. P., 162 Fisher, A. E., 586
Deegan, J., 559 Edinger, H. M., 154
Collier, G., 28-51, 4, 32, 33, 35, Fitzgerald, R. D., 606
De Good, D. E., 603 Edwards, S. B., 15, 582
36, 38, 39, 40, 129, 134, 193 Fitzsimons, J. T., 31, 32, 36
Deguchi, T., 606 Egger, M. D., 56, 57
Collins, D., 600, 601 Flaherty, C. F., 509
Delbruck, N., 248 Ehrenfreund, D., 38 Flamer, R., 563
Collins, J. P., 438 Delgado, J. M. R., 589 Ehrlich, W., 606
Conger, J. J., 597 Fleshier, M., 120, 351, 407, 426,
Delini-Stula, A., 606 Eimas, P. D., 487 482, 483, 489
Author Index 657

Fletcher, F. G., 58 Gilden, L., 532 Hager, J. L., 14, 91, 407, 436 Hemmes, N. S., 80, 82, 83, 86,
Flint, G. A., 319 Gill, C. A., 78, 452 Hagguist, W. W., 258, 278 90, 438
Flory, R. K., 129, 130, 131, 132, Gill, T. V., 640 Hahn, W. W., 597, 608 Hendersen, R. W„ 153-173, 4
135, 136, 137, 139 Gillet, A., 153, 166 Hailman, J. P., 435 Hendry, D. P., 318, 319, 323,
Flourens, M. J. P., 8 Gilliam, W. J., 598, 601, 602, Hain, J. D., 604 326, 336, 350, 352, 354
Flynn, J. P., 14, 15, 16, 21, 582 609, 610 Hake, D. F., 23, 89, 129, 136, Henke, P. G., 74
Fodor, J. A., 632, 635, 651 Gilmore, L., 600, 601 182, 183, 358, 359, 380, 384, Henry, P„ 22, 143, 296, 324
Folz, E. L., 604 Githens, S. H., 130 385, 386, 409, 417, 418, 426, Hensel, H., 161
Fonberg, E., 20 Giulian, D., 404 429, 468 Henton, W. W„ 89, 360, 517
Fontaine, O., 559 Giurintano, L. P., 440 Halas, E. S., 605 Herberg, L. J., 579, 581
Foote, W. 582 Glass, D. H., 86 Halgren, C. R., 487 Herbert, E. W., 251, 252
Foree, D. D., 70, 120, 407, 408, Glaucheva, L., 20 Hall, G., 503, 508 Herd, J. A., 598, 601, 602, 603,
485, 487 Glazer, H., 141, 142, 143 Hall, J. F., 639 604, 606, 609
Forgione, A. G., 409 Glazer, R. D., 113, 115, 116 Halliday, M. S., 76, 77, 87, 90 Herman, P. N., 516
Forrest, M. S., 603 Gleitman, H., 632, 651 Halmi, K. A., 7 Herrnstein, R. T., 28, 61 , 63,
Forsyth, R. P„ 598, 601, 602, Gleitman, L. R., 632, 651 Halpern, B. P., 144, 527, 529 78, 79, 88:, 99/ 133, 185, 204,
603, 604, 606 Glenn, M. F., 434 Hamilton, B.; 87 205, 200, 207, 208, 209, 210,
Fowler, H., 261, 281, 282 Glickman, S. E„ 571, 580, 581, Hamilton, C. L., 162 211, 212, 214, 233, 234, 235,
Fowler, R. L., 606 586, 588 Hamilton, E. L., 219 236, 240, 242, 243; 245, 246,
Fraenkel, G. S., 158 Goesling, W. J., 608, 610 Hamlin, P„ 32, 33, 36, 41, 42, 247, 248, 251, 252, 253, 254,
Frank, J., 140, 141, 142, 143, Goff, W. R., 523 47 257, 258, 259, 260, 261, 262,
147 Goldberg, A. M., 542 Hamm, H. D., 307 263, 264, 265, 266, 267, 268,
Frankenheim, J. M., 553 Goldberg, S. R., 179, 192, 193, Hammel, H. T., 153, 154, 157, 269, 270, 272, 274, 275, 277,
Franklin, S., 56 542, 554 158 278, 297, 298, 306, 324, 327,
Frazier, T. W., 603 Goldiamend, I., 222 Hammer, C,, 219 328, 329, 330, 332, 333, 334,
Freed, L. M., 138 Goldman, L„ 603 Handley, S. L., 167 336, 357, 373, 374, 375* 380,
Freedman, D. X., 554 Goldstein, M. D., 573 Hanford, P. V., 260, 281, 307, 381, 382, 383, 386, 387, 397,
Freeman, B. J., 74, 75, 76, 78, Goldstein, R., 573, 579 317 400, 401, 402, 407, 532, 542*
82 Gollub, L. R., 288-312, 2, 137, Hanford, P. W., 219 543, 559
Freeman, F., 482, 490, 498, 499 181, 193, 194, 226, 270, 271, Hankins, W, G„ 8 Hess, W. R i, 14, 570
Freeman, M. E., 162 272, 302, 313, 315, 318, 322, Hanna, B., 102 Higgins, D. J., 603
Freeman, W. J,, 154, 155, 156 324, 460, 402, 540 Hansen, M, G., 168, 169 Hilgard, E . R„ 028, 637, 638
Freud, S., 11 Gonzales, F. A., 61, 72 Hanson, H. M., 249, 292, 441, Hill, F. A., 607
Friedman, H„ 438, 472 Goodman, I, J., 590 442, 443, 448, 460, 494 Hill, J. H„ 33, 86
Frieman, J., 498 Goodrich, K. P., 258, 278 Hardy, J. D., 154, 158 Hill, R. T„ 560
Fritsch, G„ 570 Gordon, M, S„ 162 Harlow, H** 159 Hill, S. Y,j,579
Frommer, G. P., 58, 60, 81, Gordon, T. P., 603 Harman, J., 406 Hilton, A., 61
120 Gormezano, L., 61 HarrG R., 276, 277 Hinde, J. S 91, 430
Fry, F. E. J., 157 Gott, C. T.? 213, 223, 302, 562 Harr el, N., 604, 605 Hinde, R. A., 3, 8, 90, 91, 133,
Fry, W. T., 143, 212, 226, 290, Gottlieb, S. H., 608 Harris, A. H., 596-618, 5, 597, 139, 142, 436
293, 295, 559 Gottwald, P,, 357 60S, 606, 600, 610 Mine, B,, 577
Fusco, M. M., 154 Gourevitch, G., 519, 523 Harris, J. D., 522, 523 Hineline, P. H., 13
Fuxe, K„ 585 Goyette, G. H., 498 Harris, L. S„ 553 Hineline, P. N., 364=414. 5, 13,
Grabowski, J., 395 Harris, R. E., 609, 610 260, 267, 374, 375, 380, 381,
Gaebelein, C. J., 611 Graeff, F. G., 120, 406, 408 Harris, R. T , 437, 556 400, 401, 402, 403, 405, 406,
Galambos, R., 597 Graham, L. A., 603 Harrison, H,, 154 409
Galbraith, K., 509 Granger, L., 603 Harrison, J. M.? 261, 282, 367 Hirsch, E., 28-51, 4, 32, 33, 34,
Gale, C. C., 153, 154, 166 Grant, D. A, 61 Harrison, R. H., 440, 441, 445, 35, 36, 38, 39, 40, 48, 193
Gallistel, C. R., 259, 279, 580 Grastydn, E,, 590 449, 450, 483, 487, 488, 489, Hi55, R, H„ 214, 434, 316
Galosy, R. A., 611 Gray, V. A., 482, 501, 502, 503, 491, 493, 494, 495, 528 Hitzig, E„ 570
Gamzu, E., 53-97, 3, 55, 56, 58, 505, 508 Harsh, J„ 397 Hochachka, r. W., 157
60, 61, 62, 63, 65, 67, 68, 69, Green, D. M., 533, 534, 537 Harvey, J. A., 202, 574 Hodge, H. G., 5Sg
80, 81, 85, 87, 126, 127, 128, Green, H. L., 32, 35 Hariem, P., 141, 142 Hoebei, B. C., 14, 32, 571, 570,
133, 185, m, 273, 273, 359 Green, K., F,, 113 Hawkes, 225, 250, 046 501, 583, 585
Gandelman, R., 574, 576 Greenberg, I., 130 Hawkins, T. D., 120, 131, 164, Ilodos, W., 141, 521, 572, 57S,
Gantt, W, FL, 596 Greene* Ik, 580 ]79, 181, 572, 573, 575, 570, 574, 597
Ganz, L,, 486 Green way, L,, 380 578 Ho£erf K. G.f 14
Garcia, E, 639 Greer, G L, 158 Headlee, C. P., 557 Hoffbrand, B. I., 603
Garcia, J.; 14, 91, 113, 484, 485 Gregor, G. L., 468 ff^apSj R. 2|11 Hoffman, ff, §„ IgO, 051,
Gardner, B. T., 640 Griffiths, R. R., 561, 563 Hearst, E., 54, 55, 56, 58, 60, 353, 357, 365, 407, 400, 426,
Gardner D, R„ 158 Gross, 8. 604 02; 03; 65; 68; 69; 81, 85, 90, 450, 482, 483, 489
Gardner, E, L., 590 Grossen, N. E., 117, 393 120, 127, 314, 347, 361, 376, Hoffmeister, F., 179, 192
Gardner, L. G., 379, 590 Grossman, 5. P., 89, 116, 162 436, 439, 444, 445, 446. 447, Hogan, J. A., 59, 60, 121, 159
Gardner, R. A., 640 Grove, R., 548, 549 449, 450, 451, 453, 454, 455, Hnkanson, J, E,, 690
Gardner, W. M., 58 Groves, L. C., 127 456, 466, 470, 471, 472, 482, Hokfelt, T., 585
Garrett, M. F., 632, 651 Guess, D., 639 487, 499, 503, 560, 597 Holbrook, J. W., 135
Garrick, L. D„ 157, 162 Guilkey, M., 216, 226, 302, 308 Heath, J. E., 157, 161 Holland, C. H** 245, 282
Gaudilliere, J. P., 44 Gunn, D. L., 158 Heath, R. G.; 589, 590 Holland, J. G., 236, 263
Geller, I., 558, 560 Gustavson, C. R., 408 Hebb, D. O., 571, 641, 642 Hollard, V., 248, 252, 277, 573,
Gentry, W. D., 136 Guthrie, E. R., 29 Hebert, D. L., 75, 448 590
Gerbrandt, L. A., 606 Guttman, N., 45, 84, 86, 249, Hefferline, R. F,, 606, 610 Hollis, J. H., 563, 564
German, D. C., 571, 585 259, 263, 279, 280, 434, 438, Heinemann, E. G., 489, 490, Holloway, S. M., 135
Gerry, J. E., 486 442, 443, 445, 456, 460, 461, 524, 536 Holly, B. L., 564
Gibbon, J„ 57, 58, 127, 128, 472, 489, 493, 527 Heise, G. A., 561 Holstein, S. B., 102
375, 399, 400, 401,410 Heistad, G. T„ 136, 561 Holz, W. C., 182, 194, 253, 254,
Gibson, D. A., 320, 322 Haberlandt, K., 495 Heller, H., 17 357, 384, 385, 386, 394, 429
Gibson, W. E„ 575, 576 Hackett, E. R., 168 Hellon, R. F„ 167 Honig, W. K., 1-6, 7, 53, 318,
Gilbert, R. M., 135, 144 Hackett, J. T., 605, 606 Hellstrom, B., 154 319, 435, 445, 447, 449, 456,
Gilbert, T. F., 131 Hagamen, W. D., 589, 590 Helson, H., 4 464, 486, 493, 496, 497, 498,
658 Author Index

Honig (Cont.) Johnson, D. G., 598 Kimmel, H. D., 606, 610 Leander, J. D., 306
499, 500, 503, 505, 507, 508, Johnson, D. M., 221, 222 King, J. E., 589, 590 Lee, J. K., 226, 302
525, 531, 572, 573, 630 Johnson, R., 470 Kinnard, W. J,, 356 Leeming, F. C., 262, 282
Honour, A. J., 167 Johnson, R. J., 607 Kintsch, W. A., 33 Lehrer, R., 33, 35
Horn, P. W., 597 Johnson, T. A., 600, 606 Kirby, A. J., 119, 120 Leifer, L., 606
Horowitz, Z. P., 585 Johnson-Laird, P., 651 Kirkpatrick, M. A., 564 Leitenberg, H., 1
Hosking, K. E., 604 Johnston, J. M., 294, 316 Kirkpatrick, W. E., 168, 169 Le Magnen, J., 32, 33, 35, 36,
Hothersall, D., 607, 610 Jolley, A., 603 Kissileff, H. R., 32, 35, 36, 40, 44
Hovland, C. I., 493 Jones, L. G., 598, 601 43, 131, 135 Lenzer, I. I., 574, 582
Howard, B., 36 Jones, B. E., 586, 588 Kitzes, L. A., 522 Leonard, C. M., 159
Howard J. L., 168, 586, 598, Jones, H„ 225, 226 Kleiber, M., 32 Leslie, J., 350, 355, 356
610, 611 Jouvet, M., 22, 586, 588 Klein, M., 407, 448, 450, 451, Levak, M., 14
Howard, R., 446, 467, 471, 472 Justesen, R., 590 453 Leveille, R., 137
Hubbard, R., 10 Jwaideh, A. R., 293, 294, 295, Kleinknecht, R. A., 137 Levin, R., 33, 44
Howarth, C. I., 575, 576, 581, 296, 297, 318, 320 Kling, J. W., 258, 259, 278, 575, Levine, M. D., 603
586, 588 578 Levine, S., 33, 44, 603
Huang, Y. H„ 582, 590 Kachanoff, R., 137 Klinger, J. M., 517 Levitsky, D., 32, 33, 34, 35, 38,
Huey, R. B., 164 Kadden, R. M., 376, 377, 380, Kluger, M. J., 166 40, 129, 134
Huff, R. G„ 461, 462 399, 410, 610 Knarr, F., 35 Levy, C., 14
Hughes, B. O., 35 Kahn, M., 610 Knott, J. K., 606 Lewin, L., 407
Hughes, J. E., 303, 305 Kakolewski, J. W., 15, 571 Knott, P. D., 576 Lewis, D., 61
Hughes, L. H., 261, 282, 297, Kalat, J., 8, 14, 91, 484, 485 Knutson, J. F., 137 Lewis, P., 407
304, 318, 320 Kalish, H. I., 434, 442, 445, 489, Koch, D. L., 262, 282 Libby, M. E., 400
Hull, C. L., 10, 29, 30, 31, 36, 527 Kodera, T., 465, 467, 473, 474 Liddell, H. S., 138, 421
75, 99, 101, 146, 217, 218, Kamin, L. J., 54, 342, 351, 353, Koelle, G. B., 165 Lieberman, D. A., 318, 319,
234, 315, 370, 371, 373, 374, 354, 357, 393 , 492, 507 Koelling, R. A., 14, 91, 484, 485 320, 321, 322, 326
377, 448, 453, 483, 491, 494, Kamiya, J., 604 Kohler, W., 29 Liebman, J. M., 586
495, 506 Kanarek, R., 28-51, 4, 33, 34, Komisaruk, B. R., 21 Liebson, I., 563
Humphreys, L. G., 61 35, 36, 38, 39, 40, 47, 193 Konorski, J., 138, 144, 483, 605 Lilie, N., 561
Hundt, A. G., 40, 102 Kanner, R. M., 606 Kop, D., 610 Lilly, J. C„ 589, 590
Hunt, G. L., 69 Kaplinsky, M., 583 Koresco, M., 436, 454 Lindsley, O., 563
Hunt, G. M., 23, 136, 561, 562, Karpicke, J., 90, 466, 472 Koster, E. D., 261, 282 Lindy, J., 577
563 Karpman, M., 103 Kowalski, M., 604 Linholm, E., 575
Hunt, H. F., 351, 353, 354, 357 Kasatkin, N. I., 20 Kraeling, D., 261, 263, 280, 281 Lipman, E. A., 421, 426
Hursh, S. R., 61, 324, 332, 336 Katkin, E. S„ 610 Krahenbuhl, G. S., 603 Lippa, A. S., 585
Hurwitz, H. M. B., 248, 358, Kaufman, A., 291 Krai, P. A., 14, 485 Lipscombe, H. S., 603
372, 376, 377, 380 Kausler, D. H., 638 Kramer, T. J., 85, 141, 255, Lipton, J. M., 153, 154, 162,
Huston, J. P., 582 Kazdin, A. E., 7 447, 466, 467, 468, 469 166, 167
Hutchison, J., 21, 129 Keehn, J. D., 127, 137, 368, Kramis, R. C., 589 Lisina, M. I., 606
Hutchinson, R. R., 415-431, 4, 381, 407,408, 409 Krantz, J. A., 448 Litner, J. S„ 390, 391, 392, 393
23, 136,183,409,417,418, 420, Keenan, B., 606, 610 Krasnegor, N. A., 366, 396 Lloyd, K. E., 243, 244
421, 422, 423, 426, 468, 561 Keesey, R. E., 258, 259, 263, Krausman, D., 606, 610 Logan, F. A., 258, 259, 260,
Hutt, P. J., 258, 278 278, 573, 575 Krecek, J., 17 262, 263, 278, 281, 495, 509
Hyden, H., 606 Kelleher, R. T., 174-200, 4, 28, Krecekova, J., 17 LoLordo, V, M., 70, 89, 90,
143, 178, 179, 183, 184, 185, Kreuter, C., 360 120, 272, 273, 275, 359, 392,
Iglauer, C., 250, 251 186, 187, 188, 189, 190, 191, Krost, K., 44 407, 408, 485, 487, 579
Ingram, D. L., 153 192, 193, 194, 195, 196, 197, Kruuk, H., 33, 47, 49 Lomax, P., 165, 168, 169
Innis, N. K., 127, 136, 138, 301 202, 212, 213, 222, 226, 229, Kubena, R. K., 555, 561 Long, D. H., 604
Inskeep, E. K., 162 288, 289, 290, 293, 294, 295, Kuenen, D. J., 20 Longstreth, L. E., 305, 309
Irwin, R. J., 267, 518, 537 297, 298, 299, 300, 301, 302, Kulli, 244, 276 Looney, T. A., 136
Isaac, D., 144 303, 304, 305, 306, 307, 309, Lorens, S. K., 574
lsom, K. E., 606 313, 314, 315, 318, 345, 356, LaBounty, C. E., 247, 254, 266 Lorenz, K., 11, 91, 138
lson, J. R., 86 358, 366, 367, 381, 384, 385, Lachman, R., 610 Loth, E., 598
Iverson, L., 540 406, 410, 428, 429, 540, 552, Lachter, G. D., 201 Lott, D. F., 289, 551
Iverson, S., 540 555, 556, 557, 558, 559, 598, LaForge, H., 603 Lotter, E. C., 129, 130, 131, 132
601, 603, 604, 609, 643 Lakoff, G., 622 Lotti, V. J., 167, 168, 169
Jacob, J., 167 Keller, A. D„ 154 Lambert, J. V., 381, 382, 405, Loughead, T. E., 224
Jacobs, G. H., 518 Keller, F. S., 219, 303, 314, 367, 406 Louis-Sylvestre, J., 44
Jacobs, H. L., 557 407 Lamoreaux, R. R., 369, 426 Louw, G. N., 162
Jacobson, F. H., 154 Keller, K., 83, 84, 86, 87, 88, Lander, D. G., 267, 435, 447 Loveland, D. H„ 61, 63, 245,
Jackson, D. C., 154 273, 274, 275, 359 Lang, P. J., 610 246, 247, 254, 257, 269, 272,
Jacquet, Y. F., 132 Kelley, M. J., 117 Lange, K. O., 367 274, 306
Jaffe, M. L., 74 Kellicut, M. H., 445 Lanzetta, J. T., 323 Lovejoy, E., 492, 507
Jaynes, J., 29 Kello, J. E., 301, 304 La Rue, C. G., 35 Lower, J. S., 604
Jeddi, E., 157, 158, 159 Kelly, D. D., 89, 252, 358, 359, Lashley, K. S., 222, 483, 485, Lubow, R. E., 487
Jenkins, H. M., 2, 7, 23, 54, 55, 598 487, 488, 492, 641, 642 Luce, R. D., 530
58, 59, 60, 61, 62, 63, 65, 68, Kemp, F. D., 158 Lat, J., 43 Luke, D., 551
69, 81, 84, 85, 119, 120, 127, Kemp, J. M., 588 Laties, V. G„ 142, 143, 154, 164, Lutzker, J. R., 639
140, 220, 221, 272, 318, 320, Kendall, S. B., 292, 296, 297, 166, 167, 189, 367, 542, 555, Lynch, G. S., 105
325, 326, 359, 361, 398, 405, 318, 320, 322, 324, 325 556, 560, 562 Lyon, D. O., 209, 210, 213, 217,
406, 439, 440, 441, 445, 449, Kenshalo, D. R., 135 Lattal, K. A., 75 341, 342, 345, 346, 350, 351,
450, 483, 487, 488, 489, 491, Keys, N. W., 576 Lawler, J. E., 603 355, 357
492, 493, 494, 495, 499, 509, Kiess, H. O., 15 Lawrence, C. E., 314, 318, 320 Lyons, J. E., 439, 441, 446, 447,
528, 646 Killam, K. F., 597 Lawrence, D. H., 38, 487 454, 494, 496
Jennings, J. R., 603 Killeen, P., 79, 134, 217, 246, Lazarus, R. S., 603
Jensen, C., 131 265, 272, 276, 277, 292, 298, Lea, S. E. G., 276, 342 Maatshe, J. L., 226
Johanson, C., 306 332, 333, 334, 336, 646 Leach, L. J., 562 MacCorquodale, K. B. K., 633,
John, E. R., 597 Kimble, G. A., 54, 67 Leaf, R. C., 370, 397 634, 635, 637, 638, 639, 640,
Johnson, D. F., 499, 500 Kimmel, E., 606 Leahy, J. P., 586 648, 649
Author Index 659

MacDonnell, M. F., 15, 16, 21 195, 222, 345, 380, 384, 404, Moreno, O. M., 578 O’Boyle, M. K„ 131, 132, 135
MacFarlane, W. V., 36 429, 558 Morgan, C. T., 641, 642 Obrist, P. A., 598, 603, 610, 611
Maclnnes, J. W., 606 McLaughlin, R., 420 Morgan, C. W., 577, 578, 583, O’Connell, M. E., 410
Mackintosh, N. J., 481-513, 5, McLean, J. H., 162, 249, 324, 584 Oden, D. L„ 130
54, 343, 441, 482, 488, 495, 460 Morgan, M. J., 207, 219, 342, O’Donahue, N. F., 589
497, 498, 500, 501, 502, 503, McLelland, J. P., 137 435 Odum, E. P., 46
504, 507 McLeod, A., 225 Morin, L. P., 164 Ogawa, N., 603
MacKinnon, J. R., 509 McMichael, J. S., 323 Morishima, M. S., 154 Ogilvie, D. M„ 159
Mach, E., 30 McMillan, D. E., 553, 558 Morrison, S. D., 47, 58, 65 Olds, J., 570, 571, 572, 574, 575,
MacPhail, E. M., 407, 590 McMillan, J. C., 89, 90, 322, Morse, C. W., 134 576, 577, 579, 581, 583, 584,
Magnus, R., 8 323, 324, 359 Morse, W. H„ 174-200, 4, 28, 585, 589, 590, 605
Magnuson, J. J., 157 McNeill, D., 633 176, 177, 178, 179, 182, 183, Olds, M. E., 572, 576, 585, 589,
Magoun, H, W., 154 McSweeney, F. K., 236, 240 184, 185, 180, 187, 188, 189, 590
Maier, S. F., 375 Meacham, J., 324 190, 191, 192, 193, 194, 195, O’Leary, K. D., 309
Malagodi, E. F., 294, 306, 316 Mead, R. M., 183, 4g8, 429 196, 197, 201, 202, 204, 205, Olson, K., 535
Malcuit, G., 603 Meehl, P. E., 100, 101 206, 207, 208, 209, 210, 211, Olson, L., 585
Mails, J. L., 578 Meisch, R., 553, 554, 561 gig, gi7, ggg, gg3, gge, gga, Olson, R. D., 378
Malmo, R. B., 597 Mellgren, R. L., 487 229, 265, 288, 305, 309, 345, Opton, E. M., 603
Malone, J. C., 75, 77 Mello, N. K., 528, 563 356, 366, 367, 381, 384, 385, OrlofT, E, R„ 428} 429
Malott, M. K., 434, 486, 525, Melmon, K. L„ 603 406, 410, 428, 429, 540, 542, Orme-Johnson, D. W., 357
526 Melt2er, D., 89, 359 543, 544, 546, 552, 556, 557, o§t, J, W,, 487
Malott, R. W., 224, 434, 525, Melvin, K, B„ 113, 116 558, 559, 598, 001, 003, 004, Overmann, S. R., 222. 226
526 Mendelson, J. H., 14, 138, 563, 609, 646 Overton, D. A., 553, 556
Malsbury, C., 372, 383, 590 383 Mgskgwitz, H„ 522
Mandell, C„ 536 Mentzer, T. L., 519 Mougey, E. H., 597 Pacliomov, N., 154
Mandler, G.f 610 Metzner, R. J., 578 Moulton, J. M., 6gg Padilla, s. G., 09
Mangan, G., 597 Meyer, D. R., 116 Mount, L. E., 159 Palmer, J. A., 456
Manocha, S. N., 553 Meyers, K. A., 603 Mowrer, O. H., 225, 226, 369, Pankseppi JM 32* 33s 35, 44,
Mansfield, R. J. W., 532 Meyers, W. J., 574 420 574, 575, 576
Marczynski, X, J,, 605, 606 Miczek, K. A„ 89, 356, 357, 358 Mrosovsky, N., 46 Paolini, R, M*, 585
Margules, D. L„ 571, 579, 581 Middaugh, 5., 611 Muehleisen, P„ 407 PappaB, B. A., 606, 60§, 609,
Mariner, A., 65 Migler, B., 347, 438 Muir, D., 435, 447 619
Mariner, R, W„ 249, 461, 462, Miles, C, G., 492, 495, 499 Mulvaney, D. E., 297, 318, 320, Pare, W., 604, 606
487 Milestone, R., 153 324, 326 Parrish, J,, 598
Marler, P., 144 Mill, J. L„ 5 Murgatroyd, D., 154 Paul, G., 564
Marmaroff, S., 103, 105 Millar, R. D., 346 Murray, C. S., 59, 65 Pavlov, I, P„ 9, 22, 29, 31, 53,
Marowitz, L, A., 144 Millenson, J., 13, 132, 135, 243, Murray, E. N„ 010 54, 55, 56, 58, 59, 60, 61, 62,
Marr, M. T., 201, 226, 295, 297, 244, 248, 249, 253, 348, 350, Myer, J. S., 408, 409 08, 70, 71, 75, 80, 81, 84, 85,
305 352, 354, 355, 356, 372, 376, Myers, J. L., 289, 308 88? 89f 90, 91f 118, 119, 120,
Marsh, D. F., 548 377, 438 Myers, R. D., 167, 169 121, 122, 127. 138, 144, 340,
Marsh, G., 449, 451, 453, 470 Miller, A. M , 590 Myhre, K„ 157, 158 341, 34g, HB, Hi, 347, 349,
Marsh, G. R,, 606 Miller, F. E., 604 351, 357, 358, 359, 360, 397,
Marsh, J. T„ 598 Miller, J. S„ 137 Nakamura, K., 168 400, 401, 447, 470, 483, 48$,
Marshall, J. F„ 16, 17, 586 Miller, J. T„ 457, 458 Nathan, M. A., 598 492, 493, 496, 596
Martin, J. M., 201 Miller, L. II., 606 Nathan, M, S,, 598 Fear, J. J., 74, 80, 409
Martin, J. R , 165 Miller, N. E., 14, 56, 57, 61, 99, Nauta, W. J. H., 597, 603 Peck, S., 373, 380
Martinez, E, S,, 252 112, 289, 308, 426, 571, 581, Navarick, D, J„ 01, 324, 320, Peden, B. F., 65
Marvin, S., 597, 606 582, 590, 603, 604, 607, 608, 327, 332, 336 Peele, T, L„ 589, 596
Marwine, A, G.# 33, 34, 35, 36, 609, 610,611 Navarro, A. P., 544 PeindaneS, R., 167
40, 42 Miller, R. E., 604, 606 Neffinger, G. G., 375, 398, 401 Peirce, G., 020
Marx, M H., 249, 260 Milligan. W. L., 523, 529 Neill, W. H„ 157 Pellegrino, L. J., 572
Marx, R. A., 153 Milner, P. M., 570, 57l, 572, Ncimcck, W„ 597 Penn, P. E., 168, 169
Mason, J. W., 589, 597, 598, 575, 576, 584, 586, 588 Nelson, J. A., 249, 460
Milsum, J. H., 160 Fennypackcr, 11 5:i 445
599, 600, 601, 604, 606 Nelson, K„ 137, 144 Peretz, B,? 390
Mason, W. A., 38 Mishkin, M., 521,525 Ncuringer, A. J., 185, 205, 207,
Miyata, ¥,, 004 Porei Cruet, J„ 597, 598, 000
Masserman, j, H,* 557 208, 210, 211, 252, 301. 304,
Massonet, B., 163 Moffatt, G. U.f 262, 282 308? 316, 333, 334 Pcrim G, T», 200, 2§l
Pd'kins, C. C., 82, 00, 133, 273
Mathews, M„ 153, 166 Mofflttj Mit 25L 256i 266, 275, Nevin, J. A., 75, 77, 78, 141,
Matsumiya, Y., 575 276 glS, g363 g4gj g45} g46j g48, Perlmutter, L. G., 07
Matthews, T. J., 153, 165 Mogenson, G. J., 570-595, 5, 255, 266, 267, 268, 260, 274, Persingsr, M; A., 409
571, 573, 576, 577, 578, 579, Persson, N., 580
Maxey, G, C„ 75 275, 270, 315, 525, 535, 537
Mayer, J., 32, 33, 35, 36, 153, 581, 582, 583, 584, 585, 586, Newman, B. L., 573 Peters, J. V., 000
158 587 Newman, F. L., 482, 490, 494 Peters, R. J.,
Maynard, E. A., 562 Mole, J. S„ 77 Newton, J, E. 0„ 598, 606 Peterson, C., 248

McClelland, D., 570 Moody, D. B., 526, 527, 532, Nichols, J. R., 557 Peterson, G. B., 56, 58, 60, 81,

McConahey, O. L., 565, 566 534 Nickerson, M., 165 90, 120
McCoy, F. B., 610 Moody, J. E., 409 Niemegeer, C. J., 586 Peterson, J. L., 136
McCracken, S. B., 57 Mooney, R. D,, 165 Ninteman, F. W„ 585 Peterson, N., 485, 486, 487
McCutcheon, E. P., 611 Moore, A. U., 487 Nissen, H. W., 641, 64g Peterson, R. H., 603
McDiarmid, C. G., 222 Moore, B. R., 9, 23, 81, 113, Noel, R. C., 496 Pfaffman, C.; 582, 5837 585,
McDonald, D. G., 597 118, 119, 120, 121, 122, 123, Norborg, J„ 300, 316, 317 586, 588
McFarland, D. J., 139, 140, 144 127, 359 Norton, P. R., 605 Phillips, A. G., 571, 577, 578,
McGeer, E. G., 586 Moore, J. W., 55, 58, 59, 61, 62, Notterman, J. M., 494 582, 583, 584, 585, 586, 587,
McGill, W. J., 141 67, 303, 334, 489, 493, 494 Nowicki, L. E„ 589, 590 588, 589
McGillis, D. B„ 116 Moore, K. E., 540 Nunes, D. L., 211 Phillips, M. L., 605
McGinty, D. J., 21, 156 Moore, R. A., 168 Nunnally, J. C., 449 Piaget, J., 636, 640
McGowan, B. K., 113 Moore, T. E., 633, 634 Nye, P. W., 520 Pickens, R., 543, 552, 554, 557,
McHugh, P. R., 589 Moot, S. A., 604 558, 563
McKearney, J. W., 185, 193, Moran, G., 132 Oatley, K„ 32, 36, 45 Pierce, C. H., 219, 260, 281
660 Author Index

Pierrel-Sorrentino, R. A., 434, 61, 80, 84, 85, 88, 120, 127, Russell, D. B., 492 130, 131, 132, 135, 137, 147,
515 128, 315, 341, 342, 343, 344, Russo, P., 102 182, 296, 620, 635, 636, 647
Pike, R., 537 350, 357, 358, 390, 391, 392, Rutherford, G., 639 Segal, M., 588
Pin, C., 586 393, 397, 398, 400, 409, 445, Rutstein, J., 154, 155 Seifter, J., 558, 560
Pisani, P., 640 446, 447, 448, 451, 452, 457, Selekman, W., 440, 455
Platt, J. R., 221, 222, 224, 226 470, 487, 507, 508 Sachar, E. J., 597 Seligman, M. E. P., 14, 84, 91,
Pliskoff, S. S., 79, 139, 164, 179, Reus, J. F., 606 Sadowsky, S., 524, 575 113, 375, 407, 436, 520
181, 222, 237, 238, 239, 240, Revusky, S. H., 153, 484, 507 Sailor, W., 639 Sepinwall, J., 74
241, 242, 243, 244, 246, 248, Reynierse, J. H., 131, 132 Sainsbury, R. S., 528 Seraganian, P., 435, 447
249, 250, 252, 572, 573, 575, Reynolds, G. S., 2, 72, 73, 74, Sakai, M., 575 Setzer, J., 441
576, 578 76, 77, 79, 84, 88, 140, 142, Sallery, R. D., 418 Sevenster, P., 16, 118, 120, 121,
Plumlee, L., 252, 359, 598, 606, 213, 214, 215, 216, 217, 218, Salzinger, K., 641 122, 123,137
609 223, 225, 247, 254, 257, 258, Sandlie, M. E., 605 Seward, J. P., 261, 263, 282,
Pointer, F. A., 606 259, 263, 265, 266, 267, 268, Sanger, D., 347, 349 575, 576
Poli, M., 87, 90 292, 472, 573 Santibanez, H. G., 605 Sewell, W. R., 209, 307
Polish, E., 597 Reynolds, M. D., 143 Saslow, C. A., 522 Seyffarth, H., 19
Polk, D. L., 167 Ricci, J. A., 57, 90, 127 Satinoff, E., 153-173, 4, 146, Shaffer, M. M., 178
Pomerleau, O. F., 358, 393, 606 Riccio, D. C., 367 154, 155, 167, 168 Shan, S. Y. Y., 154, 155
Popp, R. J., 372, 397 Rice, H. K., 597 Saunders, J., 516, 605 Shanab, M. E., 136, 139
Poppen, R., 436 Richards, R. W., 129, 137, 447, Sawrey, J. M., 604 Shannon, C. E., 322
Porter, J. H., 135, 138 461, 462, 466, 469 Sawrey, W. J., 597, 604 Shapiro, A. P., 597
Porter, P. B., 575, 589, 590, Richardson, K., 224 Scerni, A., 586 Shapiro, D., 606, 610
597 Richelle, M., 559 Schachter, S., 583 Sharp, J. C., 590
Porter, R, W., 598, 604, 610 Richter, C. P., 28, 29, 30, 31, Schadler, M., 440, 454 Shea, R. A., 261, 282
Poschel, B. P. H., 585 32, 33, 34, 35, 36, 44, 46 Schaeffer, R. W., 40, 102, 136 Shearn, D. W., 607
Potts, M. A., 162 Ricketts, P., 600, 601 Schalch, D. S., 603 Sheatz, G., 597
Pouthas, V., 143 Riddle, W. C., 185, 358, 427 Schaller, G. B., 33, 36, 47, 48 Sheffield, F. D., 63, 581
Powell, D. A., 529 Riesen, A. H., 486 Schaub, R. E., 318, 319, 320, Shepard, R. N., 530, 531
Powell, R. W., 209, 213, 217, Riess, D. A., 404, 406, 407 321, 342 Shepp, B. E., 487
375, 380, 404 Riley, A. L., 89, 90, 359, 409, Schipper, L. M., 61 Sheriff, W., 162
Powell, T. P. S., 588 410 Schlichting, U. U., 179, 192 Sherman, J. A., 307, 637, 639
Powers, P., 7 Riley, D. A., 447 Schiff, B. B., 571, 580, 581, Sherman, j. E., 462
Premack, D., 4, 28, 33, 40, 99, Rilling, M., 432-480, 5, 76, 80, 586, 588 Sherrington, C. S., 8, 9, 14, 25,
101, 102, 103, 104, 105, 106, 85, 129, 137, 141, 222, 255, Schlesinger, K., 606 29
107, 108, 109, 110, 111, 112, 407, 446, 447, 450, 451, 453, Schlosberg, H., 467, 493 Sherry, G., 487
113, 115, 122, 123, 180, 181, 455, 465, 466, 467, 468, 469, Schmaltz, L. W., 404 Shettleworth, S., 8, 14, 69, 75,
267, 268, 276, 288, 624, 631, 471, 472, 473, 474, 475, 482 Schmitt, D. R., 237, 276 77, 78, 91, 113, 114, 115, 118,
633, 640, 641, 650, 651 Rinaldi, P., 608 Schneider, B. A., 140, 207, 208, 123, 140, 268, 436, 483, 580
Pribram, K. H„ 517 Ringler, R., 606 210, 211, 213, 216, 249, 333, Shimp, C. P., 79, 224, 225, 245,
Price, T., 495 Rioch, D., 597 530, 531 251, 255, 256, 263, 264, 266,
Prokasy, W. F., 318, 340, 639 Ritter, S., 585 Schneider, J. W., 265, 298, 335 267, 269, 271, 272, 274, 275,
Proni, T. J., 417 Roberts, A. E., 358, 380 Schneiderman, N., 604 276, 398, 643
Proppe, D. W., 154 Roberts, E., 586, 588 Schnelle, J. F., 248 Shmavonian, B. M., 603
Purtle, R. B., 442 Roberts, W. W., 14, 15, 165, Schnitzer, S. B., 589 Shnidman, S. R., 378, 610
Pylyshyn, Z. W., 650, 651 582 Schnur, P., 507 Shull, R. L„ 79, 140, 141, 207,
Robinson, A. E., 5, 35, 36 Schoener, T. W., 33, 46, 47, 49 209, 216, 226, 237, 242, 243,
Rachlin, H., 13, 59, 75, 80, 86, Robinson, G. M., 619-627, 622, Schoenfeld, W. N., 48, 201, 203, 244, 246, 249, 302, 308
87, 90, 99, 127, 128, 133, 138, 627, 630, 633, 634, 635, 640, 208, 213, 222, 303, 314, 376, Shumaker, J., 639
141, 180, 235, 238, 239, 241, 647, 650 377, 398, 409, 610 Sibly, R., 144
246, 248, 253, 254, 264, 267, Robinson, J. A., 597, 600, 601 Schrier, A. M., 260, 263, 280, Sidley, N. A., 516, 519
272, 273, 274, 275, 276, 297, Robinson, J. E., 262, 282 487 Sidman, M., 2, 30, 58, 119, 165,
298, 328, 333, 386, 406 Robinson, W. W., 597, 598, Schroeder, S. R., 236, 242 185, 201, 234, 260, 280, 289,
Rackham, D., 59, 119 601, 602 Schrot, B., 79 342, 344, 349, 357, 368, 369,
Ramer, R. G., 470 Roby, T. B., 581 Schrot, S. H., 130 370, 371, 372, 373, 374, 375,
Ramsay, D. A., 606 Rocha e Silva, M. I., 219 Schulman, A., 578 376, 377, 378, 380, 381, 388,
Ramsden, M., 461, 462 Rodgers, W. L., 20 Schuster, C. R„ 540, 541, 545, 390, 393, 394, 395, 396, 400,
Randolph, J. J., 209, 307 Roeper, T., 632 553, 554, 556, 557, 563 401, 407, 426, 427, 438, 450,
Ranson, S. W., 154 Roll, D. L., 14 Schuster, R. H., 289, 299, 303, 504, 517, 540, 542, 551, 578,
Rashotte, M. E., 509 Rolls, E. T., 580, 583 305, 309, 317, 333, 334 579, 597, 598, 599, 600, 605
Raskin, D. C., 261, 282 Rose, R. M., 597, 603, 604 Schusterman, R., 519 Sidman, R. L., 517
Raslear, T. G., 515, 524 Rosen, A. H., 605 Schwam, E., 58, 65 Siegel, P. S., 36
Rasmussen, A. J., 598 Rosen, A, P., 456 Schwartz, B., 53-97, 3, 24, 63, Silberberg, A., 58, 65, 79, 81,
Ray, B. A., 438, 504, 517, 522, Rosenberger, P. B., 522, 523 64, 65, 67, 69, 75, 80, 81, 82, 87, 237, 240, 241, 242, 244,
525 Rosenblith, J. Z., 135 83, 85, 86, 87, 88, 90, 91, 126, 246
Ray, O. S., 356, 574, 577 Rosenblum, M. A., 606 127, 128, 133, 141, 143, 185, Silver, M. P., 260, 281, 606
Razran, G., 607 Rosenquist, H., 437 273, 275, 329, 359, 407, 408 Silverman, P. J., 304, 435, 464
Rebert, C. S., 606 Ross, G. S., 141, 597 Schwartz, G. E., 610 Simmelhag, V., 69, 81, 91, 113,
Rech, R. H., 540 Ross, L. E., 61 Schwartzbaum, J. S., 445 127, 129, 130, 133, 136, 137,
Rechtschaffen, A., 162 Rossi, R. R., 577 Schwing, R. C., 589, 590 139, 140, 185, 272, 273, 274,
Redford, M. E„ 82, 90, 133, 273 Routtenberg, A., 572, 577, 585, Scobie, S. R., 358 406, 464, 580
Regal, P. J., 161, 162 589 590 Scott, J. K., 562 Simmonds, M. A., 168
Rege, V., 490 Rozin, P., 8, 14, 18, 90, 153, Scott, T. R., 523, 529 Simpson, C. W., 139
Reichlin, S., 603 157, 158, 484, 485 Scruton, P. M., 347, 348, 350 Simson, L. R., 598
Reid, L. IE, 575, 589 Rozkowska, E., 20 Scucki, H., 610 Singh, B., 607
Reiss, S., 487 Rubin, H. B., 409 Scull, J., 73 Singh, S. D., 141, 142, 143, 553
Renfrew, T. W., 183, 418, 420, Rudolph, R. L., 486, 489, 490, Sears, G. W., 391, 392, 398, 400, Siskel, M., 45
422 491,492, 493,494 401, 407 Skinner, B. F., 1, 2, 5, 7, 8, 9,
Renner, K. E., 575 Rumbaugh, D. M., 640 See vers, M. H., 192 23, 24, 28, 29, 30, 31, 32, 33,
Rescorla, R. A., 54, 55, 56, 58, Rusiniak, K. W., 8 Segal, E. F., 628-653, 5, 129, 36, 38, 44, 45, 46, 48, 49, 53,
 Author Index
661

Skinner (Cont.) Stern, J. A., 597, 604 292, 294, 434, 437, 438, 439, VanVeen, W. J., 606
54, 63, 88, 91, 127, 128, 129, Stevens, S. S., 242, 262 440, 441, 453, 455, 456, 460, Vardaris, R. M., 606
134, 140, 143, 174, 175, 176, Stevenson, J. A. F., 571, 579, 461, 462, 482, 483, 487, 488, Vasselli, J. R., 129
177, 180, 182, 184, 185, 187, 581, 582, 583 489, 490, 493, 494, 496, 499, Vasta, R., 647
201, 204, 205, 206, 207, 208, Stevenson, J. G., 443 500, 501, 502, 503, 504, 505, Vaughan, H. G., 532
209, 210, 212, 213, 216, 217, Stevenson-Hinde, J., 3, 8, 133 507, 508, 509, 526, 527 Vaughn, L. K., 166
218, 219, 222, 223, 224, 225, Stiers, M., 58, 65, 81 Thomas, D. W., 32, 33, 35, 36, Verhave, T., 377, 388, 389, 390,
227, 234, 253, 290, 291, 293, Stinson, R. H., 159 61 392, 560
295, 296, 298, 309, 341, 345, Stitt, C., 353 Thomas, G., 305, 307, 308 Vickery, C., 201
347, 358, 393, 406, 409, 434, Stoddard, L. T., 438 Thomas, J. R., 295, 302, 306 Viemeister, N. F., 319
467, 469, 504, 572, 573, 580, Stokes, L. W., 393 Thomas, P. R., 441 Villareal, J., 137, 553
598, 619, 621, 622, 628, 629, Stolerman, I. P., 560 Thompson, D. M., 139, 355 Vogt, C. P., 290
632, 633, 634, 635, 636, 637, St. Omer, V. V., 564 Thompson, J., 633 Vom Saal, W., 499
638, 639, 640, 641, 642, 643, Stone, E. A., 604, 605, 608 Thompson, L, W„ 606 Von Glasersfeld, E., 640
646, 647, 648, 650, 651 Stoyak, M., 407 Thompson, R. L„ 58, 59, 375
Slangen, J. L., 138 Stoyva, J. M., 604, 610 Thompson, T, I„ 5, 15, 128, Waddell, T. R., 306
Slaughter, J„ 608 Stretch, R., 428, 429 136 Wade, G. N., 162
Sloan, M., 590 Strieker, E. M., 586 Thompson, TfJ 540-569, 395, Wade, M., 483, 485, 487, 488,
Slobin, D., 633 Stromme, S. B., 158 540, 541, 543, 545, 548, 549, 492
Smith, C. B., 188, 559, 560 Strong, C., 564, 565 551, 552, 553, 554, 557, 558, Wagner, A. R„ 61, 344, 397,
Smith, G. D., 379, 380, 404, 405 Stubbs, D, A., 218, 237, 239, 559, 561, 562, 563, 565 451, 452, 457, 487, 495, 496,
Smith, J. B., 90, 134, 143, 192, 240, 243, 244, 248, 249, 250, Thomson, D. M.? 638
359, 517 498, 499, 501, 502, 507, 508
252, 302, 303, 304, 305, 307, Thone, L., 559 Wagner, M. J., 128, 137
Smith, J. C., 14 308,316, 435, 464 Thorndike, E. L.f 53, 99, 100,
Smith, N. S., 580 Wald, G., 10, 22
Stuckey, H, L,, 30 101, 634 Walker, E. L., 99, 101
Smith, O. A., 598 Stunkard, A. J., 7 Thorne, P., 610
Smith, R. F., 319, 407, 408 Sturm, T., 59 Walker, S, F„ 248
Thorpe, 5, A., 517, 522 Wallace, R. F., 335
Smith, R. M., 598, 604 Stutz, R. M., 577 Tinbergen, N., 14, 15, 20, 22,
Smith, S. G„ 58 Sullivan, M. A., 324 Waller, M, B„ 185, 186, 358,
23, 142 556
Smith, W. M., 58 Sulzbacher, S. I., 564, 565 Timberlake, W., 29, 106, 107
Smith, W. S., 69 Sutherland, N. §., 367, 529 Waller, P, F;, 185, 358, 544.
Tobias, L. L„ 564 546, 556
Snapper, A. G., 376, 377, 398, Sutterer, J. R„ 86, 598, 610 Todarov, J. C., 79, 120, 244,
606, 610 Svensson, L., 139 Walsh, M„ 409
249, 253, 271, 272, 274, 382, Walter, W. C., 606
Snowden, G. T„ 32, 33 Svinicki, J. G., 482, 489, 498 406, 408
Snyder, E., 164, 577 Svinicki, M, D,, 482, 489 Walters, G. G., 113, 115, 116
Tolliver G., 139, 597, 606 Wampler, R. g.; 156
Solomon, R. L., 54, 61, 88, 341, Swadlow, H. A., 604 Tolson, W., 597, 598, 599
342, 358, 375, 467, 493 Ward, J. S., 509
Swets, J, A., 533, 534, 537 Tomie, A., 507, 508
Soltysik, A., 604 Swinnen, M. E. T.; 598, 606 Warner, H,, 040
Toshcff, J,, 602 Wasserman, E. A., 56, 57, 59,
Spanier, D., 131, 132 Switalski. R. W., 290, 291, 439, Tracy, W. H., 367
Sparber, S. B.f 542 441, 456, 494 60. 61, 65, 69, 120, 121, 160,
Tracy, W. K., 486, 487 273, 320
Spealman, R. D., 270, 271, 272 Symmes, D., 522 Trapold, M. A,, 201, 281, 282
Spear, N, E., 86, 457 Wauquier, A., 586
Szwejkowska, G., 483 Trattner, J , 103
Spector, O. J., 378, 379 Wayner, M, J„ 130, 133
Travis, R. P„ 589, 590 Weaver, W., 322
Spence, K. W., 61, 75, 218, 234, Tallen, R. R., 296 Trevitt, A. J., 240, 241, 242,
447, 448, 449, 450, 451, 453, Tallon, S., 32, 33, 44 Webb, R. a., 610
255, 266, 275, 276 Webbe? F. M.? 306
456, 462, 463, 476, 483, 493 Tapper, D. N., 527, 529 Trombley, J., 551
Spencer, P. S. J., 16? Tarpy, R, M„ 201,282 Webster, D. M„ 572, 589, 590
Trowiil, J. A., 574, 575, 576, Weijnenj J, A„ Iftj
Sperling, H. G, 516 Tatum, R., 604 578, 607, 610 Weii-Melherfee, II., 60S
Spies, G„ 511 Taub, E„ 222 Tucker, L., 606
Spragg, S. D. S., 557 Taus, S, E,? 314, 472 Weiner, H„ 140
TlilVing, E., 638 Wcri*krantz, L.J 3525 Sgl,
Spyker, J, M„ 542 Taylor, E. D., 597 Turner, B, N,, 16, 17
Squier, L. H., 58 Taylor, L, H,, 590 525
Turner, C., 501, 502, 503. 504, Wsisnaaii, ii;. 77. 05 102.
Squires, N. K„ 248, 298, 300, Teague, R. S., 154 507
316, 317, m, 332, 333, 334 Teccc, J, Jw 00(5 390, 391, 392, SOS, 456, 460,
Turner, M. E., 651
Squires, R. D., 154 TeghtSS&nian, R , 421 10U 402
Staddon, J, E, R., 125-152, Teitelbaum, P_, 7-27, 3, 8, 10
Tyrrell, £, s„ 597
Tversky, A., 277
Weiss, a. b., m? m
1-6, 68, 69, 73, 75, 77, 80, 81f 16, 17, 18f 20, 355 46, 136, Weiss, B., 142, 153, 154, 164,
Twltchcll, T. E., 19, 22 16^ 167? 189, 215, 225, 5os,
89, 91, 113. 126, 127, 128, 571, 579, 581, 583, 586, 597
129, 130, 131? 132, 1333 134; Temple, W,, 298 367, 542, 555, 556, 560, 562,
Ulrich, R. E., 394, 418, 421,
135, 137, 138, 139, 140, 141, Templer, D. I., 579 4^ 4s>$
142, 143, 147, 185, 192, 204, Templeton, J. R., 150 Wfiiss, J. D., 597
Ungerstedt, U., 585, 586
217, 222, 224, 235, 255, 256, Ten Eyck, R. L., 298, 329 Weiss, J. M., 604, 605, 608
Uphouse, L. L., 606 Weiss, S. J., 74, 4m, 449, 444,
205, 206, 272, 273, 274, 275, Terhune, J. G., 103, 112 Uretsky, N. J.; 168
276, 277? 289, 298, BOO, 301, Texman, J, S„ 510, 518, 535, 453, 459, 460, 461, 462, 463
Ursin, H., 589
304, 305, 359, 406, 458, 464, 536, 537 Weisz, J. D., 597
Ursin, R., 589 N, W.,
580, 646 Terman, M., 516, 521, 524, 575 Weissman,
Uyeda, A. A„ 261, 282, 575, Wells, J. A., 551
Stamm, J. S., 517 Terrace, H. S., 57, 61, 73, 75, 576
Stark, P., 589 76, 77, 84, 85, 127, 186, 268, Welty, R„ 606
Starr, B. C., 301, 304 270, 271, 275, ,441, 446, 447, Wenger, M. A., 607
Yalenstein, E. S., 15, 571, 572, Wenzel, B. M., 597
Stebbins, W. C., 519, 522, 525, 453, 454, 456, 460, 461, 462, 573, 574, 577
526, 527, 532, 598 465, 466, 467, 468, 469, 470, Werdegar, D., 598, 606
Vanderwart, M. L., 604 Werner, J. S., 138
Stein, L., 129, 156, 342, 344, 471, 472, 473, 474, 481, 483, Vanderwolf, C. H., 118, 604
345, 349, 350, 353, 571, 574, 486, 489, 507, 553 Wertheim, G. A., 383, 562
Van Dyne, G. C., 89 Wesemann, A. F., 116
576, 580, 585, 586 Tersky, B., 606, 610 Vanegas, H., 582
Steinbaum, E. A., 14 Teyler, T. J., 606 Wessells, M. G., 23, 61, 65, 464,
Van Houten, R., 490, 491, 492, 465, 470
Steiner, J., 324 Thach, J. S., 367, 597 493, 494 Westbrook, R. F., 73, 82, 83,
Steiner, S. S., 178 Theios, J., 365 Van Ost, S. L., 459 86, 90, 275, 495
Stellar, E., 33 Thoenen, H., 168 Van Sommers, P., 160, 532 Westoby, M., 36, 47, 48
Stephens, J. H., 603 Thomas, D. R., 249, 290, 291, Van Toller, C., 350, 352 Wetzel, M. C., 572, 589, 590
662 Author Index

Wheatley, W. L., 79, 269, 270, Willis, F. N., 45 Wright, A. A., 518, 530, 532, Young, G. A., 117, 119, 120,
271, 272, 274, 460, 461, 462 Willis, R. D., 252 536 183, 418, 420, 522
Wherry, F., 600, 601 Wilson, A., 388 Wright, B., 40 Young, J., 153, 166
Whishaw, I. Q., 168 Wilson, D. M., 142 Wright, B. A., 164 Young, P. T., 32, 582
White, A. J., 255, 275, 276 Wilson, M., 517 Wright, J. E., 164, 576, 578 Young, W., 21
Whitehurst, G. J., 639, 640, 647 Wilson, R. G., 21 Wurtman, R. J., 168 Younger, M. S., 393
White, K. G., 437 Wilton, R. N., 78, 127, 319, Wiittke, W., 136, 179, 192, 356, Yum, K. S., 557
Wiener, N., 322 322, 323, 324, 325 558 Yunger, L. M., 574
Wiepkema, P. R., 32, 35 Winokur, S., 56, 127 Wyckoff, L. B., 30, 313, 314,
Wike, E. L., 289, 305 Winton, A. S. W., 435, 464 318,322 Zeaman, D., 258, 278
Wilcoxon, H. C., 14, 99, 485 Wise, C. D., 571, 585, 586 Wynne-Edwards, V. C., 33, 34 Zeigler, H. P., 32, 35
Wilkie, D. M., 74, 86, 458, 459, Witoslawski, J. J., 292 Wyrwicka, W., 14 Zeiler, M. D., 201-232, 2, 125,
461, 462, 470 Witty, W., 216, 226, 302, 308 140, 176, 185, 193, 204, 207,
Wilkinson, H. A., 589, 590 Xhenseval, B., 559 214, 216, 224, 227, 301, 305
Wolin, B. R., 67
Williams, B. A., 496, 497 Zener, K., 61, 138
Wong, I. G., 589, 590
Williams, D. R., 2, 3, 24, 55, Yager, D., 517, 535 Zentall, T., 482
56, 62, 63, 64, 65, 66, 67, 68, Wood, K. A., 252, 264
Yaksh, T. L., 167, 169 Zeilke, S., 138
70, 75, 78, 80, 87, 88, 119, Wood-Gush, D. G. M., 35 Yamamoto, W. S., 31 Zigmond, M. J., 586
127, 128, 141, 143, 268, 272, Woodruff, G., 58, 60, 62 Yanagita, T., 192 Zimmerman, D. W., 175, 299,
597, 598 Woods, J. H., 250, 251, 263, Yarczower, M., 357, 447, 455, 304
Williams, H., 2, 3, 62, 63, 64, 553, 554 456, 457, 460, 461, 462 Zimmerman, J., 219, 260, 281,
119, 128, 272 Woods, P. J., 262, 282 Yarensky, P., 536 306, 307, 308, 316, 317
Williams, J. L., 598 Woods, S. C., 129 Yeh, S. D. J., 154 Zis, A. P., 586
Williams, R. J., 240, 241, 242, Wool, M., 600, 601 Yehuda, S., 168 Zolman, J. F., 153, 160
255 Word, T. J., 604 York, J. L., 606 Zuriff, G. E., 143
 Subject Additive difference model, and matching,
277
Additive summation, and
generalization, 443-444, 459
stimulus
Amount of training, and stimulus control,
455-456, 483
Amphetamine (see also d-amphetamine):
classification, 544-545

Index Additivity theory of contrast, 80-91

Adipsia, hypothalamic, 17, 20, 156
Adjunctive behavior (see also Collateral
behavior, Interim activities)
and clock schedule, 555
and conditioned suppression, 355-356
and fixed-interval behavior, 559-560
and fixed-ratio behavior, 562
as displacement activities, 23 and locomotor activity, 554
Abrupt procedure, and errorless learning, eating in doves, 139 and punishment, 558, 560
466 Adjusting schedule: as reinforcer, 557
Absolute generalization gradient: avoidance, 394 and schedule control, 188-191, 55g, 557
description, 435 fixed-ratio, 205, 212~213 self-administration, 543
and stimulus control, 501-502 Adrenal hormones, and avoidance, 599- and self-stimulation of brain, 571, 585
Absolute rate o£ responding: 601 and stimulation-induced drinking, 581-
and conditioned suppression, 345, 348- Adrenergic blocking agent, and thermo¬ 582
351,354 regulation, 165 and thermoregulation, 165, 556
equations for, 257, 264-267 Adventitious punishment: tolerance for, 550
and law of effect, 257-263 and conditioned suppression, 351, 357— Amphibians, thermoregulation, 153, 156~
and magnitude of food reinforcement, 358, 360 158, 161-164
258-259 and history of animal, 184—185 Amygdala:
and power function law, 277 Adventitious reinforcement: activation by stimulation, 583
Absorption rate, drugs, 543, 547-548 in automamtenance, 63—64, 66, 68 EEG activity, 665
Abstract propositional knowledge, 650- in autoshaping, 119 self-stimulation, 590
651 bursting on shock schedule, 388 Amyl acetate, odor stimulus, 517
Abstraction ability, and animal psycho¬ on concurrent schedules, 234-235, 243 Amytal (see Amobarbital)
physics, 525 and history of animal, 184—185 Anaclitic operant, 67
Abundance, food, and population, 34 history of concept, 127-128 Anaesthetics:
Abuse, of drugs, 192-193, 542, 557-558, of induced attack, 136 excretion of, 550
563 and induced drinking, 130, 134 and thermoregulation, 165
Acceleration of pecking, and postrein- Interresponse times, 225 Analgesic drugs (see also Heroin, Mor¬
forcement pause, 142-143 and nonchaining delay, 218 phine, Narcotics, Opiates) ;
Acceptability, of language utterances, 624 and positive conditioned suppression, classification, 545
Access, to negative reinforcement, 376-377 359-360 as reinforcers, 557
Acetylcholine; problems with, 127-128, 132 Analgesic effect, electrical brain stimula¬
and stimulus control, 555-556 and running, 147 tion, 574
and thermoregulation, 168-169 and superstitious behavior, 127, 204 Androgen, and central reinforcement) 579
ACh (see Acetylcholine) and terminal response, 127-128 Angerthas, 101
Acquired distinctiveness of cues, 487-488 Aftereffect, motion: Animal psychophysics, 515“537
Acquisition; in monkeys, 523 Animal psychophysics:
autoshaping, 54—58, 62 Afterimage, colored; methodological problems, 523-525
avoidance, and drugs, 561 in animal psychophysics, 526 perception,
with central reinforcement, 575, 589 Aggressive behavior (see also Attack, scaling in, 530-532
on chained schedules, 290-291, 299 Biting attack, Schedule-induced be¬ signal detection theory, 532-537
and conditioned reinforcement, 305 havior): stimulus presentation methods, 520-525
control by incidental stimulus, 505 and aversive control, 416-430 threshold determination, 515-525
dimensional control, 454-456, 475 by-product of discrimination, 467-469 Animate motion, 29
drug effects on, 560-562 and drugs, 552, 562 Annoyer, m Law of Effect, 99, 101
errorless discrimination, 464-480 and errorless learning, 468-469 Anolis cristellus (see Lizard)
fixed-ratio performance, 227 extinction-induced, 468—469 Anorexia (see also Aphagia):
key pecking, 55-57 pigeons, 59, 120, 136-137, 139 and amphetamines, 190, 350
language, 619-627, 634 schedule-induced, 129, 134-137, 139 Antagonist, to drugs, 179, 553, 554? 557,
lexicon, 620 Siamese fighting fish, 59? 65 ? 116 563
with negative reinforcement, 374-375, suppression by punishment, 115-116 Antecedent variables, in psych9pharaia-
404-409 Air flow, stimulus control by, 491 cology, 553-554 5
overshadowing during, 492-493 Air licking, ratsj 138 Aniianxiety drugs, 545 (see also Tran,
speed, compared to extinction, 205 Alcohol (see also Ethanol); quilners)
stimulus control, 483-484 ingestion Antldwliticrgic drugs, 555
syntax, 619-627 Rhesus monkey, 597 Anticipation, of food needs, 33j 46
Activity, as das? of motor patterns, 144 reinforcer Antidepressant drugs, clamfication, Ml
Activity wheel (see Running wheel) 3i?ib 501 Antipsyckoth drugs3 cla§§|flsay8nj gyg
Acuity, visual; and stimulus control, 555-556 Anxiety:
pigeon, 517-518 and coa4j£jonc[1 gupprcSSlOR 353, 355-
Alcoholism, human, 563
sea lion, 519-520 357, 380, §9§
Alert posturing, gerbil, 115—110
Adaptation, and rate of eatings £71? and drugs, 355-357, 559, 562
Algorithm, in natural language, 6^2
Adaptation-level theory, 4 4s hypothetical drive StatE, 186
Alleyway (see Runway)
Adaptive specialization, and stimulus con¬ Anxiolytic agents, 360
trol, 484-485 Alligator, thermoregulation, 153
Allocation of resources, in feeding pat¬ Aortic flow probe, 003
Added clock, 296-297 (see also Clock Aphagia, hypothalamic, 17, 20, 156
terns, 47
schedule) Aplysia, 9
All-or-none law, 11
Added counter, on chained schedule, 295 Apomorphine, self-administration, 586
Added cues: Alpha-methyl-p-tyrosine, and self-stim¬ Apparatus:
correlated with shock, 365, 390-392 ulation of brain, 585
animal psychophysics, 524
and dimensional control, 391-392, 398- Alpha waves, 606 aversive control, 416-418
400 Altered physiological states, 596-611 central reinforcement, 572-574
in escape procedure, 383-387 Alternation, on concurrent schedules, 234 Appetitive behavior:
and negative reinforcement, 365, 381- 244, 276 and biological constraints, 114-115
398 Alternative response assumption, 99 vs. consumatory behavior, 22
and Pavlovian conditioning, 393, 395 Amitryptaline, 545 as operant, 22
and shock averaging, 397-398 Amobarbital: in sleep, 22
shock-delay procedures, 392-398 classification, 544-545 Appetitive motivation, and conditioned
and two-factor theory, 396-398 and punishment, 558 suppression, 358

663
664 Subject Index

Approach-avoidance conflict: formal model, 507 rhesus monkeys, 58, 60

and gastric ulcers, 597, 604 selective, 507-510 secondary reinforcement in, 56-57
and schedule induction, 139-140 and stimulus control, 481-510 sexual behavior, 59, 119
Approach behavior, pigeons, in auto¬ Attentiveness, and discrimination train¬ significance, 3, 7, 13, 53-71, 91
shaping, 65, 69 ing, 500-502, 505, 507-510 sign tracking in, 69, 81
Arbitrary response: Auditory cortex, EEG activity, 605 * species differences, 58, 60, 70, 91
in autoshaping, 61 Auditory intensity: stimulus substitution in, 62
in chain of reflexes, 29-30 difference limen, 524 and temporal control, 140
interchangeability of, 23 discrimination, theories, 68-70, 119
and negative reinforcement, 402-409 cat, 516 tone stimulus, 69, 273
operant as, 3, 23, 29 chinchilla, 515-516 topography of response, 58-60, 67, 81,
problems with concept, 15, 23, 29, 91, monkey, 534-535 119, 121-122
180, 408 rat, 516 trial duration, 57-58, 78, 90
prototype of behavior, 11, 54-55 Auditory learning, language, 643-647 types of US, 58
Artie sculpin, thermoregulation, 158 Auditory sensitivity: water reinforced, 58-60, 62, 81, 119
Area shift, generalization gradient, 442, bats, 519, 523 Availability:
453 mouse, 517 food, and eating patterns, 33, 36-43, 47-
Arithmetic VI schedule, 215 rat, 518 48
Arousal: Auditory thresholds, monkey, 516, 519, water, and drinking patterns, 36, 40
and blood pressure, 603 521 Availability, Law of (see Law of avail¬
and brain damage, 586 Australian lizard (see Lizard) ability)
and conditioned suppression, 352 Autoclitic: Averaging:
and electrical brain stimulation, 583 as discriminative stimulus, 621 generalization data, 437-439
and motivation, 584-586 verbal behavior, 635, 640, 642, 647-649 shock, and added cues, 387, 397-398
theoretical concept, 99 Autoclitic frames, 635 Aversive conditioning (see Avoidance,
and thermoregulation, 155-156 Automaintenance (see also Negative auto¬ Escape, Negative reinforcement,
Arsenicals, excretion of, 550 maintenance): Electric shock)
Artificial mother, and thermoregulation, conditions for, 54 Aversive contingencies, and matching,
159 and contrast, 80 243-244, 253-254
Ascending order of stimuli, in animal heat reinforced, 65 Aversive control, 415-431 (see also Avoid¬
psychophysics, 521 negative (see Negative automaintenance) ance, Electric shock, Negative rein¬
Associability (see also Conditionability, Pavlovian-operant interaction, 62-63, forcement)
Preparedness): 66-71, 119 Aversive control:
response-reinforcer, 483-485 pigeons, 62-68, 119 and brain biochemistry, 606
stimulus-response, 520 rats, 65 by-products, 415-430
Association: response-reinforcer relations, 64-69, 91 and generalization gradients, 450, 455-
CS and US in autoshaping, 55, 70 Siamese fighting fish, 65 456
linguistic, 619 sign tracking in, 69 human, 417- 420
Associationistic model, language structure, squirrel monkey, 65-66 methods, 416-418
5, 628-633, 647-649 stimulus-reinforcer relations, 63-69, 91 operant baseline, 5, 415-430
Associationists, 100 water reinforced, 65 patterns of responding, 418-430
Associative competition, theory of, 507 Automatons, animals as, 29-30 and peptic ulcers, 597, 604-605
Associative learning, and verbal behavior, Autonomic responses, operant condition¬ Aversiveness:
634 ing, 606-611 assessment, 415, 422
Associative shifting, and verbal behavior, Autoshaping, 53-91 in behavioral contrast, 76-77, 84-86
634, 640 Autoshaping: and biological constraints, 115
Asymmetry: acquisition, 54-58 errors in programmed learning, 467,
in multiple schedules, 268 aggressive behavior, 59, 65 469-470
reinforcement and extinction, 205 chicks, 59-60, 65, 70, 120 forced running, 103
Atropine: and classical conditioning, 2-3, 54—55, heat, 165
and avoidance, 542 58, 60-61, 68, 80, 91 interim period, 139-140
classification, 545 contingency vs. pairing, 54-56 and relativity of reward, 102, 111-112
and fixed-interval behavior, 544 and contrast, 73-91, 272-274 stimuli with negative outcomes, 314,
and learning, 561 control procedures, 55-56, 61 319, 325
and stimulus control, 555 directedness of responses, 61 taste qualities, 527
and thermoregulation, 169 dog, 58 thermal stimuli, in human, 162-164
Attack (see also Aggressive behavior. electrical brain stimulation in, 60 US, and conditioned suppression, 341
Biting attack, Schedule-induced be¬ with electric shock, 70 Aversive situation, as drive operation, 368,
havior) : and errorless learning, 464-465, 470-471 378-380
and aversive control, 416-430 fish, 58 Avoidance (see also Aversive control.
brain-stimulation-induced, 14 heat reinforced, 59-60 Electric shock, Negative reinforce¬
in cat, 15 informativeness of stimuli in, 56-57, 80, ment, Shock delay. Shock deletion)
drug effects, 552, 562 84 Avoidance (of shock, by rats, unless other¬
electrically evoked, 14-15 necessary and sufficient conditions, 54- wise specified), 367-410
extinction-induced, 468-469 56, 61, 67 Avoidance:
hypothalamic lesions and, 17 and operant conditioning, 13, 23, 54-55, abortable sequence schedule, 366
as interim activity, 135-139 62, 70-71, 91 and alcohol ingestion, 599
in pigeon, 136-137, 139 partial reinforcement in, 61 and aversive control, 425-427
in rat, 17, 136, 139 as Pavlovian conditioning, 55, 60-63, and blood pressure conditioning, 608
in rhesus monkey, 380 68-71, 80-81, 91, 119. 121-122, 127, by-product of escape, 364-365
schedule-induced, 135-139 138 cardiovascular changes in, 599-604
shock-elicited, 183-184, 380, 409, 417— peck duration in, 24, 67-68, 81 cat, 193, 605
425 pigeons, 3, 7, 13, 24, 54-62, 68-70, 81, and conditioned reinforcement, 319
in squirrel monkey, 136, 139, 183-184, 119 and conditioned suppression, 358, 360
380, 409, 417-425, 428 and positive conditioned suppression, discriminated
Attention: 359 and animal psychophysics, 516
in animal psychophysics, 517, 523-524, quail, 58-59 and EEG activity, 605-606
528, 536 rat, 58, 60, 65, 81 and endocrine changes, 599
concept, 5, 30 redundant stimuli, 56-57 drug effects, 555-557
Subject Index 665

EEG activity in, 605-606 and autoshaping, 73-91, 272-274 havior, Attack, Schedule-induced
endocrine changes in, 599-601, 604 by-product of discrimination, 467 behavior):
free-operant, 5, 119, 185, 369-371, 380, definition, 73 apparatus, 416-418
597-606 (see also Avoidance, Sid- determinants, 267, 274 intensity, 418
man; Shock delay, Shock deletion) equations, 267, 270 mouse, 416-417
functional relevance and, 113-119 and errorless learning, 472-474 patterns, 418-430
gastrointestinal changes in, 597, 604- and generalization, 442, 449, 454 punishment schedule, 428
605 Herrnstein’s account, 266-268, 270-275 schedule-induced, 136
and generalization, 450, 455-456 and induced aggression, 469 shock-elicited, 183-184, 380
heat, by rats, 156 inhibition and, 75-76, 84-85 and shock reduction, 420, 426
and infectious disease, 597-598 and matching, 267, 272—275 squirrel monkey, 136, 183-184, 4l7~425,
long-term effects, 380, 427, 429, 599- necessary and sufficient conditions, 73, 428
605 76, 78, 83-85 Blackout (see also Timeout):
multiple-variable-interval, 267~27l with negative reinforcement, 382 as brief stimulus, second-order sched¬
operant-Pavlovian interactions, 88 and Pavloviati conditioning, 84—85, 88— ules, 301, 304
and peptic ulcers, 597, 604 89 delay of reinforcement, and matching,
pigeon, 13, 120, 450-451 and peak shift, 442, 454 251-252
problems with concept, 364, 369-370 quantitative account, 267, 270 fixed-interval schedule, 207-208
rhesus monkeys, 185, 366, 597, 599-601, and reinforcement frequency, 76-78, 84, fixed-ratio schedule, 210—Q12
604 86 iwnrial interval, and stimulus control,
and response strength, 200 response-independent schedules, 132- 489-490
as shock-frequency reduction, 387 133, 373 multiple schedule, and matchjngj
squirrel monkeys, 185, 193-190 and response suppression, 76-77, 84 Blocking, and stimulus control, 499-500,
temporal pattern of behavior, 425-427 as schedule interaction, 88 507
theories, 364-365, 393, 396-398 temporal properties, 75-78, 86 Blood flow, instrumental conditioning,
thermal change, 165 theories, 75-91, 266-275 571, 608
variable-interval, g60, 267~27l Behavioral final common path, 144 Blood pressure:
Avoidance, Sidman, 119, 185, 369-371, 380, Behavioral homeostasis, and temperature- and conditioned suppression, 35fj, 598
426 regulation, 153 conditioning, 571
and physiological changes, 597-606 Behavioral ph armacplggy, 5, 153~155, lQi~ and escape-avoidance, 601—604
and thermoregulation, 165
169, 188-193, 349-360, 540-569 (see instrumental conditioning, 608-610
also Drugs, Psychgpharmagglogy) measurement and recording, 598, 606
Baboon;
basic variables, 543-551 Boa constrictor, thermoregulation, 162
blood pressure conditioning, 609-610 Body weight;
behavioral mechanisms, 551 “500, 562
escape-avoidance, 601-602
principles of drug action, 543-551 and changeover responses, 244
thermoregulation, 153, 166 and escape-avoidance, 605
Behavioral state, in sequence Of activities,
Background stimuli (see also Contextual
,144 and matching, 269
stimulus): and polydipsia, 13^
Behavioral stream, and negative reinforce¬
and stimulus control, 457-458 ment, 409, 410 “Botanizing,” 2, 12, 30, 91
Backward conditioning, 55, 61
Behavioral toxicology, 562-563 Bout duration, induced activities, 141-
Baconian induction, 2, 12, 28 Behavioral variation, 127 142, 146
Bad news, and conditioned reinforcement,
Behaviorism: Bowing response, pigeon, in automain-
319, 322 aild language, 519-527, 633-640 tenanee, 65
Barbital, 544
Skinner’s radical, 1-2 Boyle's law, 276
Barbiturates: Bcnzadrine (see d-amphetamine) Brachium corqunctivum., self-stimulation
chemistry, 544
Benzodiazepine, and conditioned sup¬ Of, 590
classification, 544-545 Brain chemistry, 800
pression, 356
and conditioned suppression, 355-356 Beta adrenergic blockade, and blood pres¬ Brain damage:
and punishment, 558
sure, 603 dogs, 20
reinforcer, 557 Bella splcndens (see Siamese fighting fish) humans, 20
and schedule performances, 188, 191
Bias: hypothalamus, and thermoregulation,
stimulus control by, 555 154-156
tolerance, 550
animal psychophysics, and signal de¬
tection theory, 532-533 monkey, and spiral aftereffect, 520
Barking, in sea lion, 519-520 physiological stydj,
concurrent schedules, 238-239, 247-248,
Bar pressing (see Leyer pressing) 251-256 preoptic area, and thermoregulation,
Basal ganglia, and eleetrleal stimulation,
drug selection, 544 134
585 rats, 16-18
generalization gradient in extinction,
Base phrase marker, language theory,
438 and self-stimulation, 582, 586-587
629-630 stages m recovery from, 8, 17, 2®
Baseline; response, in animal psychophysics, 516,
518, 521, 5247534 Brain stem circuits, 588
stability, and drug effects, 7, 543, 551-
Biconditional behavior, 70 Brain temperature, and thermoregulation*
552 154, 158
Binary fractionation model, language
stimulus control of, and drugs, 555 Break and run, on FI, 140
structure, 643-647
Basic strings, in language, 629-630 Break point, on El, 265
Biochemistry, of brain, 606
Basking, in thermoregulation, 162, 164 Brief stimulus;
Biofeedback, 571, 580
Bat, auditory sensitivity, 519, 523 and chained schedule, 296-297, 299
Batesian mimicry, 484 Biological control systems, 160
on concurrent chains, 334
Beak movement, as interim activity, 128 Biological predispositions, 113, 122
and conditioned reinforcement, 316-318,
Behavior, modification: Biological relevance, 127
334
and brain stimulation, 580 Biology of association, 70
discriminative effects, 300—309
and drugs, 565 Biotransformation of drugs, 550
in extinction, 305-306
Behavioral arrest, and aversive control, Birds (see also Chick, Pigeon, Quail);
hopper presentation as, 316-317
425 thermoregulation, 159-160
physical properties, 304-305
Behavioral clock, and induced activities, Birth rate, and food availability, 34
and second-order schedule, 299-309,
140-148 Bisection, animal psychophysics, 532 315-316
Behavioral contrast (see also Local con¬ Biting, rats: Brightness:
trast, Negative behavioral contrast, in automaintenance, 65 discrimination,
Positive behavioral contrast. Tran¬ in contrast experiments, 82 pigeons, 521, 532
sient contrast): Biting attack (see also Aggressive be¬ rats, 526-527
666 Subject Index

Brightness (cont.) generalization gradient, 452, 482, 502 Changeover response:

stimulus control by, pigeon, 490 Central grey matter, self-stimulation, 590 punishment, 243-244, 253
stimulus generalization, rat, 493 Central limit theorem, and behavioral rate, 233, 243-244, 249-250, 254, 276
Bristol board, 98, 114 chaining, 142 Chemical structure, drugs, 544
Brook trout, thermoregulation, 158 Central motive state: Chewing, schedule-induced, 128-133
Burst duration, in contingent-response behavioral state as, 144 Chick:
experiments, 110-112 and electrical brain stimulation, 582- autoshaping, 59-60, 65, 70, 120-121
Bursting: 589 nuzzling response, 65
on concurrent VI schedules, 238, 246 Central nervous system, behaviorally in¬ omission training, 65, 120-121
on interval vs. ratio schedules, 223 duced changes, 605-606 thermoregulation, 153, 159-160
in postshock period, 388, 394 Central reinforcement, 570-590 (see also wavelength generalization, 486
Button pressing, human, and matching, Electrical brain stimulation) Chicken:
234-235 acquisition with, 575 meal patterning, 35
Buzzer, stimulus control by, 484 advantages, 580 misbehavior, 13
By-products: vs. conventional reinforcement, 574-581, Chick-peas, reinforcer for pigeons, 259
aversive control, 415-430 589 Chimpanzee:
discrimination learning, 467, 475 extinction after, 575-576 language behavior, 640, 650-651
measurement, 572-574 schedule-induced polydipsia, 129
CA (see Catecholamine) methodology, 572-574 second-order schedule, 299, 302
Caffeine, drug classification, 545 and motivation, 570, 576, 579-580, 582- token reinforcement, 306
Caloric density, 33, 35, 43 589 Chinchilla, auditory intensity discrimi¬
Caloric regulation, 43, 47 persistence of behavior, 576-578 nation, 515-516, 519
Cannabis (see Marijuana, Tetrahydro¬ and secondary reinforcement, 576 Children, relativity of reward, 102
cannabinol) species and sites used, 589-590 Chloral hydrate, 545
Cannula, implanted, 60 theories, 581-588 Chlordiazepoxide:
Carbachol, and thermoregulation, 169 Central reward pathways, 582 acquisition with, 561
Carbon monoxide, behavioral effects, 563 CER (see Conditioned emotional response, classification, 545
Cardiac output, and escape-avoidance, Conditioned suppression) and conditioned suppression, 349, 356-
602-603 Cerebral cortex: 357
Cardiorespiratory system, transient and inhibition, 588 as depressant, 555-556
changes, 597 integrative functions, 641-642 and punishment, 558
Cardiovascular responses, operant con¬ stimulation of, 570 schedule control, 189, 191, 557
ditioning, 606-611 Cerebral peduncle, self-stimulation, 590 stimulus control, 555-556
Cardiovascular system, experimentally in¬ Cervidae, 621 Chlorpromazine:
duced changes, 598, 604 Chain pulling: acquisition with, 561
Carnivores: squirrel monkey, 420-421 classification, 544-545
caloric regulation, 47 and thermoregulation, 164 and clock schedule, 555
feeding patterns, 36, 47-49 Chained schedule, 289-299 and deprivation, 553
Castration, and central reinforcement, 579 central reinforcement, 573 errorless learning, 553
Cat: and choice, 335 escape behavior, 384
attack behavior, 14-15 complex behavior on, 292 human retardates, 564
auditory intensity discrimination, 516 concurrent (see Concurrent chain negative reinforcement, 367, 384
avoidance, 193, 605 schedule) psychiatric use, 540, 565-566
caloric regulation, 43 conditioned reinforcement on, 288-316 and punishment, 558
carnivorous eating pattern, 49 controlling variables, 289-290, 293 schedule control, 188-189, 556-557
color vision, 515 description, 289 shock avoidance, 556-557
EEG activity, 605-606 discriminative stimuli, 289, 293-299 stimulus control, 555
electrical brain stimulation, 14-15, 589- drug effects, 555 thermoregulation, 167, 556
590 and electrical brain stimulation, 181 Choice, 233-282, 313-337 (see also Con¬
meal patterning, 35-39, 49 interresponse times, 223 current schedules, Matching)
meal size, 34 interval schedules, response rate, 291- absolute response rate in, 257-263
patterning of drinking, 36 292 additive difference model, 277
psychophysics, 515, 516, 520 maintained responding, 291-293 central vs. conventional reinforcer, 577—
rat-killing behavior, 14-15 order of stimuli, 295-296 578
self-stimulation of brain, 589-590 ratio schedules, response rate, 292-293 concurrent chains, 329-337
shock avoidance, 193, 605 and relativity of reinforcement, 181 concurrent schedules, 235, 239, 245-246,
shock-delay procedure, 393-394 and tandem schedule, 290, 293-295, 298 253-254
thermoregulation, 153, 166-167 thermal reinforcement, 164 and conditioned reinforcement, 313-337
visual discrimination, 520 transition performances, 290-291 discrete trial procedure, 236
Catch trials, in animal psychophysics, 519 Chaining: immediate vs. delayed reinforcement,
Catecholamine: and external stimuli, 142 251, 252
and avoidance, 600 fixed-interval schedules, 221-222 interresponse time, 255-256
and heart rate conditioning, 608 interim activities, 142 and matching law, 275-276
and self-stimulation of brain, 571, 585- sequence of behaviors, 141-143 measurement, 235, 239
588 and temporal discrimination, 141-143 and negative reinforcement, 366, 374-
Catecholamine pathways: Chaining delay, and gradient of reinforce¬ 392
inhibition, 587-588 ment, 217-218 and punishment, 253-254
and self-stimulation of brain, 585-589 Chaining hypothesis, schedule control, qualitatively different reinforcers, 252,
Catecholaminergic neurons, 571, 585-588 221-222 262
Category symbols, in language, 630 Changeover delay: and required response rate, 334
Catheter, intravenous, drug administra¬ animal psychophysics, 523 Cholinergic blocking agent, 169
tion by, 555-556 concurrent schedules, 79, 242-244, 250, Cholinesterase inhibitors, and thermo¬
Caudate nucleus, self-stimulation of, 590 276 regulation, 165
Causal factors, behavioral sequences, HS¬ minimum for matching, 242-244, 249 Cholinomimetic agents, 168-169
US and observing responses, 321 Cingulate gyrus, self-stimulation, 589
Cebus monkey, contingent-response ex¬ response rate during, 244 Circadian rhythms, and thermoregulation,
periment, 102 role in matching, 242-244, 250, 276 161-162
Ceiling effect: use, 235 Circularity, of weak law of effect, 100
in conditioned suppression, 351, 354 Changeover key, use, 234, 237 Classical conditioning, 340-361 (see also
Subject Index 667

Pavlovian conditioning, Respondent Competing responses: Conditioned emotional response (see also
conditioning) choice situation, 257, 262 Conditioned suppression):
autoshaping, 119-122 and conditioned suppression, 351-353, and conditioned suppression, 353-357,
and brain stimulation, 576 358-359, 360 360
and conditioned suppression, 342, 344, and delay of reinforcement, 218 and latent inhibition, 487
351, 358, 360, 517 and inhibitory control, 463-464 and overshadowing, 492
descriptions, 9, 53, 61, 138, 340, 489 shock situation, 260-261, 408-410 partial reinforcement effect, 61
drug effects, 553, 554 and stimulus generalization, 438-439 physiological changes, 598-599
fear, and avoidance, 393-397 Competition: Conditioned enhancement, 88-90
galvanic skin response, 493 associative, 507 Conditioned flexion response:
gerbils, 115-116 behavioral states, 144—147 dog, 483-484
human, 20, 493 stimuli, 496, 498, 507-508 goats and sheep, 487
and induced behavior, 125-126 terminal and interim activities, 132-135, Conditioned inhibition:
misbehavior of organisms, 3 145-146 and contrast, 84-85
necessary and sufficient conditions, 343- Complex symbol, in language, 648
and errorless learning, 475
344 Compound stimulus, and stimulus and latent inhibition, 487
and operant behavior, 340-361 generalization, 439, 443—444, 459— measurement, 445-447
pigeons, auditory frequency, 489 463 Conditioned reflex (see Reflex)
rabbit: Comprehension, language, 649—651 Conditioned reinforcement, 288-309, 313-
auditory frequency, 489 Concept learning: 337 (see also Secondary reinforce¬
eyelid conditioning, 493-496, 498 and animal psychophysics, 525 ment)
Click rate, stimulus control by, 496 and verbal behavior, 634 chained schedules, 288-289, 293-299,
Clock schedule: Concepts, linguistic, 619 309, 313, 315
and chain schedules, 296-297, 299 Conceptual nervous system, 32 choice and, 313-314, 326-337
drug effects, 555 Concurrent chain schedule: concepts, 180, 288-289, 309, 313-315
and gastric ulcer, 604-605 and conditioned reinforcement, 303, concurrent chain schedules, 326—337
CNV (see Contingent negative variation) SI3, $27-337 conjoint schedules, 307-308
CO (see Changeover) description, 297, 327 discriminative effects, 289, 309, 314—315
Cocaine: matching on, 252, 297-298, 329-333 discriminative stimulus hypothesis, 181
classification, 545 strengths and weaknesses, 327-329 drug effects, 558
and concurrent matching, 250 terminological problem, 298 hypotheses, 313-315, 325-326, 336-337
reinforcer, 192-193, 250, 557 Concurrent generalization tost, 435 and information, $0$, 313-337
self-administration, 543 Concurrent models, altered physiological mathematical treatment, 4
and self-stimulation of brain, 585 states, 596-606 observing responses, 313-315, 318-326,
Cochlear nucleus, EEG activity, 605 Concurrent schedules, 233-282 336-337
Cockroaches, meal patterning, 35 Concurrent schedules; pairing hypothesis, 313, 315-318, 334,
COD (see Changeover delay) absolute response rates, 265-266 336-337
Codeine, rein forcer, 192 behavioral pharmacology, 551 paradigm experiment, 288
Coding, language, 620, 622-627 brief stimulus presentation, 306-308
relation to primary, 289
Cognitive dissonance, 9 central reinforcement, 572-573 schedule control, g, g80-$O9
Cognitive learning, and verbal behavior, conditioned reinforcement, 326-337 second-order schedules, 302-309, 313-
636, 649 and conditioned suppression, 355
Cognitive processes, and central reinforce¬ and contrast, 78-80 310
uncertainty reduction, 318-326, 306—337
ment, 582 description, 233-234
and verbal behavior, 633
Cognitive theories of language, 628-633, electric shock, 374-375, 386, 388-392 Conditioned suppression, 340-303 (see also
638, 640-642, 651-652 and extradimensional training, 501-503 Conditioned emotional response,
CO-key concurrent schedule, 234-237 generalization test, 435
Cold escape, rats, 261, 263, 282
Positive conditioned suppression)
and. inhibitory control, 403-404 and anxiety, 355-357, 360, 393
Collateral behavior (see also Adjunctive interactions, 78-80, 233—256, 265-266
and autoshaping, 90, 359
behavior, interim activities, Sched= matching, 78-80, 233-256, 272-276, 329-
ule-induced behavior): 338 and avoidance behavior, 35§, 309
classical conditioning, 340-361
and choice on IRT schedule, 255, 266 generality, 248-256, 275, 276
and conditioned suppression on DRL.
concurrent choice, Z53
and multiple schedule, tlZ and contrast, 88—90
347 and negative reinforcement, 254
description, 341-342
and temporal discrimination, 143 and punishment, 253-254 drug effects, 342, 350, 355-957, 360, 554
College students: reinforcement frequency, 235—244, 247,
Estcs*5kinner procedure, 341-344
lever pressing, 103, 107 249,252 gerbils, 11$
relativity of reward, 102 reinforcement immediacy, 233, 251—252 hypotheses, 351-358
wheel cranking, 103, 107 reinforcement magnitude, 233-234, 248-
interference hypothesis, 351-353, 357-
Color circle, animal psychophysics, 530- 251 359
531 negative reinforcement, 254, 374—375,
and latent inhibition, 487
Color coding, animal psychophysics, 530 386, 388-392
measurement, 348-351, 300
Color vision: and observing responses, 321 monkey, 89, 352, 358-360
and peak shift, 463-464
cat, 515 motivational hypotheses, 353-357, 358
punishment, 253-254
goldfish, 517, 522 mouse, 517
types, 234-235, 254-255
monkey, 518 and olfactory sensitivity in pigeon, 513
Concurrent superstitions, 234-235, 243
Combined cues method, generalization and operant parameters, 5, 344-348, 351
Conditionability, negative reinforcement,
gradient, 446-447 as Pavlovian fear conditioning, 315
407-408
Combined cues test, for inhibitory control, physiological changes, 598-599
Conditionable response unit, 222-227
470-472 Conditional discrimination, and verbal punishment hypothesis, 357-358
Comfort activities, as facultative behavior, behavior, 636, 646 schedule effects, 345-351, 360
135 Conditioned aversive stimulus: shock deletion procedure, 409
Command neurons, 588 and concurrent choice, 253 shock intensity, 342, 346
Common fate, law of, 623 and negative reinforcement, 393-396, shock probability, 343-344
Comparator device, 638 400 Siamese fighting fish, 116
Compatibility, instrumental and uncondi¬ and stimulus generalization, 447 animal psychophysics, 516-517, 522, 525
tioned response, 118-122 Conditioned cardiac respondent, 598 Conditioning context, and stimulus
Competence, linguistic, 628-629 Conditioned confusion, 317 generalization, 444
668 Subject Index

Conflict: and shock probability, 375-377, 398-399 in contrast, 88

approach -avoidance, shock, and extinction, 379-380 definition, 340
and gastric ulcer, 597, 604 shock, and frequency, 398-399 fear conditioning, 315
and schedule induction, 139-140 temporal, in classical conditioning, 343 informativeness, 58, 80
and de-encephalization, 23 Contingency models, altered physiological Cue (see Stimulus)
Conflict procedure, and gastric ulcers, 597, states, 606-611 Cued escape, 383-387
604 Contingency strength: * * Cues, acquired distinctiveness, 487-488
Conjoint schedule: quantitative measure, 128 Cumulation, drug effects, 550-551
brief stimulus presentation, 306-308 and terminal responses, 127 Cumulative record, advantages, 11
and conditioned reinforcement, 304 Contingency table: Cumulative recorder, history, 29
Conjunctive schedule: reinforcement and punishment, 180 Curare, and autonomic conditioning, 607-
and delay of reinforcement, 219 Contingent variation in EEG: 611
number of responses, 206-211 Negative, 606 Curiosity, 31
Consequence variables, psychopharma¬ Positive, 606 Cyclicities, fixed-interval behavior, 208-
cology, 556-559 Contingent-response (Premack-type) ex¬ 209
Constant probability schedule, and in¬ periments, 98-123 Cylert (see Magnesium pemoline)
duced drinking, 135 Continuity, behavior in time, 177-178
Constant probability VI schedule, 215 Continuous choice procedure (see Con¬ d', signal detection theory, 532-537
Constraints: current schedule) DA (see Dopamine)
biological, on learning, 3, 7, 91, 101, Continuous reinforcement schedule, cen¬ d-amphetamine (see also Amphetamine):
111-123 tral reinforcement, 573, 577 classification, 545
biological, on reinforcement, 112-123 Contrast, behavioral (see Behavioral con¬ and learning, 561
environmental, on feeding, 37, 44 trast, Local contrast. Negative and learning disability, 564
genetic, on language, 641 behavioral contrast, Positive be¬ and negative reinforcement, 367
on reinforcer availability, 40-41, 47 havioral contrast, Transient con¬ reinforcer, 192
species, trast) and schedule performance, 190-191
on autoshaped response, 61 Control (see Aversive control, Schedule and thermoregulation, 164
and central reinforcement, 580 control, Stimulus control) Dark adaptation, pigeons, 521-522
on generalization gradients, 435-436 Controlling variables, types, on schedules, Darkness, rearing in, 486
on stimulus control, 483-485 203-204, 228 Deafferentation, 146
systems (see Systems constraint) Control procedures (see also Truly Debilitation, and thermoregulation, 155
Consummatory behavior (see also Drink¬ random control, Yoked controls): Decay, rate of contingent response, 108
ing, Eating, Feeding behavior): animal psychophysics, 515-532 Decerebrate, 8, 11
vs. appetitive behavior, 22 behavioral pharmacology, 551-552 Decremental generalization gradient,
autoshaping, 62, 70 classical conditioning, 55-56, 61, 84, 120, definition, 445
in contrast experiments, 81-82 343 De-encephalization, 22-23
monkey, 58, 65 concurrent matching experiments, 242- Deep structure, of language, 629-633, 648-
pigeon, in autoshaping, 58, 65, 69-70 243 651
as reflexive, 22, 28, 49, 62 contingent-response experiments, 106- Defecation, and conditioned suppression,
Consummatory force, 144, 146 110, 115-116 351
Consummatory response: and stimulus generalization, 435-436, Defensive behavior, 115-117
relation to operant behavior, 3, 20, 22, 442, 445 Deficit, sensory, and thermoregulation,
62-68, 127 Control systems, and thermoregulation, 155-156
and terminal response, 127 160-161 Deficit theory of motivation, 31, 49
and theta waves, 119 Control theory, and thermoregulation, Delayed conditioning, classical, 343
Consummatory response theory, and cen¬ 153-154, 160-162 Delayed escape, 261-262, 281-282
tral reinforcement, 581 Coping response, and gastrointestinal Delay of reinforcement (see also Rein¬
Contact comfort, and thermoregulation, changes, 604-605 forcement, immediacy):
159 Copulation, 109, 122 delayed escape, 261
Context, and conditioned reinforcement, Correction procedure, animal psycho¬ and habit strength, 217-218
315, 330 physics, 524, 536 human, 262
Contextual stimulus (see also Situational Correlation-based law of effect. 111 and matching, 234, 251-252
stimulus): Cortical evoked activity, 605 and observing responses, 320, 322, 325
and stimulus control, 444, 457-458, 488- Cost: and response strength, 260-262, 266, 281
509 procurement vs. use, 47 and schedule control, 218-222
Contiguity: response, 239, 563 theory, 217-220
and conditioned reinforcement, 314— Counterexperiment, method of synthesis, Delay of reward procedure, 125
318, 334, 336 Delay reduction hypothesis:
10
induced drinking and food, 132 Counting schedule, conditioned suppres¬ conditioned reinforcement, 313-315,
response-reinforcer, sion, 347 324-337
interval and time schedules, 214, 219 Courtship: equations, 329, 332
problems with, 127-128 lock and key sequence, 142 quantitative analysis, 329-333
on response-dependent schedules, 204, pigeons Demerol (see Meperidine)
207, 228 autoshaping, 59, 119 Density:
in verbal behavior, 637-638 as interim activity, 137 negative reinforcement, 369-370, 375,
Contingencies of reinforcement, 3, 98-124, sticklebacks, 121-122 399, 402
175-176 CR, definition, 340 reinforcement
Contingency: CRF (see Continuous reinforcement sched¬ in autoshaping, 61
and aversive control, 425-430 ule) and conditioned reinforcement, 314,
categories, 99, 101 Criteria, reinforcement vs. punishment, 316, 325-327, 329, 332, 336-337
negative, 63, 99 186-188 differentiation schedule, 224
vs. pairing Criterion, in animal psychophysics, 521, drug effects, 552
in autoshaping, 54-56 524 and induced behavior, 140
reinforcement Cross tolerance, drugs, 550 in time, 214-215
in animal psychophysics, 525 CS (see also Warning stimulus): Dependence:
response, and schedule induction, 125— in autoshaping, 55-61, 69-70, 80 on drugs, 185, 192-193, 550, 553
128, 130-136, 140-141 and conditioned enhancement, 90 human, 557-558, 563
among responses, 98-123 in conditioned suppression, 89-90, 342- Depletion-repletion model:
response-shock deletion, 398-399 345, 350, 359 problems with, 31-32
Subject Index 669

in regulation of eating, 31, 45-46, 49 food and shock, 485 classical conditioning, 61, 138, 483-484,
Depressant drugs, 544, 555-556, 561-562 generalization gradient, 435 489-492
Deprivation: go/no-go method, 516 conditioned flexion response, 483-484
autoshaping, 59 and immediacy of reinforcement, 260, drugs as discriminative stimuli, 555-556
and central reinforcement, 575-576, 579— 281 gastric motility, 20
580, 584 matching, 236, 244, 246 hippocampal activity, 605
and conditioned suppression, 354 observing responses on, 320 hypothalmic syndrome, 20
contingent-response experiments, 102- and overshadowing, 492 meal patterning, 35
105, 115, 118, 122, 181 shock deletion, 399 negative reinforcement
effect of reinforcement, 178, 186 and stimulus control, 495, 505 shock-delay procedure, 390, 392-393
and magnitude of reinforcement, 259 threshold tracking, 522 omission training, 63
operation, 30-31 Discriminability, index of, 534 patterning of drinking, 36
and polydipsia, 130, 132 Discriminated avoidance (see Avoidance, Pavlovian conditioning, 61, 138, 483—
psychopharmacology, 553 discriminated) t 484, 489
and rate of eating, 175 Discrimination: salivation
and response-produced shock, 182 components of second order schedules, classical conditioning, 138, 340, 489
and thirst, in induced drinking, 138— 316, 318 instrumental conditioning, 607
139 extinction, and negative reinforcement, self-stimulation of brain, 589-590
as unnecessary, 36 378-380 thermoregulation, 153, 159, 166—167
and verbal behavior, 633 problems with concept, 433 Dolphin, self-stimulation of brain, 590
Descending order of stimuli, in animal simultaneous vs, successive, 487 Dominance, and feeding behavior, 34
psychophysics, 521 Discrimination learning: Domino theory, sequence of behaviors, 141
Desoxyn (see Mcthamphctamine) errorless (see Errorless discrimination Dopamine:
Development, gape response, in thrushes, learning) and self-stimulation of brain, 585-586
19 and escape from S~, 469-470 and thermoregulation, 168
Development, stages: free operant, 499—505 Dopamine-^-hydroxylase, and schizo¬
compared to recovery, 17-22 patterns, by pigeons, 528 phrenia, 571
grasp, in human, 19 sign tracking in, 69 Dopaminergic pathways, 585—588
rat, 18 Spence’s theory, 447-451, 453, 456, 462, Dorsal noradrenergic pathway, 586
Diabetes, chronic, and meal frequency, 35 476 Dose-effect relations, drugs, and schedule
Diabetes mellitus, 10 and stimulus control, 499-505 performance, 189, 192-193, 202
Diazepam: Discrimination training: Dose-response curve, behavioral pharma¬
classification, 545 and attentiveness, 500—502, 505 cology, 541, 545-547
and punishment, 558. definition, 433 Dove:
Diencephalon: and generalization, 439-453 schedule-induced eating, 139
stimulation, 577 and inhibition, 432, 439-453 schedule-induced polydipsia, 129, 136
and thermoregulation, 158 schedule effects, 503-505 thermoregulation, 153
Diet: and stimulus control, 493-505 DRH schedule (see also Differentiation
selection (see Self-selection) types, 439^440 schedule):
and thermoregulation, 162 Discriminative control (see also Stimulus and choice, 334
Dieter’s nucleus, 606 control): and conditioned suppres¬ negative reinforcement, 391
Difference concept, in animals, 518 sion, 347—348 Drinking (see also Licking, Polydipsia,
Differential probability rules, 102-108, Discriminative stimulus: Schedule-induced behavior):
111-113, 122, 181 chained schedules, 289, 293-299 adjunctive (see also Drinking, interim;
Differential reinforcement: concept, 180 Drinking, schedule-induced* Poly¬
animal psychophysics, 528-529 concurrent chains, 336 dipsia, Schedule-induced behavior)
definition, 433 and conditioned reinforcement, 314-315 and conditioned suppression, 350
and generalization gradient, 439-441 drugs as, 192-193, 555-556 aversive control schedule, 423-425
implicit, 489-491 negative reinforcement, 381-396, 400 contingent-response experiments, 102-
in shaping, 177 in polydipsia, 129, 131, 133 109, 181
and stimulus control, 432-476, 493-505 second order schedules, 302-309 elicited by brain stimulation, 571, 579,
Differentiation schedule (see also DRL in verbal behavior, 633-635 581=583, 589
schedule, Interresponse time sched¬ Discriminative stimulus hypothesis, con¬ instruments^ 163-166
ule): ditioned reinforcement, 181 instrumental avoidance, 118
definitions, 263 Disinhibitcd activities, 139-140 interim, 128-130, 134-133; 142, \ii-\il
and IRT as conditionable unit, 224 Dismhihition; patterning
Diffuseness, and stimulus control, 490, 508 and conditioned suppression, 347=348 and FR requirement, 40, 42
Digestion: and induced activities, i39, 143-145 guinea pigs, 42
herbivores, 33-34, 47-48 and motor control, 588 rati, §6, 40, 42
and thermoregulation} 16^
by novel stimulus, 116 postprandial (see Postprandial drinking)
Digging:
and temporal discrimination, l43 rdnforccr i/j. punisher, l§i
gerbil,
115-110 Displacement activities, 23, 139 relation to feeding, §5-36, 42-43, 131,
hamster, 113—118
Distribution: 135, 139
Dimensional control:
drugs, 543, 547-550 schedule induced (see also Polydipsia)
and early experience, 485-487
compared to attack ^ 137
intcrrvspopsc tim<?s, 224-225
inhibitory, 453—456
Disulfuram, 563 development, 131-132, 138
and negative reinforcement, 391-392,
Diurnal rhythm: fixed-interval schedules, 135
398, 400
and stimulus generalization, 444-447 drinking hypotheses, 130-132
Dimension of generalization, 433 rats, 36, 44 interaction with running, 146-148
Diphenhydramine, 564 eating, as interim activity, 128-130, 134-135,
Direct variables, and schedule control, guinea pigs, 35 142, 144-147
203-206, 228 rats, 33, 444 monkey, 138
Directedness: Dog: motivation, 132, 138-139
autoshaped response, 60-61, 69, 81 auditory stimulus control, 483-484 rat, 126, 129-137, 143
Pavlovian responses, 61, 81, 138 autoshaping, 58 rate, 129-135, 138
Discrete-trial procedure: avoidance, 406, 602-605 temporal locus, 133, 135, 140
animal psychophysics, 516 brain damage, 20 theories, 130-132
escape as, 367-368 cardiovascular changes, 602-603 and thirst, 138-139
 670 Subject Index

Drinking (cont.) and negative reinforcement, 367, 384 electrically-evoked, 15

squirrel monkey, and aversive control, principles of action, 543-551 lateral hypothalamus, 16-18
423-425 and psychotic behavior, 564-566 patterns, and evolution, 33, 46
Drinking tube, 98, 102, 104, 107, 108, 118, psychotropic, 545 periodicity, 28-30
181, 417 reinforcement vs. punishment, 178, 188— rate, and FR requirement, 41-42
Drinkometer, 181, 417 193 recovery, after lesions, 17, 20
Drive: reinforcers, 192-193, 556-558 as reflex, 29-31
emotional, conditioned suppression, and relativity of reward, 179-181, 188— relation to drinking, 35-36, 42-43, 131,
354-355, 358 193 135, 139
level: reversibility of effects, 551 schedule-induced, in doves, 139
and immediacy of reinforcement, 260, and schedule control, 188-193, 202, 542- and schedule-induced polydipsia, 129—
281 543, 546, 551-552, 555-560 - 130, 135
and magnitude of reinforcement, 259, self-stimulation procedure, 571, 574, Eatometer, 29
278 585-587 Echoic, in verbal behavior, 633-634, 637,
and matching, 262 sensation and perception, 562 639, 644
Drive induction: solubility, 549-550 Echolocation, in bat, 519
and central reinforcement, 581 specificity of action, 551-552 Ecological niche, and feeding patterns, 33,
theory, 99 and thermoregulation, 153-155, 164- 43, 46-49
Drive operation: 169, 188-189, 556 Economics, of feeding behavior, 34, 46, 48
central reinforcement, 576, 581, 586 time course of effects, 548-549 Ectostriatal brain stimulation, and choice,
negative reinforcement, 368, 378-380, toxic effects, 546, 562-563 252
387-388 type classifications, 544-545 Ectostriatum, self-stimulation of, 590
Drive reduction: Dry mouth: Ectotherm, thermoregulation in, 156-161,
and electrical brain stimulation, 571, and grooming in hamster, 144 166
581-582, 584, 586, 588-589 theory of polydipsia, 129 EEG (see Electroencephalogram)
theory, 99 Duckling, stimulus generalization, 485-486 Effort, of response, and thermoregulation,
and verbal behavior, 633 Duration: 164
Drive state: bout, of induced activity, 141-142, 146 Egg rolling, in goose, 23
and central reinforcement, 576, 579-584 brief stimulus, second-order schedules, Ego, 9
and schedule control, 188 304-305 EKG (see Electrocardiogram)
DRL schedule (see also Differentiation burst, contingent-response experiments, Elavil (see Amitryptaline)
schedule, Interresponse time sched¬ 110-112 Electrical brain stimulation (see also
ule): component, in multiple schedule Central reinforcement):
and carnivorous feeding pattern, 48 and contrast, 79, 87, 90 and animal psychophysics, 518
central reinforcement, 578-579 and matching, 269-272 autoshaping with, 60
component of chain, 293 and negative reinforcement, 382-383 and behavior, 570-590
component of conjoint, 307 CS, and conditioned suppression, 89 cat, 14-15, 589-590
and conditioned enhancement, 90 as discriminative stimulus, 218-219 choice, 252
conditioned suppression, 89, 347, 359 534-535 concurrent schedules, 237
and contrast, 73, 77 interim activities, control by food, 131 conditioned suppression, 358-359
drug effects, 552 inter trial interval, autoshaping, 57-58 diencephalon, 577
and generalization gradient, 436, 439 66, 78 and drugs, 188, 571, 574, 585-586
induced attack, 137 interval component, chained schedules, elicitor and reward, 14
induced behavior, 134-135, 141-143 291-292 gerbil, 111-112
induced drinking, 135 of peck hypothalamus, 14-15, 60, 111, 164, 178-
negative reinforcement, 391 autoshaping, 24, 58, 67-68, 81 181, 259, 570-589
in omission training, 66 and contrast, 87-88 intrapeduncular nucleus, 585
problems with terminology, 203 on fixed-interval, 67 locus coeruleus, 585
DRO schedule (see also Differentiation on fixed-ratio, 67 medial forebrain bundle, 577
schedule): food vs. water, 58-59, 67, 81, 119 methodology, 572-574
and behavioral contrast, 75, 77, 267 postreinforcement pause and motivation, 14, 570-589
component of second order, 306 on fixed-interval, 213, 216 partial reinforcement, 578-579
Drug abuse: on fixed-ratio, 209, 213, 217, 226-228 pigeons, 252
animal models, 192-193 prefood stimulus, and positive condi¬ postponement, 178
and behavioral pharmacology, 542, 557— tioned suppression, 359 rat, 14, 60, 89, 120, 164-165, 178, 180,
558, 563 preshock stimulus, and conditioned sup¬ 237, 259, 279, 518, 572-590
Drug effects, micro-analysis, 562 pression, 344-345, 350 reinforcement magnitude, 179, 259, 263,
Drugs (see also Behavioral pharmacology. of ratio run, as conditionable unit, 227 279
Psychopharmacology, specific drug): reinforcement (see also Reinforcement, reinforcer vs. punisher, 178-181
absorption and distribution, 543, 547- magnitude) and relativity of reward, 102, 181
550 and absolute response rate, 259 reward in visceral conditioning, 607-611
and aggression, 417, 552, 562 and conditioned enhancement, 90 septum, 570, 575-576, 583
behavioral mechanisms, 551-560 discrimination of, 249 species and sites used, 589-590
behavioral pharmacology, 5, 188-193, responding, contingent-response experi¬ substantia nigra, 585
349-360, 540-569 ments, 108-112 telencephalon, 577
and blood pressure, 603 of S—, and inhibition, 447 and thermoregulation, 164-165
classification, 544-545 stimulus, and conditioned suppression, Electric shock (see also Avoidance, Escape,
clinical use, 188, 540, 542, 563-566 344-345, 350, 359 Negative reinforcement, Shock de¬
and conditioned suppression, 349-350, terminal links, concurrent chains, 327- lay, Shock deletion):
355-357, 360 337 in animal psychophysics, 516-517, 523
cumulative effects, 550-551 trials, autoshaping, 57-58, 66, 78, 90 and attack behavior, 183-184, 380, 409,
dose-response curves, 189, 192-193, 202, Dynamic effects, and schedule perfor¬ 417-425
541, 545-547 mance, 204-210, 228 autoshaping with, 65, 70
and electric shock, 189-191, 552, 555- Dynamic models, stimulus control, 451— in aversive control, 415-430
560 453 avoidance (see Avoidance, Negative re¬
and electrical brain stimulation, 188, inforcement, Shock delay)
571, 574, 585-586 Eating (see also Feeding behavior): and central reinforcement, 574
fate in body, 550 curve of, in rat, 174-175 changeover response, 243-244, 253
and motivation, 188-189, 559-562 development, in rat, 17-18 concurrent matching, 253-254
Subject Index 671

and conditioned reinforcement, 304-305 Equipotentiality, premise of, 91 Extinction:

and conditioned suppression, 341-356 Error: brief stimulus during, 305-306
and contrast, 74-77 aversiveness: central reinforcement, 575-576, 589
delay of escape, 261-262, 282 in programmed learning, 467, 469-470 and conditioned reinforcement, 288-
density-frequency continuum, 369-370, in discrimination learning, definition, 289, 299
375, 383 464-465 and contrast, 71-77, 80, 82-83
density reduction (see Shock density re¬ in operant behavior, 22, 23 drug effects, 560-562
duction) Errorless discrimination learning, 5, 76, after fixed-ratio, 223-226
86, 464-476 generalization gradient in, 435, 438, 527
as discriminative stimulus, and negative
reinforcement, 368, 369, 377—380, Errorless discrimination learning: heart rate conditioning, 607
383 and agression, 468—469 massed, and peak shift, 456
and drug effects, 188-191, 555-556, 558- and behavioral contrast, 76, 86, 472- massed, and stimulus control, 456-457,
559 474 527
escape (see Escape) definition, 464-465 multiple schedule, 71-77, 80-83

gerbils, 113 drug effects, 553 after negative reinforcement, 377-380

inhibitory stimulus control, 470-474 non-passive effects, 226
intensity:
and choice, 253 and stimulus control, 186, 464-476 observing response procedures, 318-326
reconsideration, 432, 464-476 relation to reinforcement, 205
and conditioned suppression, 342, 346,
354 resistance to reinforcement, 446 resistance to (see Resistance to extinc¬
techniques, 466 tion)
and number of responses, 194
intensity reduction, negative reinforce¬ Terrace's theory, 467-468 after shock schedules, 377-380
ment, 375, 380 as transfer of control, 465—466, 475 Extinction-mduced aggression, 468—469
intermittent schedules, 375—377 Error signal, in thermoregulation, 160- Extinction methods, generalization gra¬
and latent inhibition, 487 165 dient, 434
and negative induction, 74 Escape (See also Avoidance, Negative re¬ Extinction schedule, relation to other
negative reinforcement, 364-410 inforcement) schedules, 203
postponement, 185, 194-196 (see also (from electric shock, by rats, unless Extradimensional shift, 487
Extradimensional training:
Shock delay) otherwise stated):
reinforcer vs. punisher, 178-180, 183— added cues, 383-387 analysis, 503-505
and aversive control, 90, 416, 421, 425- concurrent experiments, 501—503
184, 193-197
response-produced, 176—185, 190, 193— 427 and stimulus control, 497-505
centrifugal force, 367 successive stage experiments, 497—500
196, 380 j
Siamese fighting fish, 116 as change of situatien, 387 Extraneous cues, in animal psychophysics,
squirrel monkeys, 173-179, 183-185, 189- cold water, by rats, 261, 282 524
190, 193-196, 415-430 and conditioned reinforcement;, 319 Extrapyramidal system, and central rein¬
Electrocardiogram, operant conditioning cued, 383-387 forcement, 572
of, 608 delayed, 261-262, 261-292 Eye blink, Pavlovian conditioning, 61
Electrode implantation, methods, 572 drug effects, 557 Eyelid conditioning, rabbit, 493, 496, 498
Electroencephalogram, and avoidance, free operant, 368
605-606 heat, 165, 166, 168 Face washing, hamster, 113-115, 118
Electromyogram, 417-418, 606-607 intense light, 367 Facilitation:
Electrophysiological changes, behaviorally noise, 282, 367 by aversive stimuli, 422j 424, 428
induced, 605-606 pigeon, 254 induced behavior, 141, 144
EMG (see Electromyogram) procedure, 367-368 Facultative behavior;
and response strength, 261, 261-282, definition, 126
Emitted behavior, vs. elicited, 174-175
Emotion (see also Motivation, Condi¬ 543 examples, 135, 140
tioned emotional response): rotation, 367 grooming, 135
and conditioned suppression, 353-358 runway, 261-262, 281-262 running, 133-137, 140
Enccphalization, 20 -23 squirrel monkeys, 189, 380, 557 Facultative periods, on periodic schedules,
Encoding, in natural language, 620, 622- stability of, 543 134-138
temperature change, 162, 367 Fading procedure:
627
Encoding specificity, and memory, 638 temporal pattern of behavior, 425-427 in animal psychophysics, 520
Endocrinological system, durable changes, Escape / avoidance distinction, problems and errorless learning, 466
597-604 with, 368-370, 387 in language learning, 640
Endotherm, thermoregulation, 156 Estrus, rat, and thermoregulation, 162 Fading trace, negative reinfoxcemeintt, 400

Energy balance, regulation, 31, S3, 43, 47, Ethansi (see also Alcohol): False‘alarms, animal psychophysics, 516,
129 rein forcer, rats, 553, 561 521, 5255 533-534
Enhancement; and stimulus control, 555—556 Falee positive rQSpotigQo, animal psycho¬
conditioned (see Conditioned enhance¬ Ethology, 4, 7, 11-12, 15, 19-20, 49, 91, physics, 019
Fasting, eperatien to insure eating, S8-SS
ment) 120, 571-572
responding, by shock, 185 Ethyl alcohol (see Ethanol, Alcohol) Fat solubility, drugs, 549-550
stimulus control, 500-509 Evolution, feeding patterns, 33, 46-50 Fate, drugs in body, 550

Environment: Excitation: , Fatigue:

and feeding patterns, 33, 36, 44—50 and dimensional control, 444-445 and rate of eating, 175
simplification, 11 by reinforcement, 264, 272-274 and schedule control, 28
Environmental constraints (sec Con¬ Excitatory stimulus, definition, 444 Fear:
straints) Excretion, drugs, 550 as incentive, 100
Environmentalism, and language, 641 Expectancy, electrical signals of, 606 Pavlovian conditioning, 315, 393-397
Epinephrine: Expectation: “Feast or famine,” in large carnivores, 47
and heart rate conditioning, 608 as behavioral state, 144 Feature, stimulus control by, 528
and stimulus control, 555-556 in Pavlovian conditioning, 138 Features analysis, and language, 650-651
Episodic memory, 637 Experimental analysis of behavior, 1-2 Feedback (see also Positive feedback.
Equal-brightness contour, rat, 526-527, “Experiments of fruit,” 12 Negative feedback):
532 “Experiments of light,” 12 autonomic conditioning, 610
Equal-loudness functions, animal psycho¬ Exploratory behavior: in avoidance, and gastric ulcers, 604-
physics, 526, 532 and motivation, 584 605
Equalizing, vs. matching in concurrent rats, 60, 120 between behavior and consequences,
schedules, 272 Extended chained schedule, 291 125, 144
Equanil (see Meprobamate) Extension, verbal, 634 behaviors in sequence, 145-148
 672
Subject Index

Feedback (<cont.) temporal patterning, 213-214, 216 type and palatability, and induced
in language learning, 624 two-state analysis, 265 drinking, 132-133
and schedule-induced behavior, 135, Fixed-ratio schedule: as US in autoshaping, 58
145-146 acquisition of performance, 227 Food anticipation:
Feedback theory of reinforcement, 4 alcohol consumption, 563 interaction with running and drinking
Feeding behavior (see also Eating, Meal): central reinforcement, 578 146-148
cat, 34-35, 38-39, 49 compared to fixed-interval, 206
and dominance, 34
as terminal response, 127, 132-134, 141-
component of chain, 292-295 143, 146-147
elicited by brain stimulation, 571, 579, component of conjoint, 307
581, 583, 589
Food economy, and ecological niche, 46-
component of second-order, 299-305 49 ,
environmental constraints, 44, 49 and conditioned suppression, 346, 348-
and FR requirement, 42
Food expectancy, as induced state, 138
349 Food-gathering, gerbils, 115, 118
pecking as, in pigeon, 69 in contingent-response experiments, 104
relation to drinking, 35-36, 42-43, 131
Food intake (see also Eating, Feeding):
and contrast, 74 and water intake, 35
135, 139 definition, 202
species differences, 47, 49, 58
and thermoregulation, 162
drug effects, 188, 192, 543-544, 546, 548- Food rate (see also Rate, reinforcement):
topography, in monkeys, 59 549, 551, 556, 559, 562-564
Feeding pyramid, 47 and schedule-induced behavior, 129-138
duration requirement, 227 Forced-choice procedure, animal psycho¬
Fever, 160-166 dynamic effects, 206, 209-210
Figure-ground problem, in escape para¬ physics, 517-518, 524, 537
escape-avoidance, 601-602 Forced contingent response, 102-103, 111
digm, 383 induced behavior, 136, 139 181
Final common path, behavioral, 144 interreinforcer time, 205, 209-210, 217
Fibonacci VI schedule, 215 Forced motor activity, and central rein¬
limits on size, 38-39, 45, 206, 213 forcement, 573
First-order deviations, fixed interval matching, 246-247, 252, 254, 266 Forgetting:
schedules, 208 and meal parameters, 44-45 and stimulus control, 457-458
Fish (see also Goldfish, Siamese fighting and meal patterns, 36-39
fish. Stickleback): and thermoregulation, 155
meal reinforcement, 38-39, 45 Formal classes, verbal responses, 637-638
autoshaping, 58 negative reinforcement, 367, 385-386, Formal response unit, 222-223
failure of temporal discrimination, 140 390
thermoregulation, 156-158 Formatives, in language theory, 629, 630
observing responses, 320 648
5HT (see Serotonin) and pattern of drinking, 40
Fixation, in animal psychophysics, 523- Fourth-order deviations, fixed-interval
and pattern of running, 40 schedules, 208
524 pause duration, 209, 213 Free feeding, patterns, 28-36
Fixed action pattern, 15, 17, 23 peck duration, 67
Fixed constant number schedule, 226 Free food schedule (see Fixed time sched¬
response number, 209 ule, Variable time schedule)
Fixed-cycle procedure, shock deletion, response-produced shock, 182, 194-195 Free operant procedure:
371-373, 375, 400-402, 404-405 responses per reinforcer, 209-210, 212
Fixed-interval schedule: avoidance, 5, 119, 185, 369-371, 380
response units, 225-227 597-606
absolute response rate, 258, 265 and schedule-induced behavior, 136, 139 discrimination, 499-505
adventitious reinforcement and punish¬ and shaping history, 177 Freezing:
ment, 185 stereotypy of responding, 227 and animal psychophysics, 517
avoidance, 367, 377, 384, 388 strained performance, 37, 543
central reinforcement, 578 and conditioned suppression, 351, 353
temporal patterning, 213 rat, 116-117, 407-409
in chained schedule, 289-298, 335 and thermoregulation, 164
chaining on, 222 Frequency, of tone, generalization, 440-
Fixed-time schedule: 441, 450, 455-456
compared to fixed-ratio, 206 adventitious punishment, 184-185
compared to variable-interval, 265 Frequency of reinforcement (see Rein¬
adventitious reinforcement, 184-185 forcement, frequency)
concurrent matching, 252 204 Frog:
conditioned suppression, 341, 346-347 component of chain, 335
and contrast, 73 food response in, 121
and concurrent matching, 252 psychophysics, 515
cyclicities in responding, 208-209 definition, 202
definition, 202
thermoregulation, 157-158
excitation of pecking, 272-273 Frontal cortex, self-stimulation, 590
delay of reinforcement, 218-219
and induced behavior, 126, 128, 130, 133 Frustration, 23, 85, 468, 509
drug effects, 188, 192-193, 544, 555, 559,
139-142 Functional relevance hypothesis, 113-118
560
shock avoidance, 185 122-123
dynamic effects, 205-210
shock-elicited behavior, 183 Functional theory, language, 628-629, 633-
electric shock, 178-179, 182-184 193-
196 temporal patterning, 140-141, 214 640, 651-652
Flavor (see Taste) Functionalism, and language, 640-642
escape, 380
Flexion response, conditioned
and generalization gradient, 452-453, dog, 483-484
458
Galvanic skin response, 61, 493
goats and sheep, 487 Gamma rays, irradiation of rats, 527
Herrnstein’s equation, 265
Flicker-fusion frequency, monkeys, 522 Ganglionic blockade, 611
induced behavior, 125, 129-130, 133,
Floor effect: Gape response, development, in thrushes,
135-136, 138-142
on generalization gradient, 471-472, 482 19-20
local rate of reinforcement, 216
in generalization of extinction, 445 Gasterosteus aculeatus (see Stickleback)
matching, 242, 247, 252, 254, 255, 266
Fluphenazine, 545 Gastric motility, dog, 20
negative reinforcement, 367, 384, 388
Food: Gastric ulcer, rat, 597, 604-605
observing responses, 318, 320, 322, 325
peck duration, 67 as discriminative stimulus: Gastrocnemius muscle, 607
polydipsia, 129-130, 138 on fixed schedules, 129, 131, 133 Gastrointestinal changes:
punishment, 429 on variable schedules, 133 behaviorally induced, 604-605, 608
free: human, and avoidance, 597
reinforcement omission, 301
response-initiated, 301 and contrast, 87-90 Gaussian distribution, and generalization
gradients, 449
response-produced shock, 178-179, 182— during interim period, 128, 138 General activity:
184, 193-196 interaction with water, and matching, and estrus in rat, 162
responses per reinforcer, 205-212 252
response strength, 265
and induced behavior, 134
reinforcement, and stimulus control, and thermoregulation, 161-162, 165-168
in second-order schedule, 300-305 484-485
species differences, 12
Generalization (see also Stimulus generali¬
reinforcer vs. punisher, 178, 197 zation):
Subject Index 673

and animal psychophysics, 526-534 Gradient: Heroin:

of extinction, 445 generalization (see Generalization gra¬ addiction, human, 563
language behavior, 619, 621 dient, Stimulus generalization gra¬ classification, 545
maintained (see Maintained generaliza¬ dient) Hexagonal apparatus, 126, 133, 139, 147
tion) reinforcement, 217-219 Hibernation, 46
mediated, 509 Grammar, theory, 628-633, 647-649 Hierarchy, in language theory, 630, 633,
problems with concept, 433 Grammatical items, language theory, 630 643-647
semantic, 634 Grasp response, human, development and High carbohydrate diet, and thermo¬
Generalization gradient (see also Stimulus recovery, 19-20 regulation, 162
generalization gradient): Grimacing, human retardate, 564 High-fat diet, and thermoregulation, 162
animal psychophysics, 526-534 Grooming: High protein diet, and thermoregulation,
and chained schedule, 290-291 on concurrent schedules, 246 162
and conditioned suppression, 351 as facultative behavior, 135, 140 Higher-order schedule (see also Second-
inhibitory, and contrast, 82, 85 as interim activity, 17, 137 order schedule):
LSD effects, in rats, 554-555 normal, description, 17 and behavioral structure, 643
maintained, 529-530 Growth: Hippocampus:
postdiscrimination, 451, 493-494 and meal patterning, 38 and avoidance, 605
and stimulus control, 482 and meal size, 35 self-stimulation, 589
Generalization methods, animal psycho¬ GSR (see Galvanic skin response) Hippocampus, dorsal:
physics, 527-530 Guinea pig: theta waves, 119
Generative grammar, theory, 628-633, caloric regulation, 43 Histamine:
647-649 eating behavior, 34—33, 38, 41, 44 and curare, 611
Generative property, natural language, growth, 38, 41 and taste aversion learning, 14
621-622, 636-637, 639-640, 644-647 as herbivore, 48-49 and thermoregulation, 168
Generative semantics, 649 patterning of drinking, 36 Histofluorescence technique, and neural
Generic extension, in verbal behavior, 634 sexual behavior, 21 pathways, 585
Generic nature, stimulus and response, 30 stimulus generalization, 441 History:
Genetics: Gull chick, generalization gradients, 435— individual, as determinant of behavior,
and language, 640-641 436 174-175. 177-178, 184-186, 192, 196-
species differences in avoidance, 407-408 Gustatory pathways, 586 197
Geometric VI schedule, 215 organism, and psychopharmacology, 553
Gerbil: subject, and aversive control, 422, 427
alert posturing, 115-116 Habenular nucleus, 586 “Hits,” animal psychophysics, 516
classical conditioning, 115-116 Habit, 101 Homeostasis;
digging, 115-116 Habituation: feedback in, 31, 146
drinking, 98, 118 to aversive stimulation, 422 and feeding behavior, 32
eating, 98, 114 to drugs, 550 in hypothalamic syndrome, 17
electrical brain stimulation, 111-112 and rate of eating, 175 as motive force, 31
meal patterning in, 35 Haldol (see Haloperidol) and temperature regulation, 158, 156,
paper shredding, 98, 110-111, 114, 118- Hallucinogens: 160, 164-165
119 classification, 545 Homeotherm (see Endotherm)
punishment, 115 and perception, 562 Hopping, schedule-induced, in pigeons,
running, 98, 114, 118 as rein forcers, 557 139
self-stimulation of brain, 589 and stimulus control, 555-550 Hormones;
Gestalt: Haloperidol: and avoidance, 599—601, 604—605
law of common fate, 623 classification, 544-545 and central reinforcement, 579, 589
in verbal behavior, 637 and self-stimulation of brain, 586 and conditioned suppression, 598-599
Gill extension, Siamese fighting fish, 116 Hamster: and depletion, 32
Glandular response, instrumental condi¬ contingent-response experiment, 113-115 measurement and recording, 000
tioning, 609—610 digging, 113, 118 and thermoregulation, 161—162
Globus pallidus, self-stimulation, 590 eating,1 113-115 Huddling, and thermoregulation, 154, 158
Glucagon, and electrical brain stimula¬ face washing, 113-113, 118 Hue, scaling of, 531-332
tion, 579 grooming, 113 Human:
Glucose: rearing responses, 113, lis alcoholism, 503
concentration, and response strength ? scrabbling, 113-114, 118 artificial language learnlngj 689-640,
259-260, 263, 279^280 sexual behavior, 113 643 647
thermoregulation, 159
preference, and insulin injection, 557 autonomic conditioning, 007
“Glue,” reinforcement as, 99, 101 Handedness, 606 aversive control procedures, 417,421
Gnawing, elicited by brain stimulation, “Hard wiring,11 response to stimulus, 12! classical conditioning, galvanic skin
581 Head bobbing, schedule-induced, 128
response, 493
Goal-directed behavior, disruption by le¬ Heart rate: concurrent matching, Q34—239, 249243
sions, 586-588 and conditioned suppression, 352, 398 concurrent superstition, 234-235
Goat; and escape-avoidance, 602
in controlled environment, contingent-
latent inhibition, 487 measurement and recording, 606
operant conditioning, 607-611
response experiments, 107
self-stimulation of brain, 589 delay of reinforcement, 262, 282
Pavlovian conditioning, 61, 89
Golden hamster (see Hamster) drug dependence, 557-558, 563
Heat:
Goldfish: gastrointestinal changes, and avoidance,
as aversive stimulus, 154, 157, 162,
color vision, 517, 522 597
165-169
peak shift, 456 reinforcer grasp response, development, 19
self-stimulation of brain, 590 and drugs, 188-189 incentive motivation, 636
shape perception, 529 swimming response, 261, 263, 282 language, 619-652
thermoregulation, 153, 158 autoshaping, 59, 60, 120, 121 memory, 639-640
Go/no-go method, animal psychophysics, and thermoregulation, 153-158, 161— noise-induced behavior, 418-419
515-516, 524 166 Pavlovian conditioning, 20
Good news, and conditioned reinforce¬ Hemiplegia, 19 psychopharmacology, 563-566
ment, 319, 336 , Herbivores: reaction time, 532
Goose, greylag, egg retrieval, 23 caloric regulation, 47 recovery from hemiplegia, 19
Gonadal hormones (see Sex hormones) feeding patterns, 36, 47-48 retardates, 564
674 Subject Index

Human (cont.) and fixed-ratio behavior, 562 Infusion, intragastric:

self-stimulation of brain, 570, 580, 589- and schedule control, 189, 557 s nutrients, 35
590 Imitation, in verbal behavior, 637, 644, water, 36
shock-elicited behavior, 417-420 647 Ingestion rate (see also Eating):
temporal discrimination, carbon mon¬ Immediacy: curve, 174
oxide effects, 563 negative reinforcement polydipsia, 129-131
thermoregulation, 158, 162-164 human, 262 Inhibition:
verbal behavior, 628-629, 633-640 and response strength, 261-262 absolute vs. relative, 462
visceral conditioning, 607 primary reinforcement, and informa¬ in autoshaping, 61
Hunger: tion, 318 in behavioral contrast, 75-76, 82-85
and conditioned suppression, 353-354 reinforcement among behavioral states, 144-148
as hypothetical drive state, 188 and matching, 233, 234, 251-252, 266 by-product of discrimination, 467
and induced attack, 137 and response strength, 260-262, 266, courtship behavior, 121-122
and induced drinking, 132-133, 137-139 281 and dimensional control, 444-447
and magnitude of reinforcement, 259 Immobility, difficulty in shaping, 177 and induced activities, 141, 144-148
and matching, 236, 269 Incentive: among instrumental responses, 115
Skinnerian analysis, 32 central reinforcement, 580, 582-588 latent, 486-487
Hunting behavior: function of reinforcement, 99-101 among motivational systems, 122
hyena, 49 induced, 636 motor systems, by brain stimulation,
lion, 48 and peak shift, 458-463 587-588
Hydralic models, 11 Incentive contrast: by novel stimulus, 116, 588
Hyena, hunting behavior, 49 and induced aggression, 469 by reinforcement, 264, 272-274
Hyperphagia: and stimulus control, 458 and self-stimulation of brain, 587-588
hypothalamic, 162 Incentive function, reinforcement, 99-101 and stimulus control, 5, 432-476
premigration, 46 Incentive motivation: measurement, 445-447
Hyperstriatum, self-stimulation, 590 central reinforcement, 582-589 terminal response by interim, 142-143
Hypertension, and avoidance, 601-604 verbal behavior, 633, 636 tests for, 470
Hyperthermia: Incentive stimulus: Inhibitory generalization gradient, mea¬
avoidance-induced, 606 and induced behavior, 128 surement, 445-447
human, 162-163 Pavlovian conditioning, 62 Inhibitory stimulus, definition, 444
rat, 162,165-166 polydipsia, 131-132 Inhibitory stimulus control, 432-476
Hypnotic drugs: Incidental stimulus (see Contextual stim¬ Inhibitory stimulus control:
classification, 545 ulus) determinants, 453-456, 476
barbiturates, 544-545 Incremental generalization gradient, defi¬ and errorless learning, 470-474
Hypothalamus: nition, 445 and schedule variables, 458-466
anterior, and thermoregulation, 154, Indifference, on concurrent schedules, 243 Initiation:
156, 167 Indirect variables, and schedule control, meals, 28-40
and central reinforcement, 570, 572, 204-209, 217, 228-229 ratio runs, 38
575-577, 579, 582-583 Individual differences: responding, disruption by lesions, 586-
and depletion-repletion, 32, 46 aversive control, 422, 425 588
electrical stimulation, reinforcer vs. COD in matching, 243 Inner-ear defect, in mouse, 517
punisher, 178, 180, 181 peak shift, 442 Insect, chained reflexes in, 142
lateral: schedule control, 186-187 Instinctive act, 15, 23
and attack behavior, 14—15, 17
Individual subject: Instinctive drift, 3
and autoshaping, 60 behavioral pharmacology, 541-542 Instinctive responses, and theta waves, 119
electrical stimulation, 14-15, 60, 111 intensive study, 2 Instrumental behavior (see also Operant
feeding and drinking, 14, 16-17, 20, Indoleamine, and thermoregulation, 169 behavior):
32, 35, 46, 155 Induced behavior (see Schedule-induced contingencies among, 98-106
lesions, 16-17, 20, 35 behavior) gerbil, 98
and motivation, 14, 17 Induced states, on periodic schedules, Instrumental conditioning (see Operant
self-stimulation, 585, 589 137-138 conditioning)
and thermoregulation, 155-156 Induction (see also Negative induction. Instrumental environment, 98-99, 111, 112
posterior: Positive induction); Instrumental learning, 14, 53
electrical stimulation, 164, 259 Baconina, 2, 12, 28 Instrumental responses (see also Operant
and thermoregulation, 154, 156, 164 and discrimination theory, 75 behavior):
and thermoregulation, 154-156, 164 Infant, thermoregulation, 158-160 role as stimuli, 5
167-169 Infectious disease, and avoidance, 597-598 Insulin;
ventromedial: Inferior colliculus, EEG activity, 605 and choice of sugar solution, 557
lesions, and overeating, 35 Inflection, in language, 620, 623, 632, 639, and electrical brain stimulation, 579
and repletion, 32 643-647 Integration, learned responses, 20, 22
self-stimulation, 589 Inflection ratio, calculation, 342 Integration, levels of (see Levels of in¬
Hypothermia; Influenza, and thermoregulation, 163 tegration)
cats, 166 Intensity:
Information:
drug-induced, 166-169 current, and central reinforcement, 578,
human, 163 and conditioned reinforcement, 303,
313-337 782
rats, 154, 167-169 shock-elicited behavior, 418, 422, 427
Hysteresis effects, in concurrent matching, logical problem, 323-324
in natural language, 620-621 stimulus
242, 254 and overshadowing, 492
negative outcomes, 318-326
and psychophysical scaling, 532
Iconic mapping, in language behavior, Information processing, and stimulus con¬
trol, 458 and reaction time, 526, 532
620, 647, 650 Intensity dynamism effect, auditory
ICS (see Electrical brain stimulation) Information theory:
psychophysics, 524
Id, 9 conditioned reinforcement, 322 Intention, 30
Ideas, and language, 648, 650 language, 640 Intention movements, 139
“Idols of the marketplace,” 13 Informativeness: Interaction:
Iguana, thermoregulation, 157-158, 166 CS in Pavlovian conditioning, 343 activities and states, 144-145
Imipramine: stimuli in automaintenance, 63 behavioral states, 144-148
classification, 545 stimuli in autoshaping, 56-57, 61, 80, classical-operant, and conditioned sup¬
and errorless learning, 553 84 pression, 341, 344, 348-352, 358-360
Subject Index 675

concurrent schedules, 70-80, 233-282 and ratio responding, 212 IRTs/Op, and stimulus generalization,
(see also Matching) shock postponement, 194 437
dimensions of stimuli, 439 Intermeal intervals, 32-40 Islets of Langerhans, 10
eating and drinking, 35, 36, 42-43, 131, Intermittent reinforcement (see also Par¬ Isobias function, animal psychophysics,
135, 139 tial reinforcement): 536
excitation and inhibition, 447-451 central reinforcement, 573-574, 578-579 Isocarboxazid, 545
and stimulus generalization, 493-494 and polydipsia, 131-132, 135 Isomorphism, autoclitic and transforma¬
internal and external stimuli, and rein¬ and schedule control, 201-202 tional theories, 648-649
forcement, 582-585, 588-589 thermoregulation, 164 Isosensitivity curve, animal psychophysics,
multiple schedule, 71-85 (see also Be¬ Intermittent shock schedules, 375-377, 533-534
havioral contrast) 384-386, 397-398 Iteration, theory of grammar, 629
controls for, 72 Internal clock (see Behavioral clock) ITI (see Intertrial interval)
dehnitions, 73 Internal economy, 596
and Herrnstein’s equations, 266-268 Interoceptive discrimination, 610 Jumping stand, 497
and inhibitory control, 458-463 Interocular transfer, line tilt, 528
theories, 75-91, 266-275 Interpeduncular nucleus, 586 k;
among non-contingent responses, IM¬ Interreinforcement time: Hermstein’s amount of behavior
HO concurrent choice, 327-337 constancy, 202-263
operant-Pavlovian differentiation schedules, 224 definition, 257
and autoshaping, 53-91 fixed-ratio schedules, 205, 209-210, 217 Hulks incentive, 99-100
and conditioned suppression, 341, 344, interval and time schedules, 214 Kantian a prioris, and verbal behavior,
348-352, 358-360 and response rate, 210 638, 641
psychophysiological processes, 608, 610- and responses per reinforcement, 210- Kernel sentences, language theory, 629-
611 211 630, 032-033, 643, 648
punishment paradigm, 428 as schedule variable, 204—205, 209—211, Key pecking (see Pecking, Pigeons)
among reinforcers, and matching, 252- i 214, 228-229 Key pressing, human, and matching, 239
254, 264-265 variable interval schedules, 214—215 550
Kidney, excretion of drugs,
response-shock and shock-shock interval, Interresponse time: Kidney function, instrumental condition¬
370-371 and concurrent choice, 255-256 ing, 609-610
running and drinking, 146-148 conditionable response unit, 224 Kinesis, 158
sequential: contingent-response experiments, 109
behavior and environment, 177 drug effects, 559—560 Language:
among induced activities, 144-146 and generalization gradient, 437-438 acquisition, 619-627
shock-elicited behaviors, 420-426, 428 as reinforced response, 223-224 artificial, 639-640, 643-647
stimuli, and stimulus control, 496, 507 as response unit, 223-224, 263 Operant analysis, 5, 619-627, 633-640
stimulus-reinforcer, response-reinforcer, role in response rate, 238, 263-265 psychology of, 619-652
66-71 as schedule variable, 204, 224-225 theories, 619-652
terminal and interim activities, 132-133, as stimulus, 223 Latency, response,
139-140, 143 theoretical unit, 223-225 and interreinforcer time, 211
terminal and interim states, 139-140, unit of behavior, 222-225, 263 and stimulus intensity, 526-527, 532
143 Interresponse time schedule (see also Latent inhibition, 486M87
two kinds of peck, 67 Differentiation schedule, DRL Law of availability, 47-48
Interburst interval, in contingent-response schedule, DRO schedule): Law of common fate, 623
experiments, 110-112 concurrent, and matching, 251, 255-256 Law of effect, 233-282
Interchangeability, stimuli, responses and definition, 203 Law of effect:
rewards, 13, 91, 112, 113, 122, 407- regenerating property, 212 and absolute response rate, 257-263
408 and response strength, 263-264 autoshaping, 53~54, 62-63
Interdimensional training: Inter trial interval: correlation-based. 111
discrimination autoshaping, 57-58, 66, 78, 127 as equilibrium principle, 141
description, 439-440 escape procedure, 368 quantitative analysis, 4, 132, 140, 233~
effects, 444 food deliveries during, 63, 67-68 282
and peak shift, 454-455 Interval schedule (see also Fixed-interval, Thorndike’s statement, 99
and stimulus control, 494-497 Variable-interval, Random-inter¬ symmetrical , 101—102, 110
Interference hypothesis, conditioned sup val): weak, 100-101, 113, 122
pression, 351-353, 357-359 definition, 2^2 Laws of behavior, self-sufficiency, 28
Interfood interval, and sequential inter¬ regenerating power, 212 LaWs of Operant behavior, 148
actions, 147 and time schedule, 214 Laws of reflex strength, 30
Interim activities, 123-148 Intestinal contractions, operant condition¬
Learned rcleasers;
Interim activities: ing, 571, 608 and autoshaping, 62, 70
competition with terminal, 132-133 Intestinal load, 33 and Pavlovian conditioning, 70
and contrast, 273-274 Intracranial stimulation (see Electrical Learning:
control of, by food, 131-132, 140 brain stimulation) artificial language, 639-640, 643-647
definition, 126 Intradimensional shift, 487 Constraints on (see Constraints)
examples, 128-129, 137 Intradimensional training: drug effects, 560-562
motivation, 128, 132, 137-139 description, 439-440 as modification of habitat, 49
schedule-induced, 125-148 and errorless learning, 466 natural language, 619-627
SAperiods, 135-136 and peak shift, 453—454 two types, 54
variables affecting, 128-138 and stimulus control, 493-494 Learning disability, human, and drugs,
Interim period: Intrapeduncular nucleus, stimulation, 585 564
aversiveness, 139-140 Intraperitoneal route, drug administra¬ Learning effects, animal psychophysics,
DRL schedules, 135, 141 tion, 545 525
FI and VI schedules, 133, 135-138, 141 Intraverbal, in verbal behavior, 634, 646- Leash-pulling, squirrel monkey, 183-184
properties, 134-139 648 Lesions (see Brain damage)
Interim states: Invariance, schedule performances, 187 Leucocytes, and pyrogens, 166
interaction with terminal, 139-140, 143 Inverse hypothesis, stimulus control, 507- Levels of function, nervous system, 8
properties, 137-139, 144-146 508 Levels of integration, of operant, 7-24
Interlocking schedule: IRI (see Interreinforcement interval) Lever contact response, rats, 60, 65, 81-
fixed-ratio, fixed-interval, 213 IRT (see Interresponse time) 82, 120
 676 Subject Index

Lever pressing (by rats, for food, unless Lizard, thermoregulation, 153, 157-158, and stimulus control, 482-483, 491-492,
otherwise stated): 161-164 500, 504, 506
acquisition, 561 Local contrast (see also, Transient con¬ tone by key stimuli, 491
and aversive control, 421-425, 428 trast) : Massed extinction, and peak shift, 456-457
avoidance, 116-118, 185, 194-195, 260, additivity theory, 86-88 Masseter muscle, 417-418
267-268 definition, 77 Matched shock, extinction procedure, 379-
cebus monkey, 102 and Herrnstein’s equation, 268 380
college students, 103, 107 necessary conditions, 78 Matching:
conditioned suppression, 89, 341-356 and overall contrast, 77-78, 86-88 and behavioral contrast theories, 272-
and contrast, 73-74, 81-82, 89-91 pigeons, 75-78, 85-88 275
drug effects, 561 rats, 78, 85 concurrent chained schedules, 297-298,
escape, 261-262, 282, 319, 367-368, 383- Localizability (see also Diffuseness): 329-333
387 stimulus, and contrast, 82 concurrent schedules:
and experience, 175, 177 US in autoshaping, 60, 69 evidence, 239-243
in extinction, 205 Localization, stimulus on manipulandum, generality, 243, 248-256, 275, 276-277
hamster, 114 490 vs. maximizing 245-246
heat reinforced, 154, 162, 164 Local rate of reinforcement: measurement, 235, 248
heat avoidance, 165, 168 concurrent schedules, 246-247, 272 vs. multiple schedules, 272-275
and induced drinking, 132, 138 fixed-interval schedules, 216 necessary conditions, 234, 242-244
as interim activity, 133 and schedule control, 214-218 negative reinforcement, 254
macaque, 258 variable-interval schedules, 214-215 and punishment, 253-254
monkey, 89, 178, 185, 194-195, 352 variable-ratio schedules, 217 and reinforcement immediacy, 251-
pigeon, 73, 82, 91 Local rate of response: 252, 329-333
postreinforcement pause, 142 concurrent schedules, 235, 238, 244-247 and reinforcement magnitude, 233-
and reinforcement immediacy, 260-263, 274 234, 248-251
281 multiple schedules, 272 discrete trial procedure, 237, 244-246
and reinforcement magnitude, 258-259, and probability of reinforcement, 214- humans, 234-239, 242, 243
278-282 216 and IRT reinforcement, 225
rhesus monkey, 352 second-order schedules, 302-303 and k-parameter, 262-263
shaping, 177 Lock and key, courtship sequences, 142 and multiple schedules
squirrel monkey, 421-425, 428 Locomotion: and component duration, 269-272
stimulus generalization, 437 amphetamine effect, 554 conditions for, 269
and sugar concentration, 259-260, 263, and aversive control, 418-426 and contrast, 79-80
269, 279-280 Locus coerleus: and Herrnstein’s equations, 260-270
as terminal response, 133, 145-146 and central reinforcement, 586 negative reinforcement, 373-374, 382-
and thermoregulation, 164-165, 167 self-stimulation, 590 383, 386-388
token reinforcement, 306 Logarithm: responses, concurrent schedules, 235-
Lexeme, 620 drug dosage regimen, 551 237, 242-244, 272-274
Lexical items, 630 in matching equation, 238-239, 243, responses and time, 237
Lexical system, 620 247-250 concurrent schedules, 238, 246-248,
Lexicon: Long box, autoshaping with, 63 250, 255
definition, 620 Lordosis response, rodents, 21 multiple schedule, 272
use, 648 LSD: time:
Librium (see Chlordiazepoxide) classification, 545 concurrent schedules, 238-239, 246-
Licking (see also Drinking, Polydipsia, and stimulus control, 554, 556 248, 254, 272, 274
Schedule-induced behavior): Luminal (see Phenobarbital) in interresponse class, 264
air, by rats, 138 Lysergic acid (see LSD) Matching law:
in automaintenance, 65, 120 deviations from, 242-243, 256, 266, 270-
avoidance response, in rats, 118 Macaca nemistrina (see Macaque) 272, 276-277
contingent-response experiments, 104, Macaque: as empirical law, 275-276
108 EEG activity, 605-606 equations, 233, 236, 238, 245, 248, 253,
as interim activity, 133 magnitude of reinforcement, 258, 263, 255, 257, 275, 277, 329
rate, and polydipsia, 130-131 278-280 generality, 243, 248-256, 275-277
as terminal response, 133 thermoregulation, 153, 159 as intuitive assumption, 275-276
Light intensity (see Brightness) Mach bands, 452 linear, 233-234
Limbic pleasure area, 582 Macrosaccadic eye movement, 237 proportional ratio, 238-239, 242, 247-
Limbic system, and central reinforcement, Magnesium pemoline, 545 249, 252-257, 277
572, 585 Magnitude of reinforcement (see Rein¬ as tautology, 276
Limited availability schedule, and induced forcement, magnitude) as theoretical law, 276
behavior, 131-132, 135-136 Maintained generalization procedure: Matching to sample:
Limited hold procedure, animal psycho¬ animal psychophysics, 529-530 animal psychophysics, 518, 525
physics, 527 description, 434-435 and conditioned reinforcement, 307-308
Limited opportunity schedule, and nega¬ Maintenance (see Automaintenance) pigeons:
tive reinforcement, 398-406 Mammals, thermoregulation, 158-162, 167 chained schedule, 292
Limulus, 9 Mand, 633, 636-637 second order schedule, 301
Lindsley manipulandum, 602 Manual manipulation, and aversive con¬ and verbal behavior, 638
Linear VI schedule, 215 trol, 418-430 Mathematical analysis. Law of effect, 4,
Line orientation: Marijuana: 132, 140, 233-282
generalization, 440, 441, 445, 454-455, behavioral pharmacology, 545, 552 Mathematical model:
458, 482, 494, 501-502, 527 reinforcer, 557 avoidance, 365
and visual vertical, 527 and stimulus control, 555 generalization gradient, 449
Linguistics, 619-627, 628-652 Markov process, and behavioral sequences, Mathematical theory, discrimination
Lion: 142 learning, 448-451
feeding behavior, 47 Marplan (see Isocarboxizid) Maximization, energy yield/time expended,
hunting behavior, 48 Masking: 47-49
Listener, and verbal behavior, 649-651 airflow by key stimuli, 491 Maximization function, 47-49
Lithium carbonate, 545 line orientation by color, 482-483, 491 Maximizing vs. matching, on concurrent
Lithium chloride, 484 and overshadowing, 492-493, 500 schedules, 245-246, 254
Subject Index 677

Maze: Methodology: Monoamine oxidase inhibitors, and self-

central reinforcement, 572 central reinforcement, 572-574, 589 stimulation of brain, 585
contrast effects, 86 and schedule control, 204-229 Monochromatic light, rearing, and gen¬
drug effects, 555 Methylphenidate: eralization, 485-486
Meal: classification, 544-545 Mood, and schedule-induced behavior,
criterion, 36 and learning disability, 564 137- 138, 144
duration, 28, 33, 42 Metonymic extension, verbal behavior, Morphine:
frequency, 28, 33-38 634, 637 classification, 545
initiation, 32, 36, 38, 40 Metronome, 138, 484 dependence, 179, 554, 557
patterning, 28-30, 32-37, 42-45 Microstructure, stimulus generalization and fixed-ratio behavior, 551
and polydipsia, 130-132 gradient, 437-439 and punishment, 558
size, 32-35, 37-38, 130-132 Midbrain, and electrical stimulation, 583, Morphonemic rule book, in language be¬
termination, 32, 36, 38, 45 585 havior, 648
as unit of analysis, 31-32, 34, 45-46, 49 Milieu externe, 33 Motility, gastric, dogs, 20
Measurement: Milieu interne; 33 Motion aftereffect, monkey, 523
central reinforcement, 572-574 Milieu therapy, 565 Motivation:
conditioned suppression, 348—351, 360 Miltown (see Meprobomate) and conditioned suppression, 353~356,
drug effects, 189, 192-193, 202, 541, 545- Minimal unit hypothesis, 67-68 360
547 Minor tranquilizers, 545, 555 drug effects, 559~56Z
physiological changes, 606 Mirror image, and interocular transfer, and electrical brain stimulation, 570,
preference structure, 108-110 528 576, 579-580, 58g-589
sensory thresholds, 515-525 Misbehavior of organisms, 3, 7, 13, 23 feeding behavior, 49
stimulus control, 433^136, 482-483 ‘‘Misses/r animal psychophysics, 516 homeostasis as, 31
Mechanical models, animal behavior, 29- Mixed schedule: and interim activities, 128, 132, 137-139
31 and chaining, 221—222 and matching, 269
Mechanism, behavioral and psycho- conditioned reinforcement, 320-321, as maximization function, 49
pharmacology, 551—560 323-324, 336 and the operant, 1^—13, 13, 19,
Mechanism of action, psychopharma- and contrast, 74 problems with concept, 14-15
cology, 541 drug effects, 556 and schedule-induced behavior, 128“
Medial forebrain bundle: observing responses, 320-321, 323, 324 132, 137-139, 144
central reinforcement, 577, 585, 589, 607 Model: and schedule performances, 188
and thermoregulation, 156 central and conventional reinforcement, Motivation hypothesis!
Medial geniculate, EEG activity, 605 584-585 conditioned suppression, 351, 353-357,
Mediated generalization, 509 as method of synthesis, 10 358
Mediating responses, avoidance theory, Modulation; induced drinking, 130,132
364, 397, 400 behavior Motivational properties, induced States,
Mediation: by electric shock, 180, 183, 193, 195- 138- 140
in autonomic conditioning, 607-608, 610 196 Motivational state, as behavioral state, 144
in syntax learning, 624 by environment, 410 Motor behavior, and self-stimulation of
Mellaril (see Thioridazine) by food presentation, 182 brain, 586-588
Memory: response rate, chained schedules, 293, Mouse;
and stimulus control, 457-458 299 auditory sensitivity, 517
and verbal behavior, 637-638 Modulus, for measuring behavior, 262 biting attack, 416-417
Mentalese, and verbal behavior, 635, 648- Molecular weight, drugs, 549 brain biochemistry, 606
651 Momentary response probability, 108-109, conditioned suppression, 517
Mentalism, and language, 640-642 112 killing, by rat, 17
Mentalistic concepts, how avoided, 29 Mongolian gerbil (see Gerbil) meal patterning, 35
Meperidine, 545 neurological mutant, 517
Monkey (see also Squirrel monkey, Rhesus
Meprobamate; monkey, Cebus monkey, Macaque) thermoregulation, 153, 159,167
classification, 545 Monkey: Muller-Lyer illusion, 526
and conditioned suppression, 356 Multiple causation, verbal behavior, 646-
auditory intensity discrimination ROG
and schedule control, 189, 191, 557 curves, 534-535 647
Mercury vapor, behavioral effects, 562-563 Multiple concurrent schedule, matching,
auditory thresholds, 516, 519, 521
Meriones unquiculates (see Gerbil) avoidance, 366, 406 249
Mescaline, 545 central reinforcement, 578 Multiple schedule;
Mesocricetus auratus (see Hamster) animal psychophysics, 515-516
Metabolic rate, and thermoregulation, 154,
cocaine reinforcement, 250-251 avoidance,
color vision, 518
161-162, 168 behavior pharmacology, 551-552, 556-
conditioned suppression, 89, 352, 358-
Metaphoric extension, verbal behavior, 557, 559
634 360 central reinforcement, 575
dark-reared, 486
Metastability, response patterns, 192 and chained schedule, 290
equal-loudness contour, 526
Methadone: chained and tandem, 294
flicker discrimination, 5ZZ component duration
classification, 545 matching on concurrent schedules, 250—
and heroin addiction, 563 and contrast, 79, 87, 90
251
Methanol, 549 and matching, 269-272
motion aftereffect, 523 negative reinforcement, 382—383
Methamphetamine: negative reinforcement, 390, 395
classification, 545 and conditioned reinforcement, 317,
observing responses, 320-321
323-324, 336
and punishment, 560 pica, 137 conditioned suppression, 345, 347
and schedule control, 559 positive conditioned suppression, 358- contrast on, 75-91, 266-275
and ratio strain, 543 360 drug effects, 188-191, 551-552, 556-557,
Methaqualone, 545 schedule-induced drinking, 138 559
Methedrine (see Methamphetamine) second-order schedule, 302 and Herrnstein’s equations, 266-268,
Method of constant stimuli, animal psy¬ self-stimulation of brain, 578, 589-590 270-272, 373-374
chophysics, 521-522 spiral aftereffect, 529 induced behavior, 132, 136-137, 138
Method of limits, animal psychophysics, stimulus generalization, 490 interactions on (see Interaction)
520-522 tactual discrimination, 517 matching, 269-272, 373-374
Method of successive approximations, in thermoregulation, 164, 167 negative reinforcement, 367, 369, 381—
shaping behavior, 54, 62 visual fixation training, 523 387
678 Subject Index

Multiple schedule (cont.) and Pavlovian conditioning, 364-365, Noradrenergic pathways, 571, 585-588
and peak shift, 441-443, 458-463 390-393, 395-398, 400 Norepinephrine:
relation to concurrent, 78-80, 266-270 procedures, 364-406 and escape-avoidance, 605
relative response rates, 268 shock-delay procedure, 965, 370-371, and heart rate conditioning, 608
responses per reinforcer, 211-212 380, 383, 388-391, 400, 402, 407, 409 and stimulus control, 555-556
response-produced shock, 195 shock delay vs. frequency reduction, and thermoregulation, 167-169
Multiple-stimulus method, generalization 402-406 Nose-key pressing:
gradient, 434 shock-deletion procedure, 365, 371-373, cats, 520
375, 382, 384, 386-387, 398-401, 403 monkeys, 58
Nalorphine: shock-density reduction, 375 rats, 516
and conditioned suppression, 554 shock-frequency reduction, 365, 373-378, Not-responding, and temporal patterning,
and morphine dependence, 179, 557 380-383, 386-387, 392, 400-406 217
reinforcer vs. punisher, 179, 192 similarity to positive, 365-368, 373, 382, Novel stimulus:
Naloxone, 557 384, 386, 406 effects on ongoing response, 116
Narcosis, cold, 157 and stimulus generalization, 450 inhibition by, 588
Narcotics (see also Opiates, Analgesics): and supplementary positive reinforce¬ Noxious stimuli (see also Electric shock,
classification, 545 ment, 404-405 Escape, Avoidance, Aversive control.
as reinforcers, 557 two-factor theory, 364-365, 393, 396-398, Negative reinforcement):
tolerance, 550 400 and maintenance conditions, 182
Nardil (see Phenelzine) two modes of, 387-388 Noyes pellets, 43, 45, 89
Nativism, and language, 641 Negative stimulus, and conditioned rein¬ Nursing, rats, 18-19
Naturalistic environments, behavior in, 4, forcement, 318-326 Nutritional quality, and eating patterns,
28-50 Nembutal (see Pentobarbital) 33, 47-49
Natural selection, in language learning, Neocortex, and instrumental autonomic Nuzzling response, chicks, 65
626 conditioning, 608
NE (see Norepinephrine) Neostriatum, self-stimulation, 590 Obesity:
Necessity, as driving force of behavior, 31 Nephrectomy, bilateral, and drinking, 36 and electrical brain stimulation, 14, 579
Neck stretching, as interim activity, 137, Nervous system, development, 8, 17-19 experimentally produced, 597
139 Nest building, gerbils, 115 Object carrying, elicited by brain stimula¬
Negative automaintenance, 13, 23, 63-65, Neural basis, reinforcement, 571, 574, 580- tion, 571, 582
128 (see also Omission effect, Omis¬ 581, 585-589 Object substitution, Pavlovian condition¬
sion training) Neural pathways, and self-stimulation, ing, 62
Negative behavioral contrast, 72-75, 78, 585-588 Observation method, several responses at
85-86, 267-268 Neurochemistry, and thermoregulation, once, 4, 126-127, 139, 147-148
additivity theory, 85-86 167-169 Observing responses:
definition, 73 Neurons: animal psychophysics, 517-519, 523
logical problems, 75, 86 catecholaminergic, 571, 585-588 and attention, 30
pigeons, 75, 78, 85-86 temperature sensitive, 154, 158, 160, 166, and conditioned reinforcement, 313-315,
and positive contrast, 75, 85-86 169 318-326, 332, 336-337
rats, 78, 85-86 Neuropharmacology, catecholaminergic in discrimination training, 508
Negative contingencies, in omission train¬ pathways, 585-586 maintaining variables, 322
ing, 99, 128 Neurophysiological system, transient Obstruction box, and central reinforce¬
Negative feedback: changes, 597 ment, 572, 574
in homeostasis, 31 Neurotransmitters: Octopus:
as self-inhibition, 144-148 and drugs, 544 failure of temporal discrimination, 140
in thermoregulation, 153-154 and thermoregulation, 167-169 shape perception, 529
Negative induction: Neutral stimulus, after errorless learning, Oddity problem, animal psychophysics,
definition,72-73 467, 471 518, 525
pigeons, 82-83, 89-90 Neutral zones, and stimulus control, 453 Odorant, stimulus control by, 517, 523
rats, 74, 82 Nialamide, 545 Olfaction:
Negative reinforcement, 364-410 (see also Niamid (see Nialamide) and central reinforcement, 582
Escape, Avoidance, Electric shock. Nigrostriatal pathway, 586-588 and lateral hypothalamus, 16
Aversive control) Noise escape, 282 Olfactory bulb, self-stimulation, 589
access to, 376 Noise-induced behavior, 418-419 Olfactory bulbectomy, and meal fre¬
acquisition, 404-409 Nonchaining delay, and gradient of rein¬ quency, 35
added cues, 365, 381-396 forcement, 217-218 Olfactory discrimination:
concurrent schedules, 254, 375, 386, 388- Noncontingent schedule (see Time sched¬ pigeons, 517
392 ule) rats, 523
contingency vs. frequency, 398-399 Noncontingent shock, and negative rein¬ Olfactory pathways, 586
definition, 364 forcement, 378-384
Olfactory tubercle, self-stimulation, 589
density, 369-370, 375 Noncontingent stimuli, 2 Omission, reinforcement, second-order
discriminative stimuli, 392-396 Nondeprived animals:
schedule, 301
escape procedure, 367-368, 383-387 large ratios tolerated by, 36, 38, 43 Omission effect (see also Omission train¬
extinction after, 377-380 meal patterning, 30, 34-35, 45
ing, Negative automaintenance):
frequency, 260, 369-370, 398-399 Nondifferential reinforcement:
definition, 433 automaintenance, 63-68, 128
illustrative experiments, 365-367 Omission training (see also Omission
immediacy: and generalized gradient, 436, 439-441,
454-456 effect, Negative automaintenance):
human, 262, 282 chicks, 65, 120-121
and overshadowing, 507-508
and response strength, 261-262, 282 dogs, 63
and stimulus control, 488-490
intermittent shock schedules, 375-377, pigeons, 63-68, 119
Non-matching to sample, 307-308
384-386 rats, 65
Non-reinforcement, as de-encephalization,
magnitude: 23 squirrel monkeys, 65
and response strength, 261, 263, 281- Nonsense syllables, 639-640, 643-647 stimulus-reinforcer relations, 63-64
282 Noradrenalin, and self-stimulation of Omnivores:
scaling, 387-388 brain, 571 caloric regulation, 47
and matching, 254, 269-270, 373-374, Noradrenergic hypothesis, reward, 571 feeding patterns, 47
382-383, 386-388 Noradrenergic neurons, and self-stimula¬ Ongoing behavior:
noncontingent shock, 378-380 tion of brain, 585-586 and negative reinforcement, 409-410
 Subject Index
679

and reinforcement vs. punishment, 176, Overmatching, concurrent schedules, 242, in behavior contrast, 81, 84-85, 88-89
180-182, 191, 197 244, 248, 254-255 blood pressure, 89
Ontogenic contingencies, 580 Overshadowing: and conditioned suppression, 88-89
Operant (see also Instrumental): and blocking, 500 description, 61-63
as behavioral state, 144 and masking, 492-493 dogs, 61, 138, 340, 483-484, 489, 492
concept, 3, 7-8, 11-14 necessary conditions, 507 expectations in, 138
as conditionable unit, 222 and stimulus control, 492-493, 496, 499- eye blink, 61
criterion of motivation, 13-14 501, 503, 506 fear, and conditioned reinforcement, 315
de-encephalization, 22-23 theory, 506-508 galvanic skin response, 01
definitions, 175, 186 Overtraining, 23 heart rate, 61, 89
as emitted, 174-175 human infants, £0
encephalization, 20-21 Paced schedule, and choice, 255-256, 266 mechanisms, 61-62
as functionally identifiable class, 175 276-£77 and negative reinforcement, 364—365,
hierarchical structure, 22-23 Paced VI schedule, and IRT reinforce¬ 390-393, 395-398, 400
history, 8-11, 28-31, 53-54, 174-175 ment, 223 and operant behavior, 53-91
and human behavior, 13 Pacing procedure, 345, 354 partial reinforcement, 61
on interresponse time schedules, £55 Pacing response, schedule-induced, 128, relation to operant, 54—55, 62, 71, 88—
levels of integration, 7-24 137 89, 91
physiological thinking and, 8-11 Paired-associates learning, 637 salivation, 9, 53, 61, 138, 340, 489, 493
problems with concept, 7, 13-14, 23 Pairing: sexual behavior; 59
relation to general psychology, 2, 4—5, 8 concurrent chains, and conditioned re¬ and shock.correlated stimuli, 390=391
self-stimulation of brain as, 580 inforcement, 533-336 stimulus substitution in, 61—82
Skinner’s justification, 11—12 and conditioned reinforcement, 303-308 trace, and shock-delay, 400
and terminal response, 127 US and CS, in autoshaping, 55—58? 127 Pavlovian hypothesis, contingent-response
as unit, 2, 174-177, 222 Pairing hypothesis, conditioned reinforce. experiments, 119-123
Operant analysis, basic assumption, £9 ment, 313, 315-318, 325, 334-336 Pawing!
Operant behavior: Palatability of food: in automaintenanee, 05
and chronic somatic changes, 597 and diurtlal eating cycle, 44 schedule-induced, 128, 133
and classical conditioning, 53-91, 340- and induced behavior, 132-133, 138 PD (see P§eudoai§trinainaU9n)
361 PaleostriatUM, self-stimulation, 590 PDC (see Postdiscrimination gradient)
and conditioned suppression, 340, 341- Pallidum, self-stimulation* 590 Peak shift;
351 Pancreatectomy, 10 and area shift 442, 453
language as, 619-627, 633-640 Panel pressing, dogs, on shock delay sched¬ and behavioral contrast, 44£, 454
laws,. 148 ule, 390 by-product of discrimination, 467
and Pavlovian conditioning, 53-91, 340- Panting, and thermoregulation, 154, 167 compounding theory, 439-463
360 Paper shredding, gerbils, 98, 110-111, 114- concurrent schedules, 463-464
technical advances, 186-187 115, 118-119 determinants, 44£, 453-450, 470
and thermoregulation, 153-169 Parallels; double, 442
Operant conditioning- applied to operant, 11 and errorless learning, 467, 471
autonomic responses, 000-011 as method of synthesis, 10 goldfish, 456
and autoshaping, 62, 66-67, 70, 120 between recovery and development 8 and interdlmemional training, 454-455
history. 8-11, 28-31, 53-54, 174-175 17-22 and Intradimensional trainings 433—454
definition, 175 649-651
Paraphrase, in language behavior, and massed extinction, 456
parameters, and conditioned suppres¬ Parenteral route, drug administration, 545 mathematical constraints} 448-449
sion, 344-348, 351, 300 Paricto-occipital activity, cat, 605 necessary and sufficient conditions, 456
and physiological states, 596-611 Partial reinforcement (see also Intermit¬ negative, 443
relation'to Pavlovian, 54-55, 62, 71, 88, tent reinforcement, Reinforcement): and negative reinforcement; 451, 4£3
91, 120 advantage of schedules, 299 positive, 441-443
visceral responses, 606-611 and autoshaping, 61 and schedule variables* 458-460
Operant discrimination, and central rein¬ brain stimulation, 576; 578-579} 589 Spence Js theory, 448—449
forcement, 575 and persistence, 509 tone frequency, pigeons, 441
Operant methods: Partial reinforcement effect; wavelength; pigeons, 9, 441—44g
advantages,« 2f 5, 7 m autoshaping, 61, 64, 119 Peck, duration;
aversive control! 415-430 generalized, 509 In autyghapltig, £4, §7-§§, §1
animal psychophysics, 515-537 operant conditioning; 61 and esntMgf, 87-88
central reinforcement, 572- 580 Pavlovian conditioning, 61 on FI and FR, 67
Opiates (see also Heroin, Morphine, Ftar= and resistance to extinction, food 7JS. water reinforcement, 58—59, 67,
codes): Path-independence, feeding behavior, 46 81, 119 ' ’
classification, 544-545 Pattern: In omission training;
self-administration, 545 drinking, 36, 40, 42 Peeking (by pigeons, for food, unless
Opportunity hypothesis, induced drinking, meals, 28-30, 32-37, 42-45 otherwise stated);
130-132 responding acquisition, 55, 57, 62-63
Optimal-duration model, response prob¬ and matching, 255-256, 276 aggressive, in pigeon, 59
ability, 109-113, 122-123 in preshock stimulus, 350 automaintenance, 6£-<?8
Optimal level theory, 99 and schedule performance. 213—221, autoshaping, 3, 7, 13, 24, 54-62, 69-70,
Optimization, cost-benefit, in feeding, 46 265 81, 119
Order effects, concurrent matching experi¬ Pause: chicks *
ments, 242, 254 interresponse (see Interresponse time) autoshaping, 59, 60, 65, 120, 121
Oral route, drug administration, 545 postr enforcement (sec Post-reinforce¬ thermoregulation, 159-160
ment pause) conditioned suppression, 345, 346, 351,
Orientation response:
Pavlovian conditioning (see also Classical 353, 357
animal psychophysics, 523
conditioning): effect of experience, 177
in autoshaping, 65, 69
aggressive behavior, 59 errorless learning, 464-480
and contrast, 89
and altered physiological states, 596 extinction, 205
and hypothalamic lesions, 16-17 in automaintenance, 62-63 as interim response, 128, 133, 140, 144
Orrery, language model, 623 in autoshaping, 53-55, 58, 60-61, 80-81, maintenance, and Pavlovian condition¬
Outcome-dependency, instincts and oper¬ 119-122 ing, 63, 81, 88, 91, 119, 120
ants, 15 and avoidance, 364-365, 393, 396-398, negative reinforcement, 406-407
Overextension, verbal behavior, 634 400 operant vs. reflexive, 67
 680 Subject Index

Pecking (<cont.) Phylogeny of aggression, 417 punishment, 65, 253-254

as Pavlovian response, 54-55, 58, 63, 69, Physiological psychology, relation to reinforcement magnitude, 248-251, 259
81, 89, 91, 119 operant, 5 response-produced stimuli, 221-222
positive conditioned suppression, 359 Physiological states: schedule-induced behavior, 126-129,
postreinforcement pause, 142-143 altered, 596-618 136-139
as prototypic operant, 54-55, 71, 100 durable, 597-598 second-order schedules, 299-306
rate: experimental alteration, 596-611 self-stimulation of brain, 572, 589-590
as controlling stimulus, 504 measurement, 606 stimulus generalization, 434-443, 450,
as constant, 238-246 technological and methodological devel¬ 454-456, 458, 482, 489, 490, 493-496,
and reinforcement magnitude, 259, 278 opments, 606 501-502, 527-528, 530-532
schedule-induced, 128, 130, 140, 144 transient, 597 time discrimination, 218, 219
shaping, 177 Physiologizing, 9 treadle pressing, 82, 89-90, 236
as terminal response, 126-128, 133, 135, Physiology vs. operant in analysis of be¬ VI vs. VR performance, 210
140-144 havior, 8-12 visual acuity, 517-518, 521,
voluntary vs. reflexive, 54-55, 67 Pica, schedule-induced, 23, 137 wild, 236
Pedal pressing, pigeon, observing re¬ Pig- „ wing flapping, 59
sponses, 318, 320 rooting response, 13 Piloerection, and conditioned suppression,
Pedal pushing, dog, avoidance, 605 thermoregulation, 153, 158-159 351
Pentobarbital: Pigeon: Pimozide, 586
classification, 544-545 absolute response rate, 257-258 Piperazine, 544
clock schedule, 555 adventitious punishment, 184 Placebo effect, 554
extinction effects, 561-562 adventitious reinforcement, 185 Plasticity principle, response units, 222
fixed-interval behavior, 559 aggressive behavior, 23, 59, 120, 136-137, Pleasantness, thermal stimuli, 162-164
fixed-ratio behavior, 544, 546, 548-549, 139, 468-469, 552 Poikilotherm (see Ectotherm)
559, 562 auditory stimulus control, 485, 487-491 Polydipsia (see also Drinking, Licking,
and punishment, 558 494-495 Schedule-induced behavior):
as reinforcer, 192 automaintenance, 62-68 ethanol, in rats, 561
schedule control, 542-543, 546, 559 autoshaping, 13, 23, 54-62, 68-70, 80-81, psychogenic, 23
and stimulus control, 555 119 schedule-induced:
and thermoregulation, 556 avoidance, 13, 120, 450-451 behavioral determinants, 129-131
Peptic ulcers: behavioral contrast, 76-78, 84-86, 266- and body weight, 132
and aversive control, 597, 604-605 275, 442, 454 chimpanzees, 129
and avoidance procedure, 597, 605 blocking, 499-500 control by eating, 129-135, 138
monkeys, 605 brightness discrimination, 521, 532 doves, 129, 136
Percentage reinforcement, brief stimulus central reinforcement, 572, 589 measurement, 129-130
schedule, 316 chained schedule, 291-298 motivation hypothesis, 130, 132, 138
Percentile schedule, and IRT reinforce¬ classical conditioning, 489 pigeons, 129, 136
ment, 224 clock schedule, 555 and postprandial drinking, 129-131
Perception: COD in matching, 243 rats, 129-130, 132, 136
animals, 526-530 conditioned suppression, 345-346, 351, regulatory mechanisms, 129
and drugs, 562 353, 357, 517 squirrel monkeys, 129
Perceptual learning, and language, 643- concurrent chained schedule, 327-337 stimulus control of, 129, 131
647 concurrent schedule: theories, 130-132
Performance theory, language behavior, brief stimuli, 307-308 Polyribosomes, mouse brain, 606
629 matching, 236-256 Population, matched to available re¬
Periodicity, responding on FI, 208-209 concurrent superstition, 234-235 sources, 33
Periodic schedules (see also Fixed interval, conditioned enhancement, 89 Positive behavioral contrast:
Variable interval, Fixed time, Vari¬ conditioned reinforcement, 318-337 definition, 73
able time): conjoint schedule, 307-308 and induced drinking, 132, 138
and induced behavior, 126-140 dark adaptation, 521-522 and matching low, 266-268, 273-274
Peripheral resistance, and cardiac output, delay of reinforcement, 219 and negative contrast, 75, 85-86
602-603 drug effects, 552-555, 559-560, 562-563 pigeons, 73-74, 77-78, 80-83, 266-268,
Permitil (see Fluphenazine) electrical brain stimulation, 252 273-274
Perphenazine, 544-545 errorless learning, 464-480 and punishment, 77
Persistence: escape, 254 rats, 72-74, 78, 82-86
and central reinforcement, 576-578, 589 fixed-time schedule, 126-127 Positive conditioned suppression:
and partial reinforcement, 509 FR FI performance, 187 and autoshaping, 359
running as facultative activity, 147-148 interocular transfer, 528 and Pavlovian conditioning, 88-90
Pharmacology: interresponse time as basic, 263-264 and punishment, 357-360
behavioral (see also Behavioral phar¬ key pecking (see Pecking) schedule effects, 359
macology, Drugs, Psychopharmacol- matching: Positive feedback, in autoshaping, 70
ogy): concurrent vs. multiple, 272 Positive induction, 72-73
behavioral, 5, 188-193, 202, 540-569 concurrent schedules, 236-256 Postdiscrimination gradient, 451, 493-494
and conditioned suppression, 349, 350, multiple schedules, 269-272 Posterior marginal gyrus, EEG activity,
355-357 and reinforcement immediacy, 251— 606
thermoregulation, 167-169 252, 266 Post food period, and polydipsia, 131
Phenelzine, 545 meal patterning, 35 Postprandial drinking:
Phenobarbital, 544-545 negative contrast, 78, 85-86 and matching, 252
Phenothiazine: negative induction, 82 and polydipsia, 129-131, 135
classification, 544-545 negative reinforcement, 386-391, 406- Postreinforcement pause:
and conditioned suppression, 356 407 and acceleration of response, 142-143
and stimulus control, 555-556 observing responses, 318-326 conditionable unit, 226-228
Philanthus triangulum (see Wasp) oddity problem, 518 fixed-interval schedules, 213, 216
Phone, element of language, 629 olfactory sensitivity, 517 and polydipsia, 131
Phonology, component of language struc¬ overshadowing vs. masking, 492-493 ratio schedules, 42, 209, 213, 217, 226-
ture, 629, 632 pecking (see Pecking) 228, 293, 346, 355
Phrase marker, in language theory, 629- polydipsia, 129, 136 and ratio size, 209, 212-213, 217
630 positive conditioned suppression, 359 and rate of responding, 142-143
Phylogenic contingencies, 29, 580 positive contrast, 73-74, 78, 81-82 tandem schedule, 258
Subject Index 681

Power function: on concurrent schedules, 234, 237, Psychotropic drugs, 545

concurrent schedules 244-245 Puffer fish poison, 166
generality of, 255, 256, 277 and conditioned reinforcement, 326- Punisher, definition, 176-177
VI vs. FI, 255 327 Punishment, 174-198
human reaction time, 532 and schedule control, 214-215 adventitious (see Adventitious punish¬
and temporal differentiation, 224, 227 responses, and reinforcement vs. punish¬ ment)
visual intensity, 532 ment, 181 animal psychophysics, 523-524
Power function matching, equations, 255, shock, and shock-frequency reduction, and aversive control, 425, 427-430
277 374-377 change over in matching, 243-244, 253
Power law, sensory magnitude, 530 of US, in classical conditioning, 315, and choice on concurrent schedules,
Prandial drinking, rats, 18 343-344 253-254
PRE (see Partial reinforcement effect) Probability, differential rules (see Differ¬ and choice of reinforcer delay, 252
Pre-avoidance period, cardiovascular ential probability rules) and conditioned reinforcement, 320,
changes, 602-603 Progesterone, and thermoregulation, 162 322, 324-325, 326
Frebehavioral time, and language, 649 Programmed instruction, 7, 467 and conditioned suppression, 337-358
Predictability, shock, and gastric ulcer, Progressive ratio schedule, and central re¬ contrast experiments, 74, 76, 77
604 inforcement, 573-574 criteria for, 186-188
Prediction: Prolixin (see Fluphenazine) definitions, 176-177,182, 186
as method of synthesis, 10 Promazine, and dock schedule, 555 drug effects, 558-360
outcomes of instrumental contingencies, Prompting, in language learning, 639-640 functional relevance and, 113, 115, 116-
98-101, 115, 122 Proportional ratio matching, 238-239, 117
Predictiveness (see also Informativeness): 242, 247-249, 252-257, 277 gerbils, 115
stimulus, in autoshaping, 63, 80, 84, 1 g7 equations, 238, 248 iftstriiMeiital behavior, 98, 101—103
and terminal response, 126-128 Propranolol, and blood pressure, 603 lever pressing, and pattern of behavior,
Preening: Protein, component of self-selected diet, 428
as facultative behavior, 135, 140 44 in observing procedure, 320-325
as interim activity, 137, 146 Proximity: pigeons, 65, 253-254
Preference: food and water, and induced drinking, problems in testing for, 102
and avoidance, 366 139 as process, 176-177, 182, 186
central us. conventional reinforcement, to reinforcement in time: recovery from, 194
577-378 in chained schedule, 289, Q93 relation to reinforcement, 175-180 182
central reinforcement sites, 572-573 and conditioned reinforcement, 326- 197
color, in pigeons, 524 337 as reproducible process, 176-177 182
among concurrent schedules, 233, 235 and Herrnstein’s equations, 265 186
among events in world, 101-110 and induced behavior, 127, 140 Siamese fighting fish, 116
FI over chained schedule, 335 and interval relativity, 221 Pure operant, self-stimulation of brain,
and incentive, on multiple schedules, Stimulus to reinforcer, in autoshaping, 580
459 127 Pure stimulus act, 30
multiple over mixed schedules, 336 Pseudodiscrimination procedure, 495-505, Purposive acts, 12-43, 23
position, and animal psychophysics, 518, 507-509 Futamen, self-stimulation, 580
524,532 Psilocybin: Pyriform cortex, self-stimulation, 589
response, in animal psychophysics, 516, 545
classification, Pyrogens, and thermoregulation, 165-166
518, 521, 524 and stimulus control, 556
saccharine concentration, 106 Psychiatry, and drugs, 540, 542, 563-566 Quaalude (see Methaqualone)
scaling of, concurrent schedules, 298 Psychoanalysis, hydralic models, 11 Quail:
shorter IRTs, 256 Psychoendocrine studies, 597-604 autoshaping, 58—59
stimulus Psycholinguistics, 619-627, 628-652 sexual behavior, 59
in animal psychophysics, 524 Psychological distance, to reinforcement, wavelength generalization, 48@
and generalization gradients, 435-436, 335 Quantitative analysis:
440, 442 Psychological independence, dimensions discrimination learning, 448-451
thermal of discrimination, 439-440 law of effect, 4, 132, 140, 233-282
animals, 153, 158 159 Psychology of language, 619-652 Quantitative comparison, drugs, 547
humans, 162-164 Psychopharmacology (see also Behavioral Quiet biting attack, in cat, 14
VI over FI schedules, 255, 332 pharmacology. Drugs): Quinine:
wavelength, effect of rearing, 486 acquisition and extinction, 560-562 excretion, 330
Preference function, and generalization antecedent variables, 553—554 and thermoregulation, 155, 167
gradient, 435-436 and behavioral mechanisms, 541, 551-
Preference procedure, central reinforce¬ 560 Rabbit:
ment, 574 consequence variables, 556-559 classical conditioning, auditory fre¬
Preference structure, measurement, 108- and drug abuse, 542, 557-558, 563 quency, 489, 494
110 human, 563-566 eyelid conditioning, 493-496
Pre-loading, and induced drinking, 138 motivation, 188-189, 559-560, 562 pancreatectomized, 10
Preoptic area, and thermoregulation, 154- and schedule control, 542-543, 546, 551- self-stimulation of brain, 589
156, 167-169 552, 555-560 thermoregulation, 159, 166-167
Preparedness: and self-stimulation of brain, 571, 580 Raccoon, misbehavior, 13
sensation and perception, 562 Rage, in cat, 14
and animal psychophysics, 520
stimulus variables, 554-556 Random-interval schedule:
and negative reinforcement, 407-408
thermoregulation, 153-155, 164-169, conditioned suppression, 354
Primates (see also Monkeys, specihc pri¬
188-189, 556 definition, 202
mates):
Psychophysics, animal, 515-537 induced drinking, 135
autoshaping, 58, 60
methods, 515-532 shock avoidance, 267-268
Priming, and central reinforcement, 576
signal detection theory in, 5, 516, 532- temporal patterning, 213, 265
Primitives, in verbal behavior, 647-648 537 Random order of stimuli, animal psycho¬
Probabilistic schedule, shock-frequency re¬ Psychophysics of association, 70 physics, 521, 524
duction, 375 Psychophysiology, classical, 596-597 Random-ratio schedule:
Probability: Psychosomatic disorders, and central rein¬ definition, 202
positive stimuli, and observing re¬ forcement, 571 and positive conditioned suppression,
sponses, 322-324 Psychotic behavior, drug effects, 564-566 359
reinforcement: Psychotomimetics (see Hallucinogens) temporal patterning, 213
682
Subject Index

Random schedule: relation of eating and drinking, 131 hamster, 113, 118
definition, 202 relativity of reinforcement, 102-109, 181 rat, 117-118
shock frequency reduction, 374-375, response-produced stimuli, 222 Recall, and memory, 638
380-381 running, 100, 103-107, 129-135, 175, Receiver operating characteristic, animal
Random-time schedule, definition, 202 181-182, 258-263, 278, 281-282
Rat: psychophysics, 533-537
schedule-induced behavior, 126-139, Receptive field model, shape discrimina¬
absolute response rate, 258-259 143, 147 tion, 529
adventitious reinforcement and punish¬ schedule performance, 188-191 Recognition, and memory, 638
ment, 184 self-stimulation of brain, 572-590 Recovery, stages of:
aggressive behavior, 136 shape perception, 529 compared to development, 17-22
alcohol reinforcement, 553, 561 shock-correlated stimuli, 390-398 in hypothalamic dogs, 20
animal psychophysics, 518 shock-frequency reduction, 373-375, in hypothalamic rats, 17
automaintenance, 65 380-384, 402-406 Red nucleus, self-stimulation, 590
autonomic conditioning, 607-611 stimulus generalization, 439, 441, 493
autoshaping, 58, 60, 81 Redundancy, stimuli in autoshaping, 56-
497, 554-555
avoidance, 116-117, 165, 260, 267-271 sugar concentration, 259-260, 263, 279-
561 Reference procedure, extinction, 377-380
280, 557 Referent, of words, 619
behavioral contrast, 73-74, 78, 81-86 swimming response, 261, 263, 282 Refinement experiment, 30-31
267-268, 270-271 taste-aversion learning, 13-14, 24, 484 Reflex:
brightness discrimination, 526-527 thermoregulation, 153-156, 162-168 and aversive control, 425
caloric regulation, 43 188-189, 556 chain, in insects, 142
central reinforcement, 572-590 thigmotaxis, 117 chain, within meals, 28, 45-46
COD in matching, 243 thyroidectomy, 18 conditioned, 9
concurrent matching, 237, 248-250, 278 token reinforcement, 306 eating as, 29-31
conditional enhancement, 89 tooth chattering, 17 functional properties, 174
conditioned suppression, 89-90, 341-359 visceral conditioning, 607-611 gape, in thrushes, 19-20
contingent-response experiments, 102- Rate:
grasp, in humans, 19
109
changeover, and matching, 234, 243-244 laws of, 31
delay of reinforcement, 260-262, 281 249-250, 254, 276 and minimal unit, 67
development of infant, 18 eating (see Eating, rate) Sherrington’s definition, 29
discrimination training, 501-502 ingestion (see Eating, rate) sleep, 22
diurnal eating and drinking patterns, local (see Local rate) as source of behavior, 67
33, 36 members of reflex chain, 29 spinal, 8-9, 11
drug effects, 188-191, 349, 355-357, 544, pecking in pigeons, as constant, 238, strength, 29-30
546, 548-549, 551, 554-556, 560, 561 246
eating rate, 41 thermoregulatory (see Thermoregula¬
reinforcement tion, reflex)
electrical brain stimulation, 14, 60, 120 and conditioned reinforcement, 314 as unit of analysis, 29-31, 49, 53
178-180, 237, 259, 279, 518, 572-590 and conditioned suppression, 345, 355 as unmotivated, 9, 13, 23
equal-brightness contour, 526-527 local, 214-218 variables governing, 9
errorless learning, 467
and matching, 233-244, 247, 249, 252 Reflex arc:
escape, 117, 261-263, 281-282, 319, 367- and schedule-induced behavior, 130 and language, 641
368, 383-387 response model of learning, 53
exploratory behavior, 60, 120 absolute, 257-263 Reflex reserve, 11, 205
extinction, 378, 380
and central reinforcement, 572, 574, Refractory phase, of reflex, 29
FR behavior, 36-38, 551 576-578 Regenerating property, interval and time
FR FI performance, 187 chained vs. tandem schedule, 293-295 schedules, 212, 229
gastric ulcers, 597, 604 during COD, 244
grooming, 17 Regression toward the mean, responses
and conditioned suppression, 345, per reinforcer, interval schedules,
heart rate in CER conditioning, 352 348-351
heat reinforcement, 154-155, 188, 261
212
and FR requirement, 37 Regulation, temperature (see Thermoreg¬
263, 282 and measure of reflex strength, 28 ulation)
hypothalamic syndrome, 16-17 property of behavior, 174-176 Regulatory mechanisms, in polydipsia, 129
insulin and sugar preference, 557 and psychophysical scaling, 532 Regulatory system:
interim activities, 126-139, 143, 147 role of interresponse time, 263-264 constraints on learning, 118-119, 122
isobias functions, 535 schedule determinants, 228-229 water, in rats, 118
killing of, by cat, 14-15 shock-delay procedures, 370-371 Reinforcement, 98-123, 174-198 (see also
latent inhibition, 487 shocks received, shock-delay pro¬ Conditioned reinforcement. Central
lever contact response, 65, 81, 82, 120 cedure, 371, 373
lever pressing (see Lever pressing) reinforcement, Negative reinforce¬
Rate dependency, drug effects, 189-191, ment)
matching on concurrent schedules, 237, 555, 558-560
248-250, 278 adventitious (see Adventitious reinforce¬
Rating method, animal psychophysics, 535 ment)
meal patterning, 28, 30, 34-38, 41-44 Rayleigh distribution, in animal psycho¬ analysis and history, 98-100
mouse-killing, 17 physics, 534 central, 570-590 (see also Electrical
negative induction, 82 Ratio schedule (see also Fixed ratio. Ran¬ brain stimulation)
negative reinforcement, 373-375, 378- dom ratio, Variable ratio): similarity to conventional, 574-581
384, 390-406 components of chain, 292-293, 296 584-585, 589
nursing, 18-19
and concurrent choice, 246-247, 254 choice, central vs. conventional, 577-578
obesity, 597 definition, 202 conditioned (see Conditioned reinforce¬
olfactory discrimination, 523 non-regenerating property, 212 ment)
omission training, 65 shock delay, 377 contingency (see Contingency)
as omnivore, 47 temporal patterning, 217, 219-220
orientation response, 16-17 ^ C°^2ingent'resPonse exPerirnents, 98-
variable vs. fixed, responses per rein¬
patterning of drinking, 40 forcer, 207 criteria for, 186-188
pica, 137 Ratio strain, 37, 543 definitions, 175, 186, 202, 433, 584
polydipsia, 129-139, 147, 561 Reaction time methods, animal psycho¬ delay (see Delay of reinforcement)
prandial drinking, 18, 129-131, 135, 252 physics, 526-527, 532 density:
reinforcement immediacy, 260-262, 281 Reactive inhibition, 146 in autoshaping, 61
reinforcement magnitude, 248-250, 258- Rearing response:
261, 278-279 and conditioned reinforcement, 314,
gerbil. 111
316, 325-327, 329, 332, 336-337
Subject Index 683

on differentiation schedule, 224 relative frequency, and matching, 233- and stimulus control, 436, 445, 450, 482,
and drugs, 552 244, 247, 249, 252 _ ' 509
and induced behavior, 140 as reproducible process, 176-177, 182, Resistance to reinforcement, and stimulus
in time, 214-215 generalization, 446, 450-451
186
depletion-repletion model, 31-32 resistance to, and stimulus generaliza¬ Resistance-to-reinforcement test for in¬
by drugs, 192-193, 556-558 tion, 446, 450 hibitory control, 470-472
electrical brain stimulation (see also schedules (see Schedules) Resource allocation, in feeding patterns,
Reinforcement, central; Electrical secondary (see Secondary reinforcement, 47
brain stimulation) Conditioned reinforcement) Respondent (see also Classical, Pavlovian):
shock-frequency reduction, 365, 373-378, elicited by brain stimulation, 573, 580
magnitude, 259
and matching, 252 382, 387, 392, 400 Respondent behavior:
shock delay, 402,-406 and conditioned suppression, 341, 342,
as effect of operation, 202
excitatory effects, 264, 272-274 strength, central reinforcement, 572-574 348-352
systems of, 580-582 as interfering response in conditioned
food vs. water, and matching, 252
free, in contrast experiments, 87-90 of temporal pattern, 643 suppression, 351-353, 358-359, 360
token,306 Respondent conditioning, 3, 9, 53
frequency and conditioned suppression,
345-355 traditional theory, 99 Response:
of verbal behavior, 637 differences, and negative reinforcement,
and contrast, 76-78, 84, 87
and induced aggression, 469, 475 Reinforcements per opportunity (see 407-408
Probability, reinforcement) interchangeability, 13, 91, 112—113, 122
and IRT reinforcement, 225
and matching, 235-244, 247, 249, 252 Reinforcement theory; types, in operant conditioning, 183
and mathematical analysis, 4 Response contingency, and scbedule-m-
and peak shift, 462
functionally defined, 4 Premack’s, 101-109 duced behavior, 125-128, 130-136,
immediacy (see also Delay of reinforce¬ traditional, 99-101 140-141
ment) Reinforcer; Response cost:
definitions, 2, 175-177, 202 and alcohol consumption, 563
and central reinforcement, 575
and matching, 233, 251-252 examples, 178 in human matching, 239
and response strength, 260-262, 266, temporal placement, 213-214 Response dependency;
261 Reinforcibility (see Conditionable re¬ as stereotypic effect, 204
inhibitory effects, 264, 272-274 sponse unit, Conditionability, Asso« and temporal contiguity, 228
interchangeability, 13-14, 23, 112-113, ciability) Response-dependent schedule, 204 (see

122 Relations, and control of verbal behavior, also Specific schedule types)
638-639 Response deprivation hypothesis, 103—113,
intermittent (see also Intermittent rein¬
forcement, Partial reinforcement) Relative generalization gradient: 122-123
and drugs, 558 description, 435 Response enhancement, by electric shock,
and schedule control, 436, 438 185, 194-195
and partial reinforcement effect, 201
and thermoregulation, 164 and stimulus control, 501-502 Response frequency, variables determin¬
of IRTs, and schedule performance, Relative proximity principle, 221 ing* 206-213
223, 225 Relative proximity rule: Response functions:
magnitude; as equilibrium principle, 140 observing responses, 322-324
and absolute response rate, 257-259, and induced behavior, 140-141 in polydipsia, 130
278 Relative reciprocal, and IRT reinforce¬ in schedule-induced behavior, 132
conditioned reinforcement, 320, 322, ment, 225 Response-independent food, in response-
326-327 Relativity; dependent schedule, 1§8, 138
and contrast, 86, 267 conditioned suppression, 348-351 Response-independent schedules, 126-128,
drugs, 192-193, 250-251, 558 inhibition, 462 132-133, 141, 204 (see also Fixed
reward vs. punishment, 101-116, 178- time, Random time, Time schedule,
electrical brain stimulation, 179, 181.
259, 279 182 Variable time)
incentive, 100 stimulus control, 496 Response-independent shock, and aversive
and induced behavior, 152-133 thermal preference, 162—165 control, 415-425
and IRT reinforcement, 225 time within intervals, 220«221 Response-initiated schedule, fixed inter*
and matching, 233-234, 248-25L 262, Releasing stimuli; valj 301
278 in attack behavior, 15, 23 Response number:
with meal reinforcement, 45 ethological concept* 410 as discriminative stimulus, %%%
and observing responses, 320, 322 learned, in autoshaping, 62, 70 on FI schedules, 206-208
and polydipsia, 130-132 REM sleep, ontogeny, 22 on FR schedules, 209
thermal reinforcement, 157, 164 Repletion (see Depiction) Response-pacing procedure, 845 ? 954
model of, central and conventional, Replication, in behavioral pharmacology, Response patterning?
584-585 542 interval and time schedules, 214
negative (see Negative reinforcement, Reproducibility, of behavioral baselittC, ratio schedule, £17-820
Escape, Avoidance) and pharmacology, 551 temporal organization, 213-221
neural substrate, 571, 574, 580-581 Reproducible behavioral processes, 174— Response-produced shock (see Electric
nondifferential: 176, 182, 186-188, 197 shock, response produced)
definition, 433 Reproductive state, and thermoregula¬ Response-produced stimuli;
and generalization gradient, 436 tion, 161-162 and chaining hypothesis, 221-222
noradrenergic hypothesis, 571 Reptiles, thermoregulation, 156-158, 161- and conditioned reinforcement, 319,
omission, as second-order schedule, 301 162, 166
322-323
as operation, 176 Reserpine:
.Response rate (see Rate, response)
partial (see Partial reinforcement, Inter¬ classification, 545
Response-reinforcer relation:
mittent reinforcement) and conditioned oppression, 355-356
in automaintenance, 63-71, 91
percentage, and FI schedule, 301 psychiatric use, 540
and schedule performance, 189 as temporal, 204
Premack’s theory, 101-106, 108-109
and self-stimulation of brain, 585 Response-shock interval:
as process, 176-177, 182
Resistance to extinction: description, 369-370
quality:
and absolute response rate, 259-260 drug reinforcement, 558 ratio shock-delay schedule, 377
and generalization, 436, 445, 450, 483 relation to shock-shock interval, 370-
and choice, 252, 262
re-evaluation of concept, 4 and partial reinforcement effect, 201 371, 388
relation to extinction, 205 Pavlovian conditioning, 61 shock-delay procedure, 369-371, 388
relation to punishment, 175-182, 197 response unit hypothesis, 225-226 shock-deletion procedure, 372
 684 Subject Index

Response-shock interval (cont.) RS interval (see Response-shock interval) electric shock, 193-196
in Sidman avoidance, 369, 370-371 Ruminants, 33, 34, 47 and generalization gradient, 436, 438-
standardized 20 sec, 599 Running: 439
Responses per reinforcement: in contingent-response experiments, induced behavior, 129-138
differentiation schedules, 224 101-109, 181 and inhibitory control, 458-466
fixed-interval schedules, 205-213, 229 as emitted, 175 negative reinforcement, 367, 375-378,
and interreinforcer time, 211-213 as facultative behavior* 133-137, 140, 384-386, 390, 395-398
ratio schedules, 207-214 146 and positive conditioned suppression,
and schedule performance, 228-229 and food rate, 133-134 359
Response strength: forced, 102-103, 181 property of operant behavior, 197
and conditioned suppression, 348-351, interaction with induced drinking, 146- as reproducible process, 176-177, 186,
360 148 197
equations for, 257, 264-267 pattern, effects of schedule, 182 Schedule-induced attack (see Attack,
and meal termination, 45 rat: Aggression)
measures, 258, 262 fixed-time schedules, 126, 133-134, 147 Schedule-induced behavior, 125-148 (see
and negative reinforcement, 260-262 immediacy of reinforcement, 260, 281 also Interim activities, Polydipsia,
reinforcement parameters, 234, 239, magnitude of reinforcement, 259, 263, Attack, Drinking)
257-263, 278 278, 280 definition, 126
theories, 263-264 negative reinforcement, 407 hypotheses, 130-132
Response suppression: shock escape, 261-262, 281-282 measurement, 129-130
in behavioral contrast, 76-78, 84 temporal pattern, 175, 182 motivation, 128-132, 138-139, 144
in contingent-response situation, 103, reinforcer vs. punisher, 181 pigeon, 126-127
105-107, 112 and schedule induction, 129, 133-135, rat, 126-127, 130
by electric shock, 183, 185, 193, 195 146-147 regulatory mechanisms, 129
and forced running, 103 Running wheel: and schedule variables, 129-138
Response unit, specification of types, 222- avoidance, 116-118 temporal and sequential structure, 135,
225 gerbils, 98, 114, 115, 118 140-148
Restraint: motorized, 102, 111 types, 127
in animal psychophysics, 523, 527 rats, 102-108, 146, 181-182 Schedule-induced drinking (see Poly¬
and aversive control, 421 Run time distribution, fixed-ratio sched¬ dipsia, Drinking)
and temporal discrimination, 140-143, ule, 228 Schedule-induced running (see Running)
147 Runway: Schedule performance, 125, 140, 176, 186-
Retardates, human: central reinforcement in, 572 188, 197, 265
brain stimulation, 581 contrast effects, 86 Schedule performance:
chlorpromazine, 564 reinforcement magnitude, 259-261, 278- chained schedules, 288-299
Reticular formation, EEG activity, 605 282 direct and indirect variables, 204, 228-
Retrieval cues, in verbal behavior, 637 shock escape, 261-262, 281-282 229
Reversal, discrimination: drug effects, 188-191, 202
and errorless learning, 470 Saccharine, in contingent-response experi¬ and electric shock, 193-196
and stimulus control, 457, 470 ments, 106, 108 interval vs. ratio, and matching, 255
Reversibility: Saccharine-glucose solution, preference by as multiply determined, 228
drug effects, 551 rats, 578 and reinforced IRTs, 223-225
interactions in multiple schedules, 72 Salience, stimulus: second-order schedules, 299-306
Reversing chains schedule, and avoidance, and conditioned reinforcement, 317 theories, 217-221
366 relation to validity, 496 Schedules of reinforcement, 201-229 (see
Reward (see Reinforcement) and stimulus control, 483, 492, 496, 500, also Schedule control, specific sched¬
rg, 99-101 506 ules) :
Rhesus monkey: Salience, warning stimulus, and shock biological significance, 48
alcohol ingestion, 597 postponement, 394-395 characteristic effects, 176-177, 181, 197,
autoshaping, 58 Saline, control in psychopharmacology, 201, 213
avoidance, 185 , 366, 599-601, 604 548, 554, 556 and extradimensional training, 503-505
blood pressure conditioning, 609 Salivary conditioning: as fundamental determinants, 201-202,
conditioned suppression, 352, 598-599 classical, 9, 53, 61, 138, 340, 489, 493 229
drug dependence, 179, 553 description, 340 history, 201-202
drug-maintained responding, 192 lesions and, 20 types, 202-203
endocrine changes in avoidance, 599- operant, 607 Schizophrenia, 571
601, 604 Salivation: Scopolamine:
epinephrine and stimulus control, 556 hamsters, and grooming, 114 classification, 545
hormones and avoidance, 599-601, 604 instrumental conditioning, 607 clock schedule, 555
hypertension and avoidance, 601 and thermoregulation, 165 and learning, 561
morphine dependence, 179, 553 Sample size, behavioral pharmacology, and stimulus control, 555
peptic ulcers, 597, 605 541-542, 551 Scrabbling response, hamster, 113-114, 118
shock avoidance, 185, 366, 599-601, 609 Satiation: SA;
shock-elicited attack, 380 central reinforcement, 576-578, 580, 589
on chained schedule, 296
shock-frequency reduction, 376-377, 380 thermoregulation, 164
shock postponement, 185 in shock-frequency reduction, 381
Satisfier, in law of effect, 99, 101
sugar concentration Scalar property, timing on response-delay SA periods, and induced behavior, 132,
and response strength, 260 procedure, 400 135-136, 141
Rhodopsin, experimental synthesis, 10, 22 Scaling, animal psychophysics, 530-532 Seal, animal psychophysics, 519
Ribonucleic acid, rat brain, 606 Scallop, fixed-interval, 140, 143, 384, 458, Sea lion, visual acuity, 519-520
Ritalin (see Methylphenidate) 643 Search time, 48
RNA (see Ribonucleic acid) Schedule complex, and negative reinforce¬ Secobarbital, 544-545
ROC (see Receiver operating character¬ ment, 384 Seconal (see Secobarbital)
istic) Schedule control, 201-229, 288-309 Secondary reinforcement (see also Con¬
Rooting response: chained schedules, 289-299 ditioned reinforcement)
pig, 13 and conditioned suppression, 345-351, in autoshaping, 56
pigeon, 65 360 and brain stimulation, 576, 589
Rough grain, on chained schedules, 291 drug effects, 188-191, 202, 542-543, 546, Second-order deviations, on fixed-interval
Routes, drug administration, 543, 545 551-552, 555-560 schedules, 208-209
Subject Index 685

Second-order schedule: 17-OH-CS {see 17-hydroxycorticosteroid) relation to response-shock interval, 370-

brief stimuli, 299-306, 313, 315-318 Sex hormones: 371, 388
description, 299 and avoidance, 599-601 shock-delay procedure, 369-370, 388
discrimination of components, 316-318 and self-stimulation of brain, 579, 589 shock-deletion procedure, 372, 384-385
drug effects, 193 and thermoregulation, 162 Sidman avoidance, 369-370
electric shock effects, 193 Sexual behavior: standardized 2 sec, 599
induced drinking, 135 and electrical brain stimulation, 579, typical values, 370
interval components, 300-301, 303, 305 581 variable, 372
and pairing hypothesis, 315—318 guinea pigs, 21 Shuttlebox:
ratio components, 301-302 pigeons: central reinforcement, 572, 574, 582
and response units, 226 autoshaping, 59, 119 escape by pigeons, 254
and tandem schedule, 294, 300, 301 as interim activity, 137 and heart rate conditioning, 608
Seizure activity, and central reinforce¬ quail, Pavlovian conditioning, 59 Shuttle response:
ment, 573 rat, and thermoregulation, 162 negative reinforcement:
Selection, terminal response, 127, 133 Shape perception, animal psychophysics, pigeons, 407
Selective attention, 507-510 529 rats, 400, 406
Self-administration, drugs, 192-193, 543, Shaping: shock delay, dogs, 393
556-558, 561, 563, 586 animal psychophysics, 520 Shuttling, and thermoregulation, 157—158,
Self-regulation procedure, central rein¬ and continuity in time, 177=178 164, 166
forcement, 574 heart rate conditioning, 607 Siamese fighting fish:
Self-selection, balanced diet, 43, 44, 47-49 in language learning, 624 attack behavior, 15, 59, 65
Self-stimulation of brain (see Electrical with negative reinforcement, 406—407 automaintenance, 65
brain stimulation. Central rein¬ by response-produced shock, 196 FavloYian conditioning, 59, 65
forcement) by successive approximations, 23, 54, 62, punishment, 116
Semantic, in language structure, 629, 632- 177,182,406 Sign stimuli, 15, 23
633 unnecessary in nondepnved animals, 37, Sign tracking;
Semantic memory, 637-638 43 autoshaping and automaintenance, 69,
Semantics, natural language, 622-624 Shaping schedule, 224 81, 127
Sensation, drug effects on, 562 Sharpening, generalization gradient, 434, discrimination learning, 69
Sensitivity index, animal psychophysics, 494-495 Signal detection theory, animal psycho-
533 Sheep, latent inhibition, 487 physics, 516, 532-537
Sensory control; Shivering, and thermoregulation, 154-155, Signaled shock, and negative reinforce¬
eating, 20 164-168 ment, 397-398
transformation, 19-20 Shock (see Electric shock) Similarity;
Sensory deficit, and thermoregulation, Shock delay (see also Avoidance) perceptual, in animal psychophysics,
155-156 as reinforcer, 402-406 530-532
Sensory fields, 14-16, 21 vs* shock-frequency reduction, 402-406 S4- and § —, and errorless learning, 465-
Sensory neglect, 16-17 Shock-delay procedure: 466
Sensory scanning, and aversive control, cues added, 392-398, 400 Simultaneous discrimination, transfer to
418—430 description, 365, 370—371 successive, 487
Sensory stimuli, and central reinforce¬ extinction after, 378-380 Simultaneous method, generalization gra¬
ment, 582-583, 586, 588-589 multiple schedule, 383 dient, 455
Sensory thresholds, measurement in multiple response patterns, 388 Single-process model of conditioning, 99
animals, 515-525 negative reinforcement, 365, 370-371, Single-response methods, animal psycho¬
Septum, stimulation, 570, 575-576, 583 380, 383, 388-391, 400, 402, 407, 409 physics, 515—517
Sequence: ratio schedules, 377 Single-stimulus method, generalization
behavior, and shaping, 175=178, 180 response bursting, 388 testing, 434
behavioral states, 144-148 and stimulus generalization, 450 Situational stimulus (see also Contextual
as conditionable response unit, 226-227 temporal discrimination, 399-401 stimulus):
induced activities, 126-127, 140-143 Shock-deletion procedure (see also Avoid¬ and stimulus control, 464-509
response, during errorless learning, 464— ance): 6-hydroxydopamine (see also Dopamine)i
465 description, 365, 371=372 lesions made with, 586
stimuli: negative reinforcement^ 369} 371—373} aiid schizophrenia, 571
on chained schedule, 295—296, 334—336 375, 382, 384, 386, 387, 398-399, 400- Skeletal behavior, Favlovian conditioning,
and stimulus control, 456—457 401, 4o3 61
as theoretical response unit, 225-226 temporal discrimination, 400-401 Skeletal system, and conditioned sup¬
Sequential dependency, in animal psy¬ Shock-density reduction; 375 pression ) §§§—599
chophysics, 524 Shock-elicited behavior: Skinner box,'2, 11,*29, §72
Sequential interaction: human, 417-420 Skin temperature, and thermoregulation,
behavior and environment, 177, 180f mouse, 416 154, 156, 163-164
197 and negative reinforcement, 380, 408- Sleep:
among behaviors, 144-145 409 ontogeny, gl-gg
Sequential organization, responses between squirrel monkey, 417-430 slow-wave, 22
reinforcers, 221 Shock-free periods, concurrent schedules, stages, 22
Sequential relations, responding on FI, 390, 392 Smell (see Olfaction)
209 Shock-frequency reduction (see also Smooth curves, as criterion for laws, 11-
Sequential structure, induced behavior, Avoidance, Negative reinforce¬ 12, 29-30
140-146 ment): Snake, thermoregulation, 162
Serial order, problem in behavior, 641- as controlling variable, 365, 373-375, Snuggling response, chicks, 60, 121
642 382-383,387,392,400-406 Social behavior, and feeding behavior, 49
Serotonin, and thermoregulation, 167, 169 direct manipulation, 374-375, 380-381 Sodium chloride, excretion, 550
Set point: Herrnstein’s equations, 373-374, 386- Sodium light, ducklings reared in, 485-486
and control theory, 154, 160 387 Sodium salicylate, and thermoregulation,
definitions, 154, 160 vs. shock delay, 402-406 167
in thermoregulation, 160-169 Shock-intensity reduction, 375, 380 Solubility, drugs, 549-550
“Sets,” 9 Shock postponement (see Shock delay) Somatic effects, chronic, behaviorally-in-
17-hydroxycorticosteroid: Shock-shock interval: duced, 597-598
and avoidance, 599-601, 604 description, 369-370 Somnolence, and hypothalamic lesions,
and conditioned suppression, 598-599 fixed vs. variable, 397 156
 686
Subject Index

Somnos (see Chloral hydrate) shock escape, 189, 380, 557 by drugs, 192-193, 555-556
Sopor (see Methaqualone) shock-induced attack, 136, 417-430 dynamic models, 451-453
Spaced responding, shock-delay procedure, thermoregulation, 41, 153, 156 and EEG activity, 605-606
400 S-R relations, 11-13, 24 by elapsed time, 218-219
Spaced responding schedule (see DRL SS interval (see Shock-shock interval) and errorless discrimination, 186
schedule) SSDR (see Species-specific defense reac¬ examples, 481
Species differences: tion) experimental procedures, 488-505
acquisition with negative reinforcement, Staircase method, animal psychophysics, heart rate conditioning, 607-608
406-409 521-522 inhibitory, 432-476
animal psychophysics, 519-520 Stalking: and amount of training, 453-456
associability of responses and rein¬ carnivores, as DRL schedule, 48 concurrent schedules, 463-464
forcers, 484-485 cat, 15 determinants, 432, 453-464, 476
automaintenance, 65-66 State (see Behavioral state, Interim state, and errorless learning, 470-474
autoshaping, 58, 60, 70, 91 Terminal state) measurement, 445-447
COD in matching, 243 State-space, and motivation, 144 on multiple schedules, 458-463
conditioned enhancement, 90-91 Steady states, and schedule control, 227- and schedule variables, 458-466
conditioned suppression, 346, 359 228 by interresponse time, 223
contrast, 73-74, 78, 81-82, 91 Steepening, generalization gradient: of interresponse times, and generaliza¬
feeding patterns, 46-49, 58 and discrimination training, 439-453 tion, 437-438
generalization gradients, 435-436 in extinction, 438, 483 and intradimensionl training, 493-494
negative reinforcement, 390 Steepness, generalization gradients, and Lashley and Wade’s theory, 483, 485-
positive conditioned suppression, 359 peak shift, 448-449, 454 486
schedule control, 186 Stelazine (see Trifluoperazine) by light, and early experience, 485-486
stimulus control: Stereotaxic procedure, implantation of by line orientation, 440-441, 445, 454-
tone frequency, 441 electrodes, 570, 572 455, 458, 482, 494, 501-502, 527
wavelength, 483 Stereotypic effects: and masking, 482-483, 491-492, 500 502
thermoregulation, 153, 156, 167-169 and reinforcement, 204 506
Species-specific behavior: and schedule performance, 204-205, 228 measurement, 433-436, 482-483
aggression in pigeon, 468 Stereotypy: multiple schedule, generalization, 458-
and aversive control, 425 activities on fixed-time schedules, 126, 463
central reinforcement, 580-581 204 nature of response and reinforcer, 483-
constraints on learning, 3, 112-118, 120— collateral behavior, and conditioned 484
121 suppression, 347 and negative reinforcement, 366, 369,
contrast, 273, 275 fixed-ratio performance, 227 381-396, 399
and negative reinforcement, 406-409 interim activities, 137 by an odorant, 517
and operant conditioning, 183 operant responses, 177 overshadowing, 492-493, 496, 499-501
pecking in pigeon, 273 superstitious behavior, 127, 204 503, 506
Species-specific defense reaction, 116-118, Steroids, and avoidance, 599-601, 604 and prior experience, 485-488
120, 165, 407 Sterols, excretion, 550 resistance to extinction, 436, 445, 450
Species-typical behavior: Stickleback: 509
and brain stimulation, 571-572 aggression, 121-122 schedule-induced behavior, 129, 131
constraints on learning, 112-118, 120- sexual behavior, 15-16, 121-122 140
121 Stimulants: signal detection analysis, 536
and negative reinforcement, 408, 410 classification, 545 theory, 505-510
Specificity: self-administration, 543 and threshold tracking, 522
autonomic conditioned response, 608, and stimulus control, 555 by tone frequency, 440-441, 450, 455-
610 Stimuli: 456
drug action, 551-552 interchangeability, 13, 23 verbal behavior, 637-640
Spectral sensitivity, goldfish, 517 relations among, in control of behavior,
Spindle activity, 605 by wavelength, 434-439, 441-443, 454
3, 5 485-486
Spiral aftereffect, monkey, 529 Stimulus change:
Sprawling, and thermoregulation, 154, 165 by wavelength difference, 525, 531, 534
cue in escape, 386 Stimulus generalization (see also Generali¬
Squirrel: and maintenance of observing, 321 zation):
animal psychophysics, 519 Stimulus compounding, and generaliza¬ aversive control, 450
self-stimulation of brain, 589-590 tion, 459-463
Squirrel monkey: brightness, 490, 493
Stimulus context, and stimulus control, compound stimulus, 439
adjusting schedule, 212-213 457-458
attack behavior, 23, 136, 417-430 and conditioned reinforcement, 316-318
Stimulus contingencies, and terminal re¬ conditioned suppression, 351
automaintenance, 65-66 sponse, 128
autoshaping, 60 diffuse stimuli, 490, 508
Stimulus control, 432-476, 481-510 discrimination learning, 447-451
avoidance, 185, 193-196 Stimulus control:
blood pressure conditioning, 609 discrimination training, 493-505
acquired distinctiveness of cues, 487- after errorless learning, 471-472
color vision, 518 488 extradimensional training, 497-505
conditioned reinforcement, 304-305 acquisition, conditions affecting, 483-486
conditioned suppression, 89 line orientation, 440-441, 445, 454-455,
and acquisition speed, 454-456 458, 482, 494, 501-502, 527
drug effects, 189-191, 557 by air flow rate, 491 monkey, 490
electric shock: and amount of training, 453-456 and Pavlovian conditioning, 481, 483-
conditioned reinforcement, 304-305 attentional factors, 481-510 484, 487, 489, 492, 493
reinforcer vs. punisher, 178-179, 183— auditory, 483-495, 528 Pavlovian theory, 483
185, 193-196 aversive baselines, 450, 455-456 and shock avoidance, 395
environmental thermoregulation, 41 and avoidance, 366, 369 tone frequency, 440-441, 450, 455-456,
FR FI performance, 187 by behavior, 221-222 493
hypertension and avoidance, 601 by brightness, 490 human, 493
negative reinforcement, 367, 380, 385- and central reinforcement, 582-583 rabbit, 493
386 as continuum, 433 wavelength
omission training, 65 definitions, 433, 482 ducklings, 485-486
schedule-induced attack, 136, 417-430 dimensional, and early experience, 485- pigeons, 434-439, 441-443, 454, 489
schedule-induced drinking, 129, 138 487 Stimulus generalization gradient:
shock-elicited responses, 183-184, 380 drug effects, 554 absolute vs. relative, 435
Subject Index 687

as artifact, 438-439 “Superstitious” responding, 13, 127, 132, behavioral interactions, 144-145
compounding theory, 459-463 204, 359 TD (see True discrimination)
as continuous, 438 Suppression: tD — tA schedules, negative reinforcement,
definition, 433 behavior, prior to shock, 422, 425 376-377, 399
determinants, 432, 436-439 concurrent response, by punishment, Tegmentum, single unit recording, 605
fixed-interval schedule, 458 253 Telencephalon, stimulation, 577
inhibitory, 445-447, 453-456, 459-463 conditioned (see Conditioned suppres¬ Temperature regulation (see Thermo¬
interdimensional training, 494-497 sion) regulation)
maintained, 434, 435, 529-530 interim activities, by prevention, 140- Temporal contingency:
measurement, 433-436 143, 147 classical conditioning, 343, 349-350
microstructure, 437-439 response: and terminal responses, 127-128
necessary and sufficient conditions, 483 in behavioral contrast, 76 Temporal control:
peak shift, 441-443 by free food, 128 and conditioned reinforcement, 314,
Spence’s theory, 447-451 by S— on chained schedule, 297 317-318, 336-337
techniques for obtaining, 433-436 by shock, 182, 185, 193, 195 interval schedules, 217
transient methods, 434 running, by terminal and interim language behavior, 643—647
Stimulus presentation methods, animal activity, 133 ratio schedules, 217
psychophysics, 520-525 terminal response by interim, 143 Temporal discrimination:
Stimulus-reduction procedure, and gen¬ Suppression ratio: carbon monoxide effects, 563
eralization gradient, 447 calculation, 342 and conditioned suppression, 349-350,
Stimulus-reinforcer relations: problems with, 342, 345, 348-351 352
in automaintenance, 63-67, 70-71 Suppressive summation, and stimulus and interim activities, 140, 142-143
autoshaping, 54-56, 70-71, 80 generalization, 443-444, 459 and polydipsia, 132
behavioral contrast, 80—84, 86-88 Supraliminal stimuli, animal psychophysics restraint effects, 140, 142-143
in operant procedures, 71-75, 91 of, 525-532 and schedule control, 217-219
Stimulus set, and generalization, 443-444 Surface structure, language, 629, 631-633, and shock-delay, 399-401
Stimulus substitution: 648-649 stochastic processes, 140-141
autoshaping, 62, 70 Surrogate mothers, and thermoregulation, and theory of patterning, 217-220
Pavlovian conditioning, 61-62 159 Temporal factors, in conditioned rein¬
Stimulus surrogation, autoshaping, 62, 70 Surrogation, stimulus (see Stimulus sur¬ forcement, 314, 317-318, 326-337
Stimulus variables, psychopharmacology, rogation) Temporal frame, in verbal behavior, 635
554-556 Sweating, and thermoregulation, 154, 165, Temporal integration, in cerebral cortex,
Stochastic process: 167 641-642
in behavior sequences, 142, 146 Swimming, escape response in rats, 261, Temporal location, component of chain
and temporal discrimination, 141—142 263, 282 schedule, 289, 293
Stochastic transitivity, and choice, 322-333 Switching response (see also Changeover) Temporal pattern:
Strain, on ratio schedules, 37, 543 Switching response, concurrent schedules, in contingent-response experiments,
Strategies, feeding behavior, 38-39, 46-49 234-235 107-108
Stress, effects of, 5, 597 Syndrome: escape behavior, 425-427
Strict learning procedure, linguistic struc¬ lateral hypothalamic, 16-18 in preshock stimulus, 350, 352
ture, 624-626 morphine withdrawal, 179, 554 and proximity to reinforcement, 265
Strict training procedures, language learn¬ Syntactic behavior: punishment behavior, 427-430
ing, 624, 640 generative, 639-640 reproducible behaviors, 175-176
Stroke, recovery from, 19 theories, 640-642 responses:
Structural analysis, language, 5, 629-633, 628
Syntactic structures f and brief stimuli, 302, 303
642-651 Syntactic system, definition, 620 chained schedules, 289-291
Structure: Syntax: second-order schedules, 300-302
induced behavior, 140-146 acquisition, 619-627, 643-647 and schedule control, 213*215
in chimpanzee, 640 on schedules, theories,
language, 622-633, 642-651
Subduction, of behavioral state, 145 component of language structure, 629- shock-elicited behavior, 418-425, 428
Substantia nigra: 632 a§ unit gf behavior, 222
lesions, 582 functional approach, 639-640, 643-647 Temporal placement, reinforcer, 213-218,
self-stimulation, 585, 590 Syntax crystal,622-627, 647 229
Success, as method of synthesis, 12 Synthesis, experimental methods, 10—11 Temporal Structure*
Successive approximations, method of Synthetic VI schedule, 263^264 drug effects, 562
shaping, 54, 62 Systems, reinforcement, 580—582 induced behavior, 140—146
Successive discrimination: Systems constraint hypothesis, 118-119, verbal behavior, 643^647
122 Temporalis muscle, 417—418
multiple schedules, 71
transfer to simultaneous, 487 Temporospatial factors, and central rein¬
Successive stage experiments, extradimen- ‘‘Tables of discovery,” 12 forcement, 575
Terminal periods, on periodic schedules,
sional training, 497-500 Tact, verbal behavior, 634-635
Sucrose: Tactual discrimination, monkeys, 517 133, 137-139
concentration, and response strength, Tandem schedule: Terminal ratio, definitions, 575-574
259-260, 263, 279, 280 absolute response rate, 258 Terminal response:
pellets, and response strength, 278 and chained schedule, 290, 293-295, 298 competition with interim, 132-133, 139
solution, magnitude of reinforcement, description, 290, 315 definition, 126
248-249, 259-260 drug effects, 542-543, 555 and errorless learning, 464, 475
Sugar (see Sucrose) fixed-interval and fixed-ratio, 210—211, operant view, 127
Summation, stimulus, 443-444, 459 223 Pavlovian view, 127
Summation method, and generalization and IRT as conditionable unit, 224 relation to reinforcer, 273-274
gradient, 446-447 and second-order schedule, 300-301, 316 schedule-induced, 126, 133, 139, 141
Sunflower seed, 98, 114 Taste: Terminal state:
Superego, 9 aversiveness, in rats, 527 interaction with interim, 139-145
Superimposition procedure, errorless and central reinforcement, 582 137-138
properties,

learning, 466 pathways, 586 Termination, meals, 28, 32, 45

Superstitions, concurrent, 234, 235 Taste-aversion learning, 7, 13-14, 24, 484 Tetrahydrocannabinol (see also Mari¬
Superstitious behavior (see also Adven¬ Taxis, 158 juana):
titious reinforcement): Taxonomy: and acquisition of avoidance, 560
Skinner’s view, 127 behavior, 12, 24 classification, 545
 688 Subject Index

Tetrahydrocannabinol (cont.) Thrush, gape response, 19-20 Traditional pairing hypothesis, condi¬
and peck rate in pigeons, 552 Thwarting, 23-24 tioned reinforcement, 314, 318, 325
and stimulus control, 555 Thyroid hormones, and avoidance, 599- Tranquilizer (see also Drugs, specific
tolerance, 553 600 drugs):
Tetrodo toxin, 166 Thyroidectomy: and conditioned suppression, 349, 355-
Thalamus: rats, 18 357
EEG activity, 605 and thermoregulation, 154 punishment effects, 558-559
self-stimulation, 590 Tilt (see Line orientation) and schedule control, 188
THC (see Tetrahydrocannabinol) Time: Transfer (see also Generalization, Stim¬
The Behavior of Organisms, 9, 32 allocation, and generalization, 435 ulus generalization):
Thematic classes, verbal responses, 637 as discriminative stimulus, 217-219 in animal psychophysics, 526-530
Theoretical response unit, 222-226 distribution of responses in, 106-108 in behavioral sequences, 147-148
Therapeutic effects, drugs, 540, 542-545, as indicator of value, 101 common elements theory, 639
563-566 matching: simultaneous to successive discrimina¬
Thermal gradient, 157 on concurrent schedules, 238-239, tion, 487
Thermal preference, 153, 159, 162-164 246-248, 254, 272, 274 stimulus control, and errorless learning,
Thermal reinforcement, 153 in interresponse class, 264 465-466, 475
Thermally sensitive units, 154, 158, 160, as stimulus on FI, 218-219 in verbal behavior, 634, 649
166, 169 Time course, drug effects, 548-549 visceral learning, 608
Thermocline, 157-161 Time distribution: Transfer tests:
Thermodynamics, 31 among concurrent schedules, 233, 246- conditioned reinforcement, 288-289
Thermoregulation, 156-169 247, 272 induced states, 138
Thermoregulation: among interresponse times, 263-264 Transformational grammar, theory, 629-
alligators, 153 Time out: 649
amphibians, 156-158 on electric shock schedule, 179, 194-195 Transient contrast, 77, 268, 275 (see also
baboons, 153, 166 escape from S—, 469-470 Local contrast)
and behavioral homeostasis, 153 negative reinforcement schedule, 367, Transient generalization, methods, 434
birds, 159-160 385-386, 388-390, 398 Transition, of situation, as reinforcer, 381
cats, 153, 166-167 punishment in animal psychophysics, Transition performance, chained sched¬
chicks, 153, 159-160 523 ules, 290-291
and diet, 162 from reinforcement: Transition states, and schedule control,
digestion and, 162 and alcoholism, 563 227-228
dogs, 153, 159, 166-167 as punisher, 244, 253 Transitivity, stochastic, and choice, 332-
doves, 153 and reinforcement omission on FI, 301 333
drug effects, 153-155, 164-169, 188-189, and schedule-induced behavior, 139 Transituational reward, 100-101, 113
556 Time schedule (see also Fixed time, Ran¬ Translation, language, 650
fish, 156-158 dom time. Variable time): Transposition, in discrimination learning,
frogs, 157-158 definition, 202 447, 497
goldfish, 153, 158 and interval schedules, 214 Treadle pressing, pigeons:
hamsters, 159 Timing, induced behavior sequences, 141— avoidance, 450, 455-456
and hormones, 161, 162 143 concurrent schedules, 236
human, 158, 162-164 Titration method: conditioned enhancement, 90
and hypothalamus, 154-156, 164, 167— animal psychophysics, 521 contrast, 82, 89
169 self-stimulation of brain, 574 negative reinforcement, 386-387, 391,
iguanas, 157-158, 166 T-maze, drug effects, 555 407
infants, 158-160 Tofranil (see Imipramine) positive conditioned suppression, 359
lizards, 153, 157-158, 161-164 Token economy, 7 relation to reinforcement, 273
macaque, 153, 159 Token reinforcement, and conditioned stimulus control, 438, 455-456
mammals, 158-159, 162, 167 reinforcement, 306 Treadmill, 40
monkey, 164, 167 Tolerance: Trial, duration, in autoshaping, 57-58, 66,
mouse, 153, 159, 167 drugs, 550, 553 90
negative feedback in, 153-154 large fixed ratios, 36, 38-39, 45 Trifluoperazine:
neural controls in, 153-155, 167-168 Tone: classification, 545
operant, 153-161, 164-169 as contingent stimulus, 115 and punishment, 558
pig, 153,158-159 stimulus in autoshaping, 69, 273 Triflupromazine, 545
and preoptic area, 154-156, 167-169 stimulus in conditioned suppression, Trigeminal nerve, section of, 15
rabbit, 159, 166-167 89, 341-344, 351 Trilafon (see Perphenazine)
rats, 153-156,162-165,167-168, 188-189, stimulus in contrast, 274-275 True discrimination procedure, 495-505,
556 Tone frequency, generalization: 507-509
reflex, 153-156, 160-161, 165-168 guinea pig, 441 Truly random control procedure, 55-56,
reptiles, 156-158, 161-162, 166 pigeons, 440-441, 450, 455-456, 495, 528 61, 84, 120, 343, 391
respondent, 154-156, 160-161, 166-168 rat, 441 £— r schedules:
snake, 162 Tongue flip, frogs, 121
squirrel monkey, 41, 153, 156 Tooth chattering, rat, 17 explanation, 203
Theta waves, 119, 605 Topography: and IRT reinforcement, 223
Thigmo taxis, 117 instrumental vs. contingent response, Turtle, animal psychophysics, 519
Thiobarbiturates, 549 108 Twittering, chicks, 60
Thioridazine, 545 lever contact response, 60, 65, 81-82, Two-factor theory, negative reinforce¬
Third-order deviations, in fixed-interval 120 ment, 364-365, 393, 396-398, 400
schedules, 208, 213 pecking, on FI and VI schedules, 141 Two-key concurrent schedule, description,
Thirst, and induced drinking, 138-139 response: 234
Thorazine (see Chlorpromazine) in autoshaping, 24, 58-60, 67, 81, Two-process model, conditioning, 99
Three-term contingency, and verbal be¬ 119-122 Two-response methods, animal psycho¬
havior, 633 on differentiation schedules, 203 physics, 517-519
Threshold: and matching, 262 Two-stage model, food motivation, 32
definitions, 520-522 and negative reinforcement, 406-408 Two-state analysis, fixed-interval respond¬
sensory, in animals, 515-525 Toxic effects, drugs, 546 ing, 265
Threshold tracking, animal psychophysics, Toxicology, behavioral, 562-563
521-523, 525 Tiactus solitarius, 586 Ulcers, and aversive control, 597, 604-605
Subject Index 689

Uncertainty reduction, and conditioned temporal patterning, 213 Water:

reinforcement, 318-326, 336-337 types of distribution of intervals, 215 escape, by rats, 261, 263, 282
Uncertainty reduction hypothesis: uncorrelated shock, 350 interaction with food:
conditioned reinforcement, 313-315, Variable-ratio schedule: and induced behavior, 131, 135, 139
318, 322-325, 336-337 conditioned suppression, 346 and matching, 252
quantitative implications, 322-325 definition, 202 reinforcement, and peck duration, 58-
Undermatching, concurrent schedules, and generalization gradient, 436, 439 GO, 67, 81, 119
242-243, 248-255 matching, 246-247, 254 reinforcer in automaintenance, 65, 67
Unit, linguistic, 620-621, 629-630 observing responses, 321 reinforcer in autoshaping, 58-60, 62, 81,
Unit, response (see Response unit) relation to variable interval, 210 119
Unit schedule: and sexual behavior in sticklebacks, 121 reinforcer in induced drinking, 138-139
part of chain schedule, 289 temporal patterning on, 213, 217 Water balance, in polydipsia, 129
part of second-order schedule, 299-302 Variable-time schedule: Water bath, and thermal preference, 163
token delivery, 306 concurrent matching, 237-238 Water holes, patterning of visits, 36
Units of behavior, problem of specifica¬ contrast, 76-77, 80-81, 84-85, 90 Water intake, and food intake, 35
definition, 202 Water restriction, and meal patterning, 35
tion, 29
Universals, linguistic, 641 electric shock, 397-398 Wavelength:
Up-down method, animal psychophysics, induced behavior, 127, ISO difference, stimulus control by, 525, 531,

521 Vasoconstriction, and thermoregulation, 534

UR, definition, 340 154, 168 maintained generalization, pigeons, 530
Uridine, in mouse brain, 606 Vasodilation, and thermoregulation, 154, stimulus generalization;
Urination: 165, 167 168 chicks, 486
and conditioned suppression, 351 Vasomotor responses, and thermoregula¬ ducklings, 485-489
guinea pigs, 21 tion, 154, 161 Japanese quail, 486
US: Vector properties, responses, 177 monkeys, 486
autoshaping, 55, 58, 61, 69—70, 80 Ventral noradrenergic neurons, stimula¬ pigeon, 434, 439, 441-443, 454, 489,
conditioned suppression, 342-344 tion, 585 493-494, 301-302, 331-332
in contrast, 81, 88 Ventral tegmentum, self-stimulation, 590 Wavelength discrimination, pigeon, 519,
definition, 340 Verbal behavior; 3, 628, 633-636, 638, 639, 534
fear conditioning, 315 642, 649 Weak law of effect, 100-101,113,112
Utility; Verbal behavior, theory, 628-629, 633-640 Weber fraction, time discrimination, 218
and matching, 277 Verbal learning, and language, 637 Weight loss:
rate increase in FR, 44 Vertical, perception, of, in pigeons, 527 between meals, 33, 36
vesprin (see Triflupromazine) and performance, 00, id
Validity, stimulus, and stimulus control, Vibratory stimulus, stimulus control by, “Well-behaved” operants, 183
494=496, 500, 506 498 Wheel cranking, college students, 199, 197
Valium (see Diazepam) Visceral-alimentary system, durable Wheel running (sss Running)
Valsalva maneuver, 609 changes, 597 Wheel turning, aud negative reinfme-
Value: Visceral learning, 607-611 ment, 390-391
of events to organism, 101 Visceral responses, operant conditioning, Whltq light, and wavelength generalisa¬
intervening variable, in matching ex¬ 571,606-611 tion, 486
Vision, rats, after lesions, 19 Wing flapping;
periments, 276
scaling, concurrent schedules, 298 Visual acuity: aggressive, in pigeon, 59
Variability, behavioral sequences, 142, 146 pigeon,317-3i8, 521 schedule-induced, in pigeon,
Variable cycle procedure, shock dclction? sea lion, 519-520 139
387 Visual discrimination, LSD effegt§, 554 Wistar rats, 605

Variable delay procedure, shock delay, Visual fixation training, in monkey, 523 Withdrawal syndrome, morphine, in mon-
372=373 Visual stimulus control (see also Bright¬ key, itS, 554
Variable-interval schedule: ness, Line orientation, Wavelength, Within-meal behavior;
absolute response rates, 257-265 Stimulus control): a§ locus of analysis, 59, 48
and adventitious punishment, 184 pigeon, 482s 483, 485, 488—492 rate changes, 44
autoshaping, 127 Vitamin A, 10, 22 Words, as discriminative stimuli, 019, 021
aversive control, 428 Vitamin deficiency, and theiraoregulatigiij Worsening, conditions, and contrast, 76
avoidance, 260, 269-271 154 ’
central reinforcement, 378 vocalization response, animal psycho¬ X-ray, poisoning by, association with food,
concurrent (see Concurrent schedules) physics, 519—520 13-14,484
and conditioned suppression, 89, 341- Voltage reduction, reinforcement, in rats,
345, 356, 359, 598 261 Ye$-ii6 procedure^ animal psychophysics?
contingent-response experiments, 108- Voluntary behavior, key pecking as, 53-55, 518-522,524
109 67 Yoga, and physiological conditioning, 697
and contrast, 71-85 Voluntary mediators, in instrumental Yoked controls:
definition, 202 autonomic conditioning, 610 avoidance, and peptic ulcer, 604-605
electric shock, 178-179, 194
brain biochemistry, 606
electrical brain stimulation, 181, 259 chained schedules, 296
Walden Twof 7
escape, 384, 386
Warm up effect: conditioned reinforcement, 319
generalization gradient, 434, 436, 438,
animal psychophysics, 521 conditioned suppression, 346, 357
440, 446, 460, 462
aversive control, 426 contingent-response experiments, 106-
induced behavior on, 132-135, 141
induced drinking on, 129, 132, 135 negative reinforcement, 409 108, 111, 115
interim periods on, 133, 135 Warning signal, in avoidance; differentiation and variable-interval,
local rate of reinforcement, 216 and endocrine changes, 599 224
matching on (see Matching, Concurrent and gastrointestinal changes, 604 fixed-ratio vs. interval, 217
schedules) Warning stimulus (see also CS): and matching, 272
negative reinforcement, 384 aversiveness, 365 omission training and automaintenance,
observing responses, 319-320, 323-324 and avoidance theory, 364-365, 393-398, 65-66, 128
paced, 225 400 VI vs. VR in pigeons, 210
probability of reinforcement, 214-215 central reinforcement, 579 and stimulus generalization, 436
relation to variable ratio, 210 shock-delay procedure, 393-398
response strength, 257-265 Wasp, hunting behavior, 142 Zona incerta, self-stimulation, 590