Investigating the Psychological World: Scientific Method in the Behavioral Sciences

Investigating the Psychological World

Life and Mind: Philosophical Issues in Biology and Psychology

Kim Sterelny and Robert A. Wilson

Investigating the Psychological World: Scientific Method in the Behavioral Sciences,

Brian D. Haig, 2014

Evolution in Four Dimensions: Genetic, Epigenetic, Behavioral, and Symbolic Variation in the History of Life, revised edition,

Eva Jablonka and Marion J. Lamb, 2014

Cooperation and Its Evolution,

Kim Sterelny, Richard Joyce, Brett Calcott, and Ben Fraser, editors, 2013

Ingenious Genes: How Gene Regulation Networks Evolve to Control Development,

Roger Sansom, 2011

Yuck! The Nature and Moral Significance of Disgust,

Daniel Kelly, 2011

Laws, Mind, and Free Will,

Steven Horst, 2011

Perplexities of Consciousness,

Eric Schwitzgebel, 2011

Humanity’s End: Why We Should Reject Radical Enhancement,

Nicholas Agar, 2010

Color Ontology and Color Science,

Jonathan Cohen and Mohan Matthen, editors, 2010

The Extended Mind,

Richard Menary, editor, 2010

The Native Mind and the Cultural Construction of Nature,

Scott Atran and Douglas Medin, 2008

Describing Inner Experience? Proponent Meets Skeptic,

Russell T. Hurlburt and Eric Schwitzgebel, 2007

Evolutionary Psychology as Maladapted Psychology,

Robert C. Richardson, 2007

The Evolution of Morality,

Richard Joyce, 2006

Evolution in Four Dimensions: Genetic, Epigenetic, Behavioral, and Symbolic Variation in the History of Life,

Eva Jablonka and Marion J. Lamb, 2005

Molecular Models of Life: Philosophical Papers on Molecular Biology,

Sahotra Sarkar, 2005

The Mind Incarnate,

Lawrence A. Shapiro, 2004

Organisms and Artifacts: Design in Nature and Elsewhere,

Tim Lewens, 2004

Seeing and Visualizing: It’s Not What You Think,

Zenon W. Pylyshyn, 2003

Evolution and Learning: The Baldwin Effect Reconsidered,

Bruce H. Weber and David J. Depew, editors, 2003

The New Phrenology: The Limits of Localizing Cognitive Processes in the Brain,

William R. Uttal, 2001

Cycles of Contingency: Developmental Systems and Evolution,

Susan Oyama, Paul E. Griffiths, and Russell D. Gray, editors, 2001

Coherence in Thought and Action,

Paul Thagard, 2000

Investigating the Psychological World

Scientific Method in the Behavioral Sciences

Brian D. Haig

The MIT Press

Cambridge, Massachusetts

London, England

© 2014 Massachusetts Institute of Technology

All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.

Library of Congress Cataloging-in-Publication Data

Haig, Brian D., 1945–.

Investigating the psychological world : scientific method in the behavioral sciences / Brian D. Haig.

   p.  cm.—(Life and mind : philosophical issues in biology and psychology) (A Bradford book)

Includes bibliographical references and index.

ISBN 978-0-262-02736-6 (hardcover : alk. paper)

ISBN 978-0-262-32238-6 (retail e-book)

1. Psychology—Research—Methodology. I. Title.

BF76.5.H335 2014

150.72'1—dc23

2013032413

10 9 8 7 6 5 4 3 2 1

d_r0

The attempt to understand and improve methods, and to do so via theorizing them, is at the center of an intelligently evolving cognition.

—Clifford Hooker (1987, 291)

Above all, if a raised standard of education in methods is to be achieved, it is necessary to engender, beyond any knowledge of particular skills and formulae as such, a perspective as to what methods are most appropriate to various areas and occasions.

—Raymond Cattell (1966, 5)

Contents

Preface

Acknowledgments

1    Method, Methodology, and Realism

1.1 Introduction

1.2 Criticisms of the Idea of Scientific Method

1.3 Four Theories of Scientific Method

1.4 The Nature of Methodology

1.5 Scientific Realism

1.6 An Overview of the Abductive Theory of Method

1.7 Conclusion

2    Detecting Psychological Phenomena

2.1 Introduction

2.2 The Nature of Phenomena

2.3 Procedures for Phenomena Detection

2.4 Reasoning from Data to Phenomena

2.5 Phenomena Detection and the Nature of Psychological Science

2.6 Implications for Psychological Research

2.7 Conclusion

3    Theory Generation: Exploratory Factor Analysis

3.1 Introduction

3.2 The Inferential Nature of Exploratory Factor Analysis

3.3 Common Factor Analysis and Scientific Method

3.4 Exploratory Factor Analysis, Phenomena Detection, and Explanatory Theories

3.5 Exploratory Factor Analysis and Confirmatory Factor Analysis

3.6 Summary and Conclusion

4    Theory Development: Analogical Modeling

4.1 Introduction

4.2 Types of Models

4.3 Data, Models, and Theories

4.4 The Functions of Models

4.5 Modeling in ATOM

4.6 Analogical Modeling

4.7 Analogical Abduction

4.8 The Dramaturgical Model

4.9 Conclusion

5    Theory Appraisal: Inference to the Best Explanation

5.1 Introduction

5.2 Inference to the Best Explanation

5.3 Two Criticisms of Inference to the Best Explanation

5.4 Inference to the Best Explanation and Other Methods of Theory Appraisal

5.5 The Proper Scope of Inference to the Best Explanation

5.6 Implications for Psychology

5.7 Conclusion

6    Conclusion

6.1 Introduction

6.2 A Coda on Scientific Problems

6.3 Two Fundamental Commitments of ATOM

6.4 Phenomena Detection and Theory Construction Again

6.5 Two Applications of ATOM

6.6 ATOM Defended and Clarified

6.7 Scientific Method and Education

6.8 Final Word

Notes

References

Index

Preface

Although modern science is made up of many parts, scientific method is its centerpiece. The centrality of method to science stems from the fact that it provides scientists with the primary form of guidance in their quest to obtain knowledge about the world. As fallible inquirers, scientists face immense challenges in their efforts to learn about the complexities of nature. In good part, these challenges are met through the use of methods, which provide scientists with the cognitive assistance that they need to undertake successful inquiry.

However, despite its undoubted importance, scientific method receives less considered attention than it deserves, from both scientists and educators. Of course, scientists take method seriously, but I believe that they do not take it seriously enough. Scientists themselves, including psychologists, learn about research methods and how to use them to conduct their research. However, the nature of this learning, and of the instruction they receive about how to employ these methods, is better described as a mix of training and indoctrination than as a genuine education designed to provide a critical, in-depth understanding of the methods. Although professional science educators sometimes promote the importance of the epistemological foundations of scientific method, the influence of this source of learning on the regular teaching of research methods is minimal. Psychology, which provides extensively in its curriculum for teaching research methods, uses textbooks that make little or no effort to inform students in depth about the nature of scientific method. Nor does its curriculum foster a critical appreciation of the various research methods that its textbooks deal with. Consequently both psychological scientists and psychology students tend to have a limited understanding of scientific method, which in turn contributes to a misuse of research methods and a suboptimal level of scientific literacy.

I think that the missing key in this educational failure is scientific methodology. Methodology is the domain officially charged with fostering the evolution and understanding of scientific methods, and it is our official repository of knowledge about those methods. Scientific methodology is not the exclusive domain of any particular discipline. Rather, it is a central part of cognitive theory, which is itself regarded as an interdisciplinary endeavor. It spans the domains of statistics, the philosophy of science, the sociology of science, the various disciplines of cognitive science, and more; but it is reducible to none of them. As a practical endeavor, methodology is concerned with the mutual adjustment of means and ends. It judges whether methods are sufficiently effective for reaching certain goals. But methodology is also critically aim oriented and considers what goals the research enterprise should pursue. Clearly no single discipline can realistically aspire to cover all the tasks of methodology.

The methodological literature in psychology is dominated by the field of statistics. Quantitative methods receive the large majority of attention in both research methods textbooks and research practice. Qualitative research methods are regarded as a poor cousin and remain on the margins of methodology, although there are signs that they are gaining some acceptance. As important as statistical methods are to science, they cannot be all that there is to scientific method. Consequently the clarion call for statisticians to be the purveyors of scientific method (e.g., Marquardt, 1987) is inappropriate. The guiding assumption of this book is that treating scientific method with the seriousness it deserves requires taking scientific methodology seriously. I do this by giving special consideration to behavioral science methodology, the philosophy of science, and statistical theory. Thus the book is interdisciplinary in nature.

The philosophy of science figures more prominently in this book than is usual for methodology texts. The reason for this emphasis is that contemporary philosophy of science contains an array of important methodological insights that are impossible to ignore when coming to grips with scientific method. In recent years, philosophy of science has increasingly sought to understand science as it is practiced, and although it has much work to do in this regard, it now has important things to say about how science is, and should be, conducted. As part of this concern with scientific practice, philosophers of science have given increased attention to research methods in science. A positive development in this regard has been the focus on the methodology of experimentation over the last thirty years, although the methodology of theory construction remains the dominant focus in the philosophy of science.

Of late, philosophers of science have also shown a willingness to deal with methodological issues in sciences other than physics. Biology has been the major beneficiary, although psychology has received some philosophical attention. There is, then, a developing literature in contemporary philosophy of science that can aid both our understanding and our use of research methods and strategies in psychology (e.g., Trout, 1998). At the same time, a small number of theoretically oriented behavioral and social science methodologists have produced work on the conceptual foundations of research methods that helps illuminate those methods. Thus the work of both professional philosophers of science and theoretical scientists should be included in a philosophical examination of behavioral research methods.

Three major philosophies of science are of particular relevance to psychology: empiricism, social constructionism, and scientific realism (Greenwood, 1992; Manicas & Secord, 1983). Nineteenth-century British empiricism had a major influence on the development of British statistics in the first half of the twentieth century (Mulaik, 1985). The statistical methods developed in that intellectual milieu remain an important part of psychology’s statistical research practice. For example, Karl Pearson’s product moment correlation coefficient was taken by its founder to be the quantitative expression of a causal relation viewed in empiricist terms. Similarly, Ronald Fisher’s endorsement of inductive method as the proper view of scientific method stemmed from a commitment to the empiricism of his day. Even in today’s postpositivist philosophical climate, authors of research methods textbooks sometimes portray quantitative research as essentially positivist in its empiricist commitments (see Yu, 2006). The traditional empiricist outlook is much too limiting because it restricts its attention to what can be observed, and regards theories merely as instruments that organize claims about observables.

For their part, qualitative methodologists tend to bolster their preferred conception of qualitative research by comparing it with an unflattering positivist picture of quantitative research. At the same time, they frequently adopt a philosophy of social constructionism that is expressed in an implausibly strong form. This form is opposed to the traditional notions of truth, objectivity, and reason and maintains that our understanding of the world is determined by social negotiation. Such a view of social constructionism tends to be employed by those who are opposed or indifferent to quantitative methods. It is a view at odds with the philosophical outlook adopted in this book.

In what follows, I adopt a scientific realist perspective on research methods. Although the subject of considerable debate, and opposed by many antirealist positions, scientific realism is the dominant philosophy of science today. In addition, a commonsense version of realism seems to be the tacit philosophy of most working scientists. With its increasingly heavy emphasis on the nature of scientific practice, the philosophy of scientific realism is becoming a philosophy for science, not just a philosophy of science. Scientific realism is, in fact, a family of positions, and in chapter 1, I sketch a view of realism that I think is appropriate for psychology. Scientific realism boasts a rich conception of methodology, which can be of considerable help in understanding and guiding behavioral science research. It is a methodology that is at once naturalistic, problem focused, and aim oriented. It also promotes both generative and consequentialist reasoning, and the importance of justifying knowledge claims on both reliabilist and coherentist grounds. The influence of this conception of methodology occurs throughout the book.

In this book, I take psychology’s commitment to scientific method very seriously. I do this principally by constructing a broad theory of scientific method, which is genuinely informed by insights in contemporary scientific methodology and speaks to the conduct of psychological research. This account of method I call the abductive theory of method (hereafter ATOM) in recognition of the importance it assigns to explanatory reasoning. In contrast to the popular hypothetico-deductive method, ATOM portrays research as a bottom-up process comprising two broad phases. The first phase involves the detection of phenomena, such as empirical generalizations. The second phase involves the construction of explanatory theories to explain claims about the phenomena. The book draws from the new experimentalism (Ackerman, 1989) in philosophy of science to help illuminate the process of phenomena detection. It also examines in detail different abductive methods of theory construction, drawing, where appropriate, from the varied philosophical literature on abductive reasoning: the widely used method of exploratory factor analysis is presented as an abductive method of theory generation; the strategy of analogical modeling is presented as an abductive approach to theory development; and the neglected method of inference to the best explanation, particularly the theory of explanatory coherence, is presented as an appropriate method of theory appraisal.

An important feature of ATOM is that it functions as a broad framework theory within which a variety of more specific research methods can be located and employed. A coherent treatment of those methods is enhanced by placing them within the framework of ATOM. In turn, the specific methods help give ATOM a good deal of its operational detail. A number of the specific methods I refer to are well known to behavioral scientists, but some are not. Psychology has tended to emphasize data analytic methods at the expense of methods of theory construction. However, ATOM assigns equal importance to the two classes of method.

A subsidiary focus of this book is a concern with science education in relation to behavioral research methods. It follows John Dewey’s (1910) lead and suggests that we adopt an inquiry-oriented conception of education that accords an important place to scientific method. The narrow nature of, and uncritical approach to, the teaching and use of research methods in psychology are highlighted in some of the chapters. The need to teach for a more critical understanding of research methods is a natural consequence of acknowledging the importance of the domain of research methodology. In light of the requirements of a genuine liberal education, I make constructive proposals for reforming the methods curriculum. The nature of ATOM and its methodological foundations shape many of these curriculum proposals.

Chapter 1 introduces the topic of scientific method by providing some background material to better appreciate the more focused discussion of method in the ensuing chapters. I begin by briefly considering the idea of scientific method and different criticisms that have been leveled against it. Next I outline and provisionally assess four prominent theories of scientific method. I then move to a consideration of the nature of scientific methodology before providing a selective overview of the key elements of the philosophy of scientific realism. Finally, I present a brief overview of ATOM to provide a conceptual framework for locating and better understanding the various methods and strategies examined in the book.

Chapter 2 draws from the new experimentalism in the philosophy of science so as to reconstruct the important process of phenomena detection as it applies to psychology. In doing so, I propose a four-stage model of data analysis. The model begins with the initial examination of data, proceeds in turn through exploratory and confirmatory data analytic phases, and finishes with the stage of constructive replication. The threefold distinction between data, phenomena, and explanatory theory is drawn, and its implications for understanding the nature of psychological science are spelled out.

Chapter 3 considers the abductive nature of theory generation by examining the logic and purpose of the method of exploratory factor analysis. I argue that the common factors that result from using this method are not fictions but latent variables, which are best understood as genuine theoretical entities. I support this realist interpretation of factors by showing that exploratory factor analysis is an abductive generator of elementary theories that exploits an important heuristic of scientific methodology known as the principle of the common cause.

Science uses many different approaches to modeling. In chapter 4, I selectively examine one important approach to scientific modeling, analogical modeling. The strategy of analogical modeling is adopted by ATOM as its chief means of theory development. Accordingly, I spell out here the structure of analogical models and the use of analogical abductive reasoning both to expand and to evaluate the plausibility of models.

Chapter 5 recommends the use of inference to the best explanation for evaluating the worth of theories in psychology. I suggest that it is a more appropriate account of theory appraisal than both the popular hypothetico-deductive method and the widely heralded Bayesian approach. I discuss a number of different explications of inference to the best explanation, in particular the theory of explanatory coherence, which is the most detailed extant explication of inference to the best explanation.

The concluding chapter rounds out the extended characterization of ATOM. First I outline an account of the nature of research problems, and then I discuss the nature and limits of ATOM. This is followed by applications of ATOM to grounded theory method and to clinical reasoning. Toward the end of the chapter, I offer some thoughts about the importance of methodology for understanding research methods. The book concludes with some brief remarks about the future prospects for ATOM.

The methodology of the behavioral sciences is a subject of relative neglect in professional philosophy of science. Thus my hope is that this book will be welcomed by those in the philosophical community who want to learn about an important set of methodological practices in one of the interesting special sciences. Conversely, I would like to think that the book contains material that will enable psychological researchers to deepen their conceptual appreciation of a variety of research methods and associated methodological matters and thereby contribute to the conduct of sound psychological research. Although the book’s primary focus is on psychology, I believe its contents are relevant to the behavioral sciences more generally.

Finally, I draw the reader’s attention to two matters. First, it is sometimes important to distinguish between scientific method as a theoretical understanding of an inquiry procedure and scientific method as a material practice. Given the book’s primary concern with ATOM, it mostly focuses on a theoretical understanding of method. Second, I have endeavored to keep abbreviations to a minimum. However, for convenience, I abbreviate the abductive theory of method as ATOM throughout the book. I also use abbreviations for exploratory factor analysis and inference to the best explanation in their respective chapters.

Acknowledgments

In the mid-1990s, Hillary Clinton argued that it takes a village to raise a child. The assertion occasioned a skeptical response from a number of quarters, but there can be no doubting the claim that it takes a village to raise a book. This book is no exception, for it has depended on the support of many people, and a number of institutions, over several decades.

Although my interest in scientific method spans more than forty years, the book project that became Investigating the Psychological World began in earnest in the second half of 2011 while I was on study leave with Denny Borsboom at the University of Amsterdam. I am grateful to Denny for his stimulating intellectual hospitality during that time, and for opportunities to collaborate with him in recent years and benefit from his versatile mind.

My interest in abductive reasoning began in the mid-1960s when I read the remarkable Charles Peirce’s unpopular ideas on education, and was kindled when I started to acquaint myself with some of his work on science. That interest was nourished in the early 1970s by my doctoral supervisor, Bill Rozeboom, who was one of the first psychologists to appreciate the importance of abductive reasoning in science. Bill’s critical acumen as a technically accomplished theoretical psychologist has helped shape my thinking on a number of methodological issues in psychology.

More recently, the work of a number of contemporary philosophers of science has been invaluable in helping me understand some of the complexities and subtleties of scientific methodology. In this regard, the following individuals deserve special mention: Tom Nickles for his richly suggestive writing on scientific method and scientific discovery; Jim Woodward and Jim Bogen for their instructive conceptualization of the process of phenomena detection; Rom Harré for his insightful depiction of the use of models and analogies in science; and Paul Thagard for his writing on inference to the best explanation, in particular his theory of explanatory coherence, which I regard as a major methodological accomplishment. I judge these philosophers as seminal thinkers on the topics just mentioned, and their influence on what I have to say about scientific method will be obvious to the reader.

Other people have also helped me develop ideas about research methods, including Adrienne Alton-Lee, Paul Barrett, Neville Blampied, Russil Durrant, Cameron Ellis, Colin Evers, Garth Fletcher, David Funder, James Grice, Deak Helton, Stephen Hill, Hugh Lauder, Tom Maguire, Dannette Marie, Keith Markus, Joel Michell, Katharina Näswall, Claire O’Loughlin, Denis Phillips, Robert Proctor, Vivianne Robinson, Bruce Ryan, Ken Strongman, Anton Turner, Tony Ward, Juliane Wilcke, and Brad Woods. Although I have profited from my interactions with all these people, two of them deserve special mention. David Funder has strongly supported my writing on method, and more besides. As a prominent empirical researcher in personality and social psychology, his belief that I am on the right track, methodologically speaking, has been reassuring. I have also benefited considerably from Keith Markus’s intellectual generosity and his ability to conceptualize methodological issues in behavioral research methods in an insightful manner.

My greatest debt of gratitude is to my wife, Fran Vertue, who has nourished and sustained me for sixteen good years. Her constant intellectual companionship has contributed in innumerable ways to the development of this book and other writing projects.

I thank Kim Sterelny and Rob Wilson for deciding that a book on the methodology of the behavioral sciences belonged in their MIT Press series Life and Mind. I am grateful to Philip Laughlin, senior acquisitions editor in philosophy and cognitive science at MIT Press, for his expert guidance on this project. Judy Feldmann, senior editor at MIT Press, gave valuable help in the final preparation of the manuscript.

Fran Vertue, Ken Strongman, and Brad Miles read a complete draft of the book and gave me very helpful feedback. Brad also assisted in preparing the manuscript for submission.

The material in chapters 2, 3, and 5 draws heavily from previously published papers.