Realismasa Stancefor Mixed Methods Researchfinal
Realismasa Stancefor Mixed Methods Researchfinal
Joseph A. Maxwell
George Mason University
Different terms have been used for such versions of realism, including
"critical" realism (Bhaskar, 1989; Archer, Bhaskar, Collier, Lawson, & Norrie
1998), "experiential" realism (Lakoff, 1987), "subtle" realism (Hammersley,
1992), “emergent” realism (Henry, Julnes, & Mark, 1998; Mark, Henry, &
Julnes, 2000), "natural" realism (Putnam, 1999), “innocent” realism (Haack,
1998, 2003), and “agential” realism (Barad, 2007). We will use the term
“critical realism” in a broad sense to include all of these versions of realism.
(We provide a more detailed description of realism later in this chapter.)
influence researchers’ purposes and actions to some degree, and are often
implicit and not easily abandoned or changed.
This dichotomous and polar view of the two approaches has been
challenged by Hammersley (1992) and Howe (2003), and more recently by
Bergman (2008b), Biesta (this volume), Hammersley (2008), and Fielding
(2008), who argue for a more complex and contextualized understanding of
the potential contributions of each approach. Many of the contributors to the
volume edited by Bergman (2008a) have serious reservations about the way
mixed method research has been conceptualized, and see the
qualitative/quantitative distinction as much more problematic than has
traditionally been assumed. Bergman claims that the assumption of generic
“strengths” of each approach, based on paradigm differences, is fallacious,
and that the conventional divide between qualitative and quantitative
methods is to a considerable degree related to “delineating and preserving
identities and ideologies rather than to describe possibilities and limits of a
Maxwell & Mittapalli, Realism and mixed methods 5
Example 1 [box]
This move was consistent with the field of sociology of sport, which
was generally seen as constructionist in orientation. However, underlying
Chambliss’s argument for a constructivist interpretation of “talent” was a
realist move, identifying actual skills and practices, and “excellence” as the
outcome of these, as real phenomena rather than simply constructions. As a
result, his work was attacked by others in this field for not treating
“winning,” and the skills that led to this, as themselves social constructions.
Chambliss’s reply was that while selecting winners on the basis of elapsed
times, rather than the beauty or precision of their strokes, was certainly a
social construction, once that construction was made, the factors that lead to
Maxwell & Mittapalli, Realism and mixed methods 7
success in terms of that standard, and the outcomes of races, have a real
existence independent of how they are construed by participants and judges.
The rest of this paper explores some of the specific uses of realist
conceptual tools in social research. First, however, we need to describe
realism in more detail as a general approach in both the natural and social
sciences.
What is realism?
The distinctive feature of these forms of realism is that they deny that
we have any objective or certain knowledge of the world, and accept the
possibility of alternative valid accounts of any phenomenon. All theories
about the world are grounded in a particular perspective and world view, and
all knowledge is partial, incomplete, and fallible. Lakoff states this distinction
between "objectivist" and "realist" views as follows:
Scientific objectivism claims that there is only one fully correct way in
which reality can be divided up into objects, properties, and
relations. . . . Scientific realism, on the other hand, assumes that "the
world is the way it is," while acknowledging that there can be more
than one scientifically correct way of understanding reality in terms of
conceptual schemes with different objects and categories of objects.
(1987, p. 265)
Example 2 [box]
Like all theories, they are models, in Geertz’s (1973) terms, both
models of and models for, but especially, they are models for;
scientific theories represent in order to intervene, if only in search of
confirmation. And the world in which they aim to intervene is, first and
foremost, the world of material (that is, physical) reality. For this
reason, I prefer to call them tools. From the first experiment to the
latest technology, they facilitate our actions in and on that world,
enabling us not to mirror, but to bump against, to perturb, to
transform that material reality. In this sense scientific theories are
tools for changing the world.
But even granted that they are constructed, and even abandoning the
hope for a one-to-one correspondence with the real, the effectiveness
of these tools in changing the world has something to do with the
relation between theory and reality. To the extent that scientific
theories do in fact “work”—that is, lead to action on things and people
Maxwell & Mittapalli, Realism and mixed methods
10
that, in extreme cases (for example, nuclear weaponry), appear to be
independent of any belief system—they must be said to possess a kind
of “adequacy” in relation to a world that is not itself constituted
symbolically—a world we might designate as “residual reality.”
11
qualitative, and mixed method research, showing how a realist perspective
can provide new and useful ways of approaching problems and important
insights into social phenomena.
12
(Maxwell, 1995, 1999). It differs from these approaches primarily in its
realist ontology—a commitment to the existence of a real, though not an
“objectively” knowable, world—and its emphasis on causal explanation
(although a fundamentally different concept of causal explanation than that
of the positivists) as intrinsic to social science.
There are many aspects of mixed method research for which realism
provides a valuable perspective. For example, it is useful to view research
designs as real entities--not simply as models for research, but also as the
actual conceptualizations and practices employed in a specific study. The
latter approach helps a reader of a research publication to understand the
real design of a study, its “logic-in-use,” which may differ substantially from
its “reconstructed logic” (Kaplan, 1964, p. 8) presented in publications
(Maxwell & Loomis, 2003; Maxwell, 2005). This conception of design as a
model of, as well as for, research is exemplified in a classic qualitative study
of medical students (Becker, Geer, Hughes, & Strauss, 1961; see example
3).
Example 3 [box]
Becker et al. (1961) begin their chapter on the "Design of the Study"
by stating that
In one sense, our study had no design. That is, we had no well-
worked-out set of hypotheses to be tested, no data-gathering
instruments purposely designed to secure information relevant to
these hypotheses, no set of analytic procedures specified in advance.
Maxwell & Mittapalli, Realism and mixed methods
13
Insofar as the term "design" implies these features of elaborate prior
planning, our study had none.
In what follows, we focus on four issues for which we feel realism can
make a particularly important contribution to mixed method research: causal
explanation, mind and reality, validity, and diversity.
14
tends to be associated with research that employs experimental or
correlational designs, quantitative measurement, and statistical analysis. As
Mohr noted, "the variance-theory model of explanation in social science has
a close affinity to statistics. The archetypal rendering of this idea of causality
is the linear or nonlinear regression model" (1982, p. 42).
This approach is quite different from variance theory. Pawson and Tilley
(1997), in their realist approach to program evaluation, stated that
Maxwell & Mittapalli, Realism and mixed methods
15
When realists say that the constant conjunction view of one event
producing another is inadequate, they are not attempting to bring
further “intervening” variables into the picture . . . The idea is that the
mechanism is responsible for the relationship itself. A mechanism
is . . . not a variable but an account of the makeup, behaviour and
interrelationship of those processes which are responsible for the
regularity. (pp. 67-68)
16
is not fixed, but contingent” (p. 69); it depends on the context within which
the mechanism operates. This is not simply a claim that causal relationships
vary across contexts; it is a more fundamental claim, that the context within
which a causal process occurs is, to a greater or lesser extent, intrinsically
involved in that process, and often cannot be “controlled for” in a variance-
theory sense without misrepresenting the causal mechanism (Sayer, 2000,
pp. 114-118). For the social sciences, the social and cultural contexts of the
phenomenon studied are crucial for understanding the operation of causal
mechanisms.
17
However, realists are not dualists, postulating two different realms of
reality, the physical and the mental. In our view, the clearest and most
credible analysis of this issue has been that of Putnam (1990, 1999), who
argued for the legitimacy of both “mental” and “physical” ways of making
sense of the world. He advocated a distinction between mental and physical
perspectives or languages, both referring to reality, but from different
conceptual standpoints. He argued that “The metaphysical realignment we
propose involves an acquiescence in a plurality of conceptual resources, of
different and mutually irreducible vocabularies . . .coupled with a return not
to dualism but to the ‘naturalism of the common man.’” (1999, p. 38)
Thus, while realism rejects the idea of "multiple realities" in the sense
of independent and incommensurable worlds in which different individuals or
societies live, it is quite compatible with the idea that there are different
valid perspectives on the world. However, it holds that these perspectives,
as held by the people we study, are part of the world that we want to
understand, and that our understanding of these perspectives can be more
or less correct (Phillips, 1990).
Realism also supports the idea that individuals' social and physical
contexts have a causal influence on their beliefs and perspectives. While this
proposition is widely accepted in everyday life, constructivists have tended
to deny the “reality” of such influences, while positivism and some forms of
post-positivist empiricism tend to simply dismiss the reality or importance of
individuals’ perspectives, or to “operationalize” these to behavioral variables.
From a realist perspective, not only are both individuals’ perspectives and
their situations real phenomena, they are separate phenomena that causally
interact with one another.
Maxwell & Mittapalli, Realism and mixed methods
18
In doing this, a realist perspective can provide a framework for better
understanding the relationship between individuals’ perspectives and their
actual situations. This issue has been a prominent concern in the philosophy
of social science for many years (e.g., MacIntyre, 1967; Menzel, 1978), and
is central to “critical” approaches to qualitative research. Critical realism
treats both individuals’ perspectives and their situations as real phenomena
that causally interact with one another. In this, realism supports the
emphasis that critical theory places on the influence that social and
economic conditions have on beliefs and ideologies. Sayer (1992, pp. 222-
223) stated that the objects of “interpretive” understanding (meanings,
beliefs, motives, and so on) are influenced both by the material
circumstances in which they exist and by the cultural resources that provide
individuals with ways of making sense of their situations. However, critical
realism approaches the understanding of this interaction without assuming
any specific theory of the relationship between material and ideational
phenomena, such as Marxism.
Validity and quality are issues for which there have been substantial
disagreements between qualitative and quantitative researchers. The types
of “validity” (many qualitative researchers don’t even use this term)
employed in each tradition have little overlap (Teddlie & Tashakkori, 2003),
and the basic assumptions involved in the two approaches are radically
different. Teddlie and Tashakkori went so far as to recommend abandoning
the term “validity” entirely in mixed method research, arguing that the term
has taken on such diverse meanings that it is losing its ability to
communicate anything (pp. 12, 36-37).
Maxwell & Mittapalli, Realism and mixed methods
19
Despite these differences, there is an important similarity between the
typical quantitative and qualitative approaches to validity. Both focus largely
on the procedures used in collecting data and drawing inferences from these
data. This is particularly obvious in the movement for “evidence-based”
research, which relies almost entirely on the type of research design as the
bases for assessing the validity of the results, with randomized experiments
as the “gold standard” for design quality. However, it also characterizes
prominent approaches to validity (or its analogues) in qualitative research.
20
For these reasons, the main approach to validity in experimental
research (e.g., Shadish, Cook, and Campbell, 2002) is grounded in the
concept of a validity threat—a possible way that a conclusion might be
wrong—and ways to address these threats. However, the emphasis has
largely been on the designs and methods used to deal with these threats.
This has been facilitated by the fact that this literature has, consistently with
a regularity view of causality as inherently general, dealt mainly with types
of validity threats, rather than emphasizing the actual ways a specific
conclusion might be wrong in a given study. The importance of the latter
point is implicit in the realist argument above, that validity is not simply
determined by procedures (although procedures are obviously relevant to
the validity of a conclusion), but must be assessed in the specific context of
a particular study. It is also an implication of a realist view of causality as
inherently local rather than general.
Similar arguments about diversity have been made for the social
sciences, as described above in discussing paradigms. However, both
qualitative and quantitative research have tendencies, theoretical as well as
methodological, to ignore or suppress diversity in their goal of seeking
general accounts, though in different ways (Maxwell, 1995). Quantitative
research often aggregates data across individuals and settings, and ignores
individual and group diversity that cannot be subsumed into a general
explanation (Shulman, 1986). Because of its emphasis on general
descriptions and causal theories, it tends to impose or generate wide-
ranging but simplistic models that do not take account of individual
Maxwell & Mittapalli, Realism and mixed methods
21
variation, unique contextual influences, diverse meanings, and idiosyncratic
phenomena.
22
realist and multiplist approach, saying that this skeptical realism and holism
“provides the context and tradition for much of our work” (2005, pp. 5-6),
but doesn’t discuss how, specifically, realism did so.
As noted above, the one area in which realist perspectives have had a
major influence on mixed method studies is in program evaluation. The work
of Tilley (described in Pawson & Tilley, 1997) and Mark, Henry, and Julnes
(2000; Henry, Julnes, & Mark, 1998), much of which combined qualitative
and quantitative approaches, has provided a realist alternative to traditional
ways of conceptualizing program evaluations. In addition, Pawson’s (2006)
analysis of literature reviews for evidence-based policy constitutes a major
critique of standard ways of integrating qualitative and quantitative results in
a literature synthesis, and presents a realist alternative to these approaches.
23
In economics, critical realism points to the main limitations of
neoclassical economics (based on econometrics principles that are
reductionist in nature and presuppose that concepts can be measured,
counted, manipulated, and cross-classified), and it provides a philosophical
and methodological foundation for a broad set of alternative approaches (see
Downward, Finch & Ramsay, 2002; Downward & Mearman, 2007;
Fleetwood, 1999; Lawson, 1989, 1997, 1998, 2001). In this regard, within
economics, critical realism supports Lawson’s view that the exclusive
dependence on mathematical/statistical modelling in economics is misguided
(Castellacci, 2006).
24
from semi-structured interviews. Using mixed methods gave the inquiry a
“better sense of balance and perspective” (p. 66). Additionally, the findings
from both approaches stimulated retroductive reasoning, a process that
involves the construction of hypothetical models as a way of uncovering the
real structures, contexts and mechanisms that are presumed to produce
empirical phenomena (Bhaskar, 1978, 1986, 1989). In addition, reliance on
retroduction necessitates that the researcher is being explicit about what is
being done during the process, including data collection and analysis
(Bollingtoft, 2007), leading to the development of a theoretical model that
explained why gatekeeping decisions tended to emerge in the way they did.
References
Archer, M., Bhaskar, R., Collier, A., Lawson, T., & Norrie, A. (Eds.) (1998).
Critical realism: Essential readings. London: Routledge.
Barad, K. (2007). Meeting the universe halfway: Quantum physics and the
entanglement of matter and meaning. Durham, NC: Duke University
Press.
25
Becker, H. S. (1990). Generalizing from case studies. In Eisner, E. W. &
Peshkin, A. (Eds.), Qualitative inquiry in education: The continuing debate
(pp. 233-242). New York: Teachers College Press.
Becker, H. S., Geer, B., Hughes, E. C., and Strauss, A. L. (1961). Boys in
white: Student culture in medical school. Chicago: University of Chicago
Press. Reprinted by Transaction Books, 1977
26
Buchler, J. (1940). “Introduction,” in Buchler (Ed.), The philosophy of Peirce:
Selected writings. Routledge and Kegan Paul.
Conant, J., & Zeglen, U. M. (2002). Hilary Putnam: Pragmatism and realism.
London: Routledge.
Covaleski, M.A., & Dirsmith, M.W. (1983) Budgets as a means of control and
loose coupling. Accounting, Organizations and Society, 8, 323-340.
27
Fleetwood, S. (1999). Critical realism in economics: Development and
debate, London: Routledge.
Greene, J., & Hall, J. (this volume). The dialectical stance and pragmatism:
Interplay between philosophy and practice in mixed method research
Forthcoming in A. Tashakkori & C. Teddlie, Handbook of mixed methods
in social and behavioral research, 2nd edition. Thousand Oaks, CA: Sage.
Guba, E.G., & Lincoln, Y.S. (1989). Fourth generation evaluation. Thousand
Oaks, CA: Sage.
Henry, G., Julnes, J., & Mark, M. (1998). Realist evaluation: An emerging
theory in support of practice. New Directions for Evaluation 78. San
Francisco: Jossey-Bass.
Maxwell & Mittapalli, Realism and mixed methods
28
Hoque, Z., & Hopper, T. (1994). Rationality, accounting and politics: A case
study of management control in a Bangladeshi jute mill. Management
Accounting Research, 5, 5-30.
Hoque, Z., & Hopper, T. (1997). Political and industrial relations turbulence,
competition and budgeting in the nationalised jute mills of Bangladesh.
Accounting and Business Research, 27, 125-143
Johnson, B., & Gray, R. (this volume). The history of integrated research in
the social and behavioral sciences. Forthcoming in A. Tashakkori & C.
Teddlie, Handbook of mixed methods in social and behavioral research,
2nd edition. Thousand Oaks, CA: Sage.
Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied
mind and its challenge to western thought. New York: Basic Books.
29
Lawson, T. (2001). Reorienting Economics. London, Routledge.
30
Maxwell, J. A. (2004b). Using qualitative methods for causal explanation.
Field Methods 16(3), 243-264 (August 2004).
31
Mohr, L. B. (1982). Explaining organizational behavior. San Francisco:
Jossey-Bass.
32
Putnam, H. (1990). Realism with a human face, edited by James Conant.
Cambridge: Harvard University Press.
Putnam, H. (1999). The threefold cord: Mind, body, and world. New York:
Columbia University Press.
33
Smith, J. K., & Deemer, D. K. (2002). The problem of criteria in the age of
relativism. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of
qualitative research, 2nd edition (pp. 877-896). Thousand Oaks, CA:
Sage.
Tashakkori, A., and Teddlie, C. (2003). The past and future of mixed method
research: From data triangulation to mixed model designs. In A.
Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social
and behavioral research (pp. 671-701). Thousand Oaks, CA: Sage.
Weber, M. (1905). Critical studies in the logic of the social sciences. Archiv
fuer Sozialwissenschaft und Sozialpolitik. Reprinted in Weber, The
Methodology of the Social Sciences. New York, The Free Press, 1949.
Objectives:
34
3. Understand why the view of paradigms as logically unified sets of
premises that are shared by members of a research community is
problematic.
1. How does the realist perspective presented in this chapter fit with your
own assumptions about qualitative, quantitative, and mixed method
research? Has the chapter changed your thinking about any of these? Do
you disagree with any of the chapter’s arguments? Why?
2. How could you apply the specific realist approaches described here to
an actual study that you might conduct? What difference would these make
in how you design and carry out the study?