Evaluating Research Methods:

Assumptions, Strengths, and Weaknesses of Three Educational Research

Paradigms

Angela M. Velez, PhD

Faculty Development Coordinator
English Instructor
Davenport University
E-mail: [email protected]

Introduction

In educational research, two distinct research paradigms are common—qualitative and

quantitative. Scholars who utilize these paradigms have generally deemed themselves
either qualitative or quantitative researchers, called “purists” (Onwuegbuzie & Leech,
2005). Rarely, though, does one see a qualitative researcher cross the border to
quantitative territory, and vice versa, because the epistemological and axiological
assumptions are distinct in these research paradigms. However, there has been a recent
surge in the use of mixed methodology research in education because the data that
emerge from these studies involve crossing borders and mixing methods to understand
problematic situations; data are stronger, more valid, more generalizable, and tend to be
much more extensive than data that emerge from single-method studies. This article
seeks to explore the assumptions, strengths, and weaknesses of all three research
paradigms—qualitative, quantitative, and mixed methods—to inform doctoral students or
doctoral candidates about methodological issues that need consideration when they
consider a research study.

Assumptions, Strengths, and Weaknesses of Paradigms

Assumptions: Quantitative Research

Quantitative research, like other research paradigms, has its own set of assumptions. For
example, it seeks to understand the facts or causes of phenomena and does not regard the
subjective states of a situation or of individuals (Reichardt & Cook, 1979); it also claims
to be value-free (Stanfield, 2006). Further, quantitative research defends positivism and
believes in the benefit of hypothetical-deductive procedures (Morales, 1995).
Quantitative researchers claim that science is rooted in objective verification but do not
account for the subjective nature of the researcher’s decisions made throughout the stages
of the research process (Onwuegbuzie & Leech, 2005). In addition, often the
epistemological assumptions of positivistic quantitative inquiry are that what happens in
one social environment being studied in this method can be generalized to future social
situations (Gall, Gall, & Borg, 2003). Lastly, quantitative inquiry assumes that science is
the superior way of knowing, understanding and predicting human experiences and that

1
the positivistic scientific method rules must be adhered to or the researchers and their
findings are disregarded (Stanfield, 2006).

In sum, quantitative assumptions regard reality, experience, and situations as quantifiable,

as measurable. What is not measurable is not worthy of being reported, and if something
is measured, validated and generalizable, then that something is generalized to all
populations that are similar to what was studied and reported because reality is
independent to personal experience (Gall, Gall & Borg, 2003).

Assumptions: Qualitative Research

Qualitative researchers are not immune to significant assumptions. First, qualitative

inquiry believes that reality is subjective and that social environments are personal
constructs created by individual interpretations that are not generalizable (Gall, Gall &
Borg, 2003); these beliefs are rooted in constructivism rather than positivism. Therefore,
the assumption is that there is not a generalizable reality that is quantifiable for a larger
population than an individual case. Qualitative researchers assume that rich description
and a deep understanding are indicative of their methodology, which insinuates that other
research paradigms are not deep (Sechrest & Sidani, 1995). In terms of axiology,
qualitative researchers believe that research is influenced by the values held by the
researcher as well as by the theories, hypotheses or the framework that the researcher is
using in his or her particular situation (Tashakkori & Teddlie, 1998).

One area of chief interest and concern in qualitative inquiry is context. Qualitative
inquiry assumes that without a clear understanding of the contextual nature of a research
study the reported data are not generalizable (Sechrest & Sidani, 1995). Overall,
qualitative researchers not only believe in contextualized perspectives but also those
which are pluralistic, interpretive, and open-ended (Cresswell & Miller, 2000). To
qualitative researchers, the world is “shifting, changing, dynamic” (Filstead, 1979).

Assumptions: Mixed Methods Research

The last paradigm to explore in terms of assumptions is mixed methods. The largest
assumption that the mixed method paradigm has is that most comprehensive research has
a combination of both quantitative and qualitative methods in their studies. Researchers
who employ mixed methods are not research “purists”, and most scholars who use strictly
quantitative or qualitative designs do not welcome mixed methods with open arms
(Tashakkori & Teddlie, 1998). Hence, they assume that to have a strong study with valid
and generalizable results, one cannot use only one research method.

Mixed method research has its roots not in positivism or constructivism, but in
pragmatism. To a mixed methods researcher, the method is not as important as the type
of research question that is asked (Cresswell, 2003). Mixed methods researchers do not
subscribe to any one philosophy; thus, they can use assumptions from both quantitative
and qualitative paradigms as the research question deems appropriate (ibid.). In addition,
while quantitative and qualitative approaches may have their own limitations separately,

2
mixed methods researchers feel that those limitations can be lessened by choosing
methods that compliment each other (Hammond, 2005). Scholars who believe strongly
in mixed methodology research believe that mono-method research is a great threat to
advancement in areas like education (Onwuegbuzie & Leech, 2005).

Strengths: Quantitative Research

In quantitative research, a noted strength is that what is being studied is not affected by
the researcher as he or she is not usually involved with the subject or subjects being
studied. What this does is decrease the chances of people’s responses or behaviors being
affected or influenced by the outside researcher (Gall, Gall & Borg, 2003). This strength,
however, has been questioned by others as it is improbable that no influence at all takes
place in research, regardless of the method or paradigm used.

Another strength of quantitative study is that it has been a strong component of a great
deal of published research data, and because of its ubiquity, rules, processes, templates,
regulations and other guiding principles are available for researchers to use, copy, and
clarify their research designs (ibid.) This means that there is consistency in process and
procedure for studies that are labeled causal-comparative, for example, or quasi-
experimental, to name just a few. Quantitative studies are said to be replicable and often
the instruments created for quantitative studies are used in further research due to the
rigors of creating an instrument that effectively measures a certain construct that can be
used in different social or educational contexts while also being valid and reliable
(Creswell, 2003).

Johnson and Onwuegbuzie (2004) suggest some strengths of quantitative research are
that it tests and validates theories that are presently constructed about how and sometimes
why phenomena happen; the data collection process is fairly quick; data is precise and
numerical; it allows for generalizability when data are drawn from fairly large random
samples (p. 19). Finally, quantitative studies do have strength in that they produce
answers (if done properly) that are solid, unlike an opinion or common sense answer
(Ratnesar & Mackenzie, 2006).

Strengths: Qualitative Research

Qualitative methods tend to be rich in narrative and description, and instead of providing
an outcome they tend to discuss the process. Contrary to the goals of qualitative inquiry,
quantitative inquiry does not attempt to first understand the contexts in which the humans
being studied are interacting. While quantitative data can describe data numerically, it
cannot and does not go one step further, as qualitative inquiry does, and first make sense
of what is being observed, then understand it, and lastly discover the meaning through
detailed explanations that do not exist in quantitative studies (Filstead, 1979).

Qualitative inquiry increases individual understanding of “otherness” through its in-depth

studies of specific groups, for the goal is not to explain their reality but to understand it
(Morales, 1995). The understanding of this “otherness” is rooted in the tradition’s

3
anthropology, sociology, human geography, cultural studies, and social psychology
(Atkinson & Delamont, 2006). While they are not traditional sciences, their histories are
long and rich and provide a firm foundation for high-quality studies in the qualitative
paradigm. For Johnson and Onwuegbuzie (2004), qualitative inquiry’s strengths include
that: it is useful for describing complex phenomena; it is usually collected in naturalistic
settings; it is responsive to local conditions and the needs of those studied and those who
are informed by the study; the words of the participants lend to further studies into how
and why phenomena occur; it allows for the study of dynamic processes (p. 20). If it does
nothing except “bring about healing, reconciliation, and restoration between the
researcher and the researched” (Stanfield, 2006, p. 725), it will have made a positive
impact on the lives of at least the people involved in a qualitative study.

Strengths: Mixed Methods Research

Mixed methodology’s first strength is that it is a “workable solution” to the seemingly

unending debates between qualitative and quantitative purists (Johnson & Onwuegbuzie,
2004). Another strength lies in its purpose, which is significant (Greene, Caracelli &
Graham, 1989). Through the research of Greene, et. al. (1989), mixed methods
researchers can ground their research in meaningful theory with a meaningful purpose.
Their empirical study found five general purposes for mixed method paradigms:
triangulation, complementarity, development, initiation, and expansion (p. 255).

Quantitative and qualitative methodologies have their own biases and limitations as
separate entities, but when two methods are chosen distinctly because their biases and
limitations cancel each other out, the result is triangulation as well as a much more solid
study than if just one method was used (ibid.). The combining nature of this method is
what makes it an intriguing alternative to a mono-method study. Unlike the dichotomies
of qualitative and qualitative, mixed methods is both deductive and inductive; it is both
objective and subjective; it is entirely practical and applicable to many researchers as it
allows a researcher to study what is important to him or her, it allows the researcher to
vary the methods used to study that interest, and it allows the researcher to use the results
of the study to create positive movements in the researcher’s own specific area of interest
(Tashakkori & Teddlie, 1998). It allows a researcher to begin the study with a positive
attitude towards both quantitative and qualitative inquiry; it allows him or her to narrow
or expand a focus as needed, where he or she can dive much further into the data to
understand meaning than they could if a single method was used (Onwueguzie & Leech,
2005). Mixed methods research is flexible in that words can be used to add meaning to
the quantitative data and numbers can be used to inform or supplement the words; the
researcher can explore a broad range of questions because he or she is not restrained by
their research methodology’s individual paradigm; finally, it provides stronger
concluding evidence through convergence and corroboration of the research findings
(Johnson & Onwuegbuzie, 2004, p. 21).

Weaknesses

4
While all three methods do have similarities—they are all empirical, they all seek to
describe data, they all configure arguments to explain that data, and they all theorize
about why a phenomena occurred in the way that it did (Sechrest & Sidani, 1995)—their
differences are seen as weaknesses by paradigm purists. For example, qualitative purists
argue that quantitative inquiry does not offer any narrative explanations of the data, and
sometimes those explanations are needed to understand the contexts in which the data
were gathered. Further, quantitative inquiry claims to be value-free (free from researcher
bias, unlike the value-laden qualitative research), but some scholars argue that this is not
possible because researchers are humans and cannot be neutral or value free in any
circumstance (Stanfield, 2006).

Weaknesses: Quantitative Research

One of the first weaknesses of quantitative inquiry that sets it apart from qualitative and
mixed methods is that it is difficult to read and understand (Burns, 2000). The statistical
aspects of a quantitative report can be technical and difficult to distinguish for average
readers of educational journals.

Quantitative research has seen resurgence in the American edu-political system due to the
2002 No Child Left Behind Act and the 2001 Education Sciences Act, which base much
of their claims on quantitative data, and it is the quantitative data that matters to policy
makers (Wright, 2006). What is considered worthy of report and policy influence are
quantitative data, which leaves many educators shaking their heads as these data are
small representations of what is really going on in American schools and in American
higher education. Some scholars and philosophers note that while quantifying
information might be the “path of least resistance”, it might also be “the path of least
significance” (Wartofsky, 1968 cited in Gall, Gall & Borg, 2003). This criticism is
related to quantitative inquiry’s emphasis on statistical significance without regard for the
significance of the effects involved (Tashakkori & Teddlie, 1998).

Weaknesses: Qualitative Research

Scholars outside of the United States have criticized American qualitative research and
do not understand why it is that qualitative research is not worthy of the respect it has
earned in other countries like the United Kingdom (Atkinson & Delamont, 2006). It has
been suggested that American qualitative researchers do not look at their educational
systems as problematic but rather as institutions that need exploring in order to define or
describe the nature of the institutions versus institutions that have significant problems in
need of resolution (Atkinson & Delamont); further, qualitative researchers need to avoid
ethnocentricity and perform their research with a firm understanding of its very rich
history in order “to produce thick descriptions in a reflexive manner” (p. 750). It could
be possible that this is due to the trend of American institutions of higher education
orienting their graduate students with competencies in mainly one (quantitative) research
paradigm and not enough in any other (qualitative and mixed); what happens is that
young scholars are not taught that research is a holistic process (Onwuegbuzie & Leech,

5
2005), not a dichotomous battle between competing methodologies whose sides they
must choose between in order to fit in the inflexible walls of the American Academy.

A weakness of qualitative inquiry, from a quantitative perspective, is that its context-rich,

value-laden, narrative-filled reports contain too much fluff and not enough hard evidence.
Further, since the researcher is actively involved in the study and is likely passionate
about either who is being studied or the context of the study that passion can be, and is, a
form of bias and can perhaps lead to an inability to see other explanations for what is
taking place (Krantz, 1995).

Because qualitative inquiry, by nature, is much more focused on a small population or

case, it is criticized for its inability to include a broader range of information in its
analyses; it is also negatively noted that qualitative researchers might express personal
opinion more than report accurate findings from the study and choose to be selective in
what they report as results (Tashakkori & Teddlie, 1998).

In terms of practicality, the more insightful the qualitative research is, the more time
consuming it becomes (Atkinson & Delamont, 2006). Data gathering is not a simple
process and involves many opportunities for human error, unlike some quantitative
methods that involve inputting numerical data into a software program to get almost
immediate results. Due to its narrative aspect, errors can and do occur in all stages of
qualitative research. For example, language is the root of qualitative research; because of
that, meaning is incredibly important. That importance is significant because a certain
term or phrase might mean one thing to one participant in a study but mean another thing
to another participant, and it also might mean something entirely different to the
researcher (Gall, Gall & Borg, 2003).

A final caveat of qualitative inquiry is that it is more difficult to find scholarly resources
that actively publish qualitative manuscripts than it is to find scholarly research for
quantitative reports (Krantz, 1995). The reason might be that, unlike quantitative inquiry,
qualitative methods do not yet have rigorous standards for reporting findings; another
reason is that it is facing a great deal of backlash due to an increased focus on
quantitative empirical data in education related to new government policies, which are
fundamentalist and positivist in conception (Denzin & Lincoln, 2005).

Weaknesses: Mixed Methods Research

In terms of mixed method designs, it is important to understand that mixed method

designs are not always the appropriate choice for all research (Onwuegbuzie & Leech,
2005). The question must decide which approach is best. Besides the appropriateness for
the research question, mixed methods research is not easy and can be challenging for one
researcher to carry out both a quantitative and qualitative study, either concurrently or in
succession, as the researcher not only has to be familiar with both schools of research but
he or she must also know how to combine them appropriately (Johnson & Onwuegbuzie,
2004).

6
Like qualitative inquiry, mixed methods research is time-intensive and can be quite costly
(ibid.). If the study is done correctly, it must be planned carefully, and it must have a
clear rationale that is defensible, which is part of the time-consuming aspect of this
methodology (Greene, et.al., 1989). While it is possible to create a fantastic mixed
method study, this approach is demanding in that it requires flexibility from the
researcher to be adaptive to the needs of the problem being studied (Reichardt & Cook,
1979; Greene et. al., 1989).

It is clear by viewing the size of the paragraphs devoted to the weaknesses of the three
paradigms that mixed methods seems to have the least amount of weaknesses; this is due
to the fact that mixed methods research is not limited by epistemological, axiological and
ontological assumptions that restrain mono-method research. Because of the freedom to
choose paradigms that fit the research question, mixed methods essentially gives
researchers the best of both worlds, but with that freedom comes a great deal of
constraints for the researcher in terms of choosing methods to fit the question, as well as
financial and time concerns.

Conclusion

While many scholars and researchers believe that the quantitative, qualitative, and mixed
methods research paradigms are so divergent that they absolutely cannot be combined
with any other (Smith, 1983; Lincoln & Guba, 1985), scholars like Onwuegbuzie and
Leech (2005) offer that there are more similarities between the research paradigms than
most people have considered, as each:

• involve the use of observation to answer research questions

• use safeguards to minimize bias and invalidity
• attempt to triangulate data
• attempt to provide explanations of findings
• interpret and create narrative conclusions about findings
• select and use analytical techniques to gain maximum meaning
• attempt to explain complex relationships
• utilize techniques to verify the data
• tend to use data reduction techniques

The strengths in these research methods lie in their methodologies—the veritable steps
taken from creating a research question to answering it—which are surprisingly similar.
The weaknesses, however, lie in the paradigmatic limitations created by epistemological,
axiological, and ontological assumptions (mostly for the mono-method studies). While
mixed methods research is not a methodological panacea, it is certainly a step towards
greater methodological choice for educational researchers, both amateur and experienced,
for generations to come.

References

7
Atkinson, P., & Delamont, S. (2006). In the roiling smoke: Qualitative inquiry and
contested fields. International Journal of Qualitative Studies in Education, 19(6), 747-
755. Retrieved February 2, 2008 from SAGE.

Burns, R.B. (2000). Introduction to Research Methods. London: SAGE.

Cresswell, J. (2003). Research design: qualitative, quantitative, and mixed methods

approaches (2nd ed.). Thousand Oaks, CA: SAGE.

Cresswell, J., & Miller, D. (2000). Determining validity in qualitative inquiry. Theory
Into Practice , 39 (3), 124-130. Retrieved August 28, 2006 from Academic Search
Premier.

Denzin, N.K. & Lincoln, Y.S. (Eds.) (2005). Handbook of Qualitative Research.
Thousand Oaks, CA: SAGE.

Filstead, W. J. (1979). Qualitative methods - a needed perspective in evaluation research.

In Cook, T. D. and Reichardt, C. S. (eds). Qualitative and Quantitative Methods in
Evaluation Research (pp. 33-48). London: Sage.

Gall, M., Gall, J., & Borg, W. (2003). Educational Research. Boston: Pearson Education,
Inc.

Greene, J.C., Caracelli, V.J., & Graham, W.F. (1989). Toward a conceptual framework
for mixed-method evaluation design. Educational Evaluation and Policy Analysis, 11(3),
255-274. Retrieved on February 1, 2008 from SAGE.

Hammond, C. (2005). The wider benefits of adult learning: An illustration of the

advantages of multi-method research. International Journal of Social Research
Methodology , 8 (3), 239-255. Retrieved on January 29, 2008 from Academic Search
Premier.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research

paradigm hose time has come. Educational Researcher , 33 (7), 14-26. Retrieved on
February 9, 2008 from SAGE.

Krantz, D.L. (1995). Sustaining vs. resolving the quantitative-qualitative debate.

Evaluation and Program Planning, 18(1), 89-96. Retrieved on January 29, 2008 from
Academic Search Premier.

Lincoln, Y.S. & Guba, E.G. (1985). Naturalistic Inquiry. Thousand Oaks, CA: Sage.

Morales, M. (1995). Uses of qualitative/quantitative terms in social and educational

research. Quality and Quantity, 29(1), 39-53. Retrieved on February 5, 2008 from
Ebscohost SocIndex.

8
Onwuegbuzie, A., & Leech, N. (2005). On becoming a pragmatic researcher: The
importance of combining quantitative and qualitative research methodologies.
International Journal of Social Research Methodology , 8 (5), 375-387. Retrieved on
January 29, 2008 from Academic Search Premier.

Patton, M. Q. (1990). Qualitative evaluation and research methods, 2nd Ed. London:
SAGE.

Ratnesar, N. & Mackenzie, J. (2006). The quantitative-qualitative distinction and the null
hypothesis significance testing procedure. Journal of Philosophy of Education, 40(4),
501-509. Retrieved on February 9, 2008 from Academic Search Premier.

Reichardt, C.S. & Cook, T.D. (1979). Qualitative and Quantitative Methods in
Evaluation Research. London: SAGE.

Sechrest, L., & Sidani, S. (1995). Quantitative and qualitative methods: Is there an
alternative? Evaluation and Program Planning , 18 (1), 77-87. Retrieved February 2,
2008 from Academic Search Premier.

Smith, J. (1983). Quantitative versus qualitative research: An attempt to clarify the issue.
Educational Researcher, 12(3), 6-13. Retrieved on February 5, 2008 from JSTOR.

Stanfield II, J. (2006). The possible restorative justice functions of qualitative research.
International Journal of Qualitative studies in Education , 19 (6), 723-727. Retrieved on
February 1, 2008 from Academic Search Premier.

Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and
quantitative approaches (Vol. 46). (L. Bickman, & D. Rog, Eds.) Thousand Oaks, CA:
SAGE.

Wright, H.K. (2006). Are we (t)here yet? Qualitative research in education’s profuse and
contested present. International Journal of Qualitative Studies in Education, 19(6), 793-
802. Retrieved on February 1, 2008 from Academic Search Premier.