Research Methodology Group

UOPX Research Community

Causal-Comparative Research
April 20, 2023
Dr. Frederick P. Lawrence
Topics in Research Designs

• Correlational Research
– March 2, 2023
• Causal-Comparative Research
– April 20, 2023
• Repeated Measures Research
– July 13, 2023
• Regression Research
– August 3, 2023
Agenda

• Definitions of Causal-Comparative Research

• Comparisons with Correlational Research
• Comparisons with Experimental Research
• Causal-Comparative Research Purpose Statements
• Causal-Comparative Research Questions and Hypotheses
• Examples of Causal-Comparative Research
• References
• Discussion and Q&A
 Definition of Causal-Comparative
Research
“Causal-comparative research is a methodology used to identify cause-effect
relationships between independent and dependent variables.

Researchers can study cause and effect in retrospect. This can help determine the
consequences or causes of differences already existing among or between
different groups of people.

When you think of Casual Comparative Research, it will almost always consist of
the following:

• A method or set of methods to identify cause/effect relationships

• A set of individuals (or entities) that are NOT selected randomly – they were
intended to participate in this specific study
• Variables are represented in two or more groups (cannot be less than two,
otherwise there is no differentiation between them)
• Non-manipulated independent variables – *typically, it’s a suggested
relationship (since we can’t control the independent variable completely)”

(https://www.questionpro.com/blog/causal-comparative-research/)
 Definition of Causal-Comparative
Research (cont.)
“Casual Comparative Research is broken down into two types:

Retrospective Comparative Research: Involves investigating a particular

question…. after the effects have occurred. As an attempt to see if a specific
variable does influence another variable.

Prospective Comparative Research: This type of Casual Comparative Research is

characterized by being initiated by the researcher and starting with the causes and
determined to analyze the effects of a given condition. This type of investigation is
much less common than the Retrospective type of investigation.”

(https://www.questionpro.com/blog/causal-comparative-research/)

“A causal-comparative design is a research design that seeks to find relationships

between independent and dependent variables after an action or event has already
occurred. The researcher's goal is to determine whether the independent variable
affected the outcome, or dependent variable, by comparing two or more groups of
individuals … causal-comparative research [is] also referred to as ex post facto
research …”
(Sage Research Methods, in Encyclopedia of Research Design, 2010)
 Comparisons with Correlational
Research
• Similarities:
• Both methods are useful when experimental research has been deemed
impossible or unethical as the research design for a particular question.
• Both designs attempt to determine relationships among variables, but
neither allows for the actual manipulation of these variables.
• Thus, neither can definitively state that a true cause-and-effect relationship
occurred between these variables.
• Finally, neither type of design randomly places subjects into control and
experimental groups, which limits the generalizability of the results.
• Differences:
• In causal-comparative research, the researcher investigates the effect of an
independent variable on a dependent variable by comparing two or more
groups of individuals.
• In correlational research, the researcher works with only one group of
individuals. Instead of comparing two groups, the correlational researcher
examines the effect of one or more independent variables on the dependent
variable within the same group of subjects.

(Sage Research Methods, in Encyclopedia of Research Design, 2010)

 Comparisons with Experimental
Research
• Similarities:
• Unlike correlational research, both experimental research and causal-
comparative research typically compare two or more groups of subjects.
Research subjects are generally split into groups on the basis of the
independent variable that is the focus of the study.
• Goal of both types of research is to determine what effect the independent
variable may or may not have on the dependent variable or variables.
• Differences:
• In true experimental research designs, the researcher manipulates the
independent variable in the experimental group. Because the researcher has
more control over the variables in an experimental research study, the
argument that the independent variable caused the change in the dependent
variable is much stronger.
• In causal-comparative research, the research subjects are already in groups
because the action or event has already occurred, whereas subjects in
experimental research designs are randomly selected prior to the manipulation
of the variables. Random sampling allows for wider generalizations to be
made from the results of an experimental research study.

(Sage Research Methods, in Encyclopedia of Research Design, 2010)

Comparison of Causal-Comparative,
Correlational, and Experimental Research

(Sage Research Methods, in Encyclopedia of Research Design, 2010)

 Causal-Comparative Research
Purpose Statements
Elements of Purpose Statements:

• Type of quantitative research (e.g., quasi-experimental, ex post facto, …)

• Research variables
• Location of study [e.g., business or organization (sector or specific), place, …)

Frequently used wording:

The purpose of this quantitative causal-comparative [or, ex post facto] research

study is to determine if variable A affects variable B within the business [or,
organizational] sector.

The purpose of this quantitative causal-comparative [or, ex post facto] research

study is to identify the cause-effect relationship between independent variable A
and dependent variable B for individuals in the business [or, organization].

The purpose of this quantitative causal-comparative [or, ex post facto] research

study is to determine if treatment variable A will lead to the outcome [response]
variable B at location [or, place].
 Causal-Comparative Research
Questions (RQ) and Hypotheses
Frequently used wording:

RQ: How does variable A affect variable B in [location of study]?

RQ: What is the cause-effect relationship between independent variable A and

dependent variable B in [location of study]?

RQ: How does treatment variable A lead to outcome [response] variable B in

[location of study]?

Hypotheses:

H0: Variable A does not affect variable B in [location of study].

H1: Variable A weakly [or moderately, or strongly] affects variable B by [brief

description of the mechanism] in [location of study].

… and similarly
 Example of Causal-Comparative
Research (1)

Article title:

“Academic Procrastination and Performance in Distance Education: A Causal-

comparative Study in an Online Learning Environment”

Authors:
Hasan Ucar, Anadolu University, Turkey
Aras Bozkurt, Anadolu University, Turkey
Olaf Zawacki-Richter, University of Oldenburg, Germany

Journal:

Turkish Online Journal of Distance Education, October 2021, Volume 22,

Number 4, Article 2, 12 pages
 Example of Causal-Comparative
Research (1) (cont.)
Problem:

• To know the procrastination behaviors of the learners in online distance

learning in order to take the necessary precautions in addressing academic
procrastination tendency, as it is significantly related to course performance
and accomplishment.

Purposes:

• To explore whether male and female learners in online distance learning would
have different academic procrastination tendencies.

• To examine whether different procrastination scores produced differences in

academic performance.

• To explore whether academic procrastination tendency was able to predict the

academic performance of online learners.
 Example of Causal-Comparative
Research (1) (cont.)
Research Questions and Hypotheses:

RQ 1. Do male and female learners significantly differ in their level of

procrastination tendencies?
Research hypothesis 1: Online male and female learners will score the same
on academic procrastination.

RQ 2. Do procrastination tendency differences produce differences in the academic

performance of online learners?
Research hypothesis 2: Online learners who scored lower on academic
procrastination tendency will have better academic performance than that of
those who scored higher on academic procrastination tendency.

RQ 3. Does academic procrastination tendency predict the academic performance

of online learners?
Research hypothesis 3: The academic procrastination tendency of online
learners will predict their academic performance.
 Example of Causal-Comparative
Research (1) (cont.)
Methodology
• Research design: quantitative causal-comparative
• Participants:
• 1,200 online undergraduate learners taking an online English course at a state
university in Turkey were invited to take an online survey
• 333 survey responses (after adjustments) [27.75%]
• Instrument:
• Turkish version of the Tuckman Procrastination Scale (1991)
• Widely used, 14-item scale to measure the academic procrastination tendency
of learners
• Data analysis:
• Independent samples t-tests to compare means of:
• Male and female learners’ level of procrastination tendencies (RQ 1)
• Low and high procrastinators, to determine whether there was a statistical
difference in their academic performance scores (RQ 2)
• Simple linear regression (SLR) model to predict the relationship between
academic procrastination tendency and academic performance (RQ 3)
 Example of Causal-Comparative
Research (1) (cont.)
Results

• RQ 1:
• Although the female learners’ average mean score (M = 34.8, SD = 4.1)
was slightly higher than the male learners’ average mean score (M = 34.1,
SD = 5.2), the difference was not statistically significant (t(331) = 1.18,
p = 0.23) in terms of academic procrastination.
• Research hypothesis 1 was confirmed.

• RQ 2:
• On average, low procrastinators had better academic performance (M =
54.76, SD = 10.6, SE = 0.83) than that of high procrastinators (M = 49.65,
SD = 10.5, SE = 0.81).
• This difference was statistically significant (t(331) = 4.38, p < 0.01) with
close to a medium-sized effect (d = 0.48).
• Research hypothesis 2 was confirmed.
 Example of Causal-Comparative
Research (1) (cont.)
Results (cont.)

• RQ 3:
• SLR suggested that learners’ academic procrastination tendencies
explained 2.3% of the variance (R2 = 0.023; F(1,131) = 7.808, p < 0.05).
• In other words, the learners’ academic procrastination tendencies
significantly predicted academic performance (B = -0.35, t = -2.79, p < 0.05).
• Research hypothesis 3 was confirmed.

Findings

• Based on age span and socio-economic status of the undergraduate learners,

future research should consider different demographics to better examine and
gain more insight into the academic procrastination phenomenon.
• Low procrastinators had better academic performance [consistent with literature].
• Online learners’ academic procrastination tendencies significantly predicted
academic performance [consistent with literature].
 Example of Causal-Comparative
Research (2)
Article title:

“STEM Certification in Georgia’s Schools: A Causal Comparative Study Using

the Georgia Student Growth Model”

Authors:
David E. Proudfoot, University of Phoenix
Michael Green, University of Phoenix
Jan W. Otter, University of Phoenix
David L. Cook, University of Phoenix

Journal:

Georgia Educational Researcher, 2018, Volume 15, Issue 1, Article 2

 Example of Causal-Comparative
Research (2) (cont.)

Problem:

• As Georgia schools become STEM certified, to understand how certification

has influenced achievement in math and science as well as important
non-STEM disciplines such as English language arts and social studies.

Purpose:

• To better understand the early results of current Georgia schools who

receive either the GaDOE STEM Certification or the AdvancED STEM
Certification to guide administrators and teachers in school reform and to
improve efforts to prepare students to be college and career ready in a
globally competitive society.
 Example of Causal-Comparative
Research (2) (cont.)

Research Questions:

RQ 1: Is there a difference in the median growth percentiles in fourth grade

English language arts for STEM certified schools when compared to non-STEM
schools?

RQ 2: Is there a difference in the median growth percentiles in fourth grade

math for STEM certified schools when compared to non-STEM schools?

RQ 3: Is there a difference in the median growth percentiles in fourth grade

science for STEM certified schools when compared to non-STEM schools?

RQ 4: Is there a difference in the median growth percentiles in fourth grade

social studies for STEM certified schools when compared to non-STEM schools?
 Example of Causal-Comparative
Research (2) (cont.)

Research Questions (cont.):

RQ 5: Is there a difference in the median growth percentiles in fifth grade

English language arts for STEM certified schools when compared to non-STEM
schools?

RQ 6: Is there a difference in the median growth percentiles in fifth grade

math for STEM certified schools when compared to non-STEM schools?

RQ 7: Is there a difference in the median growth percentiles in fifth grade

science for STEM certified schools when compared to non-STEM schools?

RQ 8: Is there a difference in the median growth percentiles in fifth grade

social studies for STEM certified schools when compared to non-STEM schools?
 Example of Causal-Comparative
Research (2) (cont.)
Methodology
• Research design: quantitative causal-comparative
• Participants:
• Grades 4 and 5 English language arts (ELA), mathematics, science, and social
studies classes in STEM and non-STEM schools in Georgia
• Measurement:
• Median growth percentiles (MGP) between STEM certified and non-STEM
schools as developed through the Georgia School Growth Model (GSGM)
• Data analysis:
• A purposive sampling technique was used within the Metropolitan Regional
Educational Service Agency (Metro RESA).
• Since the data were not normal for many of the subgroups, the non-parametric
statistical analysis technique, Mann-Whitney U test, which evaluates whether the
mean ranks for two groups differ significantly from each other, was used.
• Homogeneity of variances permitted the use of a pooled variance between
the non-STEM and STEM certified data.
• When the homogeneity assumption did not hold for a set of comparative
groups (Grade 5 ELA), a random sample was taken from the non-STEM
group that was equivalent to number of STEM schools identified for study:
homogeneity failed, so the median test was used (not highly conclusive).
 Example of Causal-Comparative
Research (2) (cont.)
Results
• Fourth grade ELA was the only group to show a significant difference in MGPs
between the STEM certified schools (Mdn = 62.50) and the non-STEM schools
(Mdn = 52), U = 2122.00, p = .004, 2-tailed, η2 = .02 [estimate of effect size, meaning
about 2% of total variance of fourth grade ELA MGPs was explained by STEM
certified schools compared to non-STEM schools, i.e., low or small effect size].
• Interestingly, both the fourth and fifth grade social studies groups had a higher mean
rank for the non-STEM groups than the STEM groups.
• All other content areas showed a higher mean rank for the STEM groups.
• The greatest differences between the STEM and non-STEM groups were in the area
of ELA for both and fourth and fifth grades.
Findings
• Based on the statistical differences noted between STEM certified and non-
STEM schools, support was provided for completing a STEM school certification
process in order to increase MGPs for fourth grade ELA.
• No support was provided for completing a STEM school certification process in
order to increase MGPs for other content areas in other grade levels. [Surprising,
based on anticipated growth, given the available literature at the time.]
 References

Brewer, E.W., & Kubn, J. (2012). Sage research methods. In Encyclopedia of

research design. SAGE Publications, Inc. https://doi.org/10.4135/9781412961288

Johnson, B. (2000, April 24-28). It's (beyond) time to drop the terms causal-
comparative and correlational research in educational research methods textbooks
[Conference paper]. Annual Meeting of the American Educational Research
Association, New Orleans, LA, United States. https://eric.ed.gov/?id=ED445010

Proudfoot, D.E., Green. M., Otter, J.W, & Cook, D.L. (2018). STEM certification in
Georgia’s schools: A causal comparative study using the Georgia Student Growth
Model, Georgia Educational Researcher, 15(1).
https://digitalcommons.georgiasouthern.edu/gerjournal/vol15/iss1/2

Ucar, H., Bozkurt, A., & Zawacki-Richter, O. (2021, October). Academic

procrastination and performance in distance education: A causal-comparative study
in an online learning environment. Turkish Online Journal of Distance Education,
22(4). https://dergipark.org.tr/en/pub/tojde/issue/65206/1002726
For Further Help: Quantitative Office Hours (go to
Home page of Research Methodology Group in MS Teams)
Discussion and Q&A