Computers & Education 227 (2025) 105224

Contents lists available at ScienceDirect

Computers & Education

Does ChatGPT enhance student learning? A systematic

review and
meta-analysis of experimental studies
Ruiqi Deng a,b,*
, Maoli Jiang a, Xinlu Yu a, Yuyan Lu a, Shasha Liu c

a
Jing Hengyi School of Education, Hangzhou Normal University, Hangzhou, China
b
Chinese Education Modernization Research Institute (Zhejiang Provincial Key Think Tank), Hangzhou Normal University, Hangzhou, China
c
School of Tourism Planning and Design, Tourism College of Zhejiang, Hangzhou, China

A R T I C L E I N F
O A B S T R A C T

Keywords:
Chat Generative Pre-Trained Transformer (ChatGPT) has generated
Teaching/learning excitement and concern in education. While cross-sectional studies have
strategies Improve highlighted correlations between ChatGPT use and learning performance,
classroom teaching they fall short of establishing causality. This review examines
Elementary education experimental studies on ChatGPT’s impact on student learning to address
Secondary education this gap. A comprehensive search across five databases identified 69
Post-secondary articles published between 2022 and 2024 for analysis. The findings
education
reveal that ChatGPT interventions are predominantly implemented at
the university level, cover various subject areas focusing on language
education, are integrated into classroom environments as part of regular
educational practices, and primarily involve direct student use of
ChatGPT. Overall, ChatGPT improves academic performance, affective-
motivational states, and higher-order thinking propensities; it reduces
mental effort and has no significant effect on self- efficacy. However,
methodological limitations, such as the lack of power analysis and
concerns regarding post-intervention assessments, warrant cautious
interpretation of results. This review presents four propositions from the
findings: (1) distinguish between the quality of ChatGPT outputs and the
positive effects of interventions on academic performance by shifting from
well- defined problems in post-intervention assessments to more complex,
project-based assessments that require skill demonstration, adopting
proctored assessments, or incorporating metrics such as originality
alongside quality; (2) evaluate long-term impacts to determine whether
the positive effects on affective-motivational states are sustained or merely
owing to novelty effect; (3) pri- oritise objective measures to complement
subjective assessments of higher-order thinking; and (4) use power
analysis to determine adequate sample sizes to avoid Type II errors
and provide reliable effect size estimates. This review provides valuable
insights for researchers, instructors, and policymakers evaluating the
effectiveness of generative AI integration in educational practice.
R. Deng et al. Computers & Education 227 (2025) 105224

1. Introduction

Although large language models (LLMs) can be traced back to the early development of natural language
processing in the mid-20th century (Maatouk, A., Piovesan, N., Ayed, F., Domenico, A. D., & Debbah, M., 2024),
the release of Chat Generative Pre-Trained Transformer (ChatGPT) in late 2022 marked a turning point (Korseberg
& Elken, 2024; Yan et al., 2023). By substantially lowering the barrier for individuals without a technological
background to utilise the generative capabilities of LLMs (Pack & Maloney, 2023), the advancements of ChatGPT
and similar models have pro- pelled generative AI (GenAI) into the forefront of educational discourse, igniting a
mixed response of excitement and concern. While praised for its potential in content generation and natural
language processing (Adeshola, I., & Adepoju, A. P., 2023) concerns about academic integrity, overreliance on
technology, and potential negative impacts on essential skills such as writing, coding, and problem-solving have
fuelled apprehension among educators (Tlili et al., 2023; Wise et al., 2024) and the public (L. Li, Ma, et al., 2024;
Na et al., 2024). Despite these concerns, ChatGPT’s potential to transform learning experiences and reinforce
educational outcomes cannot be disregarded. Teaching and learning are prime areas for disruption by this
technology(Chiarello, F., Giordano, V., Spada, I., Barandoni, S., & Fantoni, G., 2024). Unlike traditional chatbots,
which often rely on predefined responses and limited interaction patterns, ChatGPT’s generative capabilities allow
for more dynamic, context-aware conversations that can adapt to diverse educational scenarios (Hyun Baek & Kim,
2023; Niloy et al., 2024; Yang & Li, 2024). This capability to generate nuanced and personalised feedback
represents a significant departure from earlier AI technologies that primarily focus on simple task completion (Su
et al., 2023). This review focuses exclusively on ChatGPT and its impact on student learning due to its widespread
recognition and familiarity among students (Hamerman, E. J., Aggarwal, A., & Martins, C., 2024). Furthermore,
ChatGPT’s output is coherent and comparable in quality to that produced by humans (Flode´n, 2024; Gencer &
Gencer, 2024; Jarry Trujillo et al., 2024; Lin & Chen, 2024), and its performance in delivering depth and accurate
answers can surpass that of other GenAI tools (Dihan, Q., Chauhan, M. Z., Eleiwa, T. K., Brown, A. D., Hassan,
A. K., Khodeiry, M. M., Elsheikh, R. H., Oke, I., Nihalani, B. R., VanderVeen, D. K., Sallam, A. B., & Elhusseiny, A.
M., 2024)
The incorporation of ChatGPT into educational settings is occurring at an unprecedented pace, with a growing
trend of student usage of ChatGPT for various academic work (Jo, 2023; Playfoot et al., 2024). Additionally,
instructors actively seek to leverage ChatGPT in teaching and assessment practices, aiming to improve work
efficiency (Laak & Aru, 2024; Shin & Lee, 2024) and optimise student performance(Bower, M., Torrington, J., Lai,
J. W. M., Petocz, P., & Alfano, M., 2024) (Bower et al., 2024). Despite calls for empirical research into the
effectiveness of ChatGPT in enhancing student learning (e.g. Law, 2024; Lee & Song, 2024), a significant
knowledge gap exists regarding its impact across different learning out- comes. The failure to understand these
impacts before widespread implementation could diminish the quality of learning, perpetuate educational
inequalities, and raise concerns about academic integrity (Jensen et al., 2024; Jiang et al., 2024). This study
addresses this gap by aggregating the relevant experimental evidence to gain a holistic understanding of the field
and identify research trends and promising directions for future research.
This systematic review and meta-analysis seeks to answer the fundamental question: What insights does the
experimental evidence provide regarding the impact of ChatGPT on various dimensions of student learning?
Gaining a nuanced understanding of this impact is crucial for informing teaching strategies and shaping the
responsible implementation of ChatGPT and similar GenAI tools in educational contexts. If the evidence
substantiates the positive influence of ChatGPT, it becomes essential to incorporate innovative pedagogical
practices that adequately prepare students for a GenAI-driven workforce (Chan, 2023; Dianova & Schultz, 2023),
such as fostering their prompt engineering skills (Walter, 2024). Positive evidence would also indicate the need for
changes in teacher preparation, as instructors will need to adapt to new technologies rather than risk being left
behind (Garofalo & Farenga, 2024). Conversely, negative evidence would necessitate a more cautious approach,
highlighting potential areas of concern and the need for further research to mitigate any adverse effects. This
review aims to provide insights for researchers, instructors, and policymakers navigating the integration of this
transformative technology into the educational landscape.

2. Literature review

2.1. Student and instructor perceptions and attitudes towards ChatGPT in education

The perceptions and attitudes of students and instructors play a crucial role in shaping the
adoption and integration of ChatGPT. Empirical investigations have been conducted to understand
students’ (Niloy et al., 2024) and instructors’ (Al-khresheh, 2024) acceptance of integrating
ChatGPT into teaching and learning. Generally, undergraduate (Tu & Hwang, 2023) and
postgraduate students (Dai et al., 2023) demonstrate a favourable reception towards ChatGPT,

2
R. Deng et al. Computers & Education 227 (2025) 105224
despite the absence of adequate guidelines for its use (Adams et al., 2023; Zou & Huang, 2023).
While major student populations embrace ChatGPT, a discernible cohort remains indifferent
or cautious about its integration into education (Sˇedlbauer et al., 2024). A prevailing view
among students is that ChatGPT can be
leveraged for ideation and cognitive offloading of routine tasks rather than as a tool for
automated writing completion (Barrett & Pack, 2023; Yan, 2023). Notwithstanding the occurrence
of illogical, problematic, and contradictory responses (Stojanov, 2023; Urhan et al., 2024), students
often exhibit trust in the capacity of ChatGPT to deliver accurate answers (Ding et al., 2023).
Instructors have exhibited ambivalent attitudes towards the use of ChatGPT in educational
settings, reflecting both concerns and opportunities for pedagogical enhancement. A subset of
instructors have voiced concerns regarding the potential adverse impact on education posed by the
promotion of ChatGPT for students (Nam & Bai, 2023), particularly the potential dissemination of
misin- formation (Su & Yang, 2023), the lack of evidence supporting the obtained results (Cooper,
2023), the fostering of copying and pasting habits (Garcia Castro et al., 2024), and the impediment
to developing higher-order thinking skills (Mohamed, 2024). Concurrently, instructors have posited
that responsible use of ChatGPT holds promise in augmenting teaching and learning (Yusuf et al.,
2024). The perception of this group is that ChatGPT exhibits efficiency in facilitating output
generation across diverse academic tasks, including the development and optimisation of course
plans (Okulu & Muslu, 2024), rubrics (Cooper, 2023), quiz questions (U. Lee, Chen, et al., 2024),
presentation slides (Galindo-Domínguez et al., 2023), content knowledge (Su & Yang, 2023), and
innovative pedagogies (Yeh, 2024). They anticipate that ChatGPT has the potential to transform
students from passive knowledge recipients into active in- vestigators (Jeon & Lee, 2023) but often
remain critical regarding its implementation in areas such as assessment and feedback (ElSayary,
2023).
ChatGPT advancements have sparked enthusiasm and concern, highlighting the necessity for empirical
inquiries into its potential impact. Compared to instructors, students tend to be more enthusiastic about
incorporating ChatGPT into education (Chan & Tsi, 2024). While student and instructor perceptions and attitudes
provide valuable insights into initial perspectives on ChatGPT in ed- ucation (Chan & Lee, 2023; Monib et al.,
2024; Moorhouse & Kohnke, 2024), they do not offer empirical evidence of its actual impact on learning
outcomes. To move beyond subjective opinions, it is imperative to turn to empirical research that can establish
the relationship between ChatGPT adoption and student learning. Cross sectional and experimental studies are
commonly used ap- proaches to identify this relationship.

2.2. Strengths and limitations of cross-sectional studies in understanding ChatGPT’s impact

Cross-sectional studies provide valuable insights into the early exploration of ChatGPT in education.
The frequent occurring topics are students’ perceptions and experiences of ChatGPT usage (Gao et al.,
2024; Gra´jeda et al., 2024), individual differences in ChatGPT perceptions and experiences based on
personal characteristics (e.g. gender; Almazrou et al., 2024; Ofem et al., 2024), and factors driving
students’ intentions (Jo, 2024; Maheshwari, 2024; Tan et al., 2024) and actual use of ChatGPT (Grassini
et al., 2024; Salifu et al., 2024; Wijaya et al., 2024). By capturing a snapshot of student perceptions,
behaviour, and performance at a specific point in time, cross-sectional studies are advantageous due to
their ability to quickly gather large-scale data from diverse student populations. They are an effective
starting point for identifying patterns in how learners perceive and interact with ChatGPT, as well as
exploring associations between ChatGPT use and learning-related factors, such as academic
performance.
While cross-sectional studies can be used to identify the relationship between the use of ChatGPT
and learning outcomes, they often fail to clarify the directionality of this relationship. For example,
Shahzad et al. (2024) conduct a cross-sectional study involving 362 university students in China,
employing structural equation modelling to explore the relationship between ChatGPT use and
academic performance. The study identifies a positive association between ChatGPT use and improved
performance, thereby recommending that higher education institutions should consider utilising GenAI
tools to enhance student learning. However, the cross-sectional nature of the research makes it difficult
to determine whether increased ChatGPT use leads to better academic performance or if students with
stronger academic results are more inclined to use ChatGPT. This ambiguity regarding the
directionality of the relationship suggests that further experimental studies are necessary to establish
causality. Despite the directionality problem, it is not uncommon for re- searchers to claim a positive
effect of ChatGPT on learning performance based solely on cross-sectional evidence (e.g. Al-Qaysi et al.,
2024; Boubker, 2024; Dahri et al., 2024).
Another example of the limitations of cross-sectional studies can be observed in a study by Crawford
et al. (2024), which analyses data from 387 undergraduate and postgraduate students. The study
employs structural equation modelling to analyse the relationships between various factors, including
the use of ChatGPT and self-reported academic performance. Crawford et al. (2024) observe a negative

3
R. Deng et al. Computers & Education 227 (2025) 105224
yet statistically insignificant correlation between ChatGPT use and performance, thereby cautioning
against the possible adverse impact of ChatGPT. However, apart from the potential effect of social
desirability bias (Paulhus, 1991) and recall bias (Coughlin, 1990) on the accuracy of self-reported
performance, cross-sectional research remains uncertain as to whether the use of ChatGPT contributes
to poor academic performance or vice versa (e.g. Al-Mamary et al., 2024). Gaining insights into the
directionality of this relationship is pivotal for informing educational policies and practices. If ChatGPT
leads to decreased learning outcomes, in- terventions may be necessary to promote responsible usage
and mitigate potential adverse effects. However, if students with lower performance are more inclined
to use ChatGPT, the focus might shift towards providing additional support to improve their success.
The directionality problem is not the only limitation faced by cross-sectional studies examining
ChatGPT’s impact. These studies may be susceptible to spurious correlation (Haig, 2003), where the
observed negative correlation between the use of ChatGPT and lower performance could be influenced
by a confounding factor like prior knowledge. For example, individuals with limited prior knowledge
may encounter difficulties comprehending the learning material and resort to ChatGPT for assistance,
resulting in lower academic performance. Moreover, there might be inherent self-selection bias (Titus,
2006), as students who opt to use ChatGPT could differ systematically from those who do not. They
might be more prone to overreliance on GenAI tools and a decrease in effort, potentially leading to
biased results and inaccurate conclusions about the impact of ChatGPT. Lastly, cross-sectional studies
often fail to contextualise ChatGPT usage within a unique learning environment, as data are typically
collected from students across various educational settings through non-probabilistic convenience
sampling (e.g. Acosta-Enriquez et al., 2024; Bouteraa et al., 2024; Bud- hathoki et al., 2024). Contextual
factors, such as the curriculum structure, assignment types, and the integration of supplementary
instructional technologies (Biggs & Tang, 2011), potentially influence student learning with ChatGPT
and are, therefore, crucial for
interpreting how and why ChatGPT impacts learning.
Although cross-sectional studies are often economical and time-saving, and contribute to identifying
potential associations between ChatGPT adoption and learning outcomes (e.g. Jaboob et al., 2024; Ngo
et al., 2024), they are limited in their ability to establish causal relationships or determine the temporal
sequence of events. Additionally, they are prone to spurious correlations and may lack the context of
specific learning environments. The limitations inherent in the cross-sectional research design
emphasise the necessity for experimental studies to determine the impact of ChatGPT adoption on
student learning.

2.3. Experimental studies related to ChatGPT’s impact on student learning

Cross-sectional studies, while useful for identifying correlations between variables such as ChatGPT
use and learning outcomes, are limited in establishing causal relationships because they assess data
at a single point in time, lack the ability to determine the temporal sequence of events, and are
susceptible to spurious correlation. In contrast, experimental studies offer stronger causal
evidence by

4
R. Deng et al. Computers & Education 227 (2025) 105224

Table 1
Previous reviews on ChatGPT in education.
Publication Author(s) Database searched End of the Number Key contributions
date data of
collection articles
period reviewe
d
April 2023 Lo (2023) Academic Search Ultimate, ACM February 50 ChatGPT’s applications and
Digital 2023
Library, Education Research limitations in education
Complete,
ERIC, IEEE Xplore, Scopus, Web of
Science,
and Google Scholar
June 2023 Perera and ScienceDirect, Springer, Web of May 2023 8 Use of ChatGPT and
Science, potential
Lankathilaka Taylor & Francis, ResearchGate, impact on higher
(2023) education
EBSCOhost, and major academic
publishers
July 2023 Grassini (2023) Google Scholar and Scopus May 2023 Not reported Benefits and challenges of
ChatGPT integration in
July 2023 I˙pek et al. (2023) ScienceDirect, ERIC, Wiley, February 40 education
Springer, Sage, 2023 Implications and concerns
Taylor & Francis, MDPI, and JSTOR regarding the use of
ChatGPT in
education
July 2023 Jahic et al. Google Scholar, IEEE Xplore, Not reported 41 ChatGPT’s educational
(2023) ScienceDirect,
and Web of Science applications, advantages,
and
disadvantages
July 2023 Montenegro-Rueda Web of Science, Scopus, and Google June 2023 12 ChatGPT’s impact on
Scholar education
et al. (2023)
July 2023 Vargas-Murillo et Scopus, ScienceDirect, ProQuest, Not reported 16 ChatGPT’s applications,
al. IEEE
(2023) Xplore, and ACM Digital Library challenges, opportunities,
and
impact in education
August Imran and Scopus, ScienceDirect, PubMed, May 2023 30 Opportunities and
2023 and Web of challenges of
Almusharraf Science using ChatGPT for academic
(2023)
writing
August Pradana et al. Google Scholar Not reported 93 Key contributors,
2023 (2023) subtopics, and
emerging research
directions in
ChatGPT’s educational
applications
September Dempere et al. PubMed, Web of Science, IEEE Not reported 143 ChatGPT’s potential and
(2023) Xplore,
2023 Scopus, Google Scholar, ACM limitation in higher
Digital education
Library, ScienceDirect, JSTOR,
ProQuest,
SpringerLink, EBSCOhost, and
ERIC
October Ansari et al. Google Scholar, Taylor & Francis, May 2023 69 Use of ChatGPT in higher
(2024) Emerald,
2023 Sage, Elsevier, ScienceDirect, and education
PubMed
December Polat et al. Scopus July 2023 212 Trends, themes, and
(2024) contributors
2023 of ChatGPT research in
education
December Zhang and Tur Web of Science, Scopus, ERIC, Not reported 13 ChatGPT’s potential and
(2023) SpringerLink,
2023 IEEE Xplore, and ACM Digital limitation in K-12
Library education
February Mai et al. (2024) Scopus, ERIC, and Google Scholar December 51 Strengths, weaknesses,
2024 2023 opportunities, and threats
of the
use of ChatGPT in education
February Mahrishi et al. Scopus December 109 ChatGPT’s development and
(2024)
2024 2023 emerging dynamics in
research
and education
February Wong et al. Web of Science and Altmetric August 2023 175 Correlation between media
(2024)
2024 attention and scholarly
citations
5
R. Deng et al. Computers & Education 227 (2025) 105224
in ChatGPT-related
education
research
April 2024 Bhullar et al. Scopus May 2023 47 ChatGPT’s applications,
(2024)
challenges, opportunities,
and
impact in higher
education
May 2024 Yun and Scopus Not reported 58 Trends, contributors, and
Surianshah themes
(2024) of ChatGPT research in
education
June 2024 Ali et al. (2024) Academic Search Premier, Web of October 2023 112 Benefits and limitations of
Science,
and IEEE ChatGPT in teaching and
learning
July 2024 Amarathunga Scopus May 2024 45 Trends, contributors, and
(2024) gaps of
ChatGPT research in
education
July 2024 Baig and Emerald, ERIC, MDPI, SAGE, January 57 Trends, measures,
Elsevier, 2024 applications,
Yadegaridehkordi SpringerLink, Frontiers, PLoS ONE, and limitations of ChatGPT
Wiley,
(2024) and Taylor & Francis research in higher
education
August Samala et al. Web of Science and Scopus December 453 ChatGPT’s applications,
2024 (2024)
2023 advantages, limitations,
ethics
considerations, and
prospects

Note: If publication date is not provided, the acceptable date is recorded.

6
R. Deng et al. Computers & Education 227 (2025) 105224

directly measuring the effects of interventions in controlled settings (Gorard & Cook, 2007). Since
the introduction of ChatGPT, discrete experimental studies have been conducted to understand the
impact of the technology on learning. These studies span a wide range of subjects, such as
language (Maghamil & Sieras, 2024), programming (Donald et al., 2024), and health (Svendsen et
al., 2024). They also cover various educational levels, such as primary schools (Almohesh, 2024),
high schools (Kim, 2024), and universities (Xue et al., 2024). This diversity provides a broad
perspective on ChatGPT’s potential impact across diverse learning environments. However,
contradictory findings have emerged, with some studies reporting significant improvements in
learning outcomes (Emran et al., 2024; Lyu et al., 2024), while others indicate significant negative
impacts (Shin et al., 2024; Zhang et al., 2024) or no measurable effects (Baˇsi´c et al., 2023; Farah
et al., 2023). Considering the mixed findings, there remains a need for comprehensive review and
meta-analysis to resolve existing contradictions and provide clearer guidance for educational
practice. The following sections explore existing reviews of ChatGPT in education.

2.4. Existing reviews of ChatGPT in education

Table 1 presents a chronological list of review articles on ChatGPT in education. Most reviews
provide comprehensive analyses of the technology’s implications for education (Chen et al., 2024;
Grassini, 2023; Lo, 2023), whereas a few adopt a more focused approach by examining specific
educational stages, such as higher education (Ansari et al., 2024; Baig & Yadegaridehkordi, 2024;
Perera & Lankathilaka, 2023), or addressing particular applications of ChatGPT, like as a writing
assistant (Imran & Almusharraf, 2023). These existing reviews provide valuable insights into the
emerging trends, potential benefits, and challenges associated with integrating ChatGPT in
educational contexts. They recognise the transformative potential of ChatGPT for personalising
learning ex- periences, enhancing student engagement, and supporting diverse learning needs.
Simultaneously, the reviews consistently raise concerns regarding academic integrity, accuracy,
and bias, highlighting the need for careful consideration and responsible imple- mentation
strategies to mitigate potential risks.
While the existing reviews contribute to a broader understanding of ChatGPT in education, they
lack a comprehensive analysis of experimental evidence. These reviews predominantly rely on
theoretical discussions, opinions, and limited cross-sectional studies that do not establish causal
relationships between the adoption of ChatGPT and student learning. For instance, while some
reviews highlight concerns that overreliance on ChatGPT may impede critical thinking skills
(Perera & Lankathilaka, 2023; Samala et al., 2024; I˙pek et al., 2023), others posit that it can
foster such skills by providing a platform for exploring ideas and engaging in deeper thinking and
analysis (Jahic et al., 2023; Montenegro-Rueda et al., 2023). This calls for a systematic review and
meta-analysis of experimental studies to evaluate the impact of ChatGPT on student learning and
provide educators with evidence-based guidance. To provide conceptual clarity, this review
operationally defines student learning as the measurable improvement in cognitive, emotional, and
psychological outcomes that result from ChatGPT interventions, as assessed through various
methods such as standardised tests, performance tasks, and self-evaluations. The review defines
the impact of ChatGPT as including not only the effects of the standard, ready-to-use ChatGPT
application, but also those of educational tools that use ChatGPT via APIs and other custom
methods on student learning.

2.5. Research opportunities

The extant literature on ChatGPT in education often exhibits homogeneity in its scope, revealing
an emerging interest in its ap- plications while also recognising both the benefits and challenges.
To move beyond conceptual exploration (e.g. García et al., 2024; Lambert & Stevens) and
understand the actual impact of ChatGPT on student learning, examining experimental research is
essential for establishing causality and drawing robust conclusions about its effectiveness in
education (Ansari et al., 2024). This review aims to aggregate findings from experimental studies
on ChatGPT’s impact on student learning, providing a deeper understanding of its integration into
educational settings and supporting evidence-based decision-making regarding its adoption.
Teaching and learning do not occur in decontextualised settings (Deng et al., 2019). Understanding
the context of interventions is essential for interpreting trends and patterns in experimental
studies examining ChatGPT’s impact and their broader implications (McGrath et al., 2024).
Accordingly, this review investigates both the characteristics and impacts of ChatGPT
interventions and proposes two research questions.
Research question 1 (RQ1). What are the educational stages, subject areas, intervention
7
R. Deng et al. Computers & Education 227 (2025) 105224
settings, durations, and application modes of ChatGPT interventions in experimental studies?
Research question 2 (RQ2). What are the differential effects of ChatGPT interventions on
various dimensions of student learning?
Given the importance of methodological rigour for drawing valid conclusions about the impact of
ChatGPT on student learning (Lo et al., 2024; Wong et al., 2024), this review examines the
methodological quality of studies alongside intervention characteristics and reported impacts.
Since this is not a methodological review, comprehensively analysing and discussing all aspects of
methodological indicators for each reviewed publication is not feasible; however, certain
methodological details are crucial and directly impact the validity of the experimental evidence.
The review specifically focuses on sample size estimation and baseline difference control, which
are considered essential for evaluating the quality of experimental studies (Shadish et al., 2002)
and commonly featured in quality assessment guidelines for experimental research (Kmet et al.,
2004; NHLBI, 2021). Adequate sample size ensures sufficient statistical power to detect
meaningful effects, thus preventing the oversight of true effects due to insufficient power
(Abraham & Russell, 2008; Peng et al., 2012). Appropriate baseline difference control increases
precision in estimating experimental effects, ensuring that

observed effects can be more accurately attributed to the intervention rather than pre-existing
differences between groups (Critical Appraisal Skills Programme, 2023; Howitt & Cramer, 2017).
This consideration is particularly vital regarding emerging technologies such as ChatGPT, where
there is a high potential for both substantial educational advantages and unintended
repercussions. Therefore, the review proposes a third research question.
Research question 3 (RQ3). How do experimental studies of ChatGPT interventions determine
sample size and control for baseline differences?

Adeshola, I., & Adepoju, A. P. (2023). The opportunities and challenges of ChatGPT in education. Interactive

Learning Environments. Advance Online Publication. https:// doi.org/10.1080/10494820.2023.2253858

Bower, M., Torrington, J., Lai, J. W. M., Petocz, P., & Alfano, M. (2024). How should we change teaching and

assessment in response to increasingly powerful generative artificial intelligence? Outcomes of the

ChatGPT Teacher Survey. Education and Information Technologies, 29, 15403–15439.

https://doi.org/10.1007/ s10639-023-12405-0

Chiarello, F., Giordano, V., Spada, I., Barandoni, S., & Fantoni, G. (2024). Future applications of generative large

language models: A data-driven case study on ChatGPT. Technovation, 133, Article 103002.

https://doi.org/10.1016/j.technovation.2024.103002

Dihan, Q., Chauhan, M. Z., Eleiwa, T. K., Brown, A. D., Hassan, A. K., Khodeiry, M. M., Elsheikh, R. H., Oke, I.,

Nihalani, B. R., VanderVeen, D. K., Sallam, A. B., & Elhusseiny, A. M. (2024). Large language models: A

new frontier in paediatric cataract patient education. British Journal of Ophthalmology, 108(10), 1470–

1476. https://doi.org/10.1136/bjo-2024-325252

Hamerman, E. J., Aggarwal, A., & Martins, C. (2024). An investigation of generative AI in the classroom and its

implications for university policy. Quality Assurance in Education. Advance Online Publication.

https://doi.org/10.1108/qae-08-2024-0149

Maatouk, A., Piovesan, N., Ayed, F., Domenico, A. D., & Debbah, M. (2024). Large language models for telecom:

Forthcoming impact on the industry. IEEE Communications Magazine. Advance Online Publication.

https://doi.org/10.1109/mcom.001.2300473

8
R. Deng et al. Computers & Education 227 (2025) 105224

References

Abraham, W. T., & Russell, D. W. (2008). Statistical power analysis in psychological research. Social and Personality Psychology
Compass, 2(1), 283–301. https://doi. org/10.1111/j.1751-9004.2007.00052.x
Abu-Bader, S. H. (2021). Using statistical methods in social science research (3rd ed.). Oxford University Press.
Acosta-Enriquez, B. G., Arbulú Ballesteros, M. A., Huamaní Jordan, O., Lo´pez Roca, C., & Saavedra Tirado, K. (2024).
Analysis of college students’ attitudes toward the use of ChatGPT in their academic activities: Effect of intent to use,
verification of information and responsible use. BMC Psychology, 12(1), 255. https://doi.org/ 10.1186/s40359-024-01764-z
Adams, D., Chuah, K.-M., Devadason, E., & Azzis, M. S. A. (2023). From novice to navigator: Students ’ academic help-seeking
behaviour, readiness, and perceived usefulness of ChatGPT in learning. Education and Information Technologies, 29, 13617–13634.
https://doi.org/10.1007/s10639-023-12427-8
Adeshola, I., & Adepoju, A. P. (2023). The opportunities and challenges of ChatGPT in education. Interactive Learning Environments. Advance online
publication. https:// doi.org/10.1080/10494820.2023.2253858
* Ahmed Moneus, A. M., & Al-Wasy, B. Q. (2024). The impact of artificial intelligence on the quality of Saudi translators ’
performance. Al-Andalus journal for Humanities & Social Sciences, 11(96), 201–230. https://doi.org/10.35781/1637-000-096-006.
Al-khresheh, M. H. (2024). Bridging technology and pedagogy from a global lens: Teachers’ perspectives on integrating
ChatGPT in English language teaching.
Computers & Education: Artificial Intelligence, 6, Article 100218. https://doi.org/10.1016/j.caeai.2024.100218
Al-Mamary, Y. H., Alfalah, A. A., Shamsuddin, A., & Abubakar, A. A. (2024). Artificial intelligence powering education:
ChatGPT’s impact on students’ academic performance through the lens of technology-to-performance chain theory. Journal of
Applied Research in Higher Education, Advance online publication. https://doi. org/10.1108/jarhe-04-2024-0179
Ali, D., Fatemi, Y., Boskabadi, E., Nikfar, M., Ugwuoke, J., & Ali, H. (2024). ChatGPT in teaching and learning: A systematic
review. Education Sciences, 14(6), 643. https://doi.org/10.3390/educsci14060643
Almazrou, S., Alanezi, F., Almutairi, S. A., AboAlsamh, H. M., Alsedrah, I. T., Arif, W. M., Alsadhan, A. A., AlSanad, D. S.,
Alqahtani, N. S., AlShammary, M. H., Bakhshwain, A. M., Almuhanna, A. F., Almulhem, M., Alnaim, N., Albelali, S., & Attar,
R. W. (2024). Enhancing medical students critical thinking skills through ChatGPT: An empirical study with medical
students. Nutrition and health. Advance online publication. https://doi.org/10.1177/02601060241273627
* Almohesh, A. R. I. (2024). AI application (ChatGPT) and Saudi Arabian primary school students’ autonomy in online
classes: Exploring students and teachers’ perceptions. International Review of Research in Open and Distance Learning, 25(3), 1–18.
https://doi.org/10.19173/irrodl.v25i3.7641.
* Alneyadi, S., & Wardat, Y. (2023). ChatGPT: Revolutionizing student achievement in the electronic magnetism unit for
eleventh-grade students in Emirates schools.
Contemporary Educational Technology, 15(4), Article ep448. https://doi.org/10.30935/cedtech/13417.
* Alneyadi, S., & Wardat, Y. (2024). Integrating ChatGPT in grade 12 quantum theory education: An exploratory study at
Emirate school (UAE). International Journal of Information and Education Technology, 14(3), 398–410.
https://doi.org/10.18178/ijiet.2024.14.3.2061.
Amarathunga, B. (2024). ChatGPT in education: Unveiling frontiers and future directions through systematic literature
review and bibliometric analysis. Asian education and development studies. Advance online publication.
https://doi.org/10.1108/AEDS-05-2024-0101
* Ameen, L. T., Yousif, M. R., Alnoori, N. A. J., & Majeed, B. H. (2024). The impact of artificial intelligence on computational
thinking in education at university.
International Journal of Engineering Pedagogy, 14(5), 192–203. https://doi.org/10.3991/ijep.v14i5.49995.
Anderson, T., & Shattuck, J. (2012). Design-based research. Educational Researcher, 41(1), 16–25.
https://doi.org/10.3102/0013189x11428813
Ansari, A. N., Ahmad, S., & Bhutta, S. M. (2024). Mapping the global evidence around the use of ChatGPT in higher education:
A systematic scoping review. Education and Information Technologies, 29, 11281–11321. https://doi.org/10.1007/s10639-023-12223-
4
* Avello-Martínez, R., Gajderowicz, T., & Go´mez-Rodríguez, V. G. (2024). Is ChatGPT helpful for graduate students in
acquiring knowledge about digital storytelling and reducing their cognitive load? An experiment. Revista de Educaci´on a
Distancia, 24(78), 8. https://doi.org/10.6018/red.604621.
* Aydın Yıldız, T. (2023). The impact of ChatGPT on language learners’ motivation. Journal of Teacher Education and Lifelong Learning, 5(2),
582–597. https://doi.org/ 10.51535/tell.1314355.
Baig, M. I., & Yadegaridehkordi, E. (2024). ChatGPT in the higher education: A systematic literature review and research
challenges. International Journal of Educational Research, 127, Article 102411. https://doi.org/10.1016/j.ijer.2024.102411
Baker, J. P., Goodboy, A. K., Bowman, N. D., & Wright, A. A. (2018). Does teaching with PowerPoint increase students’
learning? A meta-analysis. Computers &
Education, 126, 376–387. https://doi.org/10.1016/j.compedu.2018.08.003
Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37(2), 122–147.
https://doi.org/10.1037/0003-066X.37.2.122 Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman and
Company.

29
R. Deng et al. Computers & Education 227 (2025) 105224

Barrett, A., & Pack, A. (2023). Not quite eye to A.I.: Student and teacher perspectives on the use of generative artificial
intelligence in the writing process. International Journal of Educational Technology in Higher Education, 20(1), 59.
https://doi.org/10.1186/s41239-023-00427-0
* Baˇsi´c, Zˇ., Banovac, A., Kruˇzi´c, I., & Jerkovi´c, I. (2023). ChatGPT-3.5 as writing assistance in students’ essays. Humanities and Social
Sciences Communications, 10, 750.
https://doi.org/10.1057/s41599-023-02269-7.
Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50(4),
1088–1101. https://doi.org/10.2307/ 2533446
* Beltozar-Clemente, S., & Díaz-Vega, E. (2024). Physics XP: Integration of ChatGPT and gamificatio to improve academic
performance and motivation in physics 1 course. International Journal of Engineering Pedagogy, 14(6), 82–92.
https://doi.org/10.3991/ijep.v14i6.47127.
Bhullar, P. S., Joshi, M., & Chugh, R. (2024). ChatGPT in higher education - a synthesis of the literature and a future research
agenda. Education and information technologies. Advance online publication. https://doi.org/10.1007/s10639-024-12723-x
Biggs, J. B., & Tang, C. S. (2011). Teaching for quality learning at university (3rd ed.). Open University Press.
Borenstein, M. (2022). Comprehensive meta-analysis software. In M. Egger, J. P. T. Higgins, & G. D. Smith (Eds.), Systematic reviews in
health research: Meta-analysis in context (3rd ed., pp. 535–548). John Wiley & Sons. https://doi.org/10.1002/9781119099369.ch27.
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. John Wiley & Sons.
https://doi.org/10.1002/9780470743386 Boubker, O. (2024). From chatting to self-educating: Can AI tools boost student learning
outcomes? Expert Systems with Applications, 238, Article 121820. https://doi.
org/10.1016/j.eswa.2023.121820
* Boudouaia, A., Mouas, S., & Kouider, B. (2024). A study on ChatGPT-4 as an innovative approach to enhancing English as a
foreign language writing learning.
Journal of Educational Computing Research, Advance online publication. https://doi.org/10.1177/07356331241247465.
Bouteraa, M., Bin-Nashwan, S. A., Al-Daihani, M., Dirie, K. A., Benlahcene, A., Sadallah, M., Zaki, H. O., Lada, S., Ansar, R.,
Fook, L. M., & Chekima, B. (2024). Understanding the diffusion of AI-generative (ChatGPT) in higher education: Does
students’ integrity matter? Computers in Human Behavior Reports, 14, Article 100402. https://doi.org/10.1016/j.chbr.2024.100402
Bower, M., Torrington, J., Lai, J. W. M., Petocz, P., & Alfano, M. (2024). How should we change teaching and assessment
in response to increasingly powerful generative artificial intelligence? Outcomes of the ChatGPT teacher survey. Education
and Information Technologies, 29, 15403–15439. https://doi.org/10.1007/ s10639-023-12405-0
Brom, C., Dˇechteˇrenko, F., Frollova´, N., St´arkova´, T., Bromov´a, E., & D’Mello, S. K. (2017). Enjoyment or involvement?
Affective-Motivational mediation during
learning from a complex computerized simulation. Computers & Education, 114, 236–254.
https://doi.org/10.1016/j.compedu.2017.07.001 Bryman, A., & Bell, E. (2018). Social research methods (5th ed.). Oxford
University Press.
Budhathoki, T., Zirar, A., Njoya, E. T., & Timsina, A. (2024). ChatGPT adoption and anxiety: A cross-country analysis utilising the
unified theory of acceptance and use of technology (utaut). Studies in Higher Education, 49(5), 831–846.
https://doi.org/10.1080/03075079.2024.2333937
Cambra-Fierro, J. J., Blasco, M. F., Lo´pez-P´erez, M.-E. E., & Trifu, A. (2024). ChatGPT adoption and its influence on faculty well-
being: An empirical research in higher education. Education and information technologies. Advance online publication.
https://doi.org/10.1007/s10639-024-12871-0
Castillo-Manzano, J. I., Castro-Nun˜o, M., Lo´pez-Valpuesta, L., Sanz-Díaz, M. T., & Yn˜iguez, R. (2016). Measuring the effect of
ars on academic performance: A global meta-analysis. Computers & Education, 96, 109–121.
https://doi.org/10.1016/j.compedu.2016.02.007
Çelik, F., Ersanlı, C. Y., & Arslanbay, G. (2024). Does AI simplification of authentic blog texts improve reading comprehension,
inferencing, and anxiety? A one-shot intervention in Turkish efl context. International Review of Research in Open and Distance Learning,
25(3), 287–303. https://doi.org/10.19173/irrodl.v25i3.7779 Çelik, F., Yangın Ersanlı, C., & Arslanbay, G. (2024). Does AI
simplification of authentic blog texts improve reading comprehension, inferencing, and anxiety? A one-
shot intervention in Turkish efl context. International Review of Research in Open and Distance Learning, 25(3), 287–303.
https://doi.org/10.19173/irrodl. v25i3.7779
Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and learning. International Journal of
Educational Technology in Higher Education, 20(1), 38. https://doi.org/10.1186/s41239-023-00408-3
Chan, C. K. Y., & Lee, K. K. W. (2023). The AI generation gap: Are Gen Z students more interested in adopting generative AI such
as ChatGPT in teaching and learning than their Gen X and millennial generation teachers? Smart Learning Environments, 10, 60.
https://doi.org/10.1186/s40561-023-00269-3
Chan, C. K. Y., & Tsi, L. H. Y. (2024). Will generative AI replace teachers in higher education? A study of teacher and student
perceptions. Studies In Educational Evaluation, 83. https://doi.org/10.1016/j.stueduc.2024.101395
* Chandrasekera, T., Hosseini, Z., & Perera, U. (2024). Can artificial intelligence support creativity in early design
processes?. International journal of architectural computing. Advance online publication.
https://doi.org/10.1177/14780771241254637.
* Chang, C.-Y., Yang, C.-L., Jen, H.-J., Ogata, H., & Hwang, G.-H. (2024). Facilitating nursing and health education by
incorporating ChatGPT into learning designs.
Educational Technology & Society, 27(1), 215–230. https://doi.org/10.30191/ETS.202401_27(1).TP02.
Chaudhry, I. S., Sarwary, S. A. M., El Refae, G. A., & Chabchoub, H. (2023). Time to revisit existing student’s performance
evaluation approach in higher education sector in a new era of ChatGPT — a case study. Cogent Education, 10(1), Article
2210461. https://doi.org/10.1080/2331186x.2023.2210461
* Chen, C.-H., & Chang, C.-L. (2024). Effectiveness of AI-assisted game-based learning on science learning outcomes, intrinsic
motivation, cognitive load, and learning behavior. Education and Information Technologies, 29, 18621–18642.
https://doi.org/10.1007/s10639-024-12553-x.
Chen, X., Hu, Z., & Wang, C. (2024). Empowering education development through aigc: A systematic literature review. Education
and Information Technologies, 29, 17485–17537. https://doi.org/10.1007/s10639-024-12549-7
Chiarello, F., Giordano, V., Spada, I., Barandoni, S., & Fantoni, G. (2024). Future applications of generative large language
models: A data-driven case study on ChatGPT. Technovation, 133, Article 103002.
https://doi.org/10.1016/j.technovation.2024.103002
Chiasson, R. M., Goodboy, A. K., Vendemia, M. A., Beer, N., Meisz, G. C., Cooper, L., Arnold, A., Lincoski, A., George, W.,
Zuckerman, C., & Schrout, J. (2024). Does the human professor or artificial intelligence (AI) offer better explanations to
students? Evidence from three within-subject experiments. In Communication education. Advance online publication.
https://doi.org/10.1080/03634523.2024.2398105.
Chiu, T. K. F. (2023). The impact of generative AI (GenAI) on practices, policies and research direction in education: A case of
ChatGPT and midjourney. In Interactive learning environments. Advance online publication.
https://doi.org/10.1080/10494820.2023.2253861.
Cingillioglu, I., Gal, U., & Prokhorov, A. (2024). AI-experiments in education: An AI-driven randomized controlled trial for
higher education research. Education and information technologies. Advance online publication. https://doi.org/10.1007/s10639-

21
0
R. Deng et al. Computers & Education 227 (2025) 105224
024-12633-y
Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49(12), 997–1003. https://doi.org/10.1037/0003-
066X.49.12.997
Collins, K. M., Jiang, A. Q., Frieder, S., Wong, L., Zilka, M., Bhatt, U., Lukasiewicz, T., Wu, Y., Tenenbaum, J. B., Hart, W.,
Gowers, T., Li, W., Weller, A., & Jamnik, M. (2024). Evaluating language models for mathematics through interactions.
Proceedings of the National Academy of Sciences, 121(24), Article e2318124121. https:// doi.org/10.1073/pnas.2318124121
Cooper, G. (2023). Examining science education in ChatGPT: An exploratory study of generative artificial intelligence. Journal of
Science Education and Technology, 32
(3), 444–452. https://doi.org/10.1007/s10956-023-10039-y
Coughlin, S. S. (1990). Recall bias in epidemiologic studies. Journal of Clinical Epidemiology, 43(1), 87–91.
https://doi.org/10.1016/0895-4356(90)90060-3 Crawford, J., Allen, K.-A., Pani, B., & Cowling, M. (2024). When artificial
intelligence substitutes humans in higher education: The cost of loneliness, student success,
and retention. Studies in Higher Education, 49(5), 883–897. https://doi.org/10.1080/03075079.2024.2326956
Critical Appraisal Skills Programme. (2023). Randomised controlled trial. https://casp-uk.net/checklists/casp-rct-randomised-
controlled-trial-checklist.pdf.
Croes, E. A. J., & Antheunis, M. L. (2021). Can we be friends with mitsuku? A longitudinal study on the process of relationship
formation between humans and a social chatbot. Journal of Social and Personal Relationships, 38(1), 279–300.
https://doi.org/10.1177/0265407520959463
Dahri, N. A., Yahaya, N., & Al-Rahmi, W. M. (2024). Exploring the influence of ChatGPT on student academic success and
career readiness. Education and information technologies. Advance online publication. https://doi.org/10.1007/s10639-024-13148-2
Dai, Y., Lai, S., Lim, C. P., & Liu, A. (2023). ChatGPT and its impact on research supervision: Insights from Australian
postgraduate research students. Australasian Journal of Educational Technology, 39(4), 74–88. https://doi.org/10.14742/ajet.8843

21
1
R. Deng et al. Computers & Education 227 (2025) 105224

Dang, A., & Wang, H. (2024). Ethical use of generative AI for writing practices: Addressing linguistically diverse students in U.S.
Universities’ AI statements. Journal of Second Language Writing, 66, Article 101157. https://doi.org/10.1016/j.jslw.2024.101157
* Darmawansah, D., Rachman, D., Febiyani, F., & Hwang, G.-J. (2024). ChatGPT-supported collaborative argumentation:
Integrating collaboration script and argument mapping to enhance EFL students’ argumentation skills. Education and information
technologies. Advance online publication. https://doi.org/10.1007/ s10639-024-12986-4.
David, L., Biwer, F., Baars, M., Wijnia, L., Paas, F., & de Bruin, A. (2024). The relation between perceived mental effort,
monitoring judgments, and learning outcomes:
A meta-analysis. Educational Psychology Review, 36, 66. https://doi.org/10.1007/s10648-024-09903-z
de Winter, J. C. F. (2023). Can ChatGPT pass high school exams on English language comprehension? International Journal of Artificial
Intelligence in Education, Advance online publication. https://doi.org/10.1007/s40593-023-00372-z
Dempere, J., Modugu, K., Hesham, A., & Ramasamy, L. K. (2023). The impact of ChatGPT on higher education. Frontiers in
Education, 8, 1–13. https://doi.org/ 10.3389/feduc.2023.1206936
Deng, R., Benckendorff, P., & Gannaway, D. (2019). Progress and new directions for teaching and learning in MOOCs. Computers &
Education, 129, 48–60. https://doi. org/10.1016/j.compedu.2018.10.019
Deng, R., & Gao, Y. (2023). A review of eye tracking research on video-based learning. Education and Information Technologies, 28,
7671–7702. https://doi.org/ 10.1007/s10639-022-11486-7
Derakhshan, A., & Ghiasvand, F. (2024). Is ChatGPT an evil or an angel for second language education and research? A
phenomenographic study of research-active efl teachers’ perceptions. In International journal of applied linguistics. Advance online
publication. https://doi.org/10.1111/ijal.12561.
Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of
learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251–
19257. https://doi.org/10.1073/pnas.1821936116
Dianova, V. G., & Schultz, M. D. (2023). Discussing ChatGPT’s implications for industry and higher education: The case for
transdisciplinarity and digital humanities.
Industry and Higher Education, 37(5), 593–600. https://doi.org/10.1177/09504222231199989
Dihan, Q., Chauhan, M. Z., Eleiwa, T. K., Brown, A. D., Hassan, A. K., Khodeiry, M. M., Elsheikh, R. H., Oke, I., Nihalani, B. R.,
VanderVeen, D. K., Sallam, A. B., & Elhusseiny, A. M. (2024). Large language models: A new frontier in paediatric cataract
patient education. British Journal of Ophthalmology, 108(10), 1470–1476. https://doi.org/10.1136/bjo-2024-325252
Ding, L., Li, T., Jiang, S., & Gapud, A. (2023). Students’ perceptions of using ChatGPT in a physics class as a virtual tutor.
International Journal of Educational Technology in Higher Education, 20(1), 63. https://doi.org/10.1186/s41239-023-00434-1
* Donald, M. J., Will, D., & Christopher, M. E. (2024). Using ChatGPT with novice arduino programmers: Effects on
performance, interest, self-efficacy, and programming ability. Journal of Research in Technical Careers, 8(1), 1–17.
https://doi.org/10.9741/2578-2118.1152.
Dondio, P., Gusev, V., & Rocha, M. (2023). Do games reduce maths anxiety? A meta-analysis. Computers & Education, 194, Article
104650. https://doi.org/10.1016/j. compedu.2022.104650
Dube, S., Dube, S., Ndlovu, B. M., Maguraushe, K., Malungana, L., Kiwa, F. J., & Muduva, M. (2024). Students’ perceptions of
ChatGPT in education: A rapid systematic literature review. In K. Arai (Ed.), Intelligent computing (pp. 258–279). Springer.
https://doi.org/10.1007/978-3-031-62273-1_18.
Dunnigan, J., Henriksen, D., Mishra, P., & Lake, R. (2023). “Can we just please slow it all down?” School leaders take on
ChatGPT. TechTrends, 67(6), 878–884. https://doi.org/10.1007/s11528-023-00914-1
Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current
Directions in Psychological Science, 12(3), 83–87. https://doi.org/10.1111/1467-8721.01235
Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in
meta-analysis. Biometrics, 56(2), 455–463. https://doi.org/10.1111/j.0006-341x.2000.00455.x
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V.,
Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L.,
Buhalis, D., … Wright, R. (2023). Opinion paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on
opportunities, challenges and implications of generative conversational AI for research, practice and policy. International
Journal of Information Management, 71, Article 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
ElSayary, A. (2023). An investigation of teachers’ perceptions of using ChatGPT as a supporting tool for teaching and learning
in the digital era. Journal of Computer Assisted Learning, 40(3), 931–945. https://doi.org/10.1111/jcal.12926
* Emran, A. Q., Mohammed, M. N., Saeed, H., Abu Keir, M. Y., Alani, Z. N., & Mohammed Ibrahim, F. (2024). Paraphrasing
ChatGPT answers as a tool to enhance university students’ academic writing skills 2024. Bahrain: ASU International Conference in Emerging
Technologies for Sustainability and Intelligent Systems.
* Essel, H. B., Vlachopoulos, D., Essuman, A. B., & Amankwa, J. O. (2024). ChatGPT effects on cognitive skills of
undergraduate students: Receiving instant responses from AI-based conversational large language models (LLMs).
Computers & Education: Artificial Intelligence, 6, Article 100198. https://doi.org/10.1016/j. caeai.2023.100198.
* Essien, A., Bukoye, O. T., O’Dea, X., & Kremantzis, M. (2024). The influence of AI text generators on critical thinking skills in
UK business schools. Studies in Higher Education, 49(5), 865–882. https://doi.org/10.1080/03075079.2024.2316881.
* Farah, J. C., Ingram, S., Spaenlehauer, B., Lasne, F. K.-L., & Gillet, D. (2023). Prompting large language models to power educational chatbots
International Conference on Web-Based Learning. Sydney.
Flod´en, J. (2024). Grading exams using large language models: A comparison between human and AI grading of exams in
higher education using ChatGPT. In British educational research journal. Advance online publication
https://doi.org/10.1002/berj.4069.
Galindo-Domínguez, H., Delgado, N., Losada, D., & Etxabe, J.-M. (2023). An analysis of the use of artificial intelligence in
education in Spain: The in-service teacher’s perspective. Journal of Digital Learning in Teacher Education, 40(1), 41–56.
https://doi.org/10.1080/21532974.2023.2284726
Gammoh, L. A. (2024). ChatGPT risks in academia: Examining university educators’ challenges in Jordan. Education and information
technologies. Advance online publication. https://doi.org/10.1007/s10639-024-13009-y
* Gan, W., Ouyang, J., Li, H., Xue, Z., Zhang, Y., Dong, Q., Huang, J., Zheng, X., & Zhang, Y.. Integrating ChatGPT in
orthopedic education for medical undergraduates: Randomized controlled trial. Journal of Medical Internet Research, 26, Article
e57037. https://doi.org/10.2196/57037.
* Gao, S. (2024). Can artificial intelligence give a hand to open and distributed learning? A probe into the state of
undergraduate students’ academic emotions and test anxiety in learning via ChatGPT. International Review of Research in Open
and Distance Learning, 25(3), 199–218. https://doi.org/10.19173/irrodl.v25i3.7742.
Gao, L., Lo´pez-Pe´rez, M. E., Melero-Polo, I., & Trifu, A. (2024). Ask ChatGPT first! Transforming learning experiences in the age
of artificial intelligence. Studies in Higher Education, Advance online publication. https://doi.org/10.1080/03075079.2024.2323571
Gao, Q., Yan, Z., Zhao, C., Pan, Y., & Mo, L. (2014). To ban or not to ban: Differences in mobile phone policies at elementary,
middle, and high schools. Computers in Human Behavior, 38, 25–32. https://doi.org/10.1016/j.chb.2014.05.011
Garcia Castro, R. A., Mayta Cachicatari, N. A., Bartesaghi Aste, W. M., & Llapa Medina, M. P. (2024). Exploration of ChatGPT in
basic education: Advantages, disadvantages, and its impact on school tasks. Contemporary Educational Technology, 16(3), ep511.
https://doi.org/10.30935/cedtech/14615

21
2
R. Deng et al. Computers & Education 227 (2025) 105224
García, V. A., la Fuente, S. D.-d., Martín, J. I. S., & Ordax, J. M. G. (2024). A critical approach to the use of ChatGPT in higher education.
Cham: The 17th International Conference on Industrial Engineering and Industrial Management.
Garofalo, S. G., & Farenga, S. J. (2024). Science teacher perceptions of the state of knowledge and education at the advent
of generative artificial intelligence popularity. Science & education. Advance online publication. https://doi.org/10.1007/s11191-
024-00534-y
Gencer, G., & Gencer, K. (2024). A comparative analysis of ChatGPT and medical faculty graduates in medical specialization
exams: Uncovering the potential of artificial intelligence in medical education. Cureus, 16(8), Article e66517.
https://doi.org/10.7759/cureus.66517
Gorard, S., & Cook, T. (2007). Where does good evidence come from? International Journal of Research and Method in Education, 30(3),
307–323. https://doi.org/ 10.1080/17437270701614790

21
3
R. Deng et al. Computers & Education 227 (2025) 105224

Gra´jeda, A., Co´rdova, P., Co´rdova, J. P., Laguna-Tapia, A., Burgos, J., Rodríguez, L., Arandia, M., & Sanjin´es, A. (2024).
Embracing artificial intelligence in the arts classroom: Understanding student perceptions and emotional reactions to AI
tools. Cogent Education, 11(1), Article 2378271. https://doi.org/10.1080/ 2331186x.2024.2378271
Grassini, S. (2023). Shaping the future of education: Exploring the potential and consequences of AI and ChatGPT in
educational settings. Education Sciences, 13(7). https://doi.org/10.3390/educsci13070692
Grassini, S., Aasen, M. L., & Møgelvang, A. (2024). Understanding university students’ acceptance of ChatGPT: Insights from the
UTAUT2 model. Applied Artificial Intelligence, 38(1), Article 2371168. https://doi.org/10.1080/08839514.2024.2371168
Grubbs, F. E. (1969). Procedures for detecting outlying observations in samples. Technometrics, 11(1), 1–21.
https://doi.org/10.1080/00401706.1969.10490657 Haig, B. D. (2003). What is a spurious correlation? Understanding Statistics,
2(2), 125–132. https://doi.org/10.1207/S15328031US0202_03
Haindl, P., & Weinberger, G. (2024). Students’ experiences of using ChatGPT in an undergraduate programming course. IEEE
Access, 12, 43519–43529. https://doi. org/10.1109/access.2024.3380909
Halme, P., Toivanen, T., Honkanen, M., Kotiaho, J. S., Mo¨nkkO¨ nen, M., & Timonen, J. (2010). Flawed meta-analysis of
biodiversity effects of forest management.
Conservation Biology, 24(4), 1154–1156. https://doi.org/10.1111/j.1523-1739.2010.01542.x
Hamerman, E. J., Aggarwal, A., & Martins, C. (2024). An investigation of generative AI in the classroom and its implications for
university policy. Quality assurance in education. Advance online publication. https://doi.org/10.1108/qae-08-2024-0149
Hattie, J. (2008). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.
https://doi.org/10.4324/9780203887332 Hays, L., Jurkowski, O., & Sims, S. K. (2024). ChatGPT in K-12 education.
TechTrends, 68, 281–294. https://doi.org/10.1007/s11528-023-00924-z
Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics,
6(2), 107–128. https://doi.org/ 10.3102/10769986006002107
Hedges, L., & Olkin, I. (1985). Statistical methods for meta-analysis. Academic Press.
Hehir, E., Zeller, M., Luckhurst, J., & Chandler, T. (2021). Developing student connectedness under remote learning using
digital resources: A systematic review.
Education and Information Technologies, 26(5), 6531–6548. https://doi.org/10.1007/s10639-021-10577-1
Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z., & Trautsch, A. (2023). A large-scale comparison of human-written versus
ChatGPT-generated essays. Scientific Reports, 13, Article 18617. https://doi.org/10.1038/s41598-023-45644-9
Higgins, J., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M., & Welch, V. (2019). Cochrane handbook for systematic reviews of
interventions (2nd ed.). John Wiley
& Sons.
Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558.
https://doi.org/10.1002/sim.1186 Higgs, J. M., & Stornaiuolo, A. (2024). Being human in the age of generative AI: Young
people’s ethical concerns about writing and living with machines. Reading
Research Quarterly, 59(4), 632–650. https://doi.org/10.1002/rrq.552
Howitt, D., & Cramer, D. (2017). Understanding statistics in psychology with SPSS (7th ed.). Pearson.
* Hsu, M.-H. (2023). Mastering medical terminology with ChatGPT and Termbot. Health Education Journal, Advance online
publication. https://doi.org/10.1177/ 00178969231197371.
* Hu, Y.-H. (2024a). Implementing generative AI chatbots as a decision aid for enhanced values clarification exercises in online
business ethics education. Educational Technology & Society, 27(3), 356–373. https://doi.org/10.30191/ETS.202407_27(3).TP02.
* Hu, Y.-H. (2024b). Improving ethical dilemma learning: Featuring thinking aloud pair problem solving (TAPPS) and AI-
assisted virtual learning companion.
Education and information technologies. Advance online publication. https://doi.org/10.1007/s10639-024-12754-4.
* Huesca, G., Martínez-Trevin˜o, Y., Molina-Espinosa, J. M., Sanroma´n-Calleros, A. R., Martínez-Roma´n, R., Cendejas-Castro, E.
A., & Bustos, R. (2024). Effectiveness of using ChatGPT as a tool to strengthen benefits of the flipped learning strategy.
Education Sciences, 14(6), 660. https://doi.org/10.3390/educsci14060660.
Hyun Baek, T., & Kim, M. (2023). Is ChatGPT scary good? How user motivations affect creepiness and trust in generative
artificial intelligence. Telematics and Informatics, 83, Article 102030. https://doi.org/10.1016/j.tele.2023.102030
Imran, M., & Almusharraf, N. (2023). Analyzing the role of ChatGPT as a writing assistant at higher education level: A
systematic review of the literature.
Contemporary Educational Technology, 15(4), ep464. https://doi.org/10.30935/cedtech/13605
I˙pek, Z. H., Go¨züm, A.I˙. C., Papadakis, S., & Kallogiannakis, M. (2023). Educational applications of the ChatGPT AI system: A
systematic review research. Educational Process: International Journal, 12(3), 26–55. https://doi.org/10.22521/edupij.2023.123.2
* Ironsi, C. S., & Ironsi, S. S. (2024). Experimental evidence for the efficacy of generative AI in improving students’ writing
skills. Quality assurance in education.
Advance online publication. https://doi.org/10.1108/qae-04-2024-0065.
Islam, I., & Islam, M. N. (2024). Exploring the opportunities and challenges of ChatGPT in academia. Discover Education, 3(1), 31.
https://doi.org/10.1007/s44217- 024-00114-w
Jaboob, M., Hazaimeh, M., & Al-Ansi, A. M. (2024). Integration of generative AI techniques and applications in student behavior
and cognitive achievement in Arab higher education. International Journal of Human–Computer Interaction, Advance online punlication.
https://doi.org/10.1080/10447318.2023.2300016
Jahic, I., Ebner, M., & Scho¨n, S. (2023). Harnessing the power of artificial intelligence and ChatGPT in education – a first rapid literature review. Vienna,
Austria: EdMedia +
Innovate Learning. https://www.learntechlib.org/p/222670/.
Jarry Trujillo, C., Vela Ulloa, J., Escalona Vivas, G., Grasset Escobar, E., Villagra´n Guti´errez, I., Achurra Tirado, P., & Varas
Cohen, J. (2024). Surgeons vs ChatGPT: Assessment and feedback performance based on real surgical scenarios. Journal of
Surgical Education, 81(7), 960–966. https://doi.org/10.1016/j. jsurg.2024.03.012
Jensen, L. X., Buhl, A., Sharma, A., & Bearman, M. (2024). Generative AI and higher education: A review of claims from the first
months of ChatGPT. Higher education.
Advance online publication. https://doi.org/10.1007/s10734-024-01265-3
Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human
teachers and ChatGPT. Education and Information Technologies, 28(12), 15873–15892. https://doi.org/10.1007/s10639-023-11834-1
* Ji, Y., Zou, X., Li, T., & Zhan, Z. (2023). The effectiveness of ChatGPT on pre-service teachers’ STEM teaching literacy, learning performance, and cognitive load in
a teacher
training course. Guangzhou, China: The 6th International Conference on Educational Technology Management.
Jiang, Y., Xie, L., Lin, G., & Mo, F. (2024). Widen the debate: What is the academic community’s perception on ChatGPT? Education and information
technologies. Advance online publication. https://doi.org/10.1007/s10639-024-12677-0
Jo, H. (2023). Understanding AI tool engagement: A study of ChatGPT usage and word-of-mouth among university students and
office workers. Telematics and Informatics, 85, Article 102067. https://doi.org/10.1016/j.tele.2023.102067
Jo, H. (2024). From concerns to benefits: A comprehensive study of ChatGPT usage in education. International Journal of Educational
Technology in Higher Education, 21
(1), 35. https://doi.org/10.1186/s41239-024-00471-4
Jochim, J., & Lenz-Kesekamp, V. K. (2024). Teaching and testing in the era of text-generative AI: Exploring the needs of students
3
0
R. Deng et al. Computers & Education 227 (2025) 105224
and teachers. Information and learning sciences. Advance online publication. https://doi.org/10.1108/ils-10-2023-0165
Joˇst, G., Taneski, V., & Karakatiˇc, S. (2024). The impact of large language models on programming education and student
learning outcomes. Applied Sciences, 14(10). https://doi.org/10.3390/app14104115
Kapur, M., Saba, J., & Roll, I. (2023). Prior math achievement and inventive production predict learning from productive
failure. Npj Science of Learning, 8(1), 15. https://doi.org/10.1038/s41539-023-00165-y
Karatas¸, F., Abedi, F. Y., Ozek Gunyel, F., Karadeniz, D., & Kuzgun, Y. (2024). Incorporating AI in foreign language education: An
investigation into ChatGPT’s effect on foreign language learners. Education and information technologies. Advance online
publication. https://doi.org/10.1007/s10639-024-12574-6
Karkoulian, S., Sayegh, N., & Sayegh, N. (2024). ChatGPT unveiled: Understanding perceptions of academic integrity in
higher education - a qualitative approach.
Journal of Academic Ethics, Advance online publication. https://doi.org/10.1007/s10805-024-09543-6
* Kavadella, A., Dias da Silva, M. A., Kaklamanos, E. G., Stamatopoulos, V., & Giannakopoulos, K. (2024). Evaluation of
ChatGPT’s real-life implementation in undergraduate dental education: Mixed methods study. JMIR Medical Education, 10,
Article e51344. https://doi.org/10.2196/51344.

3
0
 R. Deng et al. Computers & Education 227 (2025) 105224

* Khuibut, W., Premthaisong, S., & Chaipidech, P. (2023). Integrating ChatGPT into synectics model to improve high school
student’s creative writing skill. The 31st International Conference on Computers in Education, Matsue, Japan.
* Kim, R. (2024). Effects of ChatGPT on Korean EFL learners’ main-idea reading comprehension via top-down processing.
Language Research, 60(1), 83–106. https:// doi.org/10.30961/lr.2024.60.1.83.
Kim, J., Ham, Y., & Lee, S.-S. (2024). Differences in student-AI interaction process on a drawing task: Focusing on students’
attitude towards AI and the level of drawing skills. Australasian Journal of Educational Technology, 40(1), 19–41.
https://doi.org/10.14742/ajet.8859
Kmet, L. M., Lee, R. C., & Cook, L. S. (2004). Standard quality assessment criteria for evaluating primary research papers from a variety of fields. Alberta
Heritage Foundation for Medical Research.
Koltovskaia, S., Rahmati, P., & Saeli, H. (2024). Graduate students’ use of ChatGPT for academic text revision: Behavioral,
cognitive, and affective engagement.
Journal of Second Language Writing, 65, Article 101130. https://doi.org/10.1016/j.jslw.2024.101130
Korseberg, L., & Elken, M. (2024). Waiting for the revolution: How higher education institutions initially responded to ChatGPT.
Higher education. Advance online publication. https://doi.org/10.1007/s10734-024-01256-4
* Kosar, T., Ostoji´c, D., Liu, Y. D., & Mernik, M. (2024). Computer science education in ChatGPT era: Experiences from an
experiment in a programming course for
novice programmers. Mathematics, 12(5), 629. https://doi.org/10.3390/math12050629.
Krieglstein, F., Beege, M., Rey, G. D., Ginns, P., Krell, M., & Schneider, S. (2022). A systematic meta-analysis of the reliability
and validity of subjective cognitive load questionnaires in experimental multimedia learning research. Educational Psychology
Review, 34(4), 2485–2541. https://doi.org/10.1007/s10648-022-09683-4
* Kucuk, T. (2024). Chatgpt integrated grammar teaching and learning in efl classes: A study on tishk international
university students in erbil, Iraq. Arab World English Journal, 100–111. https://doi.org/10.24093/awej/ChatGPT.6.
Kwan, V. S. Y., John, O. P., Robins, R. W., & Kuang, L. L. (2008). Conceptualizing and assessing self-enhancement bias: A
componential approach. Journal of Personality and Social Psychology, 94(6), 1062–1077. https://doi.org/10.1037/0022-3514.94.6.1062
Laak, K.-J., & Aru, J. (2024). Generative AI in K-12: Opportunities for learning and utility for teachers 25th international conference, AIED. Brazil: Recife.
Al-Qaysi, N., Al-Emran, M., Al-Sharafi, M. A., Iranmanesh, M., Ahmad, A., & Mahmoud, M. A. (2024). Determinants of ChatGPT
use and its impact on learning performance: An integrated model of BRT and TPB. International Journal of Human–Computer Interaction,
Advance online publication. https://doi.org/10.1080/ 10447318.2024.2361210.
Lambert, J., & Stevens, M. ChatGPT and generative AI technology: A mixed bag of concerns and new opportunities. Computers
in the Schools, Advance online publication. https://doi.org/10.1080/07380569.2023.2256710.
Law, L. (2024). Application of generative artificial intelligence (GenAI) in language teaching and learning: A scoping literature
review. Computers and Education Open, 6, Article 100174. https://doi.org/10.1016/j.caeo.2024.100174
* Lee, H.-Y., Chen, P.-H., Wang, W.-S., Huang, Y.-M., & Wu, T.-T. (2024). Empowering ChatGPT with guidance mechanism in
blended learning: Effect of self-regulated learning, higher-order thinking skills, and knowledge construction. International Journal
of Educational Technology in Higher Education, 21(1), 16. https://doi.org/ 10.1186/s41239-024-00447-4.
Lee, U., Jung, H., Jeon, Y., Sohn, Y., Hwang, W., Moon, J., & Kim, H. (2024). Few-shot is enough: Exploring ChatGPT prompt
engineering method for automatic question generation in English education. Education and Information Technologies, 29, 11483–
11515. https://doi.org/10.1007/s10639-023-12249-8
Lee, M., Oi-yeung Lam, B., Ju, E., & Dean, J. (2017). Part-time employment and problem behaviors: Evidence from adolescents
in South Korea. Journal of Research on Adolescence, 27(1), 88–104. https://doi.org/10.1111/jora.12258
Lee, L., Packer, T. L., Tang, S. H., & Girdler, S. (2008). Self-management education programs for age-related macular
degeneration: A systematic review. Australasian Journal on Ageing, 27(4), 170–176. https://doi.org/10.1111/j.1741-
6612.2008.00298.x
Lee, S., & Song, K.-S. (2024). Teachers’ and students’ perceptions of AI-generated concept explanations: Implications for
integrating generative AI in computer science education. Computers & Education: Artificial Intelligence, 7, Article 100283.
https://doi.org/10.1016/j.caeai.2024.100283
Lee, G.-G., & Zhai, X. (2024). Using ChatGPT for science learning: A study on pre-service teachers’ lesson planning. IEEE Transactions
on Learning Technologies, 17, 1683–1700. https://doi.org/10.1109/tlt.2024.3401457
* Li, H. (2023). Effects of a ChatGPT-based flipped learning guiding approach on learners’ courseware project performances
and perceptions. Australasian Journal of Educational Technology, 39(5), 40–58. https://doi.org/10.14742/ajet.8923.
* Li, T., Ji, Y., & Zhan, Z. (2024). Expert or machine? Comparing the effect of pairing student teacher with in-service teacher
and ChatGPT on their critical thinking, learning performance, and cognitive load in an integrated-STEM course. Asia Pacific
Journal of Education, 44(1), 45–60. https://doi.org/10.1080/ 02188791.2024.2305163.
Li, L., Ma, Z., Fan, L., Lee, S., Yu, H., & Hemphill, L. (2024). ChatGPT in education: A discourse analysis of worries and
concerns on social media. Education and Information Technologies, 29, 10729–10762. https://doi.org/10.1007/s10639-023-12256-9
* Li, Y., Sadiq, G., Qambar, G., & Zheng, P. (2024). The impact of students’ use of ChatGPT on their research skills: The
mediating effects of autonomous motivation, engagement, and self-directed learning. Education and information technologies.
Advance online publication. https://doi.org/10.1007/s10639-024-12981-9.
Lian, Y., Tang, H., Xiang, M., & Dong, X. (2024). Public attitudes and sentiments toward ChatGPT in China: A text mining
analysis based on social media. Technology in Society, 76, Article 102442. https://doi.org/10.1016/j.techsoc.2023.102442
* Liao, J., Zhong, L., Zhe, L., Xu, H., Liu, M., & Xie, T. (2024). Scaffolding computational thinking with ChatGPT. IEEE Transactions
on Learning Technologies, 17, 1668–1682. https://doi.org/10.1109/tlt.2024.3392896.
Lin, Z., & Chen, H. (2024). Investigating the capability of ChatGPT for generating multiple-choice reading comprehension
items. System, 123, Article 103344. https:// doi.org/10.1016/j.system.2024.103344
Liu, X. (2024). Navigating uncharted waters: Teachers’ perceptions of and reactions to AI-induced challenges to assessment. The asia-
pacific education researcher.
Advance online publication. https://doi.org/10.1007/s40299-024-00890-x
* Liu, Z., Vobolevich, A., & Oparin, A. (2023). The influence of AI ChatGPT on improving teachers’ creative thinking. International
Journal of Learning, Teaching and Educational Research, 22(12), 124–139. https://doi.org/10.26803/ijlter.22.12.7.
Liu, M., Zhang, L. J., & Biebricher, C. (2024). Investigating students’ cognitive processes in generative AI-assisted digital
multimodal composing and traditional writing. Computers & Education, 211. https://doi.org/10.1016/j.compedu.2023.104977
Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4).
https://doi.org/10.3390/educsci13040410 Lo, C. K., Hew, K. F., & Jong, M. S.-y. (2024). The influence of ChatGPT on student
engagement: A systematic review and future research agenda. Computers &
Education, 105100. https://doi.org/10.1016/j.compedu.2024.105100
* Lu, J., Zheng, R., Gong, Z., & Xu, H. (2024). Supporting teachers’ professional development with generative AI: The effects on
higher order thinking and self-efficacy.
IEEE Transactions on Learning Technologies, 17, 1279–1289. https://doi.org/10.1109/tlt.2024.3369690.
* Lyu, W., Wang, Y., Chung, T. R., Sun, Y., & Zhang, Y. (2024). Evaluating the effectiveness of llMs in introductory computer science education: A semester-
long field study.
Atlanta: The 11th ACM Conference on Learning @ Scale.
Ma, Q., Crosthwaite, P., Sun, D., & Zou, D. (2024). Exploring ChatGPT literacy in language education: A global perspective and
comprehensive approach. Computers &
31
R. Deng et al. Computers & Education 227 (2025) 105224
Education: Artificial Intelligence, 7, Article 100278. https://doi.org/10.1016/j.caeai.2024.100278

Maatouk, A., Piovesan, N., Ayed, F., Domenico, A. D., & Debbah, M. (2024). Large language models for telecom: Forthcoming
impact on the industry. In IEEE communications magazine. Advance online publication. https://doi.org/10.1109/mcom.001.2300473.

* Maghamil, M. C., & Sieras, S. G. (2024). Impact of ChatGPT on the academic writing quality of senior high school students. Journal
of English Language Teaching &
Applied Linguistics, 6(2), 115–128. https://doi.org/10.32996/jeltal.2024.6.2.14.
* Mahapatra, S. (2024). Impact of ChatGPT on esl students’ academic writing skills: A mixed methods intervention study.
Smart Learning Environments, 11(1), 9. https://doi.org/10.1186/s40561-024-00295-9.
Maheshwari, G. (2024). Factors influencing students’ intention to adopt and use ChatGPT in higher education: A study in the
Vietnamese context. Education and Information Technologies, 29(10), 12167–12195. https://doi.org/10.1007/s10639-023-12333-z

32
R. Deng et al. Computers & Education 227 (2025) 105224

Mahrishi, M., Abbas, A., Radovanovic, D., & Hosseini, S. (2024). Emerging dynamics of ChatGPT in academia: A scoping review.
Journal of University Teaching and Learning Practice, 21(1–31).
Mai, D. T. T., Da, C. V., & Hanh, N. V. (2024). The use of ChatGPT in teaching and learning: A systematic review through swot
analysis approach. Frontiers in Education, 9, Article 1328769. https://doi.org/10.3389/feduc.2024.1328769
McGrath, C., Farazouli, A., & Cerratto-Pargman, T. (2024). Generative AI chatbots in higher education: A review of an
emerging research area. Higher education.
Advance online publication. https://doi.org/10.1007/s10734-024-01288-w
McHugh, M. L. (2012). Interrater reliability: The Kappa statistic. Biochemia Medica, 22(3), 276–282.
* Meyer, J., Jansen, T., Schiller, R., Liebenow, L. W., Steinbach, M., Horbach, A., & Fleckenstein, J. (2024). Using LLMs to
bring evidence-based feedback into the classroom: AI-generated feedback increases secondary students’ text revision,
motivation, and positive emotions. Computers & Education: Artificial Intelligence, 6, Article 100199.
https://doi.org/10.1016/j.caeai.2023.100199.
Mohamed, A. M. (2024). Exploring the potential of an AI-based chatbot (ChatGPT) in enhancing English as a foreign language
(EFL) teaching: Perceptions of EFL faculty members. Education and Information Technologies, 29(3), 3195–3217.
https://doi.org/10.1007/s10639-023-11917-z
Monib, W. K., Qazi, A., & Mahmud, M. M. (2024). Exploring learners’ experiences and perceptions of ChatGPT as a learning
tool in higher education. Education and information technologies. Advance online publication. https://doi.org/10.1007/s10639-024-
13065-4
Montenegro-Rueda, M., Ferna´ndez-Cerero, J., Ferna´ndez-Batanero, J. M., & Lo´pez-Meneses, E. (2023). Impact of the
implementation of ChatGPT in education: A systematic review. Computers, 12(8), 153.
https://doi.org/10.3390/computers12080153
Moorhouse, B. L., & Kohnke, L. (2024). The effects of generative AI on initial language teacher education: The perceptions of
teacher educators. System, 122, Article 103290. https://doi.org/10.1016/j.system.2024.103290
Moorhouse, B. L., Yeo, M. A., & Wan, Y. (2023). Generative AI tools and assessment: Guidelines of the world’s top-ranking
universities. Computers and Education Open,
5. https://doi.org/10.1016/j.caeo.2023.100151
* Mugableh, A. I. (2024). The impact of ChatGPT on the development of vocabulary knowledge of Saudi EFL students. Arab World
English Journal, 265–281. https:// doi.org/10.24093/awej/ChatGPT.18.
* Mun, C.-y. (2024). EFL learners’ English writing feedback and their perception of using ChatGPT. Journal of English Teaching through
Movies and Media, 25(2), 26–39.
Mutlu-Bayraktar, D., Cosgun, V., & Altan, T. (2019). Cognitive load in multimedia learning environments: A systematic review.
Computers & Education, 141. https:// doi.org/10.1016/j.compedu.2019.103618
Na, H., Staudt Willet, K. B., Shi, H., Hur, J., He, D., & Kim, C. (2024). Initial discussions of ChatGPT in education-related
subreddits. Journal of Research on Technology in Education, Advance online publication. https://doi.org/10.1080/15391523.2024.2338091
* Naamati-Schneider, L., & Alt, D. (2024). Beyond digital literacy: The era of AI-powered assistants and evolving user skills.
Education and information technologies.
Advance online publication. https://doi.org/10.1007/s10639-024-12694-z.
Nam, B. H., & Bai, Q. (2023). ChatGPT and its ethical implications for stem research and higher education: A media discourse
analysis. International Journal of STEM Education, 10(1), 66. https://doi.org/10.1186/s40594-023-00452-5
Newton, P., & Xiromeriti, M. (2024). ChatGPT performance on multiple choice question examinations in higher education. A
pragmatic scoping review. Assessment &
Evaluation in Higher Education, 49(6), 781–798. https://doi.org/10.1080/02602938.2023.2299059
* Ng, D. T. K., Tan, C. W., & Leung, J. K. L. (2024). Empowering student self-regulated learning and science education through
ChatGPT: A pioneering pilot study.
British Journal of Educational Technology, 55(4), 1328–1353. https://doi.org/10.1111/bjet.13454.
Ngo, T. T. A., Tran, T. T., An, G. K., & Nguyen, P. T. (2024). ChatGPT for educational purposes: Investigating the impact of
knowledge management factors on student satisfaction and continuous usage. IEEE Transactions on Learning Technologies, 17, 1367–
1378. https://doi.org/10.1109/tlt.2024.3383773
NHLBI. (2021). Study quality assessment tools: Quality assessment of controlled intervention studies.
https://www.nhlbi.nih.gov/health-topics/study-quality- assessment-tools.
* Niloy, A. C., Akter, S., Sultana, N., Sultana, J., & Rahman, S. I. U. (2023). Is chatgpt a menace for creative writing ability? An
experiment. Journal of Computer Assisted
Learning, 40(2), 919–930. https://doi.org/10.1111/jcal.12929.
Niloy, A. C., Bari, M. A., Sultana, J., Chowdhury, R., Raisa, F. M., Islam, A., Mahmud, S., Jahan, I., Sarkar, M., Akter, S.,
Nishat, N., Afroz, M., Sen, A., Islam, T., Tareq, M. H., & Hossen, M. A. (2024). Why do students use ChatGPT? Answering
through a triangulation approach. Computers & Education: Artificial Intelligence, 6, Article 100208.
https://doi.org/10.1016/j.caeai.2024.100208
Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science,
381(6654), 187–192. https://doi.org/ 10.1126/science.adh2586
Ofem, U. J., Owan, V. J., Iyam, M. A., Udeh, M. I., Anake, P. M., & Ovat, S. V. (2024). Students’ perceptions, attitudes and
utilisation of ChatGPT for academic dishonesty: Multigroup analyses via PLS‒SEM. Education and information technologies. Advance
online publication. https://doi.org/10.1007/s10639-024-12850-5
Okulu, H. Z., & Muslu, N. (2024). Designing a course for pre-service science teachers using ChatGPT: What ChatGPT brings to
the table. Interactive learning environments. Advance online publication. https://doi.org/10.1080/10494820.2024.2322462
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2010). Cognitive load measurement as a means to advance
cognitive load theory. Educational Psychologist, 38(1), 63–71. https://doi.org/10.1207/s15326985ep3801_8
Paas, F., Tuovinen, J. E., van Merri¨enboer, J. J. G., & Aubteen Darabi, A. (2005). A motivational perspective on the relation
between mental effort and performance: Optimizing learner involvement in instruction. Educational Technology Research &
Development, 53(3), 25–34. https://doi.org/10.1007/BF02504795
Pack, A., & Maloney, J. (2023). Using generative artificial intelligence for language education research: Insights from using
OpenAI’s ChatGPT. Tesol Quarterly, 57(4), 1571–1582. https://doi.org/10.1002/tesq.3253
Pack, A., & Maloney, J. (2024). Using artificial intelligence in TESOL: Some ethical and pedagogical considerations. Tesol
Quarterly, 58(2), 1007–1018. https://doi. org/10.1002/tesq.3320
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A.,
Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hro´bjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E.,
McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews.
BMJ, 372, n71. https://doi.org/10.1136/bmj.n71
Park, Y., & Doo, M. Y. (2024). Role of AI in blended learning: A systematic literature review. International Review of Research in Open and
Distance Learning, 25(1), 164–196. https://doi.org/10.19173/irrodl.v25i1.7566
Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.),
Measures of personality and social psychological attitudes (pp. 17–59). Academic Press. https://doi.org/10.1016/B978-0-12-590241-
0.50006-X.
Peng, C.-Y. J., Long, H., & Abaci, S. (2012). Power analysis software for educational researchers. The Journal of Experimental

33
R. Deng et al. Computers & Education 227 (2025) 105224
Education, 80(2), 113–136. https://doi. org/10.1080/00220973.2011.647115
Perera, P., & Lankathilaka, M. (2023). AI in higher education: A literature review of ChatGPT and guidelines for responsible
implementation. International Journal of Research and Innovation in Social Science, 7(6), 306–314.
https://doi.org/10.47772/IJRISS.2023.7623
Perkins, M., Roe, J., Postma, D., McGaughran, J., & Hickerson, D. (2023). Detection of GPT-4 generated text in higher
education: Combining academic judgement and software to identify generative AI tool misuse. Journal of Academic Ethics, 22(1),
89–113. https://doi.org/10.1007/s10805-023-09492-6
Playfoot, D., Quigley, M., & Thomas, A. G. (2024). Hey ChatGPT, give me a title for a paper about degree apathy and student
use of AI for assignment writing. The Internet and Higher Education, 62, Article 100950.
https://doi.org/10.1016/j.iheduc.2024.100950
Polat, H., Topuz, A. C., Yıldız, M., Tas¸lıbeyaz, E., & Kurs¸un, E. (2024). A bibliometric analysis of research on ChatGPT in
education. International Journal of Technology in Education, 7(1), 59–85. https://doi.org/10.46328/ijte.606
Polyportis, A. (2024). A longitudinal study on artificial intelligence adoption: Understanding the drivers of ChatGPT usage
behavior change in higher education.
Frontiers in Artificial Intelligence, 6, Article 1324398. https://doi.org/10.3389/frai.2023.1324398
Pradana, M., Elisa, H. P., & Syarifuddin, S. (2023). Discussing ChatGPT in education: A literature review and bibliometric
analysis. Cogent Education, 10(2), Article 2243134. https://doi.org/10.1080/2331186x.2023.2243134

34
R. Deng et al. Computers & Education 227 (2025) 105224

Rawas, S. (2024). ChatGPT: Empowering lifelong learning in the digital age of higher education. Education and Information
Technologies, 29(6), 6895–6908. https:// doi.org/10.1007/s10639-023-12114-8
Rienties, B., Domingue, J., Duttaroy, S., Herodotou, C., Tessarolo, F., & Whitelock, D. (2024). What distance learning students
want from an AI Digital Assistant.
Distance education. Advance online publication. https://doi.org/10.1080/01587919.2024.2338717
Roohr, K., Olivera-Aguilar, M., Ling, G., & Rikoon, S. (2019). A multi-level modeling approach to investigating students’ critical
thinking at higher education institutions. Assessment & Evaluation in Higher Education, 44(6), 946–960.
https://doi.org/10.1080/02602938.2018.1556776
Salifu, I., Arthur, F., Arkorful, V., Abam Nortey, S., & Solomon Osei-Yaw, R. (2024). Economics students’ behavioural intention
and usage of ChatGPT in higher education: A hybrid structural equation modelling-artificial neural network approach. Cogent
Social Sciences, 10(1), Article 2300177. https://doi.org/10.1080/ 23311886.2023.2300177
Sallam, M., Al-Mahzoum, K., Almutawaa, R. A., Alhashash, J. A., Dashti, R. A., AlSafy, D. R., Almutairi, R. A., & Barakat, M.
(2024). The performance of OpenAI ChatGPT-4 and google gemini in virology multiple-choice questions: A comparative
analysis of English and Arabic responses. BMC Research Notes, 17(1), 247. https://doi.org/10.1186/s13104-024-06920-7
Samala, A. D., Rawas, S., Wang, T., Reed, J. M., Kim, J., Howard, N.-J., & Ertz, M. (2024). Unveiling the landscape of
generative artificial intelligence in education: A comprehensive taxonomy of applications, challenges, and future prospects.
Education and Information Technologies, Advance online publication.. https://doi.org/ 10.1007/s10639-024-12936-0
Sandmann, S., Riepenhausen, S., Plagwitz, L., & Varghese, J. (2024). Systematic analysis of ChatGPT, Google search and
Llama 2 for clinical decision support tasks.
Nature Communications, 15(1), 2050. https://doi.org/10.1038/s41467-024-46411-8
Saritepeci, M., & Yildiz Durak, H. (2024). Effectiveness of artificial intelligence integration in design-based learning on design
thinking mindset, creative and reflective thinking skills: An experimental study. Education and information technologies. Advance
Online Publication. https://doi.org/10.1007/s10639-024-12829-2
Schmidt, S. A. J., Lo, S., & Hollestein, L. M. (2018). Research techniques made simple: Sample size estimation and power
calculation. Journal of Investigative Dermatology, 138(8), 1678–1682. https://doi.org/10.1016/j.jid.2018.06.165
Schott, C., van Roekel, H., & Tummers, L. G. (2020). Teacher leadership: A systematic review, methodological quality
assessment and conceptual framework.
Educational Research Review, 31, Article 100352. https://doi.org/10.1016/j.edurev.2020.100352
Schroeder, N. L., Siegle, R. F., & Craig, S. D. (2023). A meta-analysis on learning from 360 ◦ video. Computers & Education, 206,
Article 104901. https://doi.org/ 10.1016/j.compedu.2023.104901
Sˇedlbauer, J., Cˇinˇcera, J., Slavík, M., & Hartlov´a, A. (2024). Students’ reflections on their experience with ChatGPT. Journal of
Computer Assisted Learning, 40, 1526–1534. https://doi.org/10.1111/jcal.12967
Seth, I., Lim, B., Cevik, J., Sofiadellis, F., Ross, R. J., Cuomo, R., & Rozen, W. M. (2024). Utilizing GPT-4 and generative artificial
intelligence platforms for surgical education: An experimental study on skin ulcers. European Journal of Plastic Surgery, 47(1), 19.
https://doi.org/10.1007/s00238-024-02162-9
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton,
Mifflin and Company. Shahzad, M. F., Xu, S., & Zahid, H. (2024). Exploring the impact of generative AI-based technologies on
learning performance through self-efficacy, fairness & ethics, creativity, and trust in higher education. Education and information
technologies. Advance online publication. https://doi.org/10.1007/s10639-024-12949-9
* Shang, S., & Geng, S. (2024). Empowering learners with AI-generated content for programming learning and computational
thinking: The lens of extended effective use theory. Journal of Computer Assisted Learning, 40, 1941–1958.
https://doi.org/10.1111/jcal.12996.
* Shi, S. J., Li, J. W., & Zhang, R. (2024). A study on the impact of Generative Artificial Intelligence supported Situational
Interactive Teaching on students’ ‘flow’ experience and learning effectiveness — a case study of legal education in China.
Asia Pacific Journal of Education, 44(1), 112–138. https://doi.org/10.1080/ 02188791.2024.2305161.
* Shin, H., De Gagne, J. C., Kim, S. S., & Hong, M. (2024). The impact of artificial intelligence-assisted learning on nursing
students’ ethical decision-making and clinical reasoning in pediatric care: A quasi-experimental study. CIN: Computers,
Informatics, Nursing, 42(10), 704–711. https://doi.org/10.1097/ cin.0000000000001177.
Shin, D., & Lee, J. H. (2024). Exploratory study on the potential of ChatGPT as a rater of second language writing. Education and
information technologies. Advance online publication. https://doi.org/10.1007/s10639-024-12817-6
* Silitonga, L. M., Hawanti, S., Aziez, F., Furqon, M., Zain, D. S. M., Anjarani, S., & Wu, T.-T. (2023). The impact of AI chatbot-based learning
on students’ motivation in
English writing classroom. Porto, Portugal: The 6th International Conference on Innovative Technologies and Learning.
Sommet, N., Weissman, D. L., Cheutin, N., & Elliot, A. J. (2023). How many participants do I need to test an interaction?
Conducting an appropriate power analysis and achieving sufficient power to detect an interaction. Advances in Methods and
Practices in Psychological Science, 6(3). https://doi.org/10.1177/ 25152459231178728. Advance online publication.
* Song, C., & Song, Y. (2023). Enhancing academic writing skills and motivation: Assessing the efficacy of ChatGPT in AI-assisted
language learning for EFL students.
Frontiers in Psychology, 14, Article 1260843. https://doi.org/10.3389/fpsyg.2023.1260843.
* Stadler, M., Bannert, M., & Sailer, M. (2024). Cognitive ease at a cost: LLMs reduce mental effort but compromise depth in
student scientific inquiry. Computers in Human Behavior, 160, Article 108386. https://doi.org/10.1016/j.chb.2024.108386.
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024).
Comparing the quality of human and ChatGPT feedback of students’ writing. Learning and Instruction, 91, Article 101894.
https://doi.org/10.1016/j.learninstruc.2024.101894
Sterne, J. A. C., Savovi´c, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H. Y., Corbett, M. S.,
Eldridge, S. M., Emberson, J. R., Herna´n, M. A., Hopewell, S., Hro´bjartsson, A., Junqueira, D. R., Jüni, P., Kirkham, J. J.,
Lasserson, T., Li, T., … Higgins, J. P. T. (2019). RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ,
366, l4898. https://doi.org/10.1136/bmj.l4898
Stojanov, A. (2023). Learning with ChatGPT 3.5 as a more knowledgeable other: An autoethnographic study. International Journal of
Educational Technology in Higher Education, 20(1), 35. https://doi.org/10.1186/s41239-023-00404-7
Su, Y., Lin, Y., & Lai, C. (2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57, Article
100752. https://doi.org/10.1016/j. asw.2023.100752
Su, J., & Yang, W. (2023). Powerful or mediocre? Kindergarten teachers’ perspectives on using ChatGPT in early childhood
education. Interactive learning environments.
Advance online publication. https://doi.org/10.1080/10494820.2023.2266490
* Suciati, S., Silitonga, L. M., Wiyaka, Huang, C.-Y., & Anggara, A. A. (2024). Enhancing engagement and motivation in English writing through
AI: The impact of ChatGPT- supported collaborative learning. Tartu: International Conference on Innovative Technologies and Learning.
* Sun, D., Boudouaia, A., Zhu, C., & Li, Y. (2024). Would ChatGPT-facilitated programming mode impact college students’
programming behaviors, performances, and perceptions? An empirical study. International Journal of Educational Technology in Higher
Education, 21(1), 14. https://doi.org/10.1186/s41239-024-00446-5.
* Svendsen, K., Askar, M., Umer, D., & Halvorsen, K. H. (2024). Short-term learning effect of ChatGPT on pharmacy students’
learning. Exploratory Research in Clinical and Social Pharmacy, 15, Article 100478. https://doi.org/10.1016/j.rcsop.2024.100478.
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational
35
R. Deng et al. Computers & Education 227 (2025) 105224
Psychology Review, 10(3), 251–296.
Tai, T.-Y. (2024). Comparing the effects of intelligent personal assistant-human and human-human interactions on EFL
learners’ willingness to communicate beyond the classroom. Computers & Education, 210.
https://doi.org/10.1016/j.compedu.2023.104965
Talsma, K., Schüz, B., Schwarzer, R., & Norris, K. (2018). I believe, therefore I achieve (and vice versa): A meta-analytic cross-
lagged panel analysis of self-efficacy and academic performance. Learning and Individual Differences, 61, 136–150.
https://doi.org/10.1016/j.lindif.2017.11.015
Tan, C. N.-L., Tee, M., & Koay, K. Y. (2024). Discovering students’ continuous intentions to use ChatGPT in higher education: A
tale of two theories. Asian education and development studies. Advance online publication. https://doi.org/10.1108/AEDS-04-2024-0096
Titus, M. A. (2006). Detecting selection bias, using propensity score matching, and estimating treatment effects: An
application to the private returns to a master’s degree. Research in Higher Education, 48(4), 487–521.
https://doi.org/10.1007/s11162-006-9034-3
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my
guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15.
https://doi.org/10.1186/s40561-023-00237-x

36
R. Deng et al. Computers & Education 227 (2025) 105224

Tsai, C.-Y., Lin, Y.-T., & Brown, I. K. (2024). Impacts of ChatGPT-assisted writing for EFL English majors: Feasibility and
challenges. Education and information technologies. Advance online publication. https://doi.org/10.1007/s10639-024-12722-y
Tu, Y.-F., & Hwang, G.-J. (2023). University students’ conceptions of ChatGPT-supported learning: A drawing and epistemic
network analysis. Interactive learning environments. Advance online publication. https://doi.org/10.1080/10494820.2023.2286370
UNESCO Institute for Statistics. (2012). International standard classification of education ISCED 2011. UNESCO Institute for Statistics.
* Urban, M., Deˇchtˇerenko, F., Lukavský, J., Hrabalova´, V., Svacha, F., Brom, C., & Urban, K. (2024). ChatGPT improves
creative problem-solving performance in university students: An experimental study. Computers & Education, 215, Article
105031. https://doi.org/10.1016/j.compedu.2024.105031.
Urhan, S., Gençaslan, O., & Dost, S¸ . (2024). An argumentation experience regarding concepts of calculus with ChatGPT.
Interactive learning environments. Advance online publication. https://doi.org/10.1080/10494820.2024.2308093
Valcea, S., Hamdani, M. R., & Wang, S. (2024). Exploring the impact of ChatGPT on business school education: Prospects,
boundaries, and paradoxes. Journal of Management Education, 48(5), 915–947. https://doi.org/10.1177/10525629241261313
van den Berg, G., & du Plessis, E. (2023). ChatGPT and generative AI: Possibilities for its contribution to lesson planning,
critical thinking and openness in teacher education. Education Sciences, 13(10), 998. https://doi.org/10.3390/educsci13100998
Vargas-Murillo, A. R., de la Asuncion Pari-Bedoya, I. N. M., & de Jesús Guevara-Soto, F. (2023). Challenges and opportunities
of AI-assisted learning: A systematic literature review on the impact of ChatGPT usage in higher education. International Journal of
Learning, Teaching and Educational Research, 22(7), 122–135. https://doi.org/10.26803/ijlter.22.7.7
V´azquez-Cano, E., Ramírez-Hurtado, J. M., S´aez-Lo´pez, J. M., & Lo´pez-Meneses, E. (2023). ChatGPT: The brightest student in the
class. Thinking Skills and Creativity,
49, Article 101380. https://doi.org/10.1016/j.tsc.2023.101380
Veenman, M. V. J., Van Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and
methodological considerations. Metacognition and Learning, 1, 3–14. https://doi.org/10.1007/s11409-006-6893-0
* Wahba, F., Ajlouni, A. O., & Abumosa, M. A. (2024). The impact of ChatGPT-based learning statistics on undergraduates’
statistical reasoning and attitudes toward statistics. Eurasia Journal of Mathematics, Science and Technology Education, 20(7), 1–14.
https://doi.org/10.29333/ejmste/14726.
Walter, Y. (2024). Embracing the future of artificial intelligence in the classroom: The relevance of AI literacy, prompt
engineering, and critical thinking in modern education. International Journal of Educational Technology in Higher Education, 21(1), 15.
https://doi.org/10.1186/s41239-024-00448-3
Waltzer, T., Pilegard, C., & Heyman, G. D. (2024). Can you spot the bot? Identifying AI-generated writing in college essays.
International Journal for Educational Integrity, 20(1), 11. https://doi.org/10.1007/s40979-024-00158-3
* Wang, X., & Feng, Y. (2023). An experimental study of ChatGPT-assisted improvement of Chinese college students’ English reading skills: A case study of dear life.
Barcelona,
Spain: The 15th International Conference on Education Technology and Computers.
* Wang, X., Zhong, Y., Huang, C., & Huang, X. (2024). ChatPRCS: A personalized support system for English reading
comprehension based on ChatGPT. IEEE Transactions on Learning Technologies, 17, 1762–1776.
https://doi.org/10.1109/tlt.2024.3405747.
* Wiboolyasarin, W., Wiboolyasarin, K., Suwanwihok, K., Jinowat, N., & Muenjanchoey, R. (2024). Synergizing collaborative
writing and AI feedback: An investigation into enhancing L2 writing proficiency in wiki-based environments. Computers &
Education: Artificial Intelligence, 6, Article 100228. https://doi.org/ 10.1016/j.caeai.2024.100228.
Wijaya, T. T., Su, M., Cao, Y., Weinhandl, R., & Houghton, T. (2024). Examining Chinese preservice mathematics teachers’
adoption of AI chatbots for learning: Unpacking perspectives through the UTAUT2 model. Education and information technologies.
Advance online publication. https://doi.org/10.1007/s10639-024- 12837-2
Williams, A. (2024). Comparison of generative AI performance on undergraduate and postgraduate written assessments in the
biomedical sciences. International Journal of Educational Technology in Higher Education, 21(1), 52. https://doi.org/10.1186/s41239-024-
00485-y
Wise, B., Emerson, L., Van Luyn, A., Dyson, B., Bjork, C., & Thomas, S. E. (2024). A scholarly dialogue: Writing scholarship,
authorship, academic integrity and the challenges of AI. Higher Education Research and Development, 43(3), 578–590.
https://doi.org/10.1080/07294360.2023.2280195
Wong, R. M., & Adesope, O. O. (2021). Meta-analysis of emotional designs in multimedia learning: A replication and extension
study. Educational Psychology Review, 33, 357–385. https://doi.org/10.1007/s10648-020-09545-x
Wong, L. H., Park, H., & Looi, C. K. (2024). From hype to insight: Exploring ChatGPT’s early footprint in education via
altmetrics and bibliometrics. Journal of Computer Assisted Learning, 40, 1428–1446. https://doi.org/10.1111/jcal.12962
Woo, D. J., Wang, D., Guo, K., & Susanto, H. (2024). Teaching EFL students to write with ChatGPT: Students’ motivation to
learn, cognitive load, and satisfaction with the learning process. Education and information technologies. Advance online
publication. https://doi.org/10.1007/s10639-024-12819-4
* Wu, C., Chen, L., Han, M., Li, Z., Yang, N., & Yu, C. (2024). Application of ChatGPT-based blended medical teaching in clinical
education of hepatobiliary surgery.
Medical Teacher, Advance online publication. https://doi.org/10.1080/0142159x.2024.2339412.
* Wu, T.-T., Lee, H.-Y., Li, P.-H., Huang, C.-N., & Huang, Y.-M. (2023). Promoting self-regulation progress and knowledge
construction in blended learning via ChatGPT-based learning aid. Journal of Educational Computing Research, 61(8), 3–31.
https://doi.org/10.1177/07356331231191125.
* Xiao, Q. (2024). ChatGPT as an artificial intelligence (AI) writing assistant for EFL learners: An exploratory study of its effects
on English writing proficiency. The 9th international conference on information and education innovations, verbania.
* Xue, Y., Chen, H., Bai, G. R., Tairas, R., & Huang, Y. (2024). Does chatgpt help with introductory programming? An experiment of students using
ChatGPT in CS1 the 46th international conference on software engineering. Lisbon: Software Engineering Education and Training.
Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information
Technologies, 28(11), 13943–13967. https://doi.org/10.1007/s10639-023-11742-4
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez-Maldonado, R., Chen, G., Li, X., Jin, Y., & Gaˇsevi´c, D. (2023). Practical and ethical
challenges of large language models in
education: A systematic scoping review. British Journal of Educational Technology, 55(1), 90–112. https://doi.org/10.1111/bjet.13370
Yang, L., & Li, R. (2024). ChatGPT for L2 learning: Current status and implications. System, 124, Article 103351.
https://doi.org/10.1016/j.system.2024.103351 Yeh, H.-C. (2024). The synergy of generative AI and inquiry-based learning:
Transforming the landscape of English teaching and learning. Interactive Learning
Environments, Advance online publication. https://doi.org/10.1080/10494820.2024.2335491
* Yilmaz, R., & Karaoglan Yilmaz, F. G. (2023). The effect of generative artificial intelligence (AI)-based tool use on
students’ computational thinking skills, programming self-efficacy and motivation. Computers & Education: Artificial
Intelligence, 4, Article 100147. https://doi.org/10.1016/j.caeai.2023.100147.
Yun, W. S., & Surianshah, S. (2024). A review of the scholarly works on ChatGPT use in education: Bibliometric analysis.
International Journal of Technology in Education, 7(3), 650–666. https://doi.org/10.46328/ijte.823
Yusuf, A., Pervin, N., & Roma´n-Gonza´lez, M. (2024). Generative AI and the future of higher education: A threat to academic
integrity or reformation? Evidence from multicultural perspectives. International Journal of Educational Technology in Higher Education,
21(1), 21. https://doi.org/10.1186/s41239-024-00453-6
Zhai, X., Nyaaba, M., & Ma, W. (2024). Can generative AI and ChatGPT outperform humans on cognitive-demanding problem-solving tasks in science? Science &
37
R. Deng et al. Computers & Education 227 (2025) 105224
education.
Advance online publication. https://doi.org/10.1007/s11191-024-00496-1
Zhai, C., & Wibowo, S. (2023). A systematic review on artificial intelligence dialogue systems for enhancing English as
foreign language students’ interactional competence in the university. Computers & Education: Artificial Intelligence, 4, Article
100134. https://doi.org/10.1016/j.caeai.2023.100134
Zhan, Y., Yan, Z., Wan, Z. H., Wang, X., Zeng, Y., Yang, M., & Yang, L. (2023). Effects of online peer assessment on higher-order
thinking: A meta-analysis. British Journal of Educational Technology, 54(4), 817–835. https://doi.org/10.1111/bjet.13310
* Zhang, J., Liu, Y., Cai, W., Wu, L., Peng, Y., Yu, J., Qi, S., Long, T., & Ge, B. (2024). Investigation of the effectiveness of applying ChatGPT in
dialogic teaching of electronic information using electroencephalography. Xi’an: The 6th International Conference on Computer Science and
Technologies in Education.
Zhang, P., & Tur, G. (2023). A systematic review of ChatGPT use in K-12 education. European Journal of Education, 59(2), Article
e12599. https://doi.org/10.1111/ ejed.12599

38
R. Deng et al. Computers & Education 227 (2025) 105224

Zhao, X., Cox, A., & Cai, L. (2024). ChatGPT and the digitisation of writing. Humanities and Social Sciences Communications, 11, 482.
https://doi.org/10.1057/s41599- 024-02904-x
* Zhou, W., & Kim, Y. (2024). Innovative music education: An empirical assessment of ChatGPT-4’s impact on student learning
experiences. Education and information technologies. Advance online publication. https://doi.org/10.1007/s10639-024-12705-z.
Zou, M., & Huang, L. (2023). The impact of ChatGPT on L2 writing and expected responses: Voice from doctoral students. Education
and Information Technologies, 29, 13201–13219. https://doi.org/10.1007/s10639-023-12397-x

39