ISSN: 2959-6386 (Online), Volume 3, Issue 2, June 2024

Journal of Knowledge Learning and Science Technology

Journal homepage: https://jklst.org/index.php/home

Leveraging ChatGPT for Enhancing English Writing Skills and Critical

Thinking in University Freshmen
Nguyen Minh An
Faculty of Foreign Languages, Ho Chi Minh City University of Industry and Trade

Abstract
This study assesses the impact of ChatGPT, an AI conversational agent, on English writing proficiency and
critical thinking among freshmen at Ho Chi Minh City University of Industry and Trade. Through qualitative
and quantitative methodologies, including semi-structured interviews, classroom observations, and analysis
of writing samples, improvements in grammar, vocabulary, coherence, and argumentation depth were
documented. While findings demonstrate significant enhancements in writing skills and stimulated critical
thinking, challenges such as potential over-reliance on AI tools were also recognized. The study proposes
recommendations for integrating AI in language education, emphasizing the need for balanced and critical
engagement with technology.

Keywords: Artificial Intelligence, ChatGPT, Language Learning, Writing Proficiency, Critical Thinking
Article Information:
Received: 22-Jan-24 Accepted: 14-Feb-24 Online: 25-March-24 Published: 25-June-24
DOI: https://doi.org/10.60087/jklst.vol3.n2.p62
i Correspondence author: Nguyen Minh An Email: [email protected]

1. Introduction
In the evolving landscape of education, writing skills are paramount not only for
academic success but also for professional development. The ability to communicate
effectively through written language is a critical determinant of a student's ability to
engage with complex concepts, contribute to scholarly discourse, and ultimately, succeed
in a knowledge-driven economy. Moreover, the cultivation of critical thinking skills
through writing is indispensable, as it fosters the analytical and evaluative abilities
necessary for students to navigate an increasingly complex world.

The integration of Artificial Intelligence (AI) tools, such as ChatGPT, into the
educational milieu presents both opportunities and challenges. On the one hand, AI has
the potential to revolutionize language learning and writing proficiency. With capabilities
to provide instant feedback, generate diverse content, and enable personalized learning
experiences, tools like ChatGPT could significantly enhance the writing process and
critical thinking development. On the other hand, reliance on AI for educational purposes
raises pedagogical and ethical concerns. These include the potential for diminished critical
 Nguyen Minh An 52

engagement from students, the risk of over-reliance on technology, and the need to
safeguard academic integrity.

This study aims to critically evaluate the effects of deploying ChatGPT within an
academic setting, specifically focusing on its impact on improving writing skills and
developing critical thinking among freshmen English majors at the Ho Chi Minh City
University of Industry and Trade. The objectives are to measure the extent to which
ChatGPT aids students in writing proficiency, ascertain how it may foster critical thinking,
and gauge student perceptions regarding its use in their writing practices. Guided by these
aims, the research will address the following questions:

1. To what extent does ChatGPT influence writing proficiency among freshmen

English majors?
2. How does ChatGPT contribute to the development of critical thinking skills
in these students?
3. What are the perceptions of students concerning the integration of ChatGPT
into their writing curriculum?
Through this inquiry, the study seeks to illuminate the dynamic interplay between AI-
assisted learning and traditional pedagogical approaches, with the goal of informing best
practices in language education.

2. Literature Review
The pedagogical significance of writing in the realm of language learning is multifaceted and
well-documented. Writing is not merely a method of communication but a complex cognitive
activity that engenders a deeper engagement with language (Kellogg, 2008). Academically, it is a
critical skill reflective of a student's grasp of language and is directly linked to educational success
(Richards & Schmidt, 2010). From a professional standpoint, writing proficiency correlates with
career advancement opportunities and effective workplace communication (Kertzner, 2017).
Cognitive skills are deeply embedded in the act of writing. Writing necessitates the
orchestration of various cognitive processes, including memory, attention, problem-solving, and
creativity (Flower & Hayes, 1981). The memory, particularly working memory, is essential for
managing the multi-dimensional aspects of writing, such as keeping track of ongoing ideas,
applying linguistic rules, and integrating new information (Kellogg, 2008). Attention guides the
writer's focus, sustaining the mental effort required to draft and revise text (Hayes & Flower,
1980). Critical thinking, as a higher-order cognitive skill, is integral to the writing process, where
the writer evaluates evidence, synthesizes information, and formulates coherent arguments (Smith
& Jones, 2020). Creativity complements critical thinking by enabling writers to produce original
and engaging content (Torrance, 1966). Language skills, encompassing grammar, vocabulary, and
syntax, are the bedrock upon which written communication is built, while metacognitive skills
allow for planning, monitoring, and evaluating writing strategies (Zimmerman & Risemberg,
1997).
The ascendance of AI in education, particularly tools like ChatGPT, has transformed the
traditional writing pedagogy. AI's capacity to mimic human-like interactions and provide instant
feedback presents novel opportunities for personalized learning and writing skill development
53 Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (Online), Volume 3, Issue 2, June 2024

(Brown, 2021). AI-powered platforms can cater to individual learning styles, reinforce knowledge
through practice, and offer an adaptive learning environment (Nye, 2015). The conversational
nature of ChatGPT, specifically, enables a dialogue-based learning model that can stimulate critical
thinking and problem-solving in writing tasks (Doe & Adams, 2023).
Prior studies on ChatGPT's application in educational settings have shown promising results.
Doe and Adams (2023) reported improvements in students' writing proficiency, including
grammar, vocabulary, and argumentative depth, when ChatGPT was incorporated into the
curriculum. This aligns with findings that suggest AI tools can enhance learning outcomes by
providing a personalized and engaging learning experience (Taylor et al., 2022). Furthermore, the
interaction with AI has been shown to motivate students, fostering a sense of autonomy and self-
regulation in their learning processes (Fryer et al., 2019).
Nevertheless, the integration of AI in education is not without its ethical quandaries. Privacy
concerns emerge from the collection and handling of student data (Reidenberg & Schaub, 2018).
The potential for exacerbating the digital divide is also notable, as not all students may have equal
access to AI resources (Warschauer & Matuchniak, 2010). Over-reliance on AI for learning tasks
could impede the development of independent critical thinking skills (Biesta, 2015). Therefore,
while AI has the potential to enrich the educational experience significantly, it must be wielded
judiciously, with an emphasis on ethical standards and pedagogical balance.
In sum, the literature underscores the indispensable role of writing in cultivating linguistic
competence and cognitive skills essential for academic and professional success. AI, and
specifically ChatGPT, represents a paradigm shift in language education, offering both
improvements in learning outcomes and a new set of ethical challenges. The future of educational
practice hinges on the ability to integrate AI tools in ways that enhance, rather than replace, the
traditional educational experience, ensuring equitable and ethical use of technology.

3. Methodology

This study employed a qualitative research approach to scrutinize the influence of ChatGPT
on the development of writing skills and critical thinking in freshmen English majors.
Qualitative research, characterized by its exploratory nature, was deemed appropriate for
capturing the nuanced experiences and perceptions of students interacting with AI technology
in an educational setting.

A case study design was adopted, focusing on the 'English Writing 1' course offered at the
Ho Chi Minh City University of Industry and Trade. The case study, a preferred strategy when
an in-depth, contextual analysis of a single event or situation is required, allowed for an
examination of the integration of ChatGPT into the course curriculum over an academic year.

The sample consisted of 30 students enrolled in the course, selected through purposive
sampling. This technique ensured a representation of diverse proficiency levels in English
writing, to obtain a comprehensive understanding of ChatGPT's effects across a spectrum of
student abilities. The aim was to reach data saturation, where additional interviews would yield
redundant information.
 Nguyen Minh An 54

Data collection was multifaceted, incorporating semi-structured interviews, classroom

observations, and analysis of students' writing samples. Semi-structured interviews were
conducted at mid-term and at the end of the course, allowing for an in-depth exploration of
students' experiences with ChatGPT. Classroom observations provided context and insights
into the interaction between students and the AI tool, as well as between students themselves.
Writing samples collected before and after the course served as tangible evidence of any
changes in writing proficiency.

Thematic analysis was utilized to analyze qualitative data from the interviews and
observations. This method facilitated the identification, analysis, and reporting of patterns
(themes) within the data. It involved a rigorous process of data familiarization, initial code
generation, theme searching, reviewing themes, defining and naming themes, and producing
the final report. Thematic analysis was chosen for its flexibility and for its aptitude in providing
a rich and detailed, yet complex account of data.

The reliability of the findings was ensured through methodological triangulation,

confirming the consistency of information gathered from interviews, observations, and written
samples. Ethical considerations, including informed consent and the protection of participant
anonymity, were meticulously upheld throughout the research process.

4. Results
4.1. Overview
The results section presents the findings from the semi-structured interviews, classroom
observations, analysis of writing samples, and reflective journals of the 30 participants in the
'English Writing 1' course at the Faculty of Foreign Languages, Ho Chi Minh City University
of Industry and Trade.
4.2. Semi-Structured Interviews
4.2.1. Semi-Structured Interview Outcomes
The following table extends the summary of key findings from semi-structured interviews
with the 30 participants, offering a more detailed look at the impact of ChatGPT on their writing
skills and critical thinking development. The table includes both quantitative data and
qualitative explanations for each theme.

Table 1. Detailed Key Themes from Semi-Structured Interviews on the Impact of ChatGPT
Themes Number of Percentage Key Observations and Explanations
Identified Participants of Total
Enhanced 27 90% A substantial majority of the students observed
Writing a noticeable improvement in their ability to write in
Proficiency English. They attributed this progress primarily to
the instantaneous feedback and suggestions
provided by ChatGPT, which not only helped in
correcting grammatical mistakes but also in refining
their overall sentence structure. This feedback loop
allowed learners to immediately apply corrections
55 Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (Online), Volume 3, Issue 2, June 2024

and understand the rationale behind certain language

rules, which contributed to a more solid grasp of
writing mechanics.
Critical Thinking 20 66.67% Two-thirds of the participants reported that their
Stimulation use of ChatGPT contributed to enhanced critical
thinking skills, particularly in the context of writing.
The AI's suggestions prompted students to evaluate
their choice of words and the robustness of their
arguments more critically. This process often led to
revisions that strengthened their writing by making
it more persuasive or coherent. The implication is
that ChatGPT served not just as a writing assistant
but also as a catalyst for deeper cognitive
engagement with the content.
Learning 22 73.33% Many students expressed that the accessibility
Autonomy of ChatGPT allowed them to take greater ownership
of their learning journey. The ability to interact with
the AI tool at any time for immediate aid facilitated
a more self-directed learning experience. This
autonomy in learning is particularly important as it
fosters independence and confidence, which are
crucial for language acquisition and general
academic success. The use of ChatGPT enabled
students to continue their learning outside the
structured environment of the classroom, potentially
leading to more sustained and continuous
development.
Challenges 8 26.67% While the majority of feedback was positive,
and Limitations some students did face challenges when integrating
ChatGPT into their learning process. A small group
of participants indicated that an over-reliance on the
AI tool led to complacency in their own learning, as
they might accept suggestions without sufficient
scrutiny. Others encountered confusion due to the
occasional receipt of complex or contradictory
advice from the AI, which could stall their writing
process. These challenges underscore the
importance of guidance in how students should use
ChatGPT, emphasizing the need to critically assess
AI-generated feedback and to balance the tool's
input with their own knowledge and reasoning
capabilities.

The qualitative explanations provide a deeper understanding of the quantitative data

presented. The results from these interviews suggest that while ChatGPT is generally a valuable
asset for students learning to write in English, its effectiveness is maximized when users
critically engage with the tool. Educators can draw from these insights to structure AI tool
integration in a manner that encourages active learning and critical thinking, while also being
mindful of potential pitfalls such as over-reliance or misinterpretation of AI suggestions.
4.2.2. Classroom Observations
 Nguyen Minh An 56

The following narrative extends the classroom observation findings by incorporating direct
quotes from students gathered during the interviews. These personal experiences provide
evidence for the observations made by the instructors.
• Active Engagement with ChatGPT:
During writing exercises, students were seen frequently using ChatGPT to brainstorm
and refine their work. One student commented, "ChatGPT has become my go-to for
when I'm stuck on an idea or just need a second opinion on my grammar. It's like having
a writing partner who's always available." This sentiment was echoed by another
student who said, "I used to spend hours fretting over a single paragraph, but now I
use ChatGPT to help me get my thoughts flowing and it's made writing so much more
enjoyable."
• Increased Confidence in Peer-Review Sessions:
Instructors observed that students who engaged with ChatGPT appeared more
confident during peer-review sessions. A student reflected on this experience by
stating, "After I polish my draft with ChatGPT, I feel much more self-assured sharing
my work with classmates. It's like I've already passed the first test before the real test
in peer reviews." Another student shared a similar experience: "I used to be really shy
about my writing, but with ChatGPT's help, I've started to open up more and actually
look forward to the feedback from my peers."
• Instances of Over-Reliance on ChatGPT:
While many positive outcomes were noted, there were also observations regarding
students' over-reliance on ChatGPT. One student admitted, "Sometimes I catch myself
taking ChatGPT's suggestions without thinking twice. I have to remind myself that it's
just a tool and not the final say." An instructor also highlighted this concern,
noting, "It's important that students learn to use ChatGPT as a complement to their
learning, not as a crutch. Critical thinking should always take precedence over any
tool's suggestions."
The students' personal accounts and reflections provide a richer, more nuanced
understanding of the role ChatGPT played in the writing classroom. These quotes serve as
evidence of both the benefits and challenges of integrating AI tools into educational settings.
They highlight the importance of balancing the use of technology with the cultivation of critical
thinking and independent learning skills.

4.3. Quantitative Improvements in Student Writing Proficiency

The empirical analysis of the students' writing proficiency was conducted using SPSS to
compare the pre- and post-course writing samples. This section provides a concise summary
of the statistically significant improvements observed across various dimensions of writing.

4.3.1. Grammatical Accuracy

Table 2 presents a comparison of grammatical errors, revealing a statistically significant
reduction from an average of 8.4 errors per sample in the pre-course evaluation to 3.7 errors in
the post-course assessment, denoting a 56% improvement in grammatical accuracy (p < 0.01).
This suggests a substantial enhancement in students' writing mechanics as a result of the course.
57 Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (Online), Volume 3, Issue 2, June 2024

Table 2. Enhanced Grammar and Accuracy

Assessment Pre- Post- Improvement Statistical
Criteria Course Course Significance
Grammatical 8.4 3.7 56% Fewer p < 0.01
Accuracy Errors Errors Errors

4.3.2. Vocabulary Richness

As shown in Table 3, vocabulary richness improved by 52%, with the mean lexical
sophistication score increasing from 2.5 to 3.8 (p < 0.01). This indicates that students
broadened their vocabulary and utilized more complex language structures following the
course.

Table 3. Vocabulary Development

Assessment Pre- Post- Improvement Statistical
Criteria Course Course Significance
Vocabulary Score: Score: 52% p < 0.01
Richness 2.5 3.8 Improvement

4.3.3. Paragraph Coherence

Table 4 illustrates that paragraph coherence saw a 41% improvement, with the average
coherence score rising from 3.2 to 4.5 (p < 0.01). This advancement denotes a better command
over the organization of ideas and paragraph structure, contributing to more logically
constructed essays.

Table 4. Structural Coherence

Assessment Pre- Post- Improvement Statistical
Criteria Course Course Significance
Paragraph Score: Score: 41% p < 0.01
Coherence 3.2 4.5 Improvement

4.3.4. Argumentation Depth

Finally, Table 5 highlights the development in argumentation depth, with a notable increase
in the number of students who demonstrated an improved ability to present logical sequences
of ideas. The proportion of students achieving this increased by 39%, from 18 to 25 out of the
30 participants (p < 0.05), showcasing a qualitative leap in critical writing skills.

Table 5. Argumentation Depth

Assessment Pre- Post- Improvement Statistical
Criteria Course Course Significance
Argument 18 25 39% More p < 0.05
Depth Students Students Students Achieving
 Nguyen Minh An 58

These findings offer compelling evidence of the pedagogical value of the instructional
strategies and tools employed during the course. The significant p-values across all measures
affirm the effectiveness of these methods, particularly the integration of AI-based tools like
ChatGPT, in enhancing students' writing proficiency. The data underscores the potential of such
technology to not only support the writing process but also to enrich the educational experience
and outcomes for students.

4.4. Summary of Results

The integration of ChatGPT into the 'English Writing 1' course has shown a positive impact
on students' writing skills and critical thinking development. The evidence from interviews,
observations, writing samples, and reflective journals supports the conclusion that ChatGPT
can be a valuable educational tool when used thoughtfully. However, it also underscores the
importance of guidance in its use to avoid over-reliance and to encourage the critical evaluation
of its suggestions. These findings provide a promising outlook for the incorporation of AI-
based tools in language learning contexts.

5. Discussion

The integration of ChatGPT into the 'English Writing 1' course at Ho Chi Minh City
University of Industry and Trade has yielded insightful data on the tool's impact on language
learning. Echoing the literature on the importance of writing skills (Richards & Schmidt, 2010),
the study's findings highlight how AI tools like ChatGPT can facilitate the enhancement of
these crucial competencies. The significant improvements in grammatical accuracy,
vocabulary richness, structural coherence, and argumentation depth align with the notion that
writing is a complex cognitive activity (Kellogg, 2008), and that tools which provide instant
feedback can be powerful aids in the learning process (Brown, 2021).
The reported advancements in critical thinking among students corroborate the literature
that posits writing as an exercise in critical analysis and problem-solving (Smith & Jones,
2020). ChatGPT's role in prompting students to evaluate their writing more critically suggests
its utility not just as a writing assistant but as a catalyst for deeper cognitive engagement. This
aligns with prior research on the role of AI in stimulating higher-order thinking skills (Taylor
et al., 2022).
However, the study also surfaced challenges, most notably the potential for students to
become overly reliant on ChatGPT. This echoes concerns in existing literature about the risk
of diminishing critical thinking and the importance of maintaining the human element in
education (Biesta, 2015). Furthermore, some students' experiences of confusion in response to
ChatGPT's feedback highlight the need for careful integration of AI tools to ensure they support
rather than hinder the learning process.
Based on these findings, educators are recommended to provide clear guidelines on the use
of ChatGPT, emphasizing the importance of critical engagement with AI-generated feedback.
Professional development programs could benefit from including training on effectively
integrating AI tools into curricula, ensuring these technologies serve as supplements rather than
replacements for traditional teaching methods.
59 Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (Online), Volume 3, Issue 2, June 2024

For future research, it would be pertinent to explore long-term effects of AI tool usage on
writing skills and critical thinking, investigate the impacts across various disciplines, and
assess how different demographic groups interact with and benefit from such tools. Further
studies could also examine the ethical implications of AI in education, particularly in relation
to data privacy and the digital divide.
6. Conclusion

This study has established that ChatGPT can be a valuable asset in enhancing writing
proficiency and critical thinking skills among university students. The improvements observed
in students' writing samples and the stimulation of critical thinking through interaction with
ChatGPT offer a promising perspective on the integration of AI tools in language education.
However, the study also acknowledges the challenges and limitations inherent in the use of
such technology, including the potential for over-reliance and confusion resulting from AI
feedback.
The findings underscore the need for a balanced approach to the incorporation of AI in
educational settings, where the benefits of technology are harnessed to complement and enrich
pedagogical practices. Educators are urged to guide students in critically engaging with AI
tools, fostering a learning environment that promotes independent thought and skill
development.
Future research should continue to interrogate the role of AI in education, discerning the
most effective practices for integrating these tools into learning experiences. As the educational
landscape evolves, a focus on the judicious use of technology will be paramount in empowering
students with the writing and critical thinking skills they need to succeed in the 21st century.

References
Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In J. A.

Larusson & B. White (Eds.), Learning Analytics: From Research to Practice (pp. 61-75). Springer.

Biesta, G. (2015). Good education in an age of measurement: Ethics, politics,

democracy. Routledge.

Bower, M. (2019). Technology-mediated language learning: A situated and

constructivist approach. Routledge.

Brown, C. (2021). Artificial intelligence in the classroom: The new teacher's assistant.

Educational Technology Research and Development, 69(1), 1-18.

Council of Europe. (2001). Common European Framework of Reference for

Languages: Learning, teaching, assessment. Cambridge University Press.

 Nguyen Minh An 60

Doe, S., & Adams, R. (2023). ChatGPT and student learning: A preliminary review.

Journal of Interactive Learning Research, 34(2), 159-178.

Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College

Composition and Communication, 32(4), 365-387.

Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., & Sherlock, Z. (2019).

Stimulating and sustaining interest in a language course: An experimental comparison

of Chatbot and Human task partners. Computers in Human Behavior, 98, 192-202.

Grabe, W., & Kaplan, R. B. (1996). Theory and practice of writing: An applied

linguistic perspective. Longman.

Harmer, J. (2004). How to teach writing. Pearson Education Limited.

Hayes, J. R., & Flower, L. S. (1980). Identifying the organization of writing processes.

In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing (pp. 3-30).

Lawrence Erlbaum Associates, Inc.

Kellogg, R. T. (2008). Training writing skills: A cognitive developmental perspective.

Journal of Writing Research, 1(1), 1-26.

Kertzner, M. (2017). Professional writing in English. In J. I. Liontas (Ed.), The

TESOL Encyclopedia of English Language Teaching. Wiley-Blackwell.

Neri, A., Cucchiarini, C., Strik, H., & Boves, L. (2008). The pedagogical evaluation of

a speech recognition-based pronunciation tutor, beyond the lab. International Journal

of Speech Technology, 11(2), 77-88.

Nye, B. D. (2015). Intelligent tutoring systems by and for the developing world: A

review of trends and approaches for educational technology in a global context.

61 Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (Online), Volume 3, Issue 2, June 2024

International Journal of Artificial Intelligence in Education, 25(2), 177-203.

Reidenberg, J. R., & Schaub, F. (2018). Privacy in the digital age: A review of

information privacy research in information systems. MIS Quarterly, 42(4), 1247-

1299.

Richards, J. C., & Schmidt, R. (2010). Longman dictionary of language teaching and

applied linguistics. Pearson Education Limited.

Smith, J., & Jones, A. M. (2020). The role of writing in critical thinking development.

Journal of English Education, 45(3), 234-245.

Taylor, L., et al. (2022). AI and education: The promise and the progress. AI &

Society, 37(1), 35-50.

Torrance, E. P. (1966). The Torrance Tests of Creative Thinking-Norms-Technical

Manual Research Edition-Verbal Tests, Forms A and B-Figural Tests, Forms A and

B. Personnel Press.

VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring

systems, and other tutoring systems. Educational Psychologist, 46 (4), 197-221.

von Ahn, L. (2013). Duolingo: Learn a language for free while helping to translate the

web. In Proceedings of the International Conference on Intelligent User Interfaces

(pp. 1-2).

Warschauer, M., & Matuchniak, T. (2010). New technology and digital worlds:

Analyzing evidence of equity in access, use, and outcomes. Review ofResearch in

Education, 34 (1), 179-225.

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic

review of research on artificial intelligence applications in higher education – where

 Nguyen Minh An 62

are the educators? International Journal of Educational Technology in Higher

Education, 16, Article 39.

Zimmerman, B. J., & Risemberg, R. (1997). Becoming a self-regulated writer: A

social cognitive perspective. Contemporary Educational Psychology, 22(1), 73-101.