dadasdadasd
dadasdadasd
55
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
Abstract
Despite widespread acknowledgement of the significance of critical thinking skills for success in today's
job market, higher education institutions face challenges in effectively nurturing these skills in students.
Educational policies, reports, and employer demands emphasise the importance of critical thinking, yet
a gap remains between its recognised value and the actual proficiency levels among university students.
This study employs a systematic literature review approach to address a research question, "What is the
impact of Generative AI on the critical thinking skills of undergraduates?" About half of the thirty
selected papers suggest that Generative AI benefits undergraduate critical thinking, but limitations in
study design prevent generalisation. Other inconclusive studies highlight the need for further research
to address research gaps.
1
Postgraduate student, Faculty of Graduate Studies, University of Kelaniya, Sri Lanka
Email: [email protected] https://orcid.org/0009-0005-1950-190X
2
Professor, Faculty of Commerce & Management Studies, University of Kelaniya, Sri Lanka
Email: [email protected] https://orcid.org/0000-0002-8057-8632
3
Executive Director CEO & Executive Dean, ICBT, Sri Lanka
Email: [email protected] https://orcid.org/0009-0009-9247-8426
The Journal of Desk Research Review and Analysis © 2024 by The Library,
University of Kelaniya, Sri Lanka, is licensed under CC BY-SA 4.0
199
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
Introduction
Critical thinking is vital in higher education, influencing students' academic, professional, personal, and
socio-political growth. Despite extensive research, there is no universally accepted definition, though it
is ecognised as a key academic skill that encourages students to question and reflect on knowledge
(Moore, 2011; Davies,2015). Research over the past four decades, particularly in Western contexts,
highlights the increasing demand for critical thinking skills to meet the needs of a complex,
technological society and the globalised economy (Moore, 2011). Philosophical perspectives emphasise
argumentation, while psychological approaches focus on cognitive skills. Educational programs have
been developed to cultivate logic, reasoning, and argumentation skills (Paul, 2011).
From a cognitive standpoint, critical thinking involves skills, attitudes, and knowledge, encompassing
abilities like evaluation, synthesis, analysis, interpretation, inference, and explanation (Sternberg &
Halpern, 2020). Critical thinkers excel in communication and teamwork, aiding in problem-solving.
Paul and Elder (2006) define critical thinking as the skill of evaluating and improving thought processes,
essential for professional success and active democratic participation.
Teaching critical thinking is vital in its contribution to a rational, civilised community, its value across
professions, and its role as a lifelong skill (Wass, 2012). Higher education aims to develop critical
thinking, a key concern for educators worldwide. Critical thinking enables graduates to participate
actively in society, tackle global issues, and foster democracy (Stanford Encyclopedia of Philosophy,
2018). However, teaching critical thinking techniques alone is insufficient; students must learn to apply
these skills in real-world situations. Instructors initially support learning through scaffolding but later
adopt a facilitator role as students grow more independent. Critical thinking is a learned skill that must
be taught, as it is not innate, emphasising the need for education to help students form independent
opinions and judgments (Fahim & Masouleh, 2012).
The Dearing Report (1997) and the Hunt Report (2011) emphasise the importance of undergraduates
acquiring skills such as "learning to learn," "critical analysis," and "independent thought," highlighting
the need for these skills to address future societal needs. Tomlinson (2010) notes that technological
advancements and new knowledge requirements have shifted employer expectations, making life skills
and critical thinking essential for graduates.
Employees who adapt to changing conditions and technological advancements are crucial in today's
dynamic workplace. Critical thinking skills enable graduates to think creatively, learn quickly, and solve
problems. According to the World Economic Forum (2023), "critical thinking and problem-solving"
are among the most essential skills for future employment.
Critical thinking and problem-solving are crucial skills in every job, as they involve analysing
situations, identifying key issues, evaluating data, and making informed decisions. According to a
NACE report (2021), these are the most sought-after skills among recent graduates. However, national
assessments in the United Kingdom (UK) and the United States (US) indicate that university students'
critical thinking abilities are not improving (Arum & Roksa, 2011; Huber & Kuncel, 2016). A survey
found that only 39% of US employers believe graduates are well-prepared in critical thinking,
highlighting a mismatch between workforce demands and educational outcomes (Y Fin Le Y. et al.,
2021).
Despite the importance of critical thinking skills for undergraduates, some barriers hinder their
development, as demonstrated by a study on South African undergraduates. (Eze. et al., 2022). The
200
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
main obstacles include lecturers' lack of proficiency, societal norms that discourage challenging
authority, students' unfamiliarity and reluctance, and an education system focused on memorisation.
Additional factors include confirmation biases, language proficiency, and an overreliance on lectures
and teachers. In summary, critical thinking in higher education is vital, but it faces significant challenges
due to various educational and societal factors (Utami et al.,2021).
Integrating GenAI in higher education presents opportunities and challenges for fostering critical
thinking. While AI tools can enhance problem-solving and creativity, they may also lead to a decline in
higher-order cognitive skills if over-relied upon (Bhosale, 2023). However, the impact of GenAI on
critical thinking remains a topic of debate. Critics argue that excessive dependence on GenAI may
reduce students' motivation to conduct independent research and analysis (Farrokhnia et al.,2023).
Therefore, it is crucial to integrate GenAI thoughtfully into educational systems to encourage inquiry,
discussion, and critical evaluation rather than passive learning. This balance is essential to ensure that
GenAI is a complement to, rather than a replacement for, traditional educational methods.
The impact of GenAI on undergraduate students' critical thinking remains underexplored, with
significant gaps in empirical research, theoretical frameworks, and practical applications (Wang, 2024).
Additionally, there is a need for more comprehensive studies on students’ and teachers’ perceptions of
GenAI across various demographics and educational contexts (Chan & Lee, 2023). With the
unprecedented interest in GenAI, it is essential to explore students' experiences and perceptions to
understand how these technologies can be integrated into higher education to enhance critical thinking
skills.
This literature review maps prior research on the influence generated by GenAI on undergraduate
students' critical thinking. The result of this Systematic Literature Review is to map out possibilities for
further research on the impact of GenAI on the development of critical thinking abilities among
undergraduates.
Methodology
Systematic Literature Review
Systematic literature reviews aim to examine a specific research question comprehensively. Unlike
standard literature reviews, which offer a general overview, systematic reviews are highly focused and
involve a more structured and transparent process. This process includes clearly defined protocols,
meticulous planning, and a detailed search strategy (Lame, 2019).
201
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
These reviews offer in-depth insights, making them exceptionally valuable to practitioners and
researchers despite being more time-consuming and detailed than standard literature reviews. The focus,
transparency, and comprehensive nature of systematic reviews ensure their reliability and relevance in
research.
Study Selection
Study selection is used to sort the references from various online publishing sources. Over a thousand
articles were initially identified, and after sorting, 30 references were identified as relevant to the
research. Two criteria were applied to the literature selection process to establish clear boundaries, as
outlined in Table 1.
202
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
Initially, a search was conducted using the keywords "Generative AI" and "Higher Education." This
search was subsequently refined by including "Critical Thinking" alongside the initial keywords.
Research papers were searched using Google Scholar, Connected Papers, Research Rabbit,
ResearchGate, and Semantic Scholar. Due to this filtering process, 68 journal articles were identified.
From these, 30 articles were selected for Review following additional filtration focused on research
questions. The selected papers were then critically examined to address the research questions.
203
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
view of the factors influencing critical thinking skills. Also, mixed-method approaches, combining both
quantitative and qualitative methods, were utilised in the studies of Ruiz-Rojas et al. (2024), Leiker et
al. (2023), and Hsiao-Ping Hsu (2023) to provide a holistic understanding of the impact of GenAI on
undergraduates' critical thinking skills, integrating diverse data sources for richer analysis and
interpretation. In addition, there are a number of studies such as Sallam (2023), George et al. (2023),
Lim et al. (2023), Kasneci et al. (2023), Farrokhnia et al. (2023), Thiga (2024), Rudolph, Tan, and Tan
(2023), Javaid et al. (2023), Dai et al. (2023), and Hutson (2024) employed literature review or desk-
based analysis. Also, a few unique approaches, such as Taiye et al. (2024) used Soft Systems
Methodology, and Tenakwaha et al. (2023) applied Competency Based Analysis.
Using its ability to produce human-like writing on a broad range of subjects, ChatGPT has become a
potent conversational AI agent (George et al., 2023). However, some worry that relying on it too much
can result in problems like plagiarism, a lack of critical thinking skills, and the dissemination of false
information (Sallam, 2023). To give students feedback on their work and encourage them to think more
carefully about the course material, Javaid et al. (2023) suggested employing ChatGPT as a teaching
assistant. They discovered that posing challenging queries and highlighting errors in students' logic
could assist them in developing their critical analysis abilities. ChatGPT was conceived by Dai et al.
(2023) as a student-driven invention with immense potential to empower students and improve their
educational resources and experiences. Due to its sophisticated conversational abilities, ChatGPT can
offer formative feedback on essays and transform it into a tutoring system, fostering critical thinking
and stimulating student debates (Farrokhnia et al.,2023). According to Michel-Villarreal et al. (2023),
educators can use simulations or virtual personas driven by ChatGPT, facilitating student involvement
in role-playing scenarios, decision-making activities, or historical recreations. These immersive
encounters can potentially boost student motivation, critical thinking, and creativity.
The study of Jia and Tu (2024) investigates AI's impact on critical thinking awareness in college
students, focusing on AI capabilities, general self-efficacy, and learning motivation. The objectives are
to explore how AI fosters critical thinking by enhancing self-efficacy and motivation, examine causal
relationships between these factors, understand AI's role in education, and provide theoretical and
practical insights. The methodology includes a literature review, hypothesis development, data
204
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
collection through surveys, questionnaire design and validation, and statistical analysis using structural
equation modelling. Key findings indicate that AI indirectly enhances critical thinking awareness by
boosting self-efficacy and motivation. General self-efficacy significantly influences learning
motivation and critical thinking, while learning motivation strongly correlates with critical thinking
skills. However, AI's direct impact on critical thinking is not significant.
The study conducted by Wang (2024) examines how native and non-native English-speaking college
students engage with ChatGPT in their writing processes, focusing on their perceptions, experiences,
and ethical dilemmas. Employing a phenomenological approach, the research reveals that students use
AI tools for convenience and enhancing their critical thinking skills. Students reported both cognitive
and emotional benefits from using ChatGPT, showing a critical engagement with AI suggestions and
reflecting on their writing practices. The study also underscores the challenges of maintaining an
authentic voice while incorporating AI, highlighting the importance of critical AI literacy (Wang,2024).
However, the research's limitations include a small sample size, a brief duration, a specific context,
varying proficiency levels, and the constraints of a phenomenological design, which may not fully
capture the diversity of student experiences.
GenAI can contribute to education by generating intricate scenarios for students to analyse and assess
using Bloom's taxonomy (JISC 2023). Additionally, engaging with GenAI text generators can prompt
students to devise innovative solutions, stimulating the highest cognitive skill level (creating) within
Bloom's taxonomy (Rudolph et al., 2023). According to the findings of Shanto et al. (2024), students,
on average, expressed that AI significantly assisted in idea generation and critical analysis compared to
working independently. This study evaluates students' responses using Lee's model (Lee, 2000, as cited
in Shanto et al.,2024) of thinking levels. However, the limitations of Shanto et al. (2024) study include
the small sample size, the controlled experimental setting, potential AI reliability issues, and the risk of
over-reliance on ChatGPT, which could hinder original thought and creativity. On the contrary, the
research on the impact of GenAI text generators on critical thinking skills in UK business schools
demonstrates that AI-based text generators enhance only the lower levels of Bloom's taxonomy (Essien
et al.,2024). Additionally, it tackles concerns regarding reliability, accuracy, and ethical implications
associated with the educational use of AI text generators.
The study by Lu et al. (2024) assessed the impact of GenAI-assisted training on 215 Chinese preservice
teachers' self-efficacy and higher-order thinking skills. One group received AI-assisted training using a
pretest-post-test design, while a control group used traditional methods. The AI group significantly
outperformed the control group in self-efficacy and higher-order thinking. Interviews with 25
participants highlighted positive perceptions, demonstrating AI's effectiveness in teacher development.
The study by Yilmaz (2023) investigates the impact of ChatGPT on students' computational thinking
skills, programming self-efficacy, and motivation. Conducted in a flipped classroom setting with hands-
on coding, the experimental group used ChatGPT for programming homework. Analysis of covariance
205
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
(ANCOVA) analysis compared the computational thinking skills of the experimental and control
groups. Key findings reveal that the experimental group demonstrated higher computational thinking
skills, self-efficacy, and motivation. ChatGPT enhanced students' creativity, algorithmic thinking,
cooperativity, and problem-solving abilities. Additionally, the experimental group had significantly
higher post-test scores in computer programming self-efficacy. Limitations of the study include the
short five-week duration, the small sample size of 45 students affecting generalizability, and the reliance
on Unified Modeling diagrams, limiting the variety of programming tasks.
The study of Berg and Plessis (2023) investigates the role of ChatGPT in teacher education, focusing
on lesson planning, critical thinking, and openness. It examines how GenAI tools like ChatGPT can
assist teachers by providing specific materials and support mechanisms, such as lesson plans, enhancing
teachers' critical thinking by encouraging diverse approaches. The qualitative methodology employed
an exploratory case study design and phenomenological research with document analysis, ensuring data
trustworthiness through credibility, transferability, dependability, and confirmability. Key findings
highlight that ChatGPT makes lesson plans accessible to all teachers, promoting equity across
geographical, social, and cultural backgrounds. However, the study emphasises the need for caution
and critically evaluating AI tools for limitations and biases (Berg & Plessis,2023).
Taiye et al. (2024) investigated using GenAI chatbots to enhance academic writing and critical thinking,
focusing on ethical challenges and stakeholder perspectives. It aims to develop CHAT4ISP-AI, a tool
to improve academic writing skills. Utilising Soft Systems Methodology (SSM), the research involved
iterative enhancements through stakeholder feedback, incorporating diverse perspectives from
developers, IT (Information Technology) support, teachers, and students. SSM employed structured
qualitative data and adaptive stakeholder engagement to refine the tool. Key findings reveal varied
expectations and concerns among stakeholders, with students engaging more in idea generation,
summarising sources, and checking grammar, while teachers have mixed reactions regarding academic
integrity (Taiye et al.,2024). The CHAT4ISP-AI tool shows promise but requires further development
and ethical considerations. Limitations include a focused scope on first-year Social Sciences students,
the initial proof-of-concept stage of the tool, divergent stakeholder objectives, and operational
challenges related to time and funding for further development.
Tenakwaha et al. (2023) assess ChatGPT's competence in answering university-level questions across
disciplines like Accounting, Social Work, Law, Management, and Education. It highlights ChatGPT's
language proficiency, reasoning ability, and response structuring but also identifies its limitations in
producing sophisticated, contextually relevant arguments. The study emphasises the importance of
human oversight in ensuring the accuracy and feasibility of AI-generated content. A multidisciplinary
team from five Australian universities evaluated ChatGPT's responses, noting that while clear and
readable, they often lacked depth and engagement. Despite its strengths, ChatGPT's responses were
sometimes basic, lacking in critical analysis and specific examples (Tenakwaha et al., 2023). The study
suggests using ChatGPT to stimulate deeper thinking rather than as a complete solution for academic
writing, underscoring the necessity of human intelligence in reviewing AI output. Limitations included
a lack of detailed analysis, correct references, and cohesive idea flow, necessitating user verification of
ChatGPT's information.
In the article "Rethinking Plagiarism in the Era of Generative AI," Hutson (2024) explores how GenAI
can enhance students' critical thinking skills in English composition courses. He argues that GenAI
tools can encourage originality by helping students generate unique ideas and content, fostering creative
thinking. Additionally, GenAI provides instant feedback on students' work, aiding in the refinement of
206
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
arguments and improving reasoning skills. The article also highlights how GenAI facilitates research
by assisting in the gathering and analysing information, which enables students to develop well-
informed perspectives.
The research findings from the study on incorporating GenAI tools into introductory programming
courses (Styve et al., 2024) indicate an improvement in students' critical thinking practices. After
engaging with AI-based programming assignments, students demonstrated a more remarkable ability
to critically evaluate AI-generated solutions, reflecting on their appropriateness and reliability.
Zaphir et al. (2024) introduce the MAGE framework (Mapping, AI vulnerability testing, Grading,
Evaluation) to evaluate the quality of critical thinking in GenAI like ChatGPT4. The framework
assesses the vulnerability of assessment tasks to AI, emphasising the importance of critical thinking
skills that AI can mimic but not fully perform. The methodology involves mapping cognitive skills,
testing AI vulnerability with different question versions, grading AI responses, and evaluating results.
A case study demonstrates the framework's application, showing how various assessment tasks can be
evaluated for AI vulnerability. Key findings of Zaphir et al.’s (2024) paper highlight the limitations of
ChatGPT4 in emulating critical thinking, suggesting that educators redesign assessments to focus on
more profound comprehension and analytical skills. The study emphasises the importance of authentic
learning to deter AI-driven shortcuts.
Integrating AI text generators into education poses distinct challenges, underscoring the intricacy of
incorporating them into existing teaching methodologies (Lancaster, 2023). Another challenge pertains
to the accuracy of AI-generated content, with nonsensical responses being common, indicating
occasional failures in grasping context accurately and raising concerns about its precision. Ensuring the
reliability of AI tools necessitates robust fact-checking procedures and continuous review mechanisms
(Leiker et al., 2023).
The paper by Ruiz-Rojas et al. (2024) explores the impact of GenAI tools on critical thinking and
collaboration among university students, aiming to identify suitable tools, evaluate their effects, and
develop effective pedagogical strategies. Using a mixed methods approach, including quantitative
surveys and qualitative thematic analysis, the study assessed 121 students selected through purposive
sampling. Key findings show high familiarity with AI tools, with 87% of respondents aware of them
and 38% using them occasionally. Additionally, 64% believe these tools significantly enhance critical
thinking. Despite this, there is a need for continuous training and technical support. Limitations include
207
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
limited sample size, a focus on quantitative data, which may not capture the full depth of experiences,
a lack of longitudinal data to assess long-term impacts, and ethical concerns related to AI biases and
data privacy.
Hsiao-Ping Hsu (2023) generated a short research paper with the support of ChatGPT 4.0 using prompt
engineering techniques, and the study found that GenAI can assist in research and education by
generating ideas and designs. However, it is crucial for scholars to evaluate its outputs to uphold
academic integrity critically. While GenAI offers efficiency and accessibility, it challenges traditional
knowledge, scholarship, and teaching concepts. More emphasis should be placed on developing critical
thinking skills to responsibly use GenAI, ensuring it complements rather than detracts from the essence
of education and scholarly work.
Existing literature highlights the potential integration of GenAI into education while also underscoring
the risks associated with excessive reliance on AI. By understanding the constraints of AI models such
as ChatGPT and designing questions, educators can encourage active participation in the learning
journey, fostering the growth of critical thinking and problem-solving abilities among students
(Elsayed,2023).
Nevertheless, it is essential to recognise that merging human intellectual capacity with the potential of
GenAI transcends the limits of human capability, accelerating the generation of innovative ideas and
fostering a more meaningful and effective educational process (Fiialka. et al.,2023).
Jia et al. Structured AI indirectly General self-efficacy Genetic factors were not
(2024). equation enhances critical significantly influences both tested.
modelling thinking learning motivation and The proposed path
awareness critical thinking, while model was only
learning motivation strongly marginally accepted.
correlates with critical The reinforcement
thinking skills. However, AI's learning theory was not
direct impact on critical utilised fully.
thinking is not significant.
208
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
209
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
Taiye et Soft System Varied Students engaging more in Limited scope. only the
al. Methodolog expectations and idea generation, summarising first-year social science
(2024). y concerns among sources, and checking students were
stakeholders grammar, while teachers have considered
mixed reactions regarding
academic integrity
Tenakwa Competence ChatGPT's Instead of viewing ChatGPT Students' perspectives
h et al. -based responses were as a solution to all academic on the motivation of
(2023) analysis of sometimes writing challenges, it should ChatGPT were not
ChatGPT's primary, lacking be seen as a tool to generate considered.
task in critical analysis ideas that spark further, more The longitudinal
response and specific in-depth thinking. approach was not
examples Human intelligence must considered.
review the output for Limited disciplines
accuracy, consistency, and considered
practicality for research-
oriented tasks.
Hutson Desk GenAI tools can GenAI provides instant The methodology was
(2024) Research encourage feedback on students' work, not strong enough.
originality by aiding in the refinement of
helping students arguments and improving
generate unique reasoning skills.
ideas and content, GenAI facilitates research by
fostering creative assisting in gathering and
thinking. analysing information,
enabling students to develop
well-informed perspectives.
Ruiz- Mixed GenAI tools Incorporating didactic and Limited sample size.
Rojas et method significantly pedagogical approaches and Lack of longitudinal
al. (2024) impact the critical generative AI tools has data.
thinking of higher enhanced the depth and
education reflectiveness of critical
students. thinking.
Styve et Survey- GenAI improves Students demonstrated a Higher-order thinking
al. based, Pre students' critical greater ability to critically was not evaluated.
(2024). and Post-test thinking practices. evaluate AI-generated The introductory course
solutions, reflecting on their will not represent all
appropriateness and levels of coding.
reliability. The results are based on
students’ self-reported
opinions and practices.
Long-term effects and
changes in critical
thinking practices were
not considered. The
focus was on specific AI
tools only.
210
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
Additionally, concerns about over-reliance on these tools have been raised, which may inhibit creativity
and original thought (Taiye et al.,2024). While students often see GenAI as helpful for personalised
learning and writing assistance, they also voice concerns about its accuracy and ethical implications,
which could affect the development of their critical thinking skills (Chan & Hu, 2023). Thus, while
GenAI has the potential to support critical thinking, its effective use requires thoughtful implementation
and continuous evaluation to mitigate potential risks (Berg & Plessis,2023; Taiye et al.,2024).
211
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
The selected studies also emphasise AI literacy and prompt engineering, which are key to developing
critical thinking skills in education. Teaching students and teachers techniques like zero-shot and few-
shot prompting encourages greater engagement with AI, promoting critical analysis and creative
problem-solving (Walter, 2024). Collaborative exercises using prompt engineering challenge students
and enhance their understanding of AI interactions, enriching their learning experience. This approach
also increases awareness of AI's limitations and capabilities, which are essential for informed decision-
making and reflective thinking. Integrating AI literacy and prompt engineering creates a more
interactive learning environment that fosters critical thinking (Jia & Tu, 2024).
According to the findings of this Review, further studies are needed to address the literature, theoretical,
practical, and contextual gaps to provide a more comprehensive understanding of the topic, particularly
considering the significance of critical thinking skills in the ever-changing higher educational
landscape.
Recommendations
Educators should prioritise AI literacy and prompt engineering to effectively leverage GenAI in
fostering critical thinking within academic activities. It is crucial to evaluate assessment tasks for
potential AI vulnerabilities and to redesign them to emphasise the testing of critical and analytical skills.
Additionally, ensuring the reliability of AI tools and providing clear guidelines on acceptable GenAI
tools are essential. Institutions should also develop robust policies to govern the responsible use of
GenAI, thereby upholding academic integrity.
Future Research
There is a pressing need for additional research on higher education institutions to assess their methods
of teaching and fostering critical thinking skills, particularly in relation to students' use of GenAI-based
tools. Future studies should examine the application of GenAI across different contexts to gain a deeper
understanding of its use and the impact of contextual factors on critical thinking skills. Moreover,
studies should focus more on how GenAI can improve higher-order thinking skills.
Prior studies have rarely utilised taxonomies beyond Bloom's to evaluate the critical thinking skills of
students using GenAI. Furthermore, research should broaden to include more disciplines, allowing for
comparisons across fields to identify factors influencing its use. Additional research is needed to explore
the link between prompt engineering and critical thinking skills among higher education students. A
longitudinal approach could help track applications like ChatGPT and identify long-term users.
212
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
Acknowledgement
This work was conducted without specific financial support from public, commercial, or not-for-profit
funding agencies.
Reference
Arum, R., & Roksa, J. (2011). Academically adrift: Limited learning on college campuses. University
of Chicago Press.
Barana, A., Marchisio, M., & Roman, F. (2023). Fostering problem-solving and critical thinking in
mathematics through generative artificial intelligence.
Berg, G., & Plessis, E. (2023). ChatGPT and Generative AI: Possibilities for Its Contribution to Lesson
Planning, Critical Thinking and Openness in Teacher Education. Education Sciences.
Bhosale, U. (2023). Responsible Use of Generative AI. Enago Academy.
https://www.enago.com/academy/responsibleuse-of-generative-ai/
Chan, C. K. Y., & Hu, W. (2023). Students’ Voices on Generative AI: Perceptions, Benefits, and
Challenges in Higher Education. https://doi.org/10.1186/s41239-023-00411-8
Chan, C. K. Y., & Lee, K. K. W. (2023). The AI generation gap: Are Gen Z students more interested in
adopting generative AI such as ChatGPT in teaching and learning than their Gen X and
millennial generation teachers? Smart Learning Environments, 10(1), 60.
https://doi.org/10.1186/s40561-023-00269-3
Davies, M. (2015). A model of critical thinking in higher education. In M.B. Paulsen (ed.), Higher
Education: Handbook of Theory and Research 30. DOI 10.1007/978-3-319-12835-1_2
Dai, Y., Liu, A., & Lim, C. P. (2023). Reconceptualising ChatGPT and generative AI as a student-
driven innovation in higher education. Procedia CIRP, 119, 84–90.
https://doi.org/10.1016/j.procir.2023.05.002 .
Dearing, R. (1997). Higher Education in the Learning Society. The National Committee of Enquiry into
Higher Education.
http://www.educationengland.org.uk/documents/dearing1997/dearing1997.html
Elsayed, S.M. (2023). Towards Mitigating ChatGPT's Negative Impact on Education: Optimising
Question Design Through Bloom's Taxonomy. 2023 IEEE Region 10 Symposium
(TENSYMP), 1-6.
Essien, A., Bukoye, O. T., O’Dea, C., & Kremantzis, M. (2024). The influence of AI text generators on
critical thinking skills in UK business schools. Studies in Higher Education.
https://doi.org/10.1080/03075079.2024.2316881
Essien, A., G. Chukwukelu, and V. Essien. (2021). Opportunities and Challenges of Adopting Artificial
Intelligence for Learning and Teaching in Higher Education. In: Fostering Communication and
Learning with Underutilized Technologies in Higher Education. IGI Global, pp. 67–78.
Eze, I.F., Iwu, C.G., & Dubihlela, J. (2022). Students’ views regarding the barriers to learning critical
thinking. International Journal of Research in Business and Social Science (2147- 4478).
Fahim, M., & Masouleh, N.S. (2012). Critical thinking in higher education: A pedagogical look. Theory
and Practice in Language Studies, 2(7), 1370-1375. doi: 10.4304/tpls.2.7.1370-1375
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT:
Implications for educational practice and research. Innovations in Education and Teaching
International, 0(0), 1–15. https://doi.org/10.1080/14703297.2023.2195846
Fiialka, S., Kornieva, Z., & Honcharuk, T. (2023). ChatGPT in Ukrainian Education: Problems and
Prospects. Int. J. Emerg. Technol. Learn., 18, 236-250.
George, A. S. H. George, and A. S. G. Martin, (2023). “A Review of ChatGPT AI’s Impact on Several
Business Sectors,” Partners Universal International Innovation Journal (PUIIJ), vol. 01, no. 01,
pp. 9–23, doi: https://doi.org/10.5281/zenodo.7644359.
Hsiao-Ping Hsu. (2023). Can Generative Artificial Intelligence Write an Academic Journal Article?
Opportunities, Challenges, and Implications. Irish Journal of Technology Enhanced Learning.
https://doi.org/10.22554/ijtel.v7i2.152
Huber, C. R., & Kuncel, N. R. (2016). Does College Teach Critical Thinking? A Meta-Analysis. Review
of Educational Research, 86(2), 431–468. https://doi.org/10.3102/0034654315605917
213
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
Hunt, C. (2011). National Strategy for Higher Education to 2030: Report of the Strategy Group.
Department of Education and Skills, Government Publications Office.
http://hea.ie/assets/uploads/2017/06/National-Strategy-for-Higher-Education-2030.pdf
Hutson, J. (2024). Rethinking Plagiarism in the Era of Generative AI. Journal of Intelligent
Communication, 4(1), 20–31. https://doi.org/10.54963/jic.v4i1.220
Javaid, M., Haleem, A., Singh, R. P., Khan, S., & Khan, I. H. (2023). Unlocking the opportunities
through ChatGPT Tool towards ameliorating the education system. Bench Council
Transactions on Benchmarks, Standards and Evaluations, 3(2),
doi:https://doi.org/10.3390/healthcare11060887.
Jia, X.-H., & Tu, J.-C. (2024). Towards a New Conceptual Model of AI-Enhanced Learning for College
Students: The Roles of Artificial Intelligence Capabilities, General Self-Efficacy, Learning
Motivation, and Critical Thinking Awareness. Systems, 12(3), 74.
https://doi.org/10.3390/systems12030074
JISC. (2023). Does ChatGPT mean the End of the Essay as an Assessment Tool? | Jisc [WWW
Document]. Accessed April 21, 2024. URL https://www.jisc.ac.uk/news/does-chatgpt-mean-
the-end-of-the-essay-as-an-assessment-tool-10-jan-2023.
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh,
G., Günnemann, S., Hüllermeier, E., et al. (2023). ChatGPT for good? On opportunities and
challenges of large language models for education. Learning and Individual Differences, 103,
102274.
Lancaster, T. (2023). “Artificial Intelligence, Text Generation Tools and ChatGPT–Does Digital
Watermarking Offer a Solution?” International Journal for Educational Integrity 19:10.
https://doi.org/10.1007/s40979-023-00131-6.
Lame, G. (2019). Systematic Literature Reviews: An Introduction. Proceedings of the Design Society:
International Conference on Engineering Design, 1(1), 1633–1642.
https://doi.org/10.1017/dsi.2019.169
Leiker, D., A. R. Gyllen, I. Eldesouky, and M. Cukurova. (2023). Generative AI for Learning:
Investigating the Potential of Synthetic Learning Videos. arXiv Prepr. ArXiv2304.03784.
Lim, W. M., A. Gunasekara, J. L. Pallant, J. I. Pallant, and E. Pechenkina. (2023). “Generative AI and
the Future of Education: Ragnarök or Reformation? A Paradoxical Perspective from
Management Educators.” The International Journal of Management Education 21 (2): 100790.
https://doi.org/10.1016/j.ijme.2023.100790.
Lu, J., Zheng, R., Gong, Z., & Xu, H. (2024). Supporting Teachers' Professional Development with
Generative AI: The Effects on Higher Order Thinking and Self-Efficacy. IEEE Transactions on
Learning Technologies, p. 17, 1279–1289.
Michel-Villarreal, R., Vilalta-Perdomo, E.L., Salinas-Navarro, D.E., Thierry-Aguilera, R., & Gerardou,
F.S. (2023). Challenges and Opportunities of Generative AI for Higher Education as Explained
by ChatGPT. Education Sciences.
Moore, T. J. (2011). Critical thinking and language: The challenge of generic skills and disciplinary
discourse. London: Bloomsbury Academic. 978144111359-web more book.pdf (secured)
NACE Center, “What is career readiness?” NACE Center for Career Development and Talent
Acquisition, 2021. https://www.naceweb.org/career-readiness/competencies/careerreadiness-
defined/ (accessed April. 23, 2024).
Paul, R. (2011). "Reflections on the Nature of Critical Thinking, Its History, Politics, and Barriers, and
Its Status Across the College/University Curriculum Part I." INQUIRY: Critical Thinking
Across the Disciplines 26 (3), 5–24.
Paul, R., & Elder, L. (2006). The miniature guide to critical thinking: Concepts and tools. Retrieved from:
http://criticalthinking.org/files/Concepts_Tools.pdf
QAA. (2023). Maintaining quality and standards in the ChatGPT era: QAA advice on the opportunities and
challenges of Generative Artificial Intelligence [WWW Document]. Accessed April 21, 2024. URL
https://www. qaa.ac.uk/docs/qaa/members/maintaining-quality-and-standards-in-the-chatgpt-
era.pdf.
Rudolph, J., Samson Tan, & Shannon Tan. (2023). "ChatGPT: Bullshit Spewer or the End of Traditional
Assessments in Higher Education?" Journal of Applied Learning and Teaching 6 (1): 342–363.
https://doi.org/10.37074/jalt.2023.6.1.9.
214
The Journal of Desk Research Review and Analysis, Vol. 2, Issue 1, 2024, 199-215
Ruiz-Rojas, L. I., Salvador-Ullauri, L., & Acosta-Vargas, P. (2024). Collaborative Working and Critical
Thinking: Adoption of Generative Artificial Intelligence Tools in Higher Education. Sustainability,
16(5367). https://doi.org/10.3390/ su16135367
Sadat Shanto, S., & Ahmed, Z. (2024). Enriching the Learning Process with Generative AI: A Proposed
Framework to Cultivate Critical Thinking in Higher Education using ChatGPT. Article in Tuijin
Jishu/Journal of Propulsion Technology, 45(1), 1001–4055.
https://doi.org/10.52783/tjjpt.v45.i01.4680
Sallam, M. (2023). “ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review
on the Promising Perspectives and Valid Concerns,” Healthcare, vol. 11(6). p. 887.
doi:https://doi.org/10.3390/healthcare11060887.
Smolansky, A., Cram, A., Raduescu, C., Zeivots, S., Huber, E., & Kizilcec, R.F. (2023). Educator and
Student Perspectives on the Impact of Generative AI on Assessments in Higher Education.
Proceedings of the Tenth ACM Conference on Learning @ Scale.
Stanford Encyclopedia of Philosophy. (2018). Critical thinking. Retrieved from
https://plato.stanford.edu/entries/critical-thinking/#DeweThreMainExam
Sternberg, R. J., & Halpern, D. F. (Eds.). (2020). Critical thinking in psychology (2nd ed.). Cambridge
University Press. https://doi.org/10.1017/9781108684354
Styve, A., Virkki, O. T., & Naeem, U. (2024). Developing critical thinking practices interwoven with
generative AI usage in an introductory programming course. In 2024 IEEE Global Engineering
Education Conference (EDUCON) (pp. 01-08). Kos Island, Greece.
https://doi.org/10.1109/EDUCON60312.2024.10578746
Tat Putjorn & Pruet Putiorn. (2023). Augmented Imagination: Exploring Generative AI from the
Perspectives of Young Learners. International Conferences on Information Technologies and
Electrical Engineering. https://doi.org/10.1109/icitee59582.2023.10317680
Tenakwah, E. S., Boadu, G., & Tenakwah, E. J. (2023). Generative AI and Higher Education Assessments:
A Competency-Based Analysis. https://orcid.org/0000-0001-6338-218X
Thiga, M. M. (2024). Generative AI and the Development of Critical Thinking Skills. IRE Journals, 7(9),
83–90.
Tomlinson, M. (2010). Investing in the self: Structure, agency, and identity in graduates' employability.
Education, Knowledge, and Economy, 4(2), 73–88. https://doi.org/10.1080/17496896.2010.499273
Utami, R.T., Saleh, M., Warsono, & Hartono, R. (2021). Factors Affecting Students’ Critical Thinking
Development in EFL Classroom. Proceedings of the 1st Paris Van Java International Seminar on
Health, Economics, Social Science and Humanities (PVJ-ISHESSH 2020).
Walter, Y. (2024). Embracing the future of Artificial Intelligence in the classroom: The relevance of AI
literacy, prompt engineering, and critical thinking in modern education. International Journal of
Educational Technology in Higher Education, 21(1). https://doi.org/10.1186/s41239-024-00448-3
Wang, C. (2024). Exploring Students' Generative AI-Assisted Writing Processes: Perceptions and
Experiences from Native and Non-native English Speakers. Technology, Knowledge, and Learning.
https://doi.org/10.1007/s10758-024-09744-3
Wass, R. T. (2012). Developing critical thinkers in higher education: A Vygotskian perspective. (Doctoral
Thesis), University of Otago, Dunedin, New Zealand. Retrieved from
https://ourarchive.otago.ac.nz/bitstream/handle/10523/2491/WassRobertT2012PhD.pdf?sequenc
e=1&isAllowed=y.
World Economic Forum. (2023). The-future-of-jobs-report-2023. www.weforum.org
Y Fin Le Y, A. le, & Finley, A. (2021). How College Contributes to Workforce Success EMPLOYER
VIEWS ON WHAT MATTERS MOST How College Contributes to Workforce Success
EMPLOYER VIEWS ON WHAT MATTERS MOST. Association of American Colleges and
Universities
Yilmaz, R., & Yilmaz, F. G. K. (2023). The effect of generative artificial intelligence (AI)-based tool use on
students’ computational thinking skills, programming self-efficacy and motivation. Computers and
Education: Artificial Intelligence, 4, 100147. https://doi.org/10.1016/j.caeai.2023.100147
Zaphir, L., Lodge, J. M., Lisec, J., Mcgrath, D., & Khosravi, H. (2024). How critically can an AI think? A
framework for evaluating the quality of thinking of generative artificial intelligence.
https://doi.org/10.48550/arXiv.2406.14769
215