Artificial Intelligence (AI) Literacy and Academic Performance of Tertiary Level Students: A Preliminary Analysis
Artificial Intelligence (AI) Literacy and Academic Performance of Tertiary Level Students: A Preliminary Analysis
Abstract: This study intends to analyze the level of AI literacy among college students and its
relationship to their academic performance. This investigation used a cross-sectional
research design to address the research objective of the study. Eight hundred sixty-nine
(869) college students served as participants in the investigation using an adapted
instrument to measure AI literacy. Data from the respondents underwent statistical analysis,
such as frequency, percentage, mean, independent t-test, Analysis of Variance, and Pearson-r
Moment of Correlation. Results show that college students have a moderately high AI literacy
level, and their academic performance was also highly satisfactory. The study also found
significant differences in AI literacy in terms of college and gender and academic
performance in terms of college, year level, and age. As for the relationship, there was a weak
positive relationship between AI literacy and the academic performance of college students.
Citation: Asio, J.M.R. (2024). Artificial Intelligence (AI) Literacy and Academic Performance
of Tertiary Level Students: A Preliminary Analysis. Social Sciences, Humanities and Education
Journal (SHE Journal), 5(2), 309 – 321.
Copyright ©2020 Social Sciences, Humanities and Education Journal (SHE Journal)
Published by Universitas PGRI Madiun. This work is licensed under the Creative Commons Attribution-
NonCommercial-ShareAlike 4.0 International License.
309
Asio SHE Journal
310
Asio SHE Journal
311
Asio SHE Journal
the participants' academic performance came from the CCS (f=324) than from
should be analyzed. The result of this CEAS (f=289) and CAHS (f=256). In
study has several implications, terms of year level, there were more
especially for students, faculty, the first-year level respondents (f=352) as
school administration, and the compared to second-year (f=260), third-
institution itself. At the same time, it can year (f=139), and fourth-year (f=118)
also be part of the growing local levels. As for the age of the respondents,
literature for AI-related studies for those less than 20 dominated the rest of
future researchers. the groups, with a frequency of 541. This
result is followed by the age bracket 21-
METHODS 25 years old with a frequency of 292,
Research Design then by the age bracket 26-30 years old
The investigator used a and 31 years old and above with a
descriptive–correlation study with an frequency of 18 apiece. As for gender,
online survey via Google form as the there were more females (f=472) than
primary data-gathering tool in this males (f=380) and those who preferred
study. The study aims to explore the not to say (f=17).
relationship between students' AI
literacy and academic performance and TABLE 1. Demographic characteristics
analyze variance in the level of AI Characteristics f %
literacy among the students. Therefore, College
the said research design was applicable CAHS 256 29.5
to current endeavors. CCS 324 37.3
CEAS 289 33.3
Year Level
Respondents
First Year 352 40.5
This study's population Second Year 260 29.9
consisted of college students from a Third Year 139 16.0
local tertiary education institution Fourth Year 118 13.6
located in Olongapo City, Philippines. Age
Eight hundred sixty-nine (869) students < 20 years old 541 62.3
voluntarily participated in the online 21-25 years old 292 33.6
survey spearheaded by the investigator 26-30 years old 18 2.1
from August to September 2023, during > 31 years old 18 2.1
the first semester of the academic year Gender
Female 472 54.3
2023-2024.
Male 380 43.7
The study's investigator used Prefer not to say 17 2.0
purposive sampling to gather enough Total 869 100.0
respondents. Also, to be part of the
survey, the respondents must possess Instrumentation
the following criteria: 1) bona fide To obtain the necessary data for
student of the participating institution, the study, the investigator adapted an
2) currently enrolled within the instrument by Carolus et al. (2023) from
semester of the academic year, 3) has an their paper MAILS-Meta AI literacy
internet connection and gadget, and 4) scale: Development and testing of an AI
willing to participate in the online literacy questionnaire based on well-
survey. Table 1 displays the summary of founded competency models and
the descriptive characteristics of the psychological change and meta-
participants for the study. competencies. In the adaptation, the
Table 1 depicts the frequency investigator considered the AI literacy
and percentage distribution of the aspect only, wherein three (3) sub-
students' demographic characteristics. variables focused on the use and
In terms of college, more respondents
312
Asio SHE Journal
313
Asio SHE Journal
314
Asio SHE Journal
the differences in AI literacy and the “know and understand AI,” the
students' academic performance when computation yield F93, 865)= 1.847, p=
grouped according to year level. The .137. Regarding “AI ethics,” it generated
table shows that only the “use and apply F(3, 865)= 0.388, p= .762. Lastly, the “AI
AI” garnered a significant finding based literacy” generated F(3, 865)= 1.315, p=
on the computation. The study obtained .268. All the probability values
F(3, 865)= 3.000, p= .030, wherein the generated during the computation were
probability value was significant at the insignificant at a .05 Alpha significance
.05 alpha significance level. As for the level. Thus, it is safe to assume that
remaining sub-variables of the A there was no substantial evidence to
literacy, the following values were F(3, prove variations in the students'
865)= 1.392, p= .244 for the “know and perceptions.
understand AI,” F(3, 865)= 1.040 for the However, for the variable
“AI ethics,” and F(3, 865)= 2.077, p= "academic performance," the study
.102 for the overall “AI literacy” variable. found a significant difference between
For the variable "academic age groups since F(3, 865)= 2.899, p=
performance," the obtained a significant .034. These findings imply that while
difference between groups since F(3, there are no significant differences in AI
865)= 12.283, p= .000. These findings literacy and specific aspects of AI
suggest that while AI literacy and knowledge and understanding across
specific aspects of AI knowledge and different age groups, there is a
understanding do not vary significantly significant difference in academic
across different year levels, there are performance.
significant differences in academic
performance. TABLE 7. Differences in the AI literacy and
academic performance of the students when
TABLE 6. Differences in the AI literacy and grouped according to gender
academic performance of the Students when Variables F Sig.
grouped according to age Use and Apply AI 10.89* .000
Variables F Sig. Know and Understand AI 5.91* .003
Use and Apply AI 2.02 .109 AI Ethics 5.07* .003
Know and Understand AI 1.85 .137 AI Literacy 8.79* .000
AI Ethics 0.39 .762 Academic Performance 0.41 .662
AI Literacy 1.32 .268 Note: *p < .05; df1=2; df2=866
Academic Performance 2.90* .034
Note: *p < .05; df1=3; df2=865 Table 7 represents the results of
the Analysis of Variance examining the
Table 6 depicts the Analysis of differences in AI literacy and students'
Variance (ANOVA) calculation results academic performance when grouped
examining the differences in AI literacy according to gender. In general,
and students' academic performance significant findings were produced by
when grouped according to age. the statistical treatment. In particular,
Generally, it is easy to decipher that the variable "use and apply AI" got a
there were no particular differences in significant difference with a result of
the students' perception of AI literacy. F(2, 866)= 10.889, p= .001. Similarly, the
The table showed the following results: variable "know and understand AI" also
for the "use and apply AI," the study produced a significant difference, with
produced F(3, 865)= 2.021, p= .109. For F(2, 866)= 5.912, p= .003. In the case of
315
Asio SHE Journal
"AI ethics," the variable also shows a .003). Furthermore, a weak positive
significant difference between genders correlation is observed for the variable
since it generated F(2, 866)= 5.067, p= "AI Literacy" (r = .096, p = .004). These
.003. Regarding "AI literacy," a findings suggest a slight positive
significant difference was detected relationship between AI literacy and
between genders due to F(2, 866)= academic performance, indicating that
8.786, p= .001. The associated individuals with higher AI literacy tend
probability values were all lower than to have slightly better academic
the .05 alpha significance level. Hence, it performance. However, the weak
is safe to assume that in terms of gender, correlations indicate that AI literacy
the study found significant differences in alone may not be the sole determinant
the students' AI literacy. However, the of academic performance, and other
table showed no significant difference factors might also contribute. Further
between genders for the "academic research is needed to explore the
performance" variable since the study complex relationship between AI
found F(2, 866)= 0.412, p= .662. These literacy and academic performance
results propose significant differences in further.
AI literacy among genders and specific
AI knowledge and understanding DISCUSSION
aspects. However, there is no significant
difference in academic performance The central aspect of this study is for the
between genders. investigator to determine AI literacy
among college students and its
TABLE 8. Relationship between AI Literacy relevance to their academic
and academic performance performance. From the generated
Variables Academic results of the study, the investigator
Performance provided some exciting and thought-
Use and Apply AI .053 provoking findings that can benefit
.121 individuals at the forefront of education.
Know and Understand AI .107* The study's findings regarding
.002 AI literacy among college students were
AI Ethics .101* unprecedented. Each latent variable of
.003 AI literacy, namely use and apply AI,
AI Literacy .096* know and understand AI, and AI ethics,
.004 generated scores corresponding to
Note: *p < .05 moderately high levels of perception
among college students. About the
Table 8 presents the relationship current findings, Zhao et al. (2022) and
between AI literacy and the Obenza et al. (2024) also generated
respondents' academic performance. almost a similar mean score from their
For the variable "Use and Apply AI," a study (above the neutral score). Another
article from Wood and colleagues
weak positive correlation is observed
(2021) disclosed that students and
with academic performance (r = .053, p
faculty reported limited AI literacy. Also,
= .121). Similarly, a weak positive Manrique and Palomares (2024) found
correlation is found for the variable that their study's respondents are highly
"Know and Understand AI" (r = .107, p = familiar with AI's ease of use and
.002). The "AI Ethics" variable also usefulness. In addition, in a literature
shows a weak positive correlation with review, ethics in AI is the least noted
academic performance (r = .101, p = term for FATE (Fairness, Accountability,
316
Asio SHE Journal
Transparency, and Ethics). They the study found a credible answer for
considered ethics a relatively broad and the study's primary aim: whether there
an umbrella term in most studies is a relationship between AI literacy and
(Memarian & Doleck, 2023; Chounta et the academic performance of college
al., 2022). There were also some students. Although it is still too early to
potential challenges for teachers, such argue the relevance of such findings, the
as vague and unclear protocols in policy current study finally laid its foundation
and curriculum, a need for more for future exploration by other
understanding of AI and its limitations researchers interested in exploring
and the emotional aspects relative to an artificial intelligence (AI) literacy. In
individual's preconception (Velander et particular, two latent variables in the
al., 2024). As for the general academic study yielded similar results regarding
performance of the respondents in the the overall AI literacy outcome. Knowing
study, most of them have relatively high and understanding AI, as well as AI
satisfactory marks. This result coincides ethics, also produced weak and positive
with the previous finding of local relationships with the academic
literature wherein students' academic performance of college students.
performance was above average These exciting findings provided
(Bancoro, 2024). an essential and vital foundation for
To make the study even more understanding students' perceptions of
interesting, variance was generated Artificial Intelligence (AI). The results
among the groups when the investigator can also be leveraged so that students
tried to group the respondents based on can consider enhancing their
their demographic characteristics. For capabilities and raising their literacy
instance, for use and applying AI, levels in this technologically diverse and
college, year level, and gender generated driven educational system.
significant differences. For the know and
understand AI, only gender yields
significant results. As for AI ethics, CONCLUSION
college and gender were the significant
findings. Moreover, for the overall AI The study revealed a moderately high
literacy, the study generated significant level of AI literacy among the
variations in college and gender respondents. The investigator also
groupings among the respondents. A observed the same AI use and
previous paper by Asirit and Hua (2023) application level, knowledge and
revealed that college students' understanding, and AI ethics. Regarding
familiarity with AI depends on age, the respondents' academic performance,
academic year, and field of study. they generally have a high satisfaction
Finally, for the relationship rate based on their grade point average
between the two variables in the study, (GPA). When the study grouped the
in general, there was a weak positive respondents according to college and
association. This result somehow gender, statistical differences were
disagreed with the findings of Bancoro found in AI literacy. As for academic
(2024), wherein the investigator found performance, the study observed
no significant association between variance in college, year level, and age.
students' academic performance and AI In general, AI literacy and the
use. However, a later study by Lestari respondents' academic performance had
and colleagues (2021) found a a weak positive association.
significant relationship between skills Furthermore, the same result was
achievement and AI use. Alshater (2022) observed for knowing and
also agrees that AI tools can significantly understanding AI and AI ethics.
enhance academic performance. Thus,
317
Asio SHE Journal
318
Asio SHE Journal
319
Asio SHE Journal
Memarian, B., & Doleck, T. (2023). Shen, Y., & Cui, W. (2024). Perceived
Fairness, Accountability, support and AI literacy: the
Transparency, and Ethics (FATE) mediating role of psychological
in Artificial Intelligence (AI), and needs satisfaction. Frontiers in
higher education: A systematic Psychology, 15, 1415248.
review. Computers and Education: https://doi.org/10.3389/fpsyg.20
Artificial Intelligence, 5, 100152. 24.1415248
https://doi.org/10.1016/j.caeai.2
023.100152 Taber, K.S. (2018). The Use of
Cronbach’s Alpha When
Mertala, P., Fagerlund, J., & Calderon, O. Developing and Reporting
(2022). Finnish 5th and 6th grade Research Instruments in Science
students' pre-instructional Education. Research in Science
conceptions of artificial Education, 48, 1273–1296.
intelligence (AI) and their https://doi.org/10.1007/s11165-
implications for AI literacy 016-9602-2
education. Computers and
Education: Artificial Intelligence, 3, Velander, J., Taiye, M. A., Otero, N., &
100095. Milrad, M. (2024). Artificial
https://doi.org/10.1016/j.caeai.2 Intelligence in K-12 Education:
022.100095 eliciting and reflecting on Swedish
teachers' understanding of AI and
Ng, D. T. K., Lee, M., Tan, R. J. Y., Hu, X., its implications for teaching &
Downie, J. S., & Chu, S. K. W. learning. Education and
(2023). A review of AI teaching Information Technologies, 29(4),
and learning from 2000 to 4085-4105.
2020. Education and Information https://doi.org/10.1007/s10639-
Technologies, 28(7), 8445-8501. 023-11990-4
https://doi.org/10.1007/s10639-
022-11491-w Wilton, L., Ip, S., Sharma, M., & Fan, F.
(2022, July). Where is the AI? Ai
Ng, D. T. K., Leung, J. K. L., Su, M. J., Yim, I. literacy for educators. In
H. Y., Qiao, M. S., & Chu, S. K. W. International Conference on
(2022). AI education and AI Artificial Intelligence in Education
literacy. In AI literacy in K-16 (pp. 180–188). Cham: Springer
320
Asio SHE Journal
International Publishing.
https://doi.org/10.1007/978-3-
031-11647-6_31
321