Frontiers in Health Informatics www.ijmi.

ir
2023; 12: 168 Open Access

Artificial intelligence literacy among healthcare professionals and students:

A systematic review
Khalil Kimiafar1 , Masoumeh Sarbaz1 , Seyyed Mohammad Tabatabaei2,3 , Kosar
Ghaddaripouri4 , Atefeh Sadat Mousavi1 , Marziyeh Raei Mehneh1,5 , Seyyedeh Fatemeh
Mousavi Baigi1,5*
1Department of Health Information Technology, School of Paramedical and Rehabilitation Sciences, Mashhad University of Medical Sciences,
Mashhad, Iran
2Department of Medical Informatics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran
3Clinical Research Development Unit, Imam Reza Hospital, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran
4Department of Health Information Management and Technology, School of Management and Medical Information Sciences, Shiraz University

of Medical Sciences, Shiraz, Iran

5Student Research Committee, Mashhad University of Medical Sciences, Mashhad, Iran

Article Info ABSTRACT

Article type: Introduction: In the digital age, since the application of artificial
Review intelligence (AI) is increasingly penetrating the world, the cultivation of AI
literacy has become increasingly important for everyone. This systematic
review investigated the level of AI literacy among healthcare professionals
Article History:
and students.
Received: 2023-10-01
Accepted: 2023-10-28
Published: 2023-11-11 Material and Methods: We searched the databases PubMed, Embase,
Scopus, and Web of Science for relevant material. The evidence gathered
* Corresponding author: from the studies included in this systematic review was reported in this
Seyyedeh Fatemeh Mousavi Baigi study using preferred reporting items for systematic reviews and meta-
analyses (PRISMA). Studies that assessed the level of AI literacy among
Department of Health Information medical and healthcare professionals and students met the inclusion criteria
Technology, School of Paramedical for this study. The quality of the included study for this review was assessed
and Rehabilitation Sciences, using the analytical cross-sectional critical assessment checklist developed
Mashhad University of Medical
by the Joanna Briggs Institute (JBI). The same standard checklist was used
Sciences, Mashhad, Iran
for data extraction.
Email: [email protected]
Results: Of the 10 included studies, 4 (40%) reported a low level of
preparation, knowledge, and literacy. In a study, it was also shown that
Keywords: radiologists had acceptable literacy about AI, and it seems that they had a
Artificial Intelligence Literacy better study of this field compared to other specialists. Another study
Digital Literacy showed that initially the level of AI literacy was not acceptable but improved
AI significantly after training. Two studies also hailed AI's contribution to
AI Competence improving healthcare.

Conclusion: Evidence from this review indicated that half of the studies on
the AI literacy of professionals and students were very low, and other
included studies also reported the basic literacy of AI acceptably. Finally, in
all included studies, AI training courses and their application in healthcare
were considered necessary for professionals and students, and they were
trying to improve the educational infrastructure.

Cite this paper as:

Kimiafar K, Sarbaz M, Tabatabaei SM, Ghaddaripouri K, Mousavi AS, Raei Mehneh M, Mousavi Baigi SF. Artificial intelligence
literacy among healthcare professionals and students: A systematic review. Front Health Inform. 2023; 12: 168. DOI:
10.30699/fhi.v12i0.524

INTRODUCTION recognition, visual perception, quick decision-

making, and language translation. AI is currently used
The primary goal of artificial intelligence (AI) is to in our daily lives to an increasingly higher level. AI is
build intelligent computers that are capable of therefore no longer a future concept. AI is
understanding computations and doing activities increasingly being used in the medical field [1]. Deep
that call for human intellect, such as precise speech learning (DL) and machine learning (ML) are two

Copyright© 2023, Published by Frontiers in Health Informatics. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0
International (CC BY) License (http://creativecommons.org/).
 Artificial intelligence literacy among healthcare professionals and students Khalil Kimiafar et al.

subcategories of AI that the health sector is training. People need to acquire specific types of AI-
investigating [2]. The field of radiology has been the related knowledge, skills, and values. Such basic AI
biggest user of AI systems. However, there are other literacy is increasingly necessary for effective
instances of its application in other fields, such as collaboration between humans and machines in life,
medicine, neurology, pathology, psychiatry, learning, and the workplace [19].
cardiology, rehabilitation, dermatology, and
Researchers have previously conducted reviews of
ophthalmology [3-10].
attitudes toward AI education among healthcare
DL refers to an area of artificial intelligence that students [20]. However, at the current stage, few
utilizes a layered algorithmic architecture for data attempts have been made to start teaching healthcare
processing [11]. DL provides a range of apps that aid students and professionals the key concepts of AI.
in identifying complex but subtle patterns in pictures. Most approaches are focused on algorithms and
This kind of ability may be used in model-oriented programming for AI training [21–25]. Accordingly,
healthcare domains like radiology, pathology, and most computer science approaches target students
dermatology for image-based automated diagnosis with computer science-related fields [26].
[4-6, 11].
However, the World Health Organization warned that
As AI continues to advance, its uses will go beyond healthcare professionals need to understand how AI
picture categorization and into fields like psychiatry's works [27]. Future healthcare workers will use AI
natural language processing and cardiology's signal concepts and algorithms based on their
processing [12]. It is expected that in the future, AI- understanding [28]. This is important because AI is
assisted systems will be able to carry out specialized not a magical tool that can be taken for granted, and
duties including patient screening and test referrals, poor implementation of AI projects can lead to
as well as becoming more autonomous and misdiagnoses that harm patients, as reported in some
recommending possible therapeutic interventions studies [29]. Previous studies have highlighted
[13]. communication gaps among healthcare workers
regarding AI, such as poor data storage, which may
In the digital age, since the application of AI is
hinder the efficient and appropriate use of this
increasingly penetrating the world, the cultivation of
technology [28].
AI literacy has become increasingly important for
everyone [14, 15]. For example, some UK medical Healthcare workers with AI knowledge are expected
schools offer limited training opportunities in to replace those without such knowledge in the
healthcare AI. However, the true prevalence of such future [30]. Apart from the fact that AI in medicine is
training and the level of AI literacy among the UK an exciting and fast-paced field of research where
medical student population remains unknown. The future doctors, specialists, and scientists can conduct
increasing pervasiveness of AI-related technologies interesting experiments, knowing how it works and
shows that the time has come for every society to being able to interpret its results is also a prerequisite
educate its citizens about AI. for their future careers. In addition, AI permeates
many daily clinical tasks as a supporting entity [31],
Additionally, as the field of healthcare AI gains
which will greatly influence the skills required for
momentum, it is becoming increasingly clear that
future performance [32].
increased awareness is needed along with AI-based
training for medical and healthcare practitioners and Despite the increasing contribution of AI in
students. A national government review was healthcare and research on the level of literacy of
conducted in 2019 on the same topic and clearly healthcare professionals and students towards AI,
emphasized the importance of arming the current few systematic reviews integrate the best evidence
and future clinical workforce with the necessary and provide an overview of the field. Therefore, this
skills to work with new digital tools, including those systematic review investigated the level of AI literacy
underpinning AI [16]. among healthcare professionals and students.
Meanwhile, the recent increase in interest in training
healthcare students in AI reflects the growing MATERIAL AND METHODS
integration of AI applications in education, research,
and clinical care. The Royal College of Physicians, Study design
Surgeons of Canada, and the Association of American The evidence gathered from the studies included in
Medical Colleges have recommended training for this systematic review was reported in this study
healthcare professionals related to AI, including using Preferred Reporting Items for Systematic
training on data sourcing and stewardship, AI ethics, Reviews and Meta-Analyses (PRISMA) [33].
critical evaluation, and interpretation of AI
applications in health [4, 17, 18]. It is clear that the Strategic search
need to properly use and apply AI while enjoying its
benefits and protecting your privacy requires proper On October 4, 2023, we searched the databases

Volume 12 | Article 168 | Nov 2023 Page 2 of 11

 Artificial intelligence literacy among healthcare professionals and students Khalil Kimiafar et al.

PubMed, Embase, Scopus, and Web of Science for measurement criteria, identification of confounding
relevant material. The search method for each factors, dealing with confounding factors strategies,
database is displayed in Appendix 1. The databases valid measured outcomes, and appropriate statistical
were searched using the following Medical Subject analysis. There were four categories for the
Headings and Emtree keywords and terms: (AI or responses to the questions: yes, no, unclear, and not
"artificial intelligence" or "machine learning" or applicable. The reaction to a question was either
"intelligence, artificial" or "computational stamped as 1 or 0, depending on whether it was yes
intelligence" or "intelligence, computational" or or no, unclear, or not pertinent. So, each included
"machine intelligence" or "intelligence, machine" or study could only get a quality score of 8, and the
"computer reasoning" or "reasoning, computer" or " exclusion only applies if the study's quality score is
AI (artificial intelligence)" ) and (literacy or skills or underneath 5.
competence or fluency) or ("AI literacy" or "artificial
intelligence literacy" or "AI skills" or "artificial RESULTS
intelligence skills" or "AI competence" or "artificial
intelligence competence" or "AI fluency" or "artificial Fig 1 displays 9,542 documents that were located
intelligence fluency") and (health or medical or through database searches. After duplications (1966)
medicine or rehabilitation). were removed, 7576 studies' titles and abstracts
were reviewed. After the titles and abstracts of the
Eligibility criteria studies were checked for relevance to the study's
goals, 7546 studies in total were discarded. Then, 30
Studies that assessed the level of AI literacy among articles were picked for full-text analysis. In the end,
medical and healthcare professionals and students 10 studies that met the criteria for inclusion were
met the inclusion criteria for this study. However, included in the review.
papers whose target audience was students other
than those studying medicine and healthcare, such as
those studying technical and computer science, for
whom the complete text was unavailable, as well as
those written in a language other than English, were
eliminated from the study.

Data extraction and synthesis

All documents were acquired using a systematic
search, and duplicate documents were eliminated
before this examination. Titles and abstracts were
independently reviewed for eligibility using the
criteria. Articles that did not meet the criteria for Fig 1: Flowchart of study selection and screening
inclusion were not included in the review. The full
texts were then retrieved and assessed in accordance Quality assessment
with the criteria for eligibility by two distinct
researchers. When there was a tie, the third author According to the quality evaluation findings provided
made the decision after discussions to resolve the in Table 1, which reveal that there was no discernible
conflict. The same standard checklist was used for bias in the studies, we only included high-quality
data extraction. Among the data items contained in studies in our analysis. One of the most prevalent
this form are the study reference, study nation, problems in these investigations was the inability to
publication year, participant characteristics (number recognize confounding factors and create mitigation
of participants and their field of study), study plans for them.
objectives, and study outcomes.
Study characteristics
Quality assessment Tables 2 and 3 list the specifications of all the
The quality of the research submitted for this review included studies. According to Fig 2, of the ten
was assessed using the analytical critical assessment included studies, two studies (20%) were conducted
checklist developed by the Joanna Briggs Institute in Saudi Arabia [34, 35], two studies (20%) were
(JBI) [34]. In order to evaluate the quality of these conducted in England [36, 37], two studies (20%)
studies, the research explicitly gathered 8 questions. were conducted in Canada [11, 38], and other studies
The following were the topics covered by the were conducted in the United States (10%) [25], Italy
questions in this checklist: a sample inclusion (10%) [24], Spain (10%) [39], and Taiwan (10%)
criterion, study subjects, the setting, valid and [40]. Fields of study for healthcare students and
reliable measurement tools (including the validity professionals include six fields of medicine [34, 36,
and reliability of questionnaires), standard 40], dentistry [34, 39], radiology [35], laboratory

Volume 12 | Article 168 | Nov 2023 Page 3 of 11

 Artificial intelligence literacy among healthcare professionals and students Khalil Kimiafar et al.

[41], ophthalmology [37], radiation therapy sample size in the included studies was 3892
ecologists, and medical physics [38], and two studies participants. The sample size was between 15 and
include healthcare fields in general [11, 42]. The total 2167 participants.
Table 1: Summary of the articles' quality ratings using the JBI critical appraisal methodology

Study/ Questions Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Score

Aboalshamat, 2022, Saudi Arabia [34] Y Y Y Y N N Y Y 6
Alelyani, 2021, Saudi Arabia [35] Y Y Y Y N N Y Y 6
Bellini, 2022, Italy [41] Y Y Y Y N N Y Y 6
Teng, 2022, Canada [11] Y Y Y Y N N Y Y 6
Russell, 2023, America [42] Y Y Y Y N N Y Y 6
Ejaz, 2022, 48 countries around the world
Y Y Y Y N N Y Y 6
[36]
Yang, 2019, Taiwan [40] Y Y Y Y N N Y Y 6
Suárez, 2022, Spain [39] Y Y Y Y N N Y Y 6
Constantin, 2023, England [37] N Y Y Y N N Y Y 5
Gillan, 2019, Canada [38] Y Y Y Y N N Y Y 6
*Abbreviations: JBI, Joanna Briggs Institute; Q, question, N, no; NA, not/applicable; U, unclear; Y, yes, Score: the quality assessment score ranged from 0 to 8 based on
each question of the JBI checklist

Taiwan
10% Saudi Arabia
20%
Spain
10%

Italy
10%

England
20%

United States
10%

Canada
20%

Fig 2: Distribution of countries in included studies

Table 2: Characteristics of participants in all included studies

Number of
Source (first author, year, country) Study method Field of participants
participants
Aboalshamat, 2022, Saudi Arabia [34] Cross-sectional study 334 Medicine and dentistry
Alelyani, 2021, Saudi Arabia [35] Cross-sectional study 714 Radiology
Biologist, laboratory specialist and
Bellini, 2022, Italy [41] Cross-sectional study 227
managers
Teng, 2022, Canada [11] Cross-sectional study 2167 Healthcare
Russell, 2023, America [42] Interview study 15 Healthcare
Ejaz, 2022, 48 countries around the
Mixed-methods 128 Medicine
world [36]
Yang, 2019, Taiwan [40] Prospective pilot study 72 Medical interns
Suárez, 2022, Spain [39] Descriptive cross-sectional 193 Dentistry
Web-based interview
Constantin, 2023, England [37] 18 Ophthalmology
study
Radiation oncologists, medical
Gillan, 2019, Canada [38] Cross-sectional study 24 physicists, treatment planners, and
radiation therapists provide treatment

Volume 12 | Article 168 | Nov 2023 Page 4 of 11

 Artificial intelligence literacy among healthcare professionals and students Khalil Kimiafar et al.

Table 3: Summary of study characteristics from all included studies

Source (first author,

Study goal Results
year, country)
Participants had low levels of readiness for AI, although dental professionals had
better literacy scores. Based on the low readiness levels measured in this study,
Assessing readiness levels for AI medical and dental professionals are not yet ready for AI.
Aboalshamat, 2022,
among medical and dental While dental professionals had better preparedness scores than medical
Saudi Arabia [34]
professionals in Saudi Arabia professionals, it is recommended that educational organizations and hospitals
include AI modules and practical training at undergraduate and postgraduate
levels.
Assessing the attitude of the
Most of the participants (61.2%) had read or heard about the role of AI in radiology.
radiology community in Saudi
The level of reported AI knowledge was moderate and acceptable. The participants
Alelyani, 2021, Saudi Arabia towards AI applications
had an acceptable level of AI knowledge. Human-machine interaction will be one of
Arabia [35] .
the most important skills in the future, and therefore AI applications should be
included in the curriculum of medical students.
Evaluation of the level of
expertise, knowledge, and
interest of Italian clinical Only 20% of respondents stated that they had a good understanding of BAI, and an
laboratory specialists in the field even smaller percentage (12%) felt that they had a competent level in this area. The
Bellini, 2022, Italy
of Big Data and Artificial overall results offer the opportunity to understand the related shortcomings in the
[41]
Intelligence (BAI) and its implementation of AI in LM. Practitioners should spend time learning the basics of
applications in laboratory these new technologies in order to evaluate clinical trial opportunities.
medicine (LM).

Investigating and identifying

More than half of the respondents did not know what AI was (1107/2167, 51.08%)
gaps in Canadian healthcare
or had a misunderstanding of it (676/2167, 31.2%). Canadian healthcare students
students' knowledge of AI, how
Teng, 2022, Canada were cautiously optimistic about the role of AI in their fields. However, many felt
these students differ across
[11] uninformed about the matter. Healthcare students in different programs identified
disciplines in their knowledge
different curricular needs, and such program-specific needs should be considered
and views of AI
when integrating the AI curriculum.
Defining AI-related clinical Six clinical competencies for AI were identified that can be used to guide future
competencies for healthcare teaching and learning programs in maximizing the potential benefits of AI-based
professionals, exploring future tools and reducing potential harms.Six AI-related clinical competencies for
Russell, 2023,
applications of healthcare AI healthcare professionals were included: 1. Basic knowledge of AI; 2. Social and
America [42]
technologies, and organizational ethical implications of AI; 3. Workflow analysis for AI-based tools; 4. AI-enhanced
responsibilities required for clinical encounters; 5. Evidence-based assessment of AI-based tools; 6. Practice-
oversight and management based training and improvement in relation to AI-based instruments
There was support for including AI education in mainstream curricula around the
Ejaz, 2022, 48
Reporting on the state of AI world, but few students had received AI education. Medical students from all
countries around the
medical education globally countries should be offered AI training as part of their curriculum to develop skills
world [36]
and knowledge about AI to ensure a patient-centric digital future in medicine.
Investigating the improvement of Literacy level was high after training with a new intervention program based on AI.
skill training of medical interns This study showed promising effects of incorporating an expert-led training course
Yang, 2019, Taiwan
with a new AI-based intervention with an AI system into the routine surgical curriculum. Increased training
[40]
program for suturing and frequency with an AI system improved medical interns' confidence and
ligature skills. performance in suturing skills.
A total of 193 students participated. A large majority of students were satisfied with
the interaction (mean 4.36), and fifth-yearstudents gave a better score to the
interaction and showed higher satisfaction values. Students who made the correct
Creating and evaluating a virtual
diagnosis evaluated the technology more positively. The results of this study show
patient through an AI chatbot to
Suárez, 2022, Spain the usefulness of simulating a VP with AI by giving students the possibility of
develop pulp pathology
[39] clinical practice and also providing an attractive and personal experience to
diagnostic skills in dental
students due to the user interface and natural language used, without
students
underestimating the economic costs. Incorporating this technology into dental
curricula has positive value for students and also ensures their training and
adaptation to new technological developments.
Identifying ophthalmologists'
expectations and concerns
regarding the National Image A strong desire to use AI to improve healthcare was found, along with concerns
Constantin, 2023,
Research Repository and their about training, costs, responsibilities, skill retention, data sharing, and disruption to
England [37]
use of AI-based decision support professional practice.
and gathering their suggestions
for improving eye healthcare
In the absence of AI-related tools, trained employees form complex views about
their value and impact. The introduction of AI strategies in radiation medicine will
Assessing radiation medicine
require changes in practice that will have ripple effects across many aspects of care
professionals' perceptions of
Gillan, 2019, Canada and will require professional groups to adjust their roles in response. To plan more
clinical and professional risks
[38] responsibly for such a complex intervention, it is crucial to consider these
and benefits and evolving roles
professional groups and their educational needs in terms of technological literacy
and responsibilities with AI
and new workflow models and empower them to participate in the leadership of
change.

Volume 12 | Article 168 | Nov 2023 Page 5 of 11

 Artificial intelligence literacy among healthcare professionals and students Khalil Kimiafar et al.

AI literacy rates among healthcare professionals highlighted as an integral advancement in healthcare

and students that should be addressed by medical and healthcare
professionals to increase their preparedness. This
According to Fig 3, of the 10 included studies, 4 (40%)
can help the next generation of medical professionals’
reported a low level of preparation, knowledge, and
cope with the age of data science and fit into the
awareness [11, 34, 36, 41]. In a study, it was also
future AI ecosystem [13, 43, 44].
shown that radiologists had acceptable literacy about
AI, and it seems that they had a better study of this Aboalshamat et al. found that dental professionals
field compared to other specialists [35]. Another were more prepared to work with AI in a clinical
study showed that initially the level of AI literacy was setting than medical professionals. But in general, the
not acceptable but improved significantly after overall level of preparation among the participants
training [40]. Two studies also hailed AI's was not satisfactory. This indicates that medical and
contribution to improving healthcare [37, 39]. dental professionals did not have sufficient literacy
Another study identified cases of clinical merit for AI and preparation to use AI at this time [34].
that could be used to guide future teaching and
Alelyani also showed in his study in Saudi Arabia that
learning programs in maximizing the potential
radiologists had an acceptable level of knowledge
benefits of AI-based tools and reducing potential
about AI technology. So, 80% of the respondents
harm [42]. In another study, professional groups
agreed that AI technologies should be validated in an
were required to adjust their roles in AI practice [38].
established clinical setting [35]. In another study in
Italy, Bellini showed that an overwhelming majority
Low literacy level
of participants (95%, 162 out of 170 respondents)
of AI 40% expressed interest in training related to the use of BAI
in laboratories. Most respondents said they would
like to improve their literacy and skills in data
management (35%) or data analysis (37%), while a
third said they would benefit from taking a course in
this area (28%).
Medium literacy
level of AI 60%
The findings of this study showed a general lack of
hardware and software infrastructure, a lack of
personal computers, a lack of corporate Wi-Fi
networks, and a low level of subjective satisfaction
regarding hardware and software equipment [41].
Low literacy level of AI Medium literacy level of AI
Teng stated that healthcare students were generally
cautiously optimistic about AI in their fields, although
Fig 3: The level of AI literacy in included studies more than half of healthcare students indicated that
they did not know what AI was or how it might be
DISCUSSION relevant to their field of work. In general, healthcare
students feel unprepared and uneducated about AI,
Principal findings which may have contributed to their fear and anxiety
about the topic [11].
Evidence from this systematic review shows that the
majority of healthcare professionals and students Similar mixed feelings have been previously
were motivated to use AI to improve healthcare. expressed by MD students from the United Kingdom
However, few participants received adequate [45] and by health professionals in France and the
training in this field and had limited literacy in the use United States [46, 47]. A 2020 European survey found
of AI. Ten studies met all the requirements for that only one-third of surveyed medical students
inclusion in this review. The level of literacy, claimed to have basic knowledge of AI [45]. A 2021
education, and skills of healthcare students towards survey of medical students in Ontario found that
AI were investigated in the studies. The JBI checklist respondents believed they understood what AI
criteria indicated that almost all studies had meant. However, when asked about specific terms
moderate to high-quality evidence. related to AI, such as machine learning or neural
networks, students did not understand them [48].
Among healthcare professionals and students, Healthcare workers seem to hope that the integration
radiologists had the most literacy in using AI [35]. of AI will bring about improvements in diagnostic
According to the findings of this study, increasing the accuracy [49], patient monitoring [46], and
frequency of training with AI systems and including reductions in medical errors [50].
the training course in the courses of healthcare
students will have promising effects in clinical work, Concerns regarding the integration of AI in
and human-machine interaction will be one of the healthcare may be attributed to the potential to
most important skills in the future. AI should also be replace health professionals [51] and additional

Volume 12 | Article 168 | Nov 2023 Page 6 of 11

 Artificial intelligence literacy among healthcare professionals and students Khalil Kimiafar et al.

knowledge requirements in their fields [52]. Medical and other studies also reported the basic literacy of
communities should address concerns about how to AI acceptably. Finally, in all included studies, AI
test AI software for therapeutic efficacy and safety training courses and their application in healthcare
before implementing a large-scale clinical application were considered necessary for professionals and
of AI [53]. Today, the superiority of computers over students, and they were trying to improve the
humans is not debatable, but this question revolves educational infrastructure. In this regard,
around how medicine can benefit from these technologies such as mobile health interventions,
capabilities [54]. digital games, simulation tools, virtual reality, and
telemedicine can be used for more useful and
With the development and growth of AI in various
effective training [62-70].
aspects of healthcare, its impact on people on the
front line of care is inevitable, but this issue has been
Strengths and weaknesses
less noticed until today. An AI-friendly healthcare
education program is therefore essential because This systematic review has many benefits. First, we
future healthcare providers will likely be responsible followed the rules for difficult systematic review
for overseeing the algorithmic interpretation of procedures [21]. Second, the quality of the evidence
patient healthcare data [55]. One of the important from each included study was assessed using the JBI
goals of the literacy course is to empower assessment criteria. This increased the quality of the
participants by increasing their ability to understand, included studies' transparency. Third, this evaluation
interact with, and communicate with AI applications offers insightful information to those who shape
on a digital course [56]. educational policy in the area of AI literacy in
healthcare and medicine. However, this study could
According to Mäkinen, the term "people
have had certain drawbacks. Limited studies have
empowerment" refers to providing people with the
explicitly examined AI literacy among professionals
tools they need to better control their lives and
and students.
expand their communication skills. The goal of AI
literacy is to provide people with new abilities and Therefore, due to the lack of evidence found, the
ways to participate in a digital society. This proof of these results needs more investigation. In
understanding refers to a goal of empowerment addition, studies used different questionnaires and
through AI or a goal of enhancing "assurance through questions to measure the level of AI literacy of
interaction with AI". Several similar concepts, such as experts and students. For example, one study reports
"digital empowerment and programming a high level of AI literacy, specifically in imaging
empowerment," have been proposed [57]. These algorithms. In another study, basic AI literacy was
terms describe the means of empowering people in evaluated, such as operational definitions. However,
different fields. Kong et al. suggested that this study included a diverse range of literacy and
programming empowerment has four main understanding among healthcare professionals and
components: planning self-efficacy, meaningfulness, students in different disciplines. Undoubtedly, with
impact, and creative self-efficacy, to understand the increasing growth of studies in this field, more
programming empowerment [58]. studies are needed to understand AI literacy, develop
educational curricula, and improve the level of
In another study on the concept of empowering AI,
knowledge and literacy specifically in each
the components of AI self-efficacy—meaningfulness,
educational field.
influence, and creative self-efficacy—were used [56].
In general, the value of AI literacy is widely
recognized because it is useful for lifelong learning CONCLUSION
and can be used in many everyday situations [59]. Evidence from this systematic review shows that the
Additionally, participants who perceive AI as majority of healthcare professionals and students
meaningful are likely to exert more effort to excel and were motivated to use AI to improve healthcare and
feel empowered. Students' understanding of AI welcomed AI in clinical contexts. However, few
literacy and its social effects is important because a students received sufficient education and literacy in
person who feels that interaction with AI creates this field. We found that interest in learning the
more impact tends to be more intrinsically motivated clinical applications, development, and evaluation of
to learn related skills. Frymier states that participants AI-based tools is ubiquitous, but current medical
can be motivated to learn AI skills after becoming curricula do not respond to this. Students and
aware of their social impact [60]. professionals believed that learning the knowledge
Empowering AI literacy education for professionals and skills necessary to navigate the field of AI in
and students has the potential to not only enable the medicine was important and emphasized the need for
use of existing technology but also inspire future multidisciplinary training on this topic. It is suggested
technology producers [61]. In summary, the results of that they provide the basic technology for the use of
this review indicated that half of the studies on the AI AI in the future. Next, they should be provided with
literacy of professionals and students were very low, training and resources to strengthen AI literacy and

Volume 12 | Article 168 | Nov 2023 Page 7 of 11

 Artificial intelligence literacy among healthcare professionals and students Khalil Kimiafar et al.

support pioneering developments in implementing AUTHOR’S CONTRIBUTION

AI-based tools in healthcare systems. This near-
universal desire to use AI to improve healthcare, All authors contributed to the literature review,
along with concerns about training, costs, design, data collection and analysis, drafting the
responsibilities, skill retention, data sharing, and manuscript, read and approved the final manuscript.
disruption to professional practice, has to date left
little work specifically on best practices for delivering CONFLICTS OF INTEREST
literacy. AI is not focused on healthcare education. To
The authors declare no conflicts of interest regarding
plan more responsibly for such a complex
the publication of this study.
intervention, it is crucial to consider these
professional groups and their training needs in terms
of technological literacy and new workflow models FINANCIAL DISCLOSURE
and empower them to participate in the leadership of No financial interests related to the material of this
change. manuscript have been declared.

ACKNOWLEDGMENT ETHICS APPROVAL

We hereby express our gratitude to the Student This study was approved after review and
Research Committee of Mashhad University of presentation in the Ethics Committee of Mashhad
Medical Sciences who helped us in conducting this University of Medical Sciences (Code: 4012496).
research.

clinical trials. Frontiers in Health Informatics. 2022;

REFERENCES 11(1): 118.
1. Jha S, Topol EJ. Adapting to artificial intelligence: 9. Mousavi AS, Mousavi Baigi SF, Dahmardeh F, Mehneh
radiologists and pathologists as information MR, Darrudi R. Teleophthalmology: A systematic
specialists. JAMA. 2016; 316(22): 2353-4. PMID: review of randomized controlled trials. Frontiers in
27898975 DOI: 10.1001/jama.2016.17438 [PubMed] Health Informatics. 2023; 12: 141.
2. Gaddi AV. eEducation and eHealth: A call for action. 10. Sarbaz M, Mousavi Baigi SF, Darroudi A, Kimiafar K.
In: Gaddi A, Capello F, Manca M. (Eds). eHealth, care Occupational exposure to blood and body fluids
and quality of life. Springer; 2014. among prehospital emergency medical services
workers in Mashhad, Northeast of Iran. Am J Infect
3. Ghaderzadeh M, Aria M. Management of COVID-19
Control. 2023; 51(9): 1004-10. PMID: 36842711 DOI:
detection using artificial intelligence in 2020
10.1016/j.ajic.2023.02.012 [PubMed]
pandemic. International Conference on Medical and
Health Informatics. ACM; 2021. 11. Wang F, Casalino LP, Khullar D. Deep learning in
medicine—promise, progress, and challenges. JAMA
4. Ghaderzadeh M, Aria M, Asadi F. X-ray equipped with
Intern Med. 2019; 179(3): 293-4. PMID: 30556825
artificial intelligence: Changing the COVID-19
DOI: 10.1001/jamainternmed.2018.7117 [PubMed]
diagnostic paradigm during the pandemic. Biomed
Res Int. 2021; 2021: 9942873. PMID: 34458373 DOI: 12. Teng M, Singla R, Yau O, Lamoureux D, Gupta A, Hu Z,
10.1155/2021/9942873 [PubMed] et al. Health care students' perspectives on artificial
intelligence: Countrywide survey in Canada. JMIR Med
5. Ghaderzadeh M, Aria M, Hosseini A, Asadi F, Bashash
Educ. 2022; 8(1): e33390. PMID: 35099397 DOI:
D, Abolghasemi H. A fast and efficient CNN model for
10.2196/33390 [PubMed]
B‐ALL diagnosis and its subtypes classification using
peripheral blood smear images. International Journal 13. Yu KH, Beam AL, Kohane IS. Artificial intelligence in
of Intelligent Systems. 2022; 37(8): 5113-33. healthcare. Nat Biomed Eng. 2018; 2(10): 719-31.
PMID: 31015651 DOI: 10.1038/s41551-018-0305-z
6. Mousavi Baigi SF, Sarbaz M, Ghaddaripouri K, Noori N,
[PubMed]
Kimiafar K. The effect of tele-rehabilitation on
improving physical activity in patients with chronic 14. Long D, Magerko B. What is AI literacy? Competencies
obstructive pulmonary disease: A systematic review and design considerations. Conference on Human
of randomized controlled clinical trials. Frontiers in Factors in Computing Systems. ACM; 2020.
Health Informatics. 2022; 11(1): 113.
15. Sabouret N. Understanding artificial intelligence. CRC
7. Mousavi Baigi SF, Mehneh MR, Sarbaz M, Aval RN, Press; 2020.
Kimiafar K. Telerehabilitation in response to critical
coronavirus: A systematic review based on current 16. National Health System. Preparing the healthcare
evidence. Journal of Isfahan Medical School. 2022; workforce to deliver the digital future [Internet].
40(678): 498-508. 2019 [cited: 15 Mar 2023]. Available from:
https://topol.hee.nhs.uk/wp-content/uploads/HEE-
8. Mousavi Baigi SF, Mousavi AS, Kimiafar K, Sarbaz M. Topol-Review-2019.pdf
Evaluating the cost effectiveness of tele-
rehabilitation: A systematic review of randomized 17. Liu Y, Chen P-HC, Krause J, Peng L. How to read articles

Volume 12 | Article 168 | Nov 2023 Page 8 of 11

 Artificial intelligence literacy among healthcare professionals and students Khalil Kimiafar et al.

that use machine learning: Users’ guides to the 10.1093/jamia/ocab148 [PubMed]

medical literature. JAMA. 2019; 322(18): 1806-16.
30. Ng DTK, Leung JKL, Chu SKW, Qiao MS.
PMID: 31714992 DOI: 10.1001/jama.2019.16489
Conceptualizing AI literacy: An exploratory review.
[PubMed]
Computers and Education: Artificial Intelligence.
18. Reznick RK, Harris K, Horsley T, Sheikh Hassani M. 2021; 2: 100041.
Task force report on artificial intelligence and
31. Ahuja AS. The impact of artificial intelligence in
emerging digital technologies. Royal College of
medicine on the future role of the physician. PeerJ.
Physicians and Surgeons of Canada; 2020.
2019: 7: e7702. PMID: 31592346 DOI:
19. Ali SA, Payne BH, Williams R, Park HW, Breazeal C. 10.7717/peerj.7702 [PubMed]
Constructionism, ethics, and creativity: Developing
32. Wartman SA, Combs CD. Medical education must
primary and middle school artificial intelligence
move from the information age to the age of artificial
education. International Workshop on Education in
intelligence. Acad Med. 2018; 93(8): 1107-9. PMID:
Artificial Intelligence. Massachusetts Institute of
29095704 DOI: 10.1097/ACM.0000000000002044
Technology; 2019.
[PubMed]
20. Mousavi Baigi SF, Sarbaz M, Ghaddaripouri K,
33. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche
Ghaddaripouri M, Mousavi AS, Kimiafar K. Attitudes,
PC, Ioannidis JP, et al. The PRISMA statement for
knowledge, and skills towards artificial intelligence
reporting systematic reviews and meta-analyses of
among healthcare students: A systematic review.
studies that evaluate health care interventions:
Health Sci Rep. 2023; 6(3): e1138. PMID: 36923372
Explanation and elaboration. Ann Intern Med. 2009;
DOI: 10.1002/hsr2.1138 [PubMed]
151(4): W65-94. PMID: 19622512 DOI:
21. Bye RT. A flipped classroom approach for teaching a 10.7326/0003-4819-151-4-200908180-00136
master’s course on artificial intelligence. International [PubMed]
Conference on Computers Supported Education.
34. Aboalshamat K, Alhuzali R, Alalyani A, Alsharif S, Qadh
Springer; 2018.
H, Almatrafi R, et al. Medical and dental professionals
22. Chesani F, Galassi A, Mello P, Trisolini G. A game-based readiness for artificial intelligence for Saudi Arabia
competition as instrument for teaching artificial vision 2030. International Journal of Pharmaceutical
intelligence. Conference of the Italian Association for Research and Allied Sciences. 2022; 11(4): 54-9.
Artificial Intelligence. Springer; 2017.
35. Alelyani M, Alamri S, Alqahtani MS, Musa A, Almater
23. de Freitas AA, Weingart TB. I'm going to learn what?!? H, Alqahtani N, et al. Radiology community attitude in
teaching artificial intelligence to freshmen in an Saudi Arabia about the applications of artificial
introductory computer science course. ACM Technical intelligence in radiology. Healthcare (Basel). 2021;
Symposium on Computer Science Education. ACM; 9(7): 834. PMID: 34356212 DOI:
2021. 10.3390/healthcare9070834 [PubMed]
24. Estevez J, Garate G, Guede J, Graña M. Using scratch to 36. Bellini C, Padoan A, Carobene A, Guerranti R. A survey
teach undergraduate students' skills on artificial on artificial intelligence and big data utilisation in
intelligence. arXiv Preprint; 2019. Italian clinical laboratories. Clin Chem Lab Med. 2022;
60(12): 2017-26. PMID: 36067004 DOI:
25. Pouly M, Koller T, Arnold R, editors. A game-centric
10.1515/cclm-2022-0680 [PubMed]
approach to teaching artificial intelligence.
International Conference on Computer Supported 37. Russell RG, Lovett Novak L, Patel M, Garvey KV, Craig
Education. CSEDU; 2019. KJT, Jackson GP, et al. Competencies for the use of
artificial intelligence–based tools by health care
26. Mishra A, Siy H. An interdisciplinary approach for
professionals. Acad Med. 2023; 98(3): 348-56. PMID:
teaching artificial intelligence to computer science
36731054 DOI: 10.1097/ACM.0000000000004963
students. Annual Conference on Information
[PubMed]
Technology Education. ACM; 2020.
38. Ejaz H, McGrath H, Wong BL, Guise A, Vercauteren T,
27. World Health Organization. Ethics and governance of
Shapey J. Artificial intelligence and medical education:
artificial intelligence for health [Internet]. 2021 [cited
A global mixed-methods study of medical students’
[22 Mar 2023]. Available from:
perspectives. Digit Health. 2022; 8:
https://www.who.int/publications/i/item/9789240
20552076221089099. PMID: 35521511 DOI:
029200
10.1177/20552076221089099 [PubMed]
28. Brouillette M. AI added to the curriculum for doctors-
39. Yang Y-Y, Shulruf B. An expert-led and artificial
to-be. Nat Med. 2019; 25(12): 1808-9. PMID:
intelligence system-assisted tutoring course to
31806886 DOI: 10.1038/s41591-019-0648-3
improve the confidence of Chinese medical interns in
[PubMed]
suturing and ligature skills: A prospective pilot study.
29. Thompson HM, Sharma B, Bhalla S, Boley R, J Educ Eval Health Prof. 2019; 16: 7. PMID: 30986892
McCluskey C, Dligach D, et al. Bias and fairness DOI: 10.3352/jeehp.2019.16.7 [PubMed]
assessment of a natural language processing opioid
40. Suárez A, Adanero A, Díaz-Flores García V, Freire Y,
misuse classifier: Detection and mitigation of
Algar J. Using a virtual patient via an artificial
electronic health record data disadvantages across
intelligence chatbot to develop dental students’
racial subgroups. J Am Med Inform Assoc. 2021;
diagnostic skills. Int J Environ Res Public Health. 2022;
28(11): 2393-403. PMID: 34383925 DOI:

Volume 12 | Article 168 | Nov 2023 Page 9 of 11

 Artificial intelligence literacy among healthcare professionals and students Khalil Kimiafar et al.

19(14): 8735. PMID: 35886584 DOI: 10.1001/amajethics.2019.146 [PubMed]

10.3390/ijerph19148735 [PubMed]
53. Geis J, Brady A, Wu C, Spencer J, Ranschaert E, Jaremko
41. Constantin A, Atkinson M, Bernabeu MO, Buckmaster J. Ethics of artificial intelligence in radiology:
F, Dhillon B, McTrusty A, et al. Optometrists' Summary of the joint European and North American
perspectives regarding artificial intelligence aids and multisociety statement. Insights Imaging. 2019;
contributing retinal images to a repository: Web- 10(1): 101. PMID: 31571015 DOI: 10.1186/s13244-
based interview study. JMIR Hum Factors. 2023; 10: 019-0785-8 [PubMed]
e40887. PMID: 37227761 DOI: 10.2196/40887
54. Celi LA, Fine B, Stone DJ. An awakening in medicine:
[PubMed]
The partnership of humanity and intelligent
42. Gillan C, Milne E, Harnett N, Purdie TG, Jaffray DA, machines. Lancet Digit Health. 2019; 1(6): e255-7.
Hodges B. Professional implications of introducing PMID: 32617524 DOI: 10.1016/s2589-
artificial intelligence in healthcare: An evaluation 7500(19)30127-x [PubMed]
using radiation medicine as a testing ground. Journal
55. Topol EJ. High-performance medicine: The
of Radiotherapy in Practice. 2019; 18(1): 5-9.
convergence of human and artificial intelligence. Nat
43. Lindqwister AL, Hassanpour S, Lewis PJ, Sin JM. AI- Med. 2019; 25(1): 44-56. PMID: 30617339 DOI:
RADS: An artificial intelligence curriculum for 10.1038/s41591-018-0300-7 [PubMed]
residents. Acad Radiol. 2021; 28(12): 1810-6. PMID:
56. Kong S-C, Cheung WM-Y, Zhang G. Evaluation of an
33071185 DOI: 10.1016/j.acra.2020.09.017
artificial intelligence literacy course for university
[PubMed]
students with diverse study backgrounds. Computers
44. Bhardwaj D. Artificial intelligence: Patient care and and Education: Artificial Intelligence. 2021; 2:
health professional's education. Journal of Clinical & 100026.
Diagnostic Research. 2019; 13(1): 1-2.
57. Mäkinen M. Digital empowerment as a process for
45. Sit C, Srinivasan R, Amlani A, Muthuswamy K, Azam A, enhancing citizens' participation. E-learning and
Monzon L, et al. Attitudes and perceptions of UK Digital Media. 2006; 3(3): 381-95.
medical students towards artificial intelligence and
58. Kong S-C, Chiu MM, Lai M. A study of primary school
radiology: A multicentre survey. Insights Imaging.
2020; 11(1): 14. PMID: 32025951 DOI: students' interest, collaboration attitude, and
programming empowerment in computational
10.1186/s13244-019-0830-7 [PubMed]
thinking education. Computers & Education. 2018;
46. Laï M-C, Brian M, Mamzer M-F. Perceptions of 127: 178-89.
artificial intelligence in healthcare: Findings from a
59. Martin A. A European framework for digital literacy.
qualitative survey study among actors in France. J
Nordic Journal of Digital Literacy. 2006; 1(2): 151-61.
Transl Med. 2020; 18(1): 4. PMID: 31918710 DOI:
10.1186/s12967-019-02204-y [PubMed] 60. Frymier AB, Shulman GM, Houser M. The
development of a learner empowerment measure.
47. Desai AN. Artificial intelligence: Promise ,pitfalls, and
Communication Education. 1996; 45(3): 181-99.
perspective. JAMA. 2020; 323(24): 2448-9. PMID:
32492093 DOI: 10.1001/jama.2020.8737 [PubMed] 61. Touretzky D, Gardner-McCune C, Martin F, Seehorn D.
Envisioning AI for K-12: What should every child
48. Mehta N, Harish V, Bilimoria K, Morgado F, Ginsburg
know about AI? AAAI Conference on Artificial
S, Law M, et al. Knowledge of and attitudes on artificial
Intelligence. 2019; 33(1): 9795-9.
intelligence in healthcare: A provincial survey study of
medical students. medRxiv Preprint. 2021. 62. Mousavi Baigi SF, Sarbaz M, Marouzi P, Kimiafar K.
49. Cho S, Han B, Hur K, Mun JH. Perceptions and attitudes Evaluating the impact of digital game on learning
medical terminology of paramedical students:
of medical students regarding artificial intelligence in
Protocol for a randomized controlled trial. Stud
dermatology. J Eur Acad Dermatol Venereol. 2021;
Health Technol Inform. 2022; 295: 51-4. PMID:
35(1): e72-3. PMID: 32852856 DOI:
35773804 DOI: 10.3233/SHTI220658 [PubMed]
10.1111/jdv.16812 [PubMed]
63. Sarbaz M, Monazah FM, Eslami S, Kimiafar K, Mousavi
50. Maassen O, Fritsch S, Palm J, Deffge S, Kunze J, Marx G,
Baigi SF. Effect of mobile health interventions for side
et al. Future medical artificial intelligence application
effects management in patients undergoing
requirements and expectations of physicians in
chemotherapy: A systematic review. Health Policy
German university hospitals: Web-based survey. Med
and Technology. 2022; 11(4): 100680.
Internet Res. 2021; 23(3): e26646. PMID: 33666563
DOI: 10.2196/26646 [PubMed] 64. Mousavi Baigi SF, Moradi F, Vasseifard F, Abadi FM,
Mazaheri Habibi MR. The effect of nutrition training
51. Gong B, Nugent JP, Guest W, Parker W, Chang PJ, Khosa
on knowledge of students at university of medical
F, et al. Influence of artificial intelligence on Canadian
sciences. Topics in Clinical Nutrition. 2022; 37(3):
medical students' preference for radiology specialty:
A national survey study. Acad Radiol. 2019; 26(4): 236-41.
566-77. PMID: 30424998 DOI: 65. Mousavi Baigi SF, Aval RN, Sarbaz M, Kimiafar K.
10.1016/j.acra.2018.10.007 [PubMed] Evaluation tools for digital educational games: A
systematic review. Acta Medica Iranica. 2022; 60(8):
52. Wartman SA, Combs CD. Reimagining medical
508-12.
education in the age of AI. AMA J Ethics. 2019; 21(2):
E146-52. PMID: 30794124 DOI:

Volume 12 | Article 168 | Nov 2023 Page 10 of 11

 Artificial intelligence literacy among healthcare professionals and students Khalil Kimiafar et al.

66. Mousavi Baigi SF, Mousavi Baigi SM, Mazaheri Habibi 10.1002/hsr2.1156 [PubMed]
MR. Challenges and opportunities of using
69. Sarbaz M, Yazdipour AB, Mousavi Baigi SF, Darroudi
telemedicine during COVID-19 epidemic: A
A, Kimiafar K. Use of digital games for educational
systematic review. Frontiers in Health Informatics.
purposes among medical and paramedical sciences
2022; 11(1): 109.
students, Mashhad, Iran. Stud Health Technol Inform.
67. Ghaddaripouri K, Mousavi Baigi SF, Noori N, Mazaheri 2023; 302: 443-7. PMID: 37203713 DOI:
Habibi MR. Investigating the effect of virtual reality on 10.3233/SHTI230169 [PubMed]
reducing the anxiety in children: A systematic review.
70. Amiri FS, Bohlouly F, Khoshkangin A, Razmi N,
Frontiers in Health Informatics. 2022; 11(1): 114.
Ghaddaripouri K, Mazaheri Habibi MR. The effect of
68. Ghaddaripouri K, Mousavi Baigi SF, Abbaszadeh A, telemedicine and social media on cancer patients'
Mazaheri Habibi MR. Attitude, awareness, and self-care: A systematic review. Frontiers in Health
knowledge of telemedicine among medical students: Informatics. 2021; 10(1): 92.
A systematic review of cross‐sectional studies. Health
Sci Rep. 2023; 6(3): e1156. PMID: 36992712 DOI:

Volume 12 | Article 168 | Nov 2023 Page 11 of 11