Teaching Clinical Reasoning To Undergraduate Medical Students by Illness Script Method - A Randomized Controlled Trial
Teaching Clinical Reasoning To Undergraduate Medical Students by Illness Script Method - A Randomized Controlled Trial
Abstract
Background: The illness script method employs a theoretical outline (e.g., epidemiology, pathophysiology, signs and
symptoms, diagnostic tests, interventions) to clarify how clinicians organized medical knowledge for clinical reasoning
in the diagnosis domain. We hypothesized that an educational intervention based on the illness script method would
improve medical students’ clinical reasoning skills in the diagnosis domain.
Methods: This study is a randomized controlled trial involving 100 fourth-year medical students in Shiraz Medical
School, Iran. Fifty students were randomized to the intervention group, who were taught clinical reasoning skills based
on the illness script method for three diseases during one clinical scenario. Another 50 students were randomized to
the control group, who were taught the clinical presentation based on signs and symptoms of the same three diseases
as the intervention group. The outcomes of interest were learner satisfaction with the intervention and posttest scores
on both an internally developed knowledge test and a Script Concordance Test (SCT).
Results: Of the hundred participating fourth-year medical students, 47 (47%) were male, and 53 (53%) were female. On the
knowledge test, there was no difference in pretest scores between the intervention and control group, which suggested a
similar baseline knowledge in both groups; however, posttest scores in the intervention group were (15.74 ± 2.47 out of 20)
statistically significantly higher than the control group (14.38 ± 2.59 out of 20, P = 0.009). On the SCT, the mean score for the
intervention group (6.12 ± 1.95 out of 10) was significantly higher than the control group (4.54 ± 1.56 out of 10; P = 0.0001).
Learner satisfaction data indicated that the intervention was well-received by students.
Conclusion: Teaching with the illness script method was an effective way to improve students’ clinical reasoning skills in the
diagnosis domain suggested by posttest and SCT scores for specific clinical scenarios. Whether this approach translates to
improved generalized clinical reasoning skills in real clinical settings merits further study.
Keywords: Students, Medical, Education, Clinical reasoning, Script concordance test, Illness script
Background
Physicians use clinical reasoning skills to gather patient
data, combine it with their previous knowledge, and then
* Correspondence: [email protected]
2
make a clinical impression, diagnosis, and management
Clinical Education Research Center, Shiraz University of Medical Sciences,
plan [1, 2]. It is a critical skill in caring for patients ef-
Shiraz, Iran
3
Foundation for Advancement of International Medical Education and fectively and efficiently and must be a part of every
Research (FAIMER), Philadelphia, USA health professional education curriculum [3]. However,
Full list of author information is available at the end of the article
© The Author(s). 2021 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,
which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if
changes were made. The images or other third party material in this article are included in the article's Creative Commons
licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons
licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain
permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the
data made available in this article, unless otherwise stated in a credit line to the data.
Moghadami et al. BMC Medical Education (2021) 21:87 Page 2 of 7
according to a survey of 123 United States internal medi- medical education curriculum, including horizontal inte-
cine clerkship directors, medical students lack clinical rea- gration of basic science courses and 36 months of clin-
soning concepts. The authors recommended that all ical rotations. Graduates are qualified for medical
undergraduate medical education curriculum incorporate practice as general practitioners, but they may continue
a structured curriculum in the clinical reasoning field [4]. their education in specialties and subspecialties [10, 11].
The conceptual framework of script theory holds that hu- The sample size had been calculated to be 32 in each
man brains interpret the world by comparing the features of group, assuming a power of 90%, the confidence interval
the mental models it makes with a real scene’s structures, of 95%, standard deviation 5.89, and 4.04 to find a 3.6 dif-
checking for consistencies and inconsistencies, patterns, and ference in the groups’ mean. Regarding at least 20% of the
irregularities. Based on the script theory, expert clinicians missing, this number had increased to 50 in each group.
make a diagnosis by considering related differential diagnoses A total of hundred fourth-year medical students were
based on comparing and contrasting key features. These ex- selected randomly from all students who entered the di-
pert clinicians activate networks of prearranged knowledge, dactic classes in the internal medicine department to
called “illness scripts” [5]. The illness script method uses a participate in the randomized controlled trial. Fifty stu-
theoretical outline to clarify how medical diagnostic know- dents were randomized to either the control group or
ledge will be organized into different categories. These cat- the intervention group. We used the CONSORT state-
egories include epidemiology, the pathophysiology of ment about randomized controlled trials [12]. A diagram
diseases, symptoms, clinical signs, and interventions, leading of the study design is shown in Fig. 1. For ethical pur-
to an accurate diagnosis [6]. poses, after the initial intervention and measurements of
As novice learners, most medical students currently de- outcomes, students crossed over to the other group to
velop clinical reasoning skills informally in clinical wards ultimately receive the educational content in both for-
with varying degrees of supervision. They generally organize mats. Both groups attended teaching sessions (work-
their medical knowledge according to the components of shops) that lasted around 7 h, excluding breaks. Both
the curriculum. When making a diagnosis, medical students workshops were guided by an internal medicine expert
often use a process of hypothesis generation and try to test who was highly familiar with teaching clinical reasoning
one symptom at a time. skills in the diagnosis domain (third author). Several tu-
Teaching clinical reasoning by illness script model could tors also helped during small group sessions during both
help medical students acquire acceptable skills in generating workshops. Descriptions, timings, and agendas of each
differential diagnoses and clinical data interpretations [7]. group’s workshops are described in Table 1.
Different assessment methods like clinical scenario-based The workshop’s goal in the intervention group was to help
multiple-choice questions, extended matching questions, and students develop a correct problem representation from the
well-known clinical reasoning tests, such as the Script Con- patient’s clinical problem and organize data into three illness
cordance Test (SCT), may provide reliable evidence of med- scripts; this was based on a clinical scenario adapted from a
ical students’ clinical reasoning skills in the diagnosis domain study by Levin et al. published in MedEdPortal [13]. .Details
[8]. SCT is one assessment of clinical reasoning skills that on the case and facilitator guide are provided in Supplemen-
emphasizes data interpretation by asking learners to estimate tary Appendix 1. To help students compare and contrast the
the impact of new information on a suggested hypothesis. findings, the ‘think aloud’ method was also used in the inter-
The SCT is based on the illness script method, which was vention group [14]. An external observer, familiar with the
developed in the field of cognitive psychology. The SCT clinical reasoning, observed both intervention and control
measures the progress of illness scripts method in medical group workshops, and confirmed using the facilitator’s guide
students as novice learners by comparing their performance by the facilitator and tutors.
on this test to a panel of expert physicians [9].
To better prepare the students for clinical rotations, we Evaluation methods
designed a brief educational intervention around clinical Knowledge test
reasoning in the diagnosis domain. We hypothesized that Before conducting the lecture and illness script sessions,
an education intervention based on the illness script the researchers designed ten multiple choice-questions
method would improve students’ clinical reasoning skills in for pretest and posttest. Each question measured a spe-
the diagnosis domain. cific teaching point. In both groups, the pretest was done
before, and the posttest was done after the teaching ses-
Methods sions. Each question had 2 points, and the total score for
Study design and participants each of the pretest and posttest was 20. Both the pretest
This Ranomized controlled trial was conducted at Shiraz and posttest were completed on paper in a proctored
Medical School, which was established in 1952 in south and closed-book setting. The scores were measured
Iran. The medical school has a seven-year undergraduate using an answer key that was developed before the
Moghadami et al. BMC Medical Education (2021) 21:87 Page 3 of 7
administration of the tests. The scorers of the pretests appropriate diagnosis option; (2) a new clinical finding; and
and posttests were blinded to the intervention. (3) a five-point Likert scale from - 2 to + 2 that should be
chosen by examinees [16]. A sample of SCT is shown in
Script concordance test Table 2. We invited ten internal medicine expert faculties
Using SCT development guidelines developed by Bernard to answer the SCT.
Charlin, internal medicine experts at Shiraz medical school The correct answer for an SCT was weighted
developed ten Script Concordance Tests (SCTs) based on based on expert response. The credit for the best
the illness scripts of these three important diseases (neph- answer was 100%, and credit for other answers was
rotic syndrome, cirrhosis, and congestive heart failure) [15]. calculated based on the expert panel’s percentage
Our SCT case-based vignettes and questions were designed who chose that answer [17]. Each SCT had 1 point,
to evaluate clinical reasoning ability in the diagnostic domain and the total score of all of the SCT was 10. The
for early medical learners. Each describes a short scenario students in the intervention and control group par-
followed by questions presented in three parts: (1) an ticipated in this SCT 4 weeks after the intervention.
Table 2 SCT sample scenario. A 50 years old female presents with both lower extremities edema since the last ten days
If you think about And you find This hypothesis becomes
q1 Nephrotic syndrome Proteinuria > 3 g/ 24 hours −2 -1 0 + 1 + 2
q2 Liver Cirhosis jaundice −2 -1 0 + 1 + 2
q3 Congestive heart failure orthopnea -2 -1 0 + 1 + 2
− 2:The hypothesis is almost eliminated, 1-:The hypothesis becomes less probable
0:The information does not affect the hypothesis
+ 1:The hypothesis becomes more probable, + 2:The hypothesis is almost approved
Satisfaction survey register with the Iranian Registry of Clinical Trials. In-
After the SCT administration, students crossed over formed written consent to participate was obtained from
from the intervention group to the control group and all participants. Participants joined the study voluntarily,
vice versa. All medical students in both groups com- and their scores remain confidential.
pleted a satisfaction questionnaire/survey about the ill-
ness script method after participating in the teaching by
illness script workshop. The questionnaire had 14 items/ Results
questions and was designed based on our previous ques- Of the hundred participating students, 47 (47%) were
tionnaires about educational workshops’ satisfaction and male, and 53 (53%) were female. The effect size was 2.04
one other study in the illness script teaching method for pretests and posttests in the intervention group, and
[10, 18, 19]. Students rated each item using a Likert scale 1.99 for the control group. There was no difference in
(1 = strongly disagree to 5 = strongly agree). Medical pretest scores between the intervention and control
education experts determined the validity of the ques- group by student t-test (10.87 + 2.49 vs. 9.84 + 2.85, p =
tionnaire using the modified Kappa variation coefficient 0.083) which was suggestive of similar baseline know-
[20]. The modified Kappa coefficient was 0.75, and the ledge in both groups. On the knowledge test, the mean
reliability was established after a pilot study (r = 0.87). posttest scores (14.38 + 2.59 in control group & 15.74 +
2.47 in intervention group) were higher than the pretest
Statistical analysis scores (9.84 + 2.85 in control group & 10.87 + 2.49 in
Data were analyzed with descriptive and analytic statis- intervention group) in both groups, and this difference
tics such as paired t-test using SPSS, version 16. The by paired t-test was statistically significant (p = 0.0001 in
alpha level was at 0.05. The effect size was measure by both groups). The comparison of posttest scores by stu-
Cohen’s d method [21]. dent t-test in the intervention group was significantly
higher than the control group (p = 0.009). The effect size
Ethical consideration for the difference between posttests was 0.54 that is a
The Ethics Committee approved of Shiraz University of moderate effect size. (Table 3). The intervention group’s
Medical Sciences approved our study by ethical code mean score was significantly higher on the SCT than the
number IR.SUMS.REC.1397.470 and did not need to control group (6.12 ± 1.95 vs. 4.54 ± 1.56, p = 0.0001).
Table 3 Comparison of pretest and posttest in each group by pair T-test and between groups by students T-test
Groups Number Mean ± SD p-value Cohen’s D effect size
Intervention Group
Pretest 50 10.87 ± 2.49 0.0001* 2.04
Posttest 50 15.74 ± 2.47
Control Group
Pretest 50 9.84 ± 2.85 0.0001* 1.99
Posttest 50 14.38 ± 2.59
Pre-test
Intervention Group 50 10.87 ± 2.49 0.083** 0.38
Control Group 50 9.84 ± 2.85
Post-test
Intervention Group 50 15.74 ± 2.47 0.009** 0.54
Control Group 50 14.38 ± 2.59
*
Pair t-test, ** Independent t-test
Moghadami et al. BMC Medical Education (2021) 21:87 Page 5 of 7
Our satisfaction survey results indicated that the inter- students showed that this education would increase stu-
vention was generally well-received by students (Table 4). dents’ participation in the learning process [23].
Most students (82%) strongly agreed or agreed that the Like the knowledge test, the SCT result showed a
tutor gave them appropriate feedback. Most students higher intervention group score than the control group.
(80%) also believed that the illness script method em- However, medical students’ SCT average score in the
phasized learning, and 78% of students reported that intervention groups was around 60% of the total possible
they were overall satisfied with the workshop. When stu- score, suggesting less proficiency even in the interven-
dents were asked how they would improve the work- tion group. These low scores might be due to their first
shop, the main suggestion was that the workshop should learning experience with the illness script method and
stress more on the “thinking aloud” approach. less clinical experience.
Other studies have used SCT for the assessment of clinical
reasoning skills. The SCT assesses illness scripts’ formation
Discussion in medical students by comparing their answers on this test
This randomized controlled trial aimed to identify the ef- to a panel of experts’ responses. In our previous studies, SCT
fect of teaching clinical reasoning skills for the diagnosis has been described as a valid and reliable assessing tool in
domain based on the illness script method. Despite the ill- the clinical reasoning field [24–27]. Compared to other stud-
ness script workshop’s briefness in the intervention group ies, we had performed SCT 4 weeks after original workshops,
and the lecturing in the control group, both the illness which helped to understand retention of knowledge for some
script method and lecturing appeared effective. Both time compared to the immediate posttest, which was better
groups showed significant improvement in posttest scores in the intervention group than the control group.
in comparison with pretest scores. The findings also Like our study, several studies have shown the effect-
showed a high effect size between pretests and posttests in iveness of teaching clinical reasoning skills during the
both groups. The results that students improved on the formal curriculum. Lee et al.’s study about teaching clin-
posttest are likely unsurprising as most success in clinical ical reasoning to medical students showed that the inter-
reasoning is attributable to knowledge gained [22]. vention group’s students scored better than the control
Our study also showed that illness script teaching inter- group on clinical reasoning tests named clinical reason-
vention helps medical students earn better scores in post- ing problems [18]. Another study by Delavari et al. about
test in contrast to the control group. This better score the educational strategies inspired by theory in develop-
maybe because teaching by illness scripts method helps ing illness scripts revealed that the medical students’
students recognize the standard and discriminating fea- scores on clinical reasoning problem tests improved after
tures in the intervention group better than the control the intervention [28]. Another study by Keemink et al.
group with a medium effect size between posttests in the about illness script development in medical students
intervention and control groups. A study by Linsen et al. showed that case-based teaching would foster the illness
about teaching clinical reasoning to first-year medical script’s richness [29].
Table 4 Results of the medical students’ satisfaction with the illness script workshop
No Statement A B C D
1 I gained a good understanding of concepts in the clinical reasoning field 72 (72%) 20 (20%) 6 (6%) 2 (2%)
2 The workshop encouraged me to improve my clinical reasoning ability in the future 69 (69%) 20 (20%) 8 (8%) 3 (3%)
3 The learning objectives of the workshop were clearly defined. 72 (72%) 26 (26%) 0 (0%) 2 (2%)
4 The amount of material delivered in the workshop was reasonable. 70 (70%) 23 (23%) 6 (6%) 1 (1%)
5 The level of difficulty of the workshop was appropriate. 63 (63%) 24 (24%) 12 (12%) 1 (1%)
6 The program was a good learning experience for me 74 (74%) 16 (16%) 9 (9%) 1 (1%)
7 The illness script model was useful for improving my clinical reasoning skills. 61 (61%) 23 (23%) 15 (15%) 1 (1%)
8 This type of education emphasized learning 80 (80%) 12 (12%) 7 (7%) 1 (1%)
9 The tutor gave us appropriate feedback 82 (82%) 14 (14%) 3 (3%) 1 (1%)
10 The “thinking aloud” method helped improve my clinical reasoning skills. 56 (56%) 22 (22%) 16 (16%) 6 (6%)
11 The workshop was boring and wasted my time 4 (4%) 10 (10%) 86 (86%) 0 (0%)
12 The students’ participation was encouraged 74 (74%) 23 (23%) 3 (3%) 0 (0%)
13 I recommend this type of teaching for other sign and symptoms 76 (76%) 18 (18%) 4 (4%) 2 (2%)
14 Overall, I am satisfied with the course. 78 (78%) 14 (14%) 6 (6%) 2 (2%)
A = Strongly agree/Agree, B=Neutral, C=Disagree/Strongly disagree, D = Missing
Moghadami et al. BMC Medical Education (2021) 21:87 Page 6 of 7
The satisfaction questionnaire results showed that the Research (FAIMER) project by the second author, Mitra Amini, a professor of
students were satisfied overall with the intervention. Shiraz University of Medical Sciences and a FAIMER fellow. The fourth author,
Bhavin Dalal, guided this project at the FAIMER Institute. This study was
They were also satisfied with the tutors’ appropriate supported in part by the Foundation for Advancement of International
feedback and believed that this course would lead to real Medical Education and Research. However, the findings and conclusions do
learning. not necessarily reflect the opinions of this organization. We thank the faculty
members and medical students at Shiraz Medical School for their
Our study’s most important strength was a random- participation in this study. We give our special thanks to all FAIMER faculty
ized controlled nature, which can eliminate several con- members and fellows for their continuous support of this project. We also
founding factors. Additionally, assessment of learning thank Jennifer Wilson at Jefferson (Philadelphia University + Thomas
Jefferson University) for editorial assistance.
was done by multiple methods like knowledge test, SCT,
and satisfaction survey showed positive results. Another Authors’ contributions
study’s strength was assessing intermediate-term know- All authors contributed to the study’s commencement and coordination,
collected data, and drafted the manuscript. MM1, MA, and MM2 participated
ledge retention tested by SCT 4 weeks after the original
in data collection, analysis, and writing of the manuscript. BD and BC
workshop. There are some limitations to the present participated in the study’s supervision, interpretation of data, and revising
study. We might have prepared the students for tests the manuscript. All authors read and approved the final manuscript.
during teaching, especially with the same instructor and
Funding
tutors for both groups; however, we used the help of an All of the funds of the present study (including the funds for performing
external observer to monitor the educational sections to teaching sections) were provided by the vice-chancellor of research at Shiraz
reduce this limitation. Another limitation is that SCT University of Medical Sciences.
was not a part of baseline testing, and we only used this
Availability of data and materials
test for posttests. The datasets used and analyzed during the current study are available from
Third, while some changes are statistically significant, the corresponding author on request.
we cannot ascertain whether these are educationally sig-
nificant. Because the results are symptom and sign- Ethics approval and consent to participate
The Ethics Committee approved this study of Shiraz University of Medical
specific, we cannot conclude that medical students de- Sciences by ethical code number IR.SUMS.REC. 1397.470 and did not require
velop better clinical reasoning skills in general, nor can registration with the Iranian Registry of Clinical Trials. Informed written
we be sure of the clinical reasoning skills application consent to participate was obtained from all students. Students participated
in the study voluntarily, and their scores remain confidential.
during actual clinical practice. Third, this study is a
single-center study for a specific group of learners with Consent for publication
small sample sizes, so generalizability is limited. Not applicable.
Competing interests
Conclusion The authors declare that they have no competing interests. Mitra Amini is
Teaching with illness scripts method provided an effect- the associate editor of the BMC Medical Education Journal, but there is no
competing interests to declare.
ive way to improve students’ clinical reasoning scores
for diagnosis domain posttest and SCT. Whether this Author details
1
approach translates to improved clinical reasoning skills 2
Shiraz Medical School, Shiraz University of Medical Sciences, Shiraz, Iran.
Clinical Education Research Center, Shiraz University of Medical Sciences,
in real clinical settings merits further study. Our findings Shiraz, Iran. 3Foundation for Advancement of International Medical Education
can serve as a rationale for implementing clinical reason- and Research (FAIMER), Philadelphia, USA. 4Noncommunicable Disease
ing education modules for undergraduate medical edu- Research Center, Shiraz University of Medical Sciences, Shiraz, Iran. 5Oakland
University William Beaumont School of Medicine, Rochester, MI, USA.
cation curricula to empower medical students in clinical 6
Medical School, University of Montreal, Montreal, Canada.
reasoning skills.
Received: 11 May 2020 Accepted: 27 January 2021
Supplementary Information
The online version contains supplementary material available at https://doi.
References
org/10.1186/s12909-021-02522-0.
1. Eva KW. What every teacher needs to know about clinical reasoning. Med
Educ. 2007;39:98–106.
Additional file 1. 2. Young M, Thomas A, Lubarsky S, Ballard T, Gordon D, Gruppen LD, et al.
Drawing boundaries: the difficulty in defining clinical reasoning. Acad Med.
2018;93(7):990–5.
Abbreviation 3. Van der Vulten CPM, Newble DI. How can we test clinical reasoning?
SCT: Script Concordance Test Lancet. 1995;345:1032–4.
4. Rencic J, Trowbridge RL, Fagan M, Szauter K, Durning S. Clinical reasoning
Acknowledgments education at US medical schools: results from a national survey of internal
This study is a part of a thesis by the first author, Mana Moghadami, for medicine clerkship directors. J Gen Intern Med. 2017;32(11):1242–6.
obtaining a Medical Doctor’s degree in Shiraz Medical School. This study was 5. Bowen JL. Educational strategies to promote clinical diagnostic reasoning. N
approved and financially supported by Shiraz University of Medical Sciences, Engl J Med. 2006;355:2217–25.
Shiraz, Iran, with an ethical code: IR.SUMS.REC.1397.470. This study was also a 6. Custers EJFM. Thirty years of illness scripts: theoretical origins and practical
Foundation for Advancement of International Medical Education and applications. Med Teach. 2014;37(5):457–62.
Moghadami et al. BMC Medical Education (2021) 21:87 Page 7 of 7
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.