Teaching Ophthalmoscopy to Medical Students

(the TOTeMS Study)

LINDA P. KELLY, PHILIP S. GARZA, BEAU B. BRUCE, EMILY B. GRAUBART, NANCY J. NEWMAN, AND
VALÉRIE BIOUSSE

PURPOSE: To determine medical student preferences 2013;156:1056–1061. Ó 2013 by Elsevier Inc. All
for learning the ocular fundus examination and to assess rights reserved.)
their accuracy using different examination modalities.

DESIGN: Prospective, randomized study of medical

E
student education approaches. XAMINATION OF THE OCULAR FUNDUS IS CRITICAL

METHODS: First-year medical students received to the accurate diagnosis of many life- and sight-
training in direct ophthalmoscopy using simulators and threatening medical conditions,1,2 and there is
human volunteers. Students were randomized to receive consensus that all graduating medical students and
vs not receive specific training on interpreting fundus generalist physicians should be proficient in the fundus
photographs prior to accuracy assessments. Students’ examination.3–5 Standards adopted by the Association of
preferences for each of the 3 methods (direct ophthalmos- University Professors in Ophthalmology (AUPO) and
copy on simulators or human volunteers, or use of fundus endorsed by the American Academy of Ophthalmology
photographs) and recognition of normal and abnormal and International Council of Ophthalmology5 specifically
fundus features were assessed. require students to be able to visualize the red reflex,

RESULTS: Of 138 first-year medical students, 119 the retina, and optic disc; to assess the optic disc for
(86%) completed all required elements. For learning cupping, color, contour, margins, vessels, and edema;
ophthalmoscopy, 85 (71%) preferred humans to simula- and specifically to recognize changes associated with glau-
tors. For learning relevant features of the ocular fundus, coma and macular degeneration.3 Despite these recom-
92 (77%) preferred photographs to ophthalmoscopy on mendations, the fundus examination is performed
simulators or humans. Accuracy of answers was better infrequently and poorly by students and most non-
when interpreting fundus photographs than when ophthalmologists.1,6,7
performing ophthalmoscopy on simulators (P < .001). In response to students’ lack of proficiency in direct
Performance improved after specific teaching about ophthalmoscopy, medical educators have primarily sug-
assessing fundus photographs before testing (P [ .02). gested 2 divergent approaches to the improvement of
Examination of the ocular fundus was found easier and undergraduate ophthalmology education. Some have
less frustrating when using photographs than when using attempted intensive, longitudinal ophthalmology training
ophthalmoscopy on simulators or humans. Eighty-four in medical school, but this has failed to produce a meaning-
students (70%) said they would prefer to have fundus ful improvement in students’ direct ophthalmoscopy skills,
photographs instead of using the ophthalmoscope during and students continued to neglect the fundus examination
upcoming clinical rotations. in their internal medicine clerkships.8 As an alternative,

CONCLUSIONS: Students preferred fundus photographs others have suggested that students master only basic direct
for both learning and examining the ocular fundus. Iden- ophthalmoscopy skills and instead focus on learning the
tification of ocular fundus features was more accurate on signs of clinical ophthalmology emergencies and be aware
photographs compared to examination by direct ophthal- of other forms of ocular fundus pathology, even those
moscopy. In the future, the increasing availability of that are not readily detectable by direct ophthalmoscopy.9
nonmydriatic ocular fundus photography may allow However, new ocular fundus imaging technologies, such as
replacement of direct ophthalmoscopy in many clinical nonmydriatic ocular fundus cameras, provide easy-to-use
settings for non-ophthalmologists. (Am J Ophthalmol and reliable alternatives to direct ophthalmoscopy and
may allow convergence of these 2 approaches. Removing
the technical challenges of direct ophthalmoscopy should
make examination of the ocular fundus easier and more
Supplemental Material available at AJO.com. effective, providing students additional time (currently
Accepted for publication Jun 12, 2013.
From the Departments of Ophthalmology (L.P.K., P.S.G., B.B.B., spent on learning how to use the direct ophthalmoscope)
E.B.G., N.J.N., V.B.), Neurology (B.B.B., N.J.N., V.B.), and to learn key ophthalmologic signs and pathology.
Neurological Surgery (N.J.N.), Emory University, Atlanta, Georgia. We propose that tomorrow’s clinicians may use ocular
Inquiries to Valérie Biousse, Neuro-Ophthalmology Unit, Emory Eye
Center, 1365-B Clifton Road NE, Atlanta, GA 30322; e-mail: vbiouss@ fundus photography instead of direct ophthalmoscopy for
emory.edu routine ophthalmic screening in appropriate clinical

1056 Ó 2013 BY ELSEVIER INC. ALL RIGHTS RESERVED. 0002-9394/$36.00

http://dx.doi.org/10.1016/j.ajo.2013.06.022
settings, and we sought to explore the integration of digital Introductory lecture and pre-test
fundus photography into medical student education.
Training sequence 1 Training sequence 2
The aim of our study was to evaluate the capabilities of Skill training with human volunteer Skill training with simulator
students to identify major anatomic features of the ocular
Quality survey
fundus on eye simulators vs fundus photographs and to
determine student preferences regarding educational tech- Skill training with simulator Skill training with human volunteer
niques (humans vs eye simulators vs fundus photographs)
and examination methods (direct ophthalmoscopy vs Quality survey

fundus photography). Testing Testing Testing Testing

sequence 1 sequence 2 sequence 1 sequence 2
Diagnostic Diagnostic
Simulator and Simulator and
training training
photograph photograph
(photograph (photograph
post-tests post-tests
slideshow) slideshow)
METHODS
Diagnostic Diagnostic
Simulator and Simulator and
training training

STUDY POPULATION: This was a prospective, random- photograph
(photograph
photograph
(photograph
post-tests post-tests
slideshow) slideshow)
ized study that was deemed exempt by the Emory Univer-
sity Institutional Review Board. The study was in Quality survey
adherence with the tenets of the Declaration of Helsinki.
Participants were all first-year medical students at Emory FIGURE 1. Study protocol and student flow for teaching
University. Inclusion criteria required informed consent, ophthalmoscopy to medical students. Students attended an
attendance at an introductory eye examination lecture, introductory lecture followed by a pretest and were then
and a small-group skill training session in which students randomized into 1 of 2 training sequences and 1 of 2 testing
sequences. Quality surveys were conducted after each training
received further eye examination training and practiced
and testing section. Skill training with human volunteers and
direct ophthalmoscopy.
simulators required direct ophthalmoscopy.

OVERVIEW OF PROTOCOL: First-year medical students
attended a 45-minute introductory lecture on the eye standardized instruction on how to use the direct ophthal-
examination. Subsequent instruction and evaluation was moscope. Each group was randomized into 1 of 2 training
conducted in 2-hour small-group sessions. Each group was sequences (Figure 1). Training sequence 1 consisted of
randomized into 1 of 2 training sequences (direct ophthal- direct ophthalmoscopy skill training on healthy human
moscopy skill training on human volunteers and then on volunteers (each had 1 dilated pupil to allow students to
anatomically and optically correct eye simulators, or vice practice ophthalmoscopy with and without pupillary dila-
versa). The groups were then randomized into 1 of 2 testing tion) and then on anatomically and optically correct eye
sequences, in which they either received or did not receive simulators. Training sequence 2 consisted of direct
specific training on assessing fundus photographs prior to ophthalmoscopy skill training in the opposite order (eye
testing their abilities to identify ocular fundus features. simulators first, and then volunteers).
Testing and surveys were performed at predefined points Eye simulators were placed in Styrofoam models in the
during the protocol. The protocol is outlined in Figure 1. shape of a human head in order to reproduce obstacles to
ophthalmoscopy, such as the nose and hair (Figure 2).

PRETEST PHASE: The introductory lecture for the first- Each eye simulator was constructed with an ocular fundus
year medical school class taught visual pathway anatomy photograph glued to the inside bottom of a white polyeth-
and the screening eye examination, including a short ylene canister (similar to a photograph film canister).10 A
demonstration of direct ophthalmoscopy. Students then 16-diopter convex lens was inserted in the mouth of the
completed a 48-item pretest, which collected demographic canister in order to reproduce an optically correct eye. A
data and information on prior exposure to ophthalmology; cover with a hole cut for a ‘‘pupil’’ was placed over the
baseline diagnostic skills in interpreting fundus features lens. Fundus photographs were correctly oriented in the
were also determined by presenting 4 ocular fundus photo- heads to simulate right or left eyes. Each head (including
graphs with questions about the appearance of the optic 2 eye simulators) was accompanied by a printed clinical
nerve, retina, and blood vessels (Supplemental Figure 1, vignette that presented a short clinical history appropriate
available at AJO.com). to its fundus findings and included fundus photographs
identical to those inside the eye simulators, which high-

TRAINING SEQUENCES: For the remainder of the study, lighted important features to identify.
students were assembled in 16 groups of 8-10 students. Twenty minutes were allotted for direct ophthalmoscopy
Each group was taught by an ophthalmology-trained training on humans and 20 minutes for the simulators.
faculty member and training was standardized in each After each skill training portion, a quality survey was
room. Each of the 16 sessions began with 5 minutes of administered. Students were asked to rate, on 10-point

VOL. 156, NO. 5 TEACHING OPHTHALMOSCOPY TO MEDICAL STUDENTS 1057

FIGURE 2. Patient simulator with optically correct ‘‘eyes’’ used to teach ophthalmoscopy to medical students. (Left) Styrofoam
model in the shape of a human head. (Middle) Ocular fundus photograph at the base of a plastic canister. (Right) Canister for optically
correct eye containing a 16-diopter convex lens capped with a cover containing a hole cut to simulate the pupil.

Likert scales, ease of viewing the ocular fundus with an students completed the simulator posttest, photograph
ophthalmoscope (10 ¼ easiest) and level of frustration in posttest, and post-photograph quality survey before
attempting direct ophthalmoscopy (10 ¼ most frustrated). receiving the diagnostic training slideshow.
They were also asked whether lack of time was the main
source of frustration and whether they would spontaneously
FINAL SURVEY: A final quality survey was administered
perform direct ophthalmoscopy routinely during their after all training and testing sequences were complete
upcoming clinical clerkships. The quality surveys also (Supplemental Figure 5, available at AJO.com). This
included the validated Positive and Negative Affect quality survey asked which methods students preferred for
Schedule (PANAS), consisting of 10 negative and 10 posi- learning how to use the direct ophthalmoscope (human
tive mood terms graded on a 5-point Likert response scale volunteers vs eye simulators with clinical vignettes) and
to document feelings engendered by the portion of training for identifying features of the ocular fundus (ophthalmos-
just completed (Supplemental Figure 2, available at AJO. copy vs ocular fundus photographs). It also asked whether
com).11 Scores were calculated by a simple sum of the 5- they would prefer to use direct ophthalmoscopy or fundus
point Likert ratings for each mood term, as per the usual photographs when evaluating a patient during their clin-
method.11 Positive affect scores range from 10-50, with ical clerkships, and how often they believed they would
higher scores representing higher levels of positive affect. attempt to evaluate the ocular fundus over the following
Negative affect scores also range from 10-50, with lower year as part of general physical examinations.
scores representing lower levels of negative affect.

DATA ANALYSIS: Medians and interquartile ranges

TESTING SEQUENCES: The students’ small groups were (IQRs) were reported for continuous data and percentages
randomized into 1 of 2 testing sequences (Figure 1). Testing were reported for categorical data. Statistical analysis was
sequence 1 began with a 10-minute diagnostic training performed using random intercept (for subject, instructor,
slideshow with instruction on how to determine if a fundus and day) mixed linear regression models for comparing
photograph depicted the right or left eye and how to assess test scores and 1-way analysis of variance with Tukey
basic features of the optic nerve, retina, and blood vessels. post hoc comparisons for affect scores using R: A Language
Students then completed a simulator posttest and a photo- and Environment for Statistical Computing (R Foundation
graph posttest as well as a quality survey. The simulator for Statistical Computing, http://www.R-project.org).
posttest consisted of 4 simulator eyes; for each eye, students
were asked if they could visualize anything inside the eye
simulator (1 of the 4 eyes was a picture of a vitreous hemor-
rhage where nothing could be visualized) and were ques-
tioned on the appearance of the optic nerve, retina, and RESULTS
blood vessels (Supplemental Figure 3, available at AJO.
com). The photograph posttest consisted of 4 fundus OF 138 FIRST-YEAR MEDICAL STUDENTS ENROLLED IN THE
photographs printed on paper, 2 of which were identical Emory School of Medicine 2016 class, 132 consented to
to fundus photographs in the simulator posttest participate in the study and 119 (86%) completed all
(Supplemental Figure 4, available at AJO.com). Fifteen required elements. Fifty-nine of the 119 (50%) were
minutes were allotted for each posttest. The post- women and the median age was 23. Eight students had
photograph quality survey asked the same questions as completed prior courses related to ophthalmology (7
the post-simulator quality survey. In testing sequence 2, undergraduate and 1 graduate).

1058 AMERICAN JOURNAL OF OPHTHALMOLOGY NOVEMBER 2013


PERFORMANCE ACCURACY: The pretest and posttests
(simulator and photograph) each had 48 items. Students
answered an average of 28.8 questions correctly on the
pretest (60%). They answered an average of 8.2 additional
questions correctly on the simulator posttest (77% correct
of 48) and 11.9 additional questions on the photograph
posttest (85% correct of 48), both significantly better
than the pretest (P < .001). Performance on the posttest
was significantly better using fundus photographs compared
to simulators (P < .001).
Students who received the diagnostic training slideshow
before completing the simulator and photograph posttests
scored an average of 1.7 more responses correctly than
students who received the diagnostic training after the
posttests (P ¼ .02). The order of training (human volun-
teers followed by simulators or vice versa) had no impact
on posttest scores (0.38 more questions correct for training
on simulators before human volunteers, P ¼ .61).

STUDENT PREFERENCES: Student-rated ease and frustra-

tion for the different examination modalities (human volun-
teers, simulators, and photographs) are shown in Figure 3.
Limited time was identified as the primary source of frustra-
tion in examining human volunteers for 18 students (15%),
in examining simulators for 21 students (18%), and in
reviewing fundus photographs for 1 student (1%).
Eighty-five students (71%) preferred human volunteers
to simulators for learning to use the direct ophthalmoscope,
FIGURE 3. Student-rated ease and frustration on 10-point
and 92 (77%) preferred photographs to simulators for
Likert scales for different modalities of ocular fundus examina-
learning the features of the ocular fundus. When given tion used by medical students. (Top) Median ease (10 [ easiest)
the choice between using the direct ophthalmoscope and of examination was 5 for human volunteers (interquartile range
directly looking at fundus photographs to assess the ocular 3-7), 7 for simulators (5-8), and 9 for photographs (8-10).
fundus, 84 (70%) indicated a preference for fundus photo- (Bottom) Median frustration level (10 [ most frustrated) was
graphs. Forty-nine percent of students indicated that they 4 for human volunteers (2-6), 3 for simulators (2-5), and 2.5
would attempt direct ophthalmoscopy 40% of the time or for photographs (2-5).
less as part of a general physical examination during clin-
ical encounters over the next year.
the ocular fundus of simulators with a direct ophthalmo-

POSITIVE AND NEGATIVE AFFECT: Positive affect scores scope. This better performance was obvious even for the
were highest for training on human volunteers (mean 31.5) group of students who did not receive specific training in
compared to 28.9 for simulators and 25.9 for photographs fundus photography. Not surprisingly, specific teaching
(all P < .001). Higher positive affect scores reflect more regarding assessment of fundus photographs resulted in
pleasurable engagement. Negative affect scores were high- additional improvement in student capability. Addition-
est for training on human volunteers (mean 13.8), ally, 70% of students indicated a preference for fundus
compared to 13.2 for simulators (P < .03) and 12.1 for photographs over direct ophthalmoscopy.
photographs (P < .001 vs human volunteers and Our results suggest that direct ophthalmoscopy may be
P ¼ .001 vs simulators). Higher negative affect scores underutilized primarily because of technical barriers,
reflect negative activation, such as irritability. despite widespread acknowledgement that the ocular
fundus examination is a key component of the general
physical examination. Numerous studies have documented
the underuse of direct ophthalmoscopy. In one study, for
DISCUSSION example, third-year American medical students attempted
direct ophthalmoscopy only 11% of the time a fundus
WE FOUND THAT FIRST-YEAR MEDICAL STUDENTS WERE examination was clinically indicated.12 In a survey of
significantly more accurate at identifying ocular fundus physicians practicing general and geriatric medicine in
features using fundus photographs than when looking at a hospital setting, all felt that ophthalmoscopy was

VOL. 156, NO. 5 TEACHING OPHTHALMOSCOPY TO MEDICAL STUDENTS 1059

important but only 3 of 72 performed it routinely. Half of only. In an assessment of third-year medical student perfor-
these physicians indicated they would perform ophthal- mance of direct ophthalmoscopy, only 32% of students
moscopy for patients with diabetes, hypertension, visual accurately described at least 1 feature of the optic disc,
impairment, and neurologic symptoms, but on review of such as cup, color, or margins; this rose to 84% in fourth-
100 case notes, ophthalmoscopy was documented on only year students after ophthalmology retraining.12 In another
3 patients; 9 with diabetes and 35 with hypertension had study, supplemental formal instruction in ophthalmoscopy
no ophthalmoscopy reported.7 In a study of pediatric physi- resulted in improved performance by medical students
cians at a university-affiliated hospital, 9 of 11 residents and compared to their peers and the increased competency
1 of 5 senior physicians responded ‘‘one to three times’’ and persisted when reassessed 2 months later.17 However, these
the rest said ‘‘never’’ when asked how many times they had types of short-term gains were not sustained during a more
examined an ocular fundus during the last year.1 In a study intensive program, which implemented a longitudinal
of the feasibility of nonmydriatic fundus photography in curriculum for ophthalmic screening examination skills
the emergency department (FOTO-ED study), only 14% across 3 years of medical school, with skill retention
of patients with conditions for which ophthalmoscopy is declining about 6 months after reinforcement in the curric-
considered important, such as headache or acute visual ulum.8 In an internet-based assessment of ophthalmoscopy
change, had direct ophthalmoscopy performed by emer- skills, medical students were tested on identifying the optic
gency room physicians.6 In a study asking patients to recall disc of one of their colleagues from an array of 15 fundus
being examined with use of a tendon hammer, ophthalmo- photographs.18 Another group also attempted to improve
scope, and stethoscope, nearly half of patients could not direct ophthalmoscopy by providing their students with
recall having ophthalmoscopy performed, while 95.7% an ocular fundus photograph of one of their peers whom
recalled being examined with a stethoscope.13 they were challenged to identify via direct ophthalmos-
These and related studies have shown not only infrequent copy; this encouraged practice of direct ophthalmoscopy
direct ophthalmoscope use, but also poor performance when with detailed observation of ocular fundi during their rota-
the ophthalmoscope is used. Indeed, most primary care tion, and 84.3% of the students trained in this manner were
interns are unable to perform an adequate screening eye able to identify a peer’s optic nerve photograph vs only
examination at the start of their postgraduate training. In 28.6% of students in the control group.19 However, no
a survey of program directors for 135 American family prac- longitudinal follow-up was provided.
tice, internal medicine, and pediatrics residency programs, In our study, positive affect scores were highest for
nearly 90% of program directors believed that fewer than training on human volunteers, reflecting pleasurable
50% of their entering residents met the standard set by engagement and interest in the activity. Interestingly,
the AUPO for a minimal level of competence expected negative affect scores were also highest for training on
when dealing with ophthalmologic problems.14 Further- human volunteers compared to use of simulators or fundus
more, in another study, hospital internists correctly diag- photographs, likely a reflection of the higher frustration
nosed ocular fundus abnormalities less than 50% of the level and greater perceived difficulty of learning on
time using direct ophthalmoscopy.7 In the FOTO-ED study, humans. Overall, however, students preferred human
emergency department physicians failed to recognize rele- volunteers to simulators for learning how to use the direct
vant ocular fundus abnormalities each time they attempted ophthalmoscope. The fact that viewing ocular fundus
to use the ophthalmoscope.6 Given the limited capabilities photographs had the lowest scores for both positive and
of most trainees and physicians with direct ophthalmos- negative affect (with the human-like simulators in the
copy, it is not surprising that they also lack self-assurance middle) may reflect the paucity of overall emotional
in performing this skill. In fact, 47% of medical student response engendered by an inert subject such as simulators.
clerks at one Canadian university expressed minimal confi- Although ocular fundus photographs produced little
dence in their ability to use direct ophthalmoscopy to affective change, students in the current study reported
examine the ocular fundus through an undilated pupil,15 that among the examination modalities used, they experi-
and 43% of general practitioners surveyed in Great Britain enced the greatest ease and least frustration with ocular
lacked confidence in using the direct ophthalmoscope.16 fundus photographs. The majority of students also preferred
In recognition of the poor accuracy and lack of confi- fundus photographs for learning features of the ocular
dence in performing direct ophthalmoscopy, a variety of fundus and for patient evaluation.
approaches have been attempted to improve ophthalmos- The increasing use of nonmydriatic fundus photography
copy education. A change in the past decade included outside of ophthalmology suggests that digital photography
the creation of the PanOptic ophthalmoscope, which may emerge as an alternative diagnostic method to direct
provides a wider field of view and may be easier to use ophthalmoscopy. Indeed, the FOTO-ED study showed
than the classic direct ophthalmoscope. However, only that nonphysician emergency department staff could be
a minority of students and practitioners are routinely using trained to obtain high-quality fundus photographs using
this more cumbersome device, and this is why we chose to a commercially available nonmydriatic ocular fundus
perform our study with the classic direct ophthalmoscope camera, and that both patients and staff rated the

1060 AMERICAN JOURNAL OF OPHTHALMOLOGY NOVEMBER 2013

technique as both comfortable and expeditious.20 It also ocular fundus photography remains difficult for patients
showed, similar to the present study, that providing ocular who are unable to cooperate, advances in technology are
fundus photographs to emergency department physicians, already beginning to provide opportunities to assess this
even without additional training in the interpretation of subset of patients at beside. Nevertheless, we believe that
ocular fundus findings, resulted in substantial improve- medical students, our future physicians, would likely be
ments in the recognition of ocular fundus findings better served if educators were able to incorporate ocular
compared to direct ophthalmoscopy, as well as in the rate fundus photograph interpretation as an important compo-
of routine ocular fundus assessment.21 However, although nent of their ophthalmology curriculum.

ALL AUTHORS HAVE COMPLETED AND SUBMITTED THE ICMJE FORM FOR DISCLOSURE OF POTENTIAL CONFLICTS OF INTEREST.
Dr Newman has been a consultant for Santhera, Trius, and Anabasis. Dr Bruce has been a consultant for the Kaiser Foundation Health Plan of Georgia.
Dr Biousse has been a consultant for Anabasis. This study was supported in part by an unrestricted departmental grant (Department of Ophthalmology)
from Research to Prevent Blindness, Inc, New York, NY and by the NIH/NEI core grant P30-EY06360 (Department of Ophthalmology). Dr Newman
received the Research to Prevent Blindness Lew R. Wasserman Merit Award. Dr Bruce received research support from the NIH/NEI (K23-EY019341).
Contributions of authors: design of the study (L.P.K., P.S.G., B.B.B., N.J.N., V.B.); conduct of the study and collection of the data (L.P.K., P.S.G.,
B.B.B., E.B.G., N.J.N., V.B.); management, analysis and interpretation of the data (L.P.K., P.S.G., B.B.B., N.J.N., V.B.); and preparation, review, or
approval of the manuscript (L.P.K., P.S.G., B.B.B., N.J.N., V.B.).
The authors would like to thank John B. Davies, who designed the eye simulators used in this study, and the Emory University School of Medicine’s
Dean’s Office and the Class of 2016, who enthusiastically participated in the study.

REFERENCES 11. Watson D, Clark LA, Tellegan A. Development and vali-

dation of brief measures of positive and negative affect:
1. Morad Y, Barkana Y, Avni I, Kozer E. Fundus anomalies: what the PANAS scales. J Pers Soc Psychol 1988;54(6):
the pediatrician’s eye can’t see. Int J Qual Health Care 2004; 1063–1070.
16(5):363–365. 12. Lippa LM, Boker J, Duke A, Amin A. A novel 3-year longi-
2. Hitchings RA. Visual disability and the elderly: time for tudinal pilot study of medical students’ acquisition and reten-
general practitioners to begin preventive screening. BMJ tion of screening eye examination skills. Ophthalmology 2006;
1989;298(6681):1126–1127. 113(1):133–139.
3. Mottow-Lippa L. Ophthalmology in the medical school 13. Nicholl DJ, Yap CP, Cahill V, Appleton J, Willetts E,
curriculum: reestablishing our value and effecting change. Sturman S. The TOS study: can we use our patients to help
Ophthalmology 2009;116(7):1235–1236. improve clinical assessment? J R Coll Physicians Edinb 2012;
4. Association of American Medical Colleges Task Force on the 42(4):306–310.
Preclerkship Clinical Skills Education of Medical Students. 14. Stern GA. Teaching ophthalmology to primary care physi-
Recommendations for preclerkship clinical skills education for cians. Arch Ophthalmol 1995;113(6):722–724.
undergraduate medical education. Available at https://www. 15. Gupta RR, Lam WC. Medical students’ self-confidence in
aamc.org/download/163788/data/recommendations_for_precler performing direct ophthalmoscopy in clinical training. Can
kship_skills_education_for_ugme.pdf. Accessed June 6, 2013. J Ophthalmol 2006;41(2):169–174.
5. International Task Force on Ophthalmic Education of Medical 16. Shuttleworth GN, Marsh GW. How effective is undergrad-
Students. Principles and guidelines of a curriculum for uate and postgraduate teaching in ophthalmology? Eye
ophthalmic education of medical students. Available at (Lond) 1997;11(5):744–750.
http://icoph.org/dynamic/attachments/resources/icocurricmed. 17. Cordeiro MF, Jolly BC, Dacre JE. The effect of formal instruc-
pdf. Accessed June 6, 2013. tion in ophthalmoscopy on medical student performance.
6. Bruce BB, Lamirel C, Wright DW, et al. Nonmydriatic ocular Med Teach 1993;15(4):321–325.
fundus photography in the emergency department. N Engl J 18. Åsman P, Lindén C. Internet-based assessment of medical
Med 2011;364(4):387–389. students’ ophthalmoscopy skills. Acta Ophthalmol 2010;
7. Roberts E, Morgan R, King D, Clerkin L. Funduscopy: 88(8):854–857.
a forgotten art? Postgrad Med J 1999;75(883):282–284. 19. Milani BY, Majdi M, Green W, et al. The use of peer optic
8. Mottow-Lippa L, Boker JR, Stephens F. A prospective study nerve photographs for teaching direct ophthalmoscopy.
of the longitudinal effects of an embedded specialty curric- Ophthalmology 2013;120(4):761–765.
ulum on physical examination skills using an ophthalmology 20. Bruce BB, Lamirel C, Biousse V, et al. Feasibility of
model. Acad Med 2009;84(11):1622–1630. nonmydriatic ocular fundus photography in the emergency
9. Benbassat J, Polak BC, Javitt JC. Objectives of teaching direct department: phase I of the FOTO-ED study. Acad Emerg
ophthalmoscopy to medical students. Acta Ophthalmol 2012; Med 2011;18(9):928–933.
90(6):503–507. 21. Bruce BB, Thulasi P, Fraser CL, et al. Diagnostic accuracy and
10. Chung KD, Watzke RC. A simple device for teaching direct use of nonmydriatic ocular fundus photography by emergency
ophthalmoscopy to primary care practitioners. Am J Ophthal- physicians: phase II of the FOTO-ED study. Ann Emerg Med.
mol 2004;138(3):501–502. doi: 10.1016/j.annemergmed. 2013.01.010.

VOL. 156, NO. 5 TEACHING OPHTHALMOSCOPY TO MEDICAL STUDENTS 1061

 2012 M1 Eye Exam Training Session Your Name:____________________________
TOTeMS Pre-Test, Page 1 of 2 SG Leader Name:________________________

Preliminary Question 1. Please provide the following demographic information about yourself:
Gender: Male Female
Age (in years): _____________
Are you an MD-PhD student? No Yes

Preliminary Question 2. Do you have any prior ophthalmologic training?

No Yes if yes, please describe: __________________________________________

Preliminary Question 3. Have you taken any undergraduate/graduate courses related to ophthalmology?
Undergraduate: No Yes if yes, please describe: ____________________________________
Graduate: No Yes if yes, please describe: _________________________________________

-----------------PHOTOGRAPH #1---------------- -----------------PHOTOGRAPH #2----------------

1. Is this the left or right eye (check ONE)? 1. Is this the left or right eye (check ONE)?
Left Right Do Not Know Left Right Do Not Know

2. Do you see the optic nerve? 2. Do you see the optic nerve?
No if no, go to #3 below No if no, go to #3 below
Yes if yes, is Yes if yes, is
that apply)? that apply)?
Normal Normal
Swollen Swollen
Pale Pale
if yes, is the cup-to- ? if yes, is the cup-to- ?
<0.2 <0.2
0.2 0.7 0.2 0.7
>0.7 >0.7

3. Do you see the retina? 3. Do you see the retina?

No if no, go to #4 below No if no, go to #4 below
Yes if yes, do Yes if yes, do
that apply)? that apply)?
Normal retina Normal retina
Hemorrhage Hemorrhage
White lesions White lesions
Other abnormal area Other abnormal area

4. Do you see the blood vessels (check ONE)? 4. Do you see the blood vessels (check ONE)?
No No
Yes Yes

PROCEED TO THE NEXT COLUMN FOR TEST CONTINUES ON REVERSE

PHOTOGRAPH #2 (PHOTOGRAPHS #3 AND #4)

SUPPLEMENTAL FIGURE 1. Pretest to determine baseline knowledge of ocular fundus features. Pretest was administered to
students immediately after the introductory lecture to determine baseline knowledge of recognition of ocular fundus features.
This also collected demographic data and prior ophthalmology experience.

1061.e1 AMERICAN JOURNAL OF OPHTHALMOLOGY NOVEMBER 2013

 2012 M1 Eye Exam Training Session Your Name:____________________________
TOTeMS Pre-Test, Page 2 of 2 SG Leader Name:________________________

-----------------PHOTOGRAPH #3---------------- -----------------PHOTOGRAPH #4----------------

1. Is this the left or right eye (check ONE)? 1. Is this the left or right eye (check ONE)?
Left Right Do Not Know Left Right Do Not Know

2. Do you see the optic nerve? 2. Do you see the optic nerve?
No if no, go to #3 below No if no, go to #3 below
Yes if yes, is Yes if yes, is
that apply)? that apply)?
Normal Normal
Swollen Swollen
Pale Pale
if yes, is the cup-to- ? if yes, is the cup-to- ?
<0.2 <0.2
0.2 0.7 0.2 0.7
>0.7 >0.7

3. Do you see the retina? 3. Do you see the retina?

No if no, go to #4 below No if no, go to #4 below
Yes if yes, do Yes if yes, do
that apply)? that apply)?
Normal retina Normal retina
Hemorrhage Hemorrhage
White lesions White lesions
Other abnormal area Other abnormal area

4. Do you see the blood vessels (check ONE)? 4. Do you see the blood vessels (check ONE)?
No No
Yes Yes

PROCEED TO THE NEXT COLUMN FOR

PHOTOGRAPH #4

SUPPLEMENTAL FIGURE 1. (Continued).

VOL. 156, NO. 5 TEACHING OPHTHALMOSCOPY TO MEDICAL STUDENTS 1061.e2

 2012 M1 Eye Exam Training Session Your Name:________________________
TOTeMS Post-Human Quality Survey, Page 1 of 1 SG Leader Name:____________________

Instructions: Please write your name and small group (SG) . Please answer
the questions honestly based on the training you have just completed on people.
1. On a scale from 1 to 10, rate the ease of viewing the ocular fundus of a person. Please circle only ONE
number.

Very difficult 1 2 3 4 5 6 7 8 9 10 Very easy

2. On a scale from 1 to 10, indicate how frustrated you were by attempting ophthalmoscopy on a person.
Please circle only ONE number.

Not 1 2 3 4 5 6 7 8 9 10 Very
frustrated frustrated

3. Was your frustration in Question #2 mainly due to insufficient time to complete the task of examining
the individuals?
Yes No

4. If not specifically requested by your attending, would you perform ophthalmoscopy on a patient as part
of a general physical exam?
Yes No

5. This scale consists of a number of words that describe different feelings and emotions. Read each item
and then circle the number from the scale below next to each word. Indicate to what extent you feel this
way right now, that is, at the present moment. (Circle only ONE number for each line.)

Very Slightly A Little Moderately Quite a Bit Extremely

or Not at All
1. Interested 1 2 3 4 5
2. Distressed 1 2 3 4 5
3. Excited 1 2 3 4 5
4. Upset 1 2 3 4 5
5. Strong 1 2 3 4 5
6. Guilty 1 2 3 4 5
7. Scared 1 2 3 4 5
8. Hostile 1 2 3 4 5
9. Enthusiastic 1 2 3 4 5
10. Proud 1 2 3 4 5
11. Irritable 1 2 3 4 5
12. Alert 1 2 3 4 5
13. Ashamed 1 2 3 4 5
14. Inspired 1 2 3 4 5
15. Nervous 1 2 3 4 5
16. Determined 1 2 3 4 5
17. Attentive 1 2 3 4 5
18. Jittery 1 2 3 4 5
19. Active 1 2 3 4 5
20. Afraid 1 2 3 4 5

SUPPLEMENTAL FIGURE 2. Post-human and post-simulator quality surveys used to assess student preferences during sessions
for teaching ophthalmoscopy to medical students. Quality surveys were administered during the training session to determine student
preferences for different modalities of examining the ocular fundus.

1061.e3 AMERICAN JOURNAL OF OPHTHALMOLOGY NOVEMBER 2013

 2012 M1 Eye Exam Training Session Your Name:_______________________
TOTeMS Post-Simulator Quality Survey, Page 1 of 1 SG Leader Name:_________________

Instructions: Please write your name and small group (SG) . Please
answer the questions honestly based on the training you have just completed using the simulators.
1. On a scale from 1 to 10, rate the ease of using the simulator to view the ocular fundus. Please
circle only ONE number.

Very difficult 1 2 3 4 5 6 7 8 9 10 Very easy

2. On a scale from 1 to 10, indicate how frustrated you were by attempting ophthalmoscopy on
the simulator. Please circle only ONE number.

Not 1 2 3 4 5 6 7 8 9 10 Very
frustrated frustrated

3. Was your frustration in Question #2 mainly due to insufficient time to complete the task of
examining the simulators?
Yes No

4. This scale consists of a number of words that describe different feelings and emotions. Read
each item and then circle the number from the scale below next to each word. Indicate to what
extent you feel this way right now, that is, at the present moment. (Circle only ONE number
for each line.)

Very Slightly A Little Moderately Quite a Bit Extremely

or Not at All
1. Interested 1 2 3 4 5
2. Distressed 1 2 3 4 5
3. Excited 1 2 3 4 5
4. Upset 1 2 3 4 5
5. Strong 1 2 3 4 5
6. Guilty 1 2 3 4 5
7. Scared 1 2 3 4 5
8. Hostile 1 2 3 4 5
9. Enthusiastic 1 2 3 4 5
10. Proud 1 2 3 4 5
11. Irritable 1 2 3 4 5
12. Alert 1 2 3 4 5
13. Ashamed 1 2 3 4 5
14. Inspired 1 2 3 4 5
15. Nervous 1 2 3 4 5
16. Determined 1 2 3 4 5
17. Attentive 1 2 3 4 5
18. Jittery 1 2 3 4 5
19. Active 1 2 3 4 5
20. Afraid 1 2 3 4 5

SUPPLEMENTAL FIGURE 2. (Continued).

VOL. 156, NO. 5 TEACHING OPHTHALMOSCOPY TO MEDICAL STUDENTS 1061.e4

 2012 M1 Eye Exam Training Session Your Name:_____________________________
TOTeMS Simulator Post-Test, Page 1 of 2 SG Leader Name:________________________

===================HEAD #1=====================
-----------LEFT EYE------------ ----------RIGHT EYE----------
1. Were you able to visualize anything inside the 1. Were you able to visualize anything inside the
? ?
Yes Yes
No if no, why? No if no, why?

2. Did you visualize the optic nerve? 2. Did you visualize the optic nerve?
No if no, go to #3 below No if no, go to #3 below
Yes if yes Yes if yes
ALL that apply)? ALL that apply)?
Normal Normal
Swollen Swollen
Pale Pale
if yes, was the cup-to- ? if yes, was the cup-to- ?
<0.2 <0.2
0.2 0.7 0.2 0.7
>0.7 >0.7

3. Did you visualize the retina? 3. Did you visualize the retina?
No if no, go to #4 below No if no, go to #4 below
Yes if yes Yes if yes
that apply)? that apply)?
Normal retina Normal retina
Hemorrhage Hemorrhage
White lesions White lesions
Abnormal area Abnormal area

4. Did you visualize the blood vessels (check 4. Did you visualize the blood vessels (check
ONE)? ONE)?
No No
Yes Yes

PROCEED TO THE NEXT COLUMN FOR TEST CONTINUES ON REVERSE (HEAD #2

HEAD #1, RIGHT EYE ON BACK)

SUPPLEMENTAL FIGURE 3. Simulator posttest assessing ability to identify fundus features using direct ophthalmoscopy on eye
simulators. Posttest was administered to determine student ability to correctly identify features of the ocular fundus on anatomically
and optically correct eye simulators using direct ophthalmoscopy.

1061.e5 AMERICAN JOURNAL OF OPHTHALMOLOGY NOVEMBER 2013

 2012 M1 Eye Exam Training Session Your Name:_____________________________
TOTeMS Simulator Post-Test, Page 2 of 2 SG Leader Name:________________________

===================HEAD #2=====================
------------LEFT EYE----------- ----------RIGHT EYE----------
1. Were you able to visualize anything inside the 1. Were you able to visualize anything inside the
? ?
Yes Yes
No if no, why? No if no, why?

2. Did you visualize the optic nerve? 2. Did you visualize the optic nerve?
No if no, go to #3 below No if no, go to #3 below
Yes if yes Yes if yes
ALL that apply)? ALL that apply)?
Normal Normal
Swollen Swollen
Pale Pale
if yes, was the cup-to- ? if yes, was the cup-to- ?
<0.2 <0.2
0.2 0.7 0.2 0.7
>0.7 >0.7

3. Did you visualize the retina? 3. Did you visualize the retina?
No if no, go to #4 below No if no, go to #4 below
Yes if yes Yes if yes
that apply)? that apply)?
Normal retina Normal retina
Hemorrhage Hemorrhage
White lesions White lesions
Abnormal area Abnormal area

4. Did you visualize the blood vessels (check 4. Did you visualize the blood vessels (check
ONE)? ONE)?
No No
Yes Yes

PROCEED TO THE NEXT COLUMN FOR

HEAD #2, RIGHT EYE

EXAMINE HEAD #1 AND ANSWER

QUESTIONS ON FRONT IF YOU HAVE NOT
YET DONE SO

SUPPLEMENTAL FIGURE 3. (Continued).

VOL. 156, NO. 5 TEACHING OPHTHALMOSCOPY TO MEDICAL STUDENTS 1061.e6

 2012 M1 Eye Exam Training Session Your Name:________________________
TOTeMS Photograph Post-Test, Page 1 of 2 SG Leader Name:__________________

--------------PHOTOGRAPH TP1--------------- --------------PHOTOGRAPH TP2---------------

1. Is this the left or right eye (check ONE)? 1. Is this the left or right eye (check ONE)?
Left Right Do Not Know Left Right Do Not Know

2. Do you see the optic nerve? 2. Do you see the optic nerve?
No if no, go to #3 below No if no, go to #3 below
Yes if yes, is Yes if yes, is
ALL that apply)? ALL that apply)?
Normal Normal
Swollen Swollen
Pale Pale
if yes, is the cup-to- if yes, is the cup-to-
(check ONE)? (check ONE)?
<0.2 <0.2
0.2 0.7 0.2 0.7
>0.7 >0.7

3. Do you see the retina? 3. Do you see the retina?

No if no, go to #4 below No if no, go to #4 below
Yes if yes (check Yes if yes
ALL that apply)? ALL that apply)?
Normal retina Normal retina
Hemorrhage Hemorrhage
White lesions White lesions
Other abnormal area Other abnormal area

4. Do you see the blood vessels (check 4. Do you see the blood vessels (check
ONE)? ONE)?
No No
Yes Yes

PROCEED TO THE NEXT COLUMN FOR TEST CONTINUES ON REVERSE

PHOTOGRAPH TP2 (PHOTOGRAPHS TP3 & TP4 ON BACK)

SUPPLEMENTAL FIGURE 4. Photograph posttest assessing ability to identify fundus features on photographs. Posttest was
administered to determine student ability to correctly identify features of the ocular fundus using fundus photographs.

1061.e7 AMERICAN JOURNAL OF OPHTHALMOLOGY NOVEMBER 2013

 2012 M1 Eye Exam Training Session Your Name:________________________
TOTeMS Photograph Post-Test, Page 2 of 2 SG Leader Name:__________________

--------------PHOTOGRAPH TP3--------------- --------------PHOTOGRAPH TP4---------------

1. Is this the left or right eye (check ONE)? 1. Is this the left or right eye (check ONE)?
Left Right Do Not Know Left Right Do Not Know

2. Do you see the optic nerve? 2. Do you see the optic nerve?
No if no, go to #3 below No if no, go to #3 below
Yes if yes, is Yes if yes, is
ALL that apply)? ALL that apply)?
Normal Normal
Swollen Swollen
Pale Pale
if yes, is the cup-to- if yes, is the cup-to-
(check ONE)? (check ONE)?
<0.2 <0.2
0.2 0.7 0.2 0.7
>0.7 >0.7

3. Do you see the retina? 3. Do you see the retina?

No if no, go to #4 below No if no, go to #4 below
Yes if yes Yes if yes
ALL that apply)? ALL that apply)?
Normal retina Normal retina
Hemorrhage Hemorrhage
White lesions White lesions
Other abnormal area Other abnormal area

4. Do you see the blood vessels (check 4. Do you see the blood vessels (check
ONE)? ONE)?
No No
Yes Yes

PROCEED TO THE NEXT COLUMN FOR

PHOTOGRAPH TP4

SUPPLEMENTAL FIGURE 4. (Continued).

VOL. 156, NO. 5 TEACHING OPHTHALMOSCOPY TO MEDICAL STUDENTS 1061.e8

SUPPLEMENTAL FIGURE 4. (Continued).

1061.e9 AMERICAN JOURNAL OF OPHTHALMOLOGY NOVEMBER 2013

SUPPLEMENTAL FIGURE 5. Final quality survey administered after completion of all sessions of teaching ophthalmoscopy to
medical students. Quality survey was administered after all training and testing sequences were complete to determine overall student
preferences for the different modalities of assessment of the ocular fundus used in this study.

VOL. 156, NO. 5 TEACHING OPHTHALMOSCOPY TO MEDICAL STUDENTS 1061.e10