JOURNAL OF MEDICAL INTERNET RESEARCH Liu et al

Original Paper

A Web-Based Telehealth Training Platform Incorporating

Automated Nonverbal Behavior Feedback for Teaching
Communication Skills to Medical Students: A Randomized
Crossover Study

Chunfeng Liu1*, BSc (Hons); Renee L Lim1,2*, GCertEd; Kathryn L McCabe1,3*, PhD; Silas Taylor4*, MEd; Rafael A
Calvo1*, PhD
1
School of Electrical and Information Engineering, The University of Sydney, Sydney, Australia
2
Sydney Medical School, The University of Sydney, Sydney, Australia
3
Psychiatry and Behavioral Sciences, University of California, Davis, Davis, CA, United States
4
UNSW Medicine, University of New South Wales, Sydney, Australia
*
all authors contributed equally

Corresponding Author:
Rafael A Calvo, PhD
School of Electrical and Information Engineering
The University of Sydney
Bldg J03, School of Electrical and Information Engineering
Maze Crescent
Sydney, 2006
Australia
Phone: 61 293518171
Fax: 61 293513847
Email: [email protected]

Abstract
Background: In the interests of patient health outcomes, it is important for medical students to develop clinical communication
skills. We previously proposed a telehealth communication skills training platform (EQClinic) with automated nonverbal behavior
feedback for medical students, and it was able to improve medical students’ awareness of their nonverbal communication.
Objective: This study aimed to evaluate the effectiveness of EQClinic to improve clinical communication skills of medical
students.
Methods: We conducted a 2-group randomized crossover trial between February and June 2016. Participants were second-year
medical students enrolled in a clinical communication skills course at an Australian university. Students were randomly allocated
to complete online EQClinic training during weeks 1–5 (group A) or to complete EQClinic training during weeks 8–11 (group
B). EQClinic delivered an automated visual presentation of students’ nonverbal behavior coupled with human feedback from a
standardized patient (SP). All students were offered two opportunities to complete face-to-face consultations with SPs. The two
face-to-face consultations were conducted in weeks 6–7 and 12–13 for both groups, and were rated by tutors who were blinded
to group allocation. Student-Patient Observed Communication Assessment (SOCA) was collected by blinded assessors (n=28)
at 2 time points and also by an SP (n=83). Tutor-rated clinical communications skill in face-to-face consultations was the primary
outcome and was assessed with the SOCA. We used t tests to examine the students’ performance during face-to-face consultations
pre- and postexposure to EQClinic.
Results: We randomly allocated 268 medical students to the 2 groups (group A: n=133; group B: n=135). SOCA communication
skills measures (score range 4–16) from the first face-to-face consultation were significantly higher for students in group A who
had completed EQClinic training and reviewed the nonverbal behavior feedback, compared with group B, who had completed
only the course curriculum components (P=.04). Furthermore, at the second face-to-face assessment, the group that completed
a teleconsultation between the two face-to-face consultations (group B) showed improved communication skills (P=.005), and
the one that had teleconsultations before the first face-to-face consultation (group A) did not show improvement.

http://www.jmir.org/2016/9/e246/ J Med Internet Res 2016 | vol. 18 | iss. 9 | e246 | p. 1

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Liu et al

Conclusions: The EQClinic is a useful tool for medical students’ clinical communication skills training that can be applied to
university settings to improve students clinical communication skills development.

(J Med Internet Res 2016;18(9):e246) doi: 10.2196/jmir.6299

KEYWORDS
nonverbal communication; nonverbal behavior; clinical consultation; medical education; communication skills; nonverbal behavior
detection; automated feedback; affective computing

it is too time consuming to be widely adopted in medical

Introduction education curricula.
There is good evidence that effective patient-clinician We have previously described a platform called EQClinic
communication can positively influence patient health outcomes [15,16]. Briefly, EQClinic is an e-learning platform that allows
[1-3]. For instance, a clinician’s supportive expressions can medical students to have recorded teleconsultations with SPs.
help the patient to develop greater feelings of trust toward their The platform uses computer vision and audio processing
clinician. These feelings of trust lead to greater patient techniques to automatically recognize, quantify, and visualize
self-efficacy, where the patient is more likely to follow selected nonverbal behaviors (as well as human feedback) for
recommended therapies, resulting in a better treatment outcome student learning and reflection. Initial pilot application of
[4]. This evidence has meant that more training programs are EQClinic has shown that medical students’ awareness of their
being offered to students to help them learn clinical nonverbal communication improved using EQClinic. However,
communication skills. Since students become competent through the platform has not been applied within a typical university
practice and feedback [5], medical students need practice with medical school curriculum.
real or standardized patients (SPs) and feedback from patients
and tutors. An SP normally refers to someone who has been Therefore, the goal of this study was to conduct a randomized
trained to act as a patient in a medical situation. However, crossover trial of the EQClinic incorporated into a university
despite the importance of communication skills, the time medical school curriculum (Multimedia Appendix 1 [17]). The
allocated to such training within medical curricula is often EQClinic platform is designed to provide clinical
limited. This is influenced, in part, by the logistics of providing communication skills training that integrates nonverbal behavior
large groups of medical students with access to SPs with whom assessment for medical students. We used a randomized
they can practice and formulate their communication techniques. crossover design to initially test the effectiveness of the
EQClinic, whereby we allocated medical students enrolled in
The traditional method for clinical communication skills training a clinical communication skills course to 1 of 2 groups and
is to provide students with feedback on video-recorded asked them to complete a teleconsultation using EQClinic at
face-to-face consultations [6]. Students benefit from reviewing different times during the semester. Interleaved with exposure
these videotapes of their clinical consultations with real patients to the EQClinic were face-to-face clinical consultation skills
or SPs [7], even more so when observers provide feedback about assessments. By staging exposure of students to the EQClinic,
the verbal or nonverbal behaviors [8]. However, organizing we evaluated the potential impact of the platform by comparing
large-scale face-to-face practice sessions and setting up the group performance on face-to-face clinical consultations before
recording environment are a challenge for medical schools. and after EQClinic exposure. To our knowledge, this is the first
Teleconferencing has been proposed as a solution for dealing application of automated nonverbal behavior detection
with this challenge [9]. For example, the WebEncounter techniques for improved medical students’ communication
teleconference platform, developed to enable medical interns skills. We hypothesized that the use of EQClinic would improve
to communicate with SPs, showed that practicing on medical students’ learning about communication skills.
WebEncounter enhanced the communication skills of the interns
when giving bad news [10]. Another recent study that related Methods
to medical students also suggested that involving telehealth
consulting between medical practices and patients enhanced Participants
students’ learning [11]. Participants were second-year undergraduate medical students
However, like WebEncounter, most clinical communication from an Australian medical school. All students were enrolled
skills training systems tend to limit training to verbal in a communication skills training course provided by the
communication skills and overlook important nonverbal medical school. Prior to this study, they were not offered any
communication behaviors. This is problematic, given that training about teleconsultations in the medical school. This
nonverbal communication is the major communication channel study was approved by the University of New South Wales
between individuals [12,13]. Manually annotating students’ Research Ethics Committee (HC Reference Number: HC16048).
nonverbal behaviors from face-to-face consultations and Students were asked to sign an online consent form when they
providing this feedback to students are common ways to improve first accessed EQClinic. No content or methodological
the learning of nonverbal communication skills [14]. However, modifications were made after study commencement.
the practicalities of providing this type of feedback means that

http://www.jmir.org/2016/9/e246/ J Med Internet Res 2016 | vol. 18 | iss. 9 | e246 | p. 2

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Liu et al

Questionnaires Nonverbal Behavior Detector

The same 5 surveys previously reported by our group were used Using audio processing and computer vision techniques,
in the present study [15]. The pre- and postinterview EQClinic automatically analyses the video recordings and
questionnaires ascertained students’ understanding of detects the following nonverbal behaviors: head movements
communication skills. The Post Interview Nonverbal Behavior (nodding, head shaking, and head tilting), facial expressions
Reflection Questionnaire asked students to estimate how often (smiling and frowning), body movements (body leaning, hand
they engaged in certain nonverbal behaviors during the gestures, and overall body movements), voice properties
interview. The Reflection Questionnaire prompted students to (volume and pitch), and speech patterns (turn taking and
reflect on the consultation. The primary outcomes measure was speaking ratio changes).
the Student-Patient Observed Communication Assessment
(SOCA) form, which is an adapted version of the Calgary
Feedback Generator
Cambridge Guides [18]. The SPs and tutors used the SOCA to Feedback information includes computer-generated nonverbal
rate students’ communication skills. The form contained four behavior feedback (NVBF) and comment feedback from the
aspects: providing structure, gathering information, building SP. EQClinic visualizes students’ nonverbal behavior using two
rapport, and understanding the patient’s needs. types of feedback reports: single-feature and combined-feature
reports. The single-feature feedback report illustrates each form
EQClinic of nonverbal behavior separately. The combined-feature
EQClinic comprises five components: an online training feedback report displays multiple kinds of nonverbal behavior
component, a personal calendar, a real-time interaction on one page. The comment feedback provides students a report
component, a nonverbal behavior detector, and a feedback that contains all the comments from the SP and tutor.
generator. In the following sections, we briefly describe each
of these.
Study Design and Procedure
The administrator of this course randomly allocated a cohort of
Training Component and Personal Calendar 268 students to group A (n=133) or group B (n=135) (see Figure
EQClinic provides training videos and documents for students 1) using a computer-generated random number sequence. One
and SPs to familiarize themselves with the platform. EQClinic student was moved from group A to group B for administrative
also provides students and SPs with an automated personal reasons. Following random allocation to a group, each
calendar. SPs can offer their availability on the calendar for participant was provided three opportunities to complete
students to make a booking. All appointments are confirmed simulated clinical consultations with SPs: a teleconsultation
using the automated messaging system without need for human using EQClinic, and two face-to-face consultations. In this
resources. study, all consultations focused on history-taking skills, to
ensure a structured and consistent interaction. The allocation
Real-Time Interaction Component of the three consultations was varied between the 2 groups. The
Once the appointment has been confirmed, videoconferencing study was conducted over 13 weeks, and it included 4 periods
enables a student and an SP to have a teleconsultation. The (see Figure 1). (1) During weeks 1–5, group A completed a
application works on most Web browsers of a personal computer teleconsultation using EQClinic and group B was blocked from
or Android tablet. During the recorded consultation, the SP can the platform. (2) During weeks 6 and 7, both groups completed
record positive and negative moments using a “thumbs” tool a face-to-face consultation. In this period, group A was still able
and comment box. to access the platform for reviewing feedback only. (3) During
To facilitate learning through reflection, online assessments weeks 8–11, group B completed an EQClinic consultation and
were included for students. The SPs evaluated student group A was blocked from the platform. (4) During weeks 12
performance immediately after the teleconsultation, during and 13, both groups were asked to complete another face-to-face
which time the students conduct a self-assessment using the consultation. In this period, group B was able to access the
same form. Students could immediately review the SP’s rating. platform for reviewing feedback only. Due to the limited
resources of setting up face-to-face consultations, not all enrolled
students completed two face-to-face consultations. However,
having a teleconsultation using EQClinic was mandatory for
every student.

http://www.jmir.org/2016/9/e246/ J Med Internet Res 2016 | vol. 18 | iss. 9 | e246 | p. 3

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Liu et al

Figure 1. Flowchart of student participation in the EQClinic medical communication training program.

external or built-in camera and microphone, (2) a good Internet

Teleconsultations Procedure connection, and (3) good lighting.
All participating SPs and students completed training via the
training component of EQClinic. In the SPs’ online training EQClinic teleconsultation comprised three sections:
component, training videos demonstrated how to book interviewing, assessing, and reviewing (see Figure 2) [15].
appointments, conduct consultations with students, provide Interview and assessment components took approximately 40
comments, and evaluate the student’s performance. The patient minutes for a student and 25–30 minutes for an SP to complete.
scenario was also included in training and detailed the main In the interviewing section, the student completed the
symptoms of the SP and other historical information. All SPs preinterview questionnaire, and then the student and the SP
were required to complete this online training. Following conducted a 15-minute consultation via the teleconference
training, the SPs listed their availability for consultations on component. The student and the SP then completed the online
their EQClinic calendar. assessments. After each interview, the SP assessed the
performance of the student using the SOCA form. Meanwhile,
Students were requested, by email, to complete one the student estimated their nonverbal behavior using the Post
teleconsultation with an SP through EQClinic. The email Interview Nonverbal Behavior Reflection Questionnaire,
described the details of the study and asked them to log in to completed a personal SOCA form, and then reviewed the SOCA
EQClinic to complete the training module. It also informed form completed by the SP and reflected on the interview using
them that, once they finished the training, they could request a the Reflection Questionnaire.
consultation time from the slots available on their personal
calendar. The SPs and students were allowed to have the Students were emailed to ask them to return to the system 24
teleconsultation anywhere as long as there was (1) a Web hours after the consultation to review different kinds of
browser on a personal computer or an Android tablet with an feedback, which included the video recording, comments from
the SP, and automated NVBF. Students also completed the
postinterview questionnaire.

http://www.jmir.org/2016/9/e246/ J Med Internet Res 2016 | vol. 18 | iss. 9 | e246 | p. 4

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Liu et al

Figure 2. Workflow for the EQClinic consultation. SOCA: Student-Patient Observed Communication Assessment; SP: standardized patient.

144 students from both group A (n=109) and group B (n=35)

Face-to-Face Consultations Procedure completed a face-to-face consultation during the period of weeks
Face-to-face consultations were conducted in consultation rooms 12 and 13. In total, 11 students (6 from group A, 5 from group
of a university-based clinical skills center. A trained tutor was B) did not complete the teleconsultation in this study. At the
present in the room to observe and assess the performance of second face-to-face consultation (weeks 12–13), 53 of 133
the student during the consultation with the SP. The tutors were (39.9%) students from group A and 30 of 135 (22.2%) from
blinded to condition allocation (group A or group B). The tutor group B had completed one teleconsultation and one face-to-face
completed a SOCA form to assess the student, and the SP did consultation before completing the second face-to-face
not provide any evaluation and feedback for the student on this consultation.
occasion. The students were asked to review the tutor’s
assessment and complete the Reflection Questionnaire. The Table 1 and Table 2 describe the mean subgroup assessment
scenario design and length of face-to-face consultations were results in the various study periods. Mean total SOCA scores
the same as those for the teleconsultations. from the first face-to-face consultation for group A (mean 13.02)
and group B (mean 12.58) did not differ significantly between
Results the groups (P=.08). To examine the influence of the NVBF
component, we compared the group A mean SOCA scores
Figure 1 shows participants’ flow through the trial. In the period (group A + NVBF: mean 13.21; 33/59, 55.9%) of those who
of week 1 to week 5, 127 (46 male, 81 female) of 133 (95.5%) had reviewed the NVBF component of the EQClinic before
group A students completed the teleconsultation on EQClinic. having their face-to-face consultations with the scores of group
In the second period (weeks 6–7), 166 of the 268 students from B students (mean 12.58). Mean SOCA scores were significantly
both group A (59/133, 44.4%) and group B (107/135, 79.3%) higher on face-to-face SOCA total score in group A + NVBF
completed a face-to-face consultation. During weeks 8–11, a (t58.25=2.13, P=.04) than in group B. However, they did not
total of 130 (62 male, 68 female) of 135 (96.3%) group B differ statistically from group A students who did not review
students completed the teleconsultation using EQClinic. Lastly, their nonverbal behavior component of the EQClinic (mean
12.77; 26/59, 44.1%, P>.05).

Table 1. Mean group medical communication skills (measured by Student-Patient Observed Communication Assessment score) assessment results
(part 1: weeks 1–7).
Component Weeks 1–5 (TCa) Weeks 6–7 (F2FCb)
Group A (n=127) Group A (n=59) Group A (NVBFc) (n=33) Group B (n=107)

Mean SD Mean SD Mean SD Mean SD

Total score 11.59 2.67 13.02 1.49 13.21 1.45 12.58 1.61
Providing structure 2.88 0.72 3.17 0.50 3.27 0.45 3.12 0.53
Gathering information 2.92 0.72 3.25 0.58 3.15 0.51 3.07 0.56
Building rapport 2.95 0.73 3.34 0.60 3.39 0.66 3.24 0.56
Understanding patient’s 2.83 0.80 3.25 0.58 3.39 0.56 3.14 0.61
needs

a
TC: teleconsultation.
b
F2FC: face-to-face consultation.
c
NVBF: students who had a face-to-face consultation and reviewed the nonverbal behavior feedback.

http://www.jmir.org/2016/9/e246/ J Med Internet Res 2016 | vol. 18 | iss. 9 | e246 | p. 5

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Liu et al

Table 2. Mean group medical communication skills (measured by Student-Patient Observed Communication Assessment score) assessment results
(part 2: weeks 8–13).
Component Weeks 8–11 (TCa) Weeks 12–13 (F2FCb)
Group B (n=130) Group A (n=109) Group A (ConAc) Group B (n=35) Group B (ConBd) Group B (NVBFe)
(n=53) (n=30) (n=13)
Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD
Total score 13.13 2.31 13.28 1.54 13.28 1.46 13.43 1.63 13.53 1.52 13.62 1.64
Providing structure 3.31 0.67 3.25 0.49 3.23 0.46 3.31 0.52 3.37 0.48 3.38 0.49
Gathering informa- 3.34 0.72 3.26 0.55 3.30 0.53 3.51 0.55 3.53 0.50 3.46 0.50
tion
Building rapport 3.16 0.73 3.46 0.58 3.47 0.54 3.34 0.58 3.37 0.55 3.46 0.63
Understanding pa- 3.32 0.68 3.31 0.57 3.28 0.59 3.26 0.44 3.27 0.44 3.31 0.46
tient’s needs

a
TC: teleconsultation.
b
F2FC: face-to-face consultation.
c
ConA: group A students who participated in two consultations (one face-to-face consultation, one teleconsultation) before week 12.
d
ConB: group B students who participated in two consultations (one face-to-face consultation, one teleconsultation) before week 12.
e
NVBF: students who had a face-to-face consultation and reviewed the nonverbal behavior feedback.

Following group B exposure to the EQClinic, the mean total Overall, adherence to the program was somewhat less than
SOCA scores from the second face-to-face consultation did not anticipated, with only 30% of student completing all components
differ between the groups (group A: mean 13.28; group B: mean of the study. Dropout increased as the semester progressed.
13.53, P>.05). Mean SOCA scores of group B students (group However, given the requirements of the undergraduate course,
B + NVBF: mean 13.62; 13/30, 43.3%) who reviewed the NVBF and the tendency for increased workload as the semester
component of the EQClinic before their second face-to-face progresses, this result is unsurprising.
consultation did not differ from those in group A or group B
The results of the first face-to-face consultation show that the
who did not complete the nonverbal review (mean 13.47; 17/30,
students who completed a teleconsultation and reviewed the
P>.05).
NVBF component scored significantly higher in their
We used paired-samples t tests to compare the SOCA assessment face-to-face consultation than did students who did not interact
scores for those students who completed EQClinic on both their with SPs on EQClinic. These results are promising. The
two face-to-face consultations. Group B alone showed difference in performance between the 2 groups seems to
significant improvement in their mean SOCA score (mean indicate that having EQClinic practice coupled with reviewing
preexposure score 12.58 vs postexposure score 13.53; t48= –2.96; feedback improved medical communication skills in group A.
P=.005). Group A showed no significant increase in SOCA As noted above, group B students achieved lower mean overall
scores (mean preexposure score 13.02 vs postexposure score SOCA score in the first face-to-face consultation. However,
13.28; P>.05). Comparison of the mean SOCA teleconsultation overall, group B students showed significant improvement from
scores rated by SPs showed that group B’s score (mean 13.13) their first to second face-to-face consultation. However, whether
was significantly higher than group A’s score (mean 11.59; students reviewed their NVBF did not influence results for this
t246.61= –4.83, P<.001). group.
These findings are interesting because they suggest improvement
Discussion in communication skills assessment after reviewing nonverbal
feedback. While the need for medical communication skills
We incorporated EQClinic into a medical communication skills
training is widely accepted within the medical teaching
teaching curriculum to provide students with additional practice
community [1-3], there is less consensus on the need for specific
opportunities with SPs. Importantly, students could review their
teaching on the nonverbal aspects of communication. This is
nonverbal communication behaviors. We examined the effects
related to the lack of adequate resources, knowledge, and
of EQClinic on medical students’ learning of communication
expertise in this aspect of communication [19]. To our
skills evaluated via the students’ assessment (SOCA) scores.
knowledge, this is the first study to systematically incorporate
Results showed that students who completed a teleconsultation
nonverbal learning feedback into medical communication skills
using EQClinic and reviewed the NVBF achieved higher SOCA
training.
scores in the first face-to-face consultation. In addition, students
accomplished higher SOCA scores in their second face-to-face Furthermore, that we showed no significant difference between
consultation if they completed a teleconsultation between the group scores in the second face-to-face assessment seems to
two face-to-face consultations. indicate that the timing of exposure to EQClinic within a
teaching curriculum did not influence students’ learning results.

http://www.jmir.org/2016/9/e246/ J Med Internet Res 2016 | vol. 18 | iss. 9 | e246 | p. 6

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Liu et al

In our study, group A was exposed to EQClinic at the beginning EQClinic led to a measurable improvement in students’ medical
of the course; whereas group B was exposed in the middle of communication skills scores, future studies will benefit from
their course. We showed that at the commencement of the an examination of the appropriate “dose” of EQClinic. This will
curriculum, when students did not have significant knowledge help determine the necessary exposure needed to provide
of clinical communication skills, exposure to EQClinic yielded sustained improvement and generalizable communication skills
a measurable bump in their clinical communication skills. For training. Finally, the growth of collected student data by
medical educators this seems to indicate that EQClinic could EQClinic will aid the refinement of rules and models using
be incorporated at any period during the teaching curriculum. machine learning algorithms to indicate to students what
nonverbal behavior is associated with positive or negative
We also showed that group B performed significantly better
responses and feedback from SPs in their clinical
than group A on the SP-rated EQClinic teleconsultations. This
teleconsultations.
difference could be explained in several ways. The first way
relates to timing of EQClinic exposure, with group B completing Study Limitations
the teleconsultation later in the semester than group A. Second, There are several limitations to our study that should be
completing face-to-face consultations before being exposed to considered when interpreting these findings. First, the absence
EQClinic, experience, and feedback garnered from the of baseline measures limited our ability to observe change over
face-to-face consultation may have improved student time. Second, all the consultations conducted in this study were
performance in the teleconsultation. The third possibility is that limited to a history-taking scenario. In reality, clinicians
the SPs who assessed students via the EQClinic increased their encounter many different scenarios. For example, when breaking
ratings across the semester. However, SP ratings neither bad news to patients, the clinician has to handle difficulties
contributed to student assessment nor were a central feature of related to emotions. In addition, all the students in this study
the EQClinic. were second-year medical students who had limited knowledge
Telehealth studies involving medical students and interns in about communication skills. Future studies may explore whether
urban, rural, and remote areas indicated that this medium was EQClinic is also useful for senior medical students and
a useful learning tool [10,11]. In EQClinic, we enhance existing professionals. A third limitation is the relatively low proportion
telehealth systems by providing students with multiple kinds of students (30%) who completed all components of the study.
of feedback. We contend that the primary functions of EQClinic While the sample was still appropriate for the statistical tests
are 2-fold: to facilitate student access to SPs to practice and conducted, future investigations will benefit from exploring in
refine their medical communication skills. The importance of greater detail the reason for student nonparticipation.
SPs to facilitate the application of clinical communication
Conclusions
theory, especially early one-on-one interactions, has been
described previously [20]. The second function of the EQClinic This study provided evidence that furnishing medical students
is to facilitate reflective practice by providing human and with opportunities to conduct teleconsultations with SPs
computer-generated feedback, in particular in regard to improved medical communication skills. In particular, offering
nonverbal behaviors, in medical communication skills training. enhanced and quantified feedback information facilitates their
reflection and enhances their learning of clinical communication
However, based on our findings, it remains unclear which of skills. Importantly, this study demonstrated that EQClinic was
the learning components were most useful to enhancing a useful and practical communication skills learning tool that
students’ learning. Moreover, although a single exposure to the is well suited to medical students within university settings.

Acknowledgments
This project was funded by the Brain and Mind Centre at The University of Sydney Australia and the Australian Government.
RAC is supported by the Australian Research Council.

Conflicts of Interest
None declared.

Multimedia Appendix 1
CONSORT-EHEALTH Checklist V1.6 [20].
[PDF File (Adobe PDF File), 1005KB-Multimedia Appendix 1]

References
1. Jackson JL. Communication about symptoms in primary care: impact on patient outcomes. J Altern Complement Med
2005;11(Suppl 1):S51-S56. [doi: 10.1089/acm.2005.11.s-51] [Medline: 16332187]
2. Arora NK. Interacting with cancer patients: the significance of physicians' communication behavior. Soc Sci Med 2003
Sep;57(5):791-806. [Medline: 12850107]

http://www.jmir.org/2016/9/e246/ J Med Internet Res 2016 | vol. 18 | iss. 9 | e246 | p. 7

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Liu et al

3. Street RL, Makoul G, Arora NK, Epstein RM. How does communication heal? Pathways linking clinician-patient
communication to health outcomes. Patient Educ Couns 2009 Mar;74(3):295-301. [doi: 10.1016/j.pec.2008.11.015] [Medline:
19150199]
4. Zolnierek KBH, Dimatteo MR. Physician communication and patient adherence to treatment: a meta-analysis. Med Care
2009 Aug;47(8):826-834 [FREE Full text] [doi: 10.1097/MLR.0b013e31819a5acc] [Medline: 19584762]
5. Mann KV. Theoretical perspectives in medical education: past experience and future possibilities. Med Educ 2011
Jan;45(1):60-68. [doi: 10.1111/j.1365-2923.2010.03757.x] [Medline: 21155869]
6. Howley L, Martindale J. The efficacy of standardized patient feedback in clinical teaching: a mixed methods analysis. Med
Educ Online 2004 Jun 01;9(18):[no page].
7. Paul S, Dawson KP, Lanphear JH, Cheema MY. Video recording feedback: a feasible and effective approach to teaching
history-taking and physical examination skills in undergraduate paediatric medicine. Med Educ 1998 May;32(3):332-336.
[Medline: 9743791]
8. Fukkink R, Trienekens N, Kramer L. Video feedback in education and training: putting learning in the picture. Educ Psychol
Rev 2010 Oct 16;23(1):45-63. [doi: 10.1007/s10648-010-9144-5]
9. Novack DH, Cohen D, Peitzman SJ, Beadenkopf S, Gracely E, Morris J. A pilot test of WebOSCE: a system for assessing
trainees' clinical skills via teleconference. Med Teach 2002 Sep;24(5):483-487. [Medline: 12474813]
10. Daetwyler CJ, Cohen DG, Gracely E, Novack DH. eLearning to enhance physician patient communication: a pilot test of
“doc.com” and “WebEncounter” in teaching bad news delivery. Med Teach 2010;32(9):e381-e390. [doi:
10.3109/0142159X.2010.495759] [Medline: 20795797]
11. Knight P, Bonney A, Teuss G, Guppy M, Lafferre D, Mullan J, et al. Positive clinical outcomes are synergistic with positive
educational outcomes when using telehealth consulting in general practice: a mixed-methods study. J Med Internet Res
2016;18(2):e31 [FREE Full text] [doi: 10.2196/jmir.4510] [Medline: 26858152]
12. Ekman P, Friesen W, O'Sullivan M, Scherer K. Relative importance of face, body, and speech in judgments of personality
and affect. J Pers Soc Psychol 1980;38(2):270.
13. O'Sullivan M, Ekman P, Friesen W, Scherer K. What you say and how you say it: the contribution of speech content and
voice quality to judgments of others. J Pers Soc Psychol 1985 Jan;48(1):54-62. [Medline: 3981393]
14. Ishikawa H, Hashimoto H, Kinoshita M, Yano E. Can nonverbal communication skills be taught? Med Teach
2010;32(10):860-863. [doi: 10.3109/01421591003728211] [Medline: 20854164]
15. Liu C, Calvo RA, Lim R. Improving medical students’ awareness of their nonverbal communication through automated
nonverbal behavior feedback. Frontiers ICT 2016 Jun 20;3:[no page]. [doi: 10.3389/fict.2016.00011]
16. Liu C, Scott KM, Lim RL, Taylor S, Calvo RA. EQClinic: a platform for learning communication skills in clinical
consultations. Med Educ Online 2016;21:31801 [FREE Full text] [Medline: 27476537]
17. Eysenbach G, Consort-EHEALTHGroup. CONSORT-EHEALTH: improving and standardizing evaluation reports of
Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [doi: 10.2196/jmir.1923]
[Medline: 22209829]
18. Simmenroth-Nayda A, Heinemann S, Nolte C, Fischer T, Himmel W. Psychometric properties of the Calgary Cambridge
guides to assess communication skills of undergraduate medical students. Int J Med Educ 2014;5:212-218 [FREE Full text]
[doi: 10.5116/ijme.5454.c665] [Medline: 25480988]
19. Nestel D, Tabak D, Tierney T, Layat-Burn C, Robb A, Clark S, et al. Key challenges in simulated patient programs: an
international comparative case study. BMC Med Educ 2011;11:69 [FREE Full text] [doi: 10.1186/1472-6920-11-69]
[Medline: 21943295]
20. Joekes K, Noble LM, Kubacki AM, Potts HW, Lloyd M. Does the inclusion of 'professional development' teaching improve
medical students' communication skills? BMC Med Educ 2011;11:41 [FREE Full text] [doi: 10.1186/1472-6920-11-41]
[Medline: 21708000]

Abbreviations
NVBF: nonverbal behavior feedback
SOCA: Student-Patient Observed Communication Assessment
SP: standardized patient

http://www.jmir.org/2016/9/e246/ J Med Internet Res 2016 | vol. 18 | iss. 9 | e246 | p. 8

(page number not for citation purposes)
XSL• FO
RenderX
 JOURNAL OF MEDICAL INTERNET RESEARCH Liu et al

Edited by G Eysenbach; submitted 04.07.16; peer-reviewed by J Sarwark, P Knight; comments to author 17.08.16; revised version
received 24.08.16; accepted 24.08.16; published 12.09.16
Please cite as:
Liu C, Lim RL, McCabe KL, Taylor S, Calvo RA
A Web-Based Telehealth Training Platform Incorporating Automated Nonverbal Behavior Feedback for Teaching Communication
Skills to Medical Students: A Randomized Crossover Study
J Med Internet Res 2016;18(9):e246
URL: http://www.jmir.org/2016/9/e246/
doi: 10.2196/jmir.6299
PMID: 27619564

©Chunfeng Liu, Renee L Lim, Kathryn L McCabe, Silas Taylor, Rafael A Calvo. Originally published in the Journal of Medical
Internet Research (http://www.jmir.org), 12.09.2016. This is an open-access article distributed under the terms of the Creative
Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly
cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright
and license information must be included.

http://www.jmir.org/2016/9/e246/ J Med Internet Res 2016 | vol. 18 | iss. 9 | e246 | p. 9

(page number not for citation purposes)
XSL• FO
RenderX