Retrieve 3
Retrieve 3
DOI: 10.1111/1747-0080.12635
ORIGINAL RESEARCH
1
Department of Dietetics, Liverpool
Hospital, Liverpool, New South Wales,
Abstract
Australia Aims: Communication is the main method used by dietitians to conduct their
2
School of Medicine, Western Sydney practice. Yet, few evidence-based tools are available to assess dietitians' com-
University, Campbelltown, New South
munication skills to guide skill development. Further, workplace peer review
Wales, Australia
3
Dietetics Department, South Western
programs for assessment of communication skills are not standard practice.
Sydney Local Health District, Sydney, DIET-COMMS is a validated tool to assess dietitians' communication skills in
New South Wales, Australia patient consultations. The aims of this study were to implement a workplace
4
Division of Nutritional Sciences, peer review program using the DIET-COMMS tool, assess dietitians' communi-
University of Nottingham,
Nottingham, UK cation skills, evaluate inter-rater reliability and dietitian satisfaction.
Methods: Single site study within Australian hospital dietetics department. Train-
Correspondence
ing for DIET-COMMS usage was undertaken with assessors (senior dietitians) and
Stephanie Notaras, Department of
Dietetics, Liverpool Hospital, Locked Bag dietitians being assessed using an online training package and face-to-face group
7103, Liverpool BC, NSW 1871, Australia. sessions. The peer review process consisted of two rounds, occurring four to six
Email: [email protected].
gov.au
months apart. The first round was undertaken with two assessors to evaluate inter-
rater reliability. An online survey was conducted to evaluate dietitian satisfaction.
Results: Seventeen dietitians completed the program. In the first round, 13 of
17 dietitians scored 76% to 100% on the DIET-COMMS tool (median = 85%,
interquartile range [IQR] = 77-93). All dietitians scored 76-100% (median =
98%, IQR = 94-100) in the second round, with significantly higher scores com-
pared to the first (98% vs 85%; P-value <.001). The intra-class correlation coef-
ficient was 0.86 (95% confidence interval = 0.64-0.95), indicating good-
excellent inter-rater reliability. All dietitians reported the tool measured the
nutrition care process adequately and was applicable to practice.
Conclusions: The peer review program using DIET-COMMS was successfully
implemented within a workplace environment. Widespread implementation of
peer review programs using DIET-COMMS is recommended as a standard
practice for the profession.
KEYWORDS
communication, DIET-COMMS, dietetics, nutrition counselling, peer review
document was updated for a second time with feedback patient to allow the consultation to be assessed. If a
provided to enhance the assessment process. Separate patient declined, the consultation proceeded without the
face-to-face training sessions were undertaken with asses- assessors and another consultation was organised to com-
sors in December 2018 and all dietitians (assessors plete the peer review.
included) in January 2019 (Table 1). Nil further changes Following the peer review assessment, the primary
to the descriptor document were made following the assessor provided initial verbal feedback to the dietitian.
training sessions. The primary assessor and second assessor were instructed
The first round of peer review was undertaken two to to complete the tool separately and debrief their scores to
four months after the training sessions (March-May each other within 48 hours. Differing scores were noted
2019). The second was conducted four to six months later and discussed by the assessors until an agreement was
(August-September 2019). For the first round, two asses- reached. If an agreement could not be reached, the lead
sors assessed each dietitian to measure inter-rater reli- researcher was invited to assist with the process. The pri-
ability of the DIET-COMMS tool. There were eight mary assessor then provided feedback to the dietitian
assessors that were paired based on working schedules. using the DIET-COMMS tool based on the method dis-
The dietitian and primary assessor (current supervisor) cussed in the training session and specific to the items
organised a time to complete the peer review with a requiring improvement. The DIET-COMMS training
newly referred patient in an outpatient clinic or inpatient package included individual item feedback sheets for use
ward. The dietitian obtained verbal consent from the by assessors and dietitians. Further discussions on
NOTARAS ET AL. 327
TABLE 1 Outline of dietitian and assessor training in the DIET-COMMS peer review program
communication skill development were encouraged COMMS items that had the most discrepancy in scoring,
within clinical supervision sessions as required. If a dieti- a summary of the department's performance and recom-
tian was found to not have met the skills and clinical rea- mendations for improvement were presented to the dieti-
soning required for the consultation, another review was tians for discussion in a face-to-face session prior to the
scheduled. In this case, the second review proceeded as second round of peer reviews. No further changes were
the first, however, without the second assessor. The made to the peer review program.
scores from the first review were included in the analysis. The peer review process utilised a global assessment
An online survey was developed and administered via to score the skills and clinical reasoning required to
SurveyMonkey Inc. for all dietitians and assessors to evaluate the dietitian-patient consultation (skills met vs
complete after the first round (Data S2). The survey was not met—see Data S1). For research purposes, the
voluntary and anonymous. Consent was implied with the 20 DIET-COMMS items were also scored in both rounds
completion of the survey. The purpose of the survey was to investigate inter-rater reliability and differences
to evaluate the revised peer review process using the between the rounds. Each DIET-COMMS item had a
DIET-COMMS tool. The dietitian survey consisted of maximum score of two. A score of 0% to 40% was poor,
22 questions focused on evaluating the DIET-COMMS 41% to 75% was average and 76% to 100% was good.7 If
training, its perceived applicability to dietetic practice, an item could not be achieved in a consultation due to
the implementation process, challenges experienced, fur- factors external to the dietitian, the item would be given
ther support required and the impact the tool has had on a “not applicable” and removed from the total possible
their communication skills. The assessor survey consisted score to ensure a fairer scoring system when calculated
of 29 questions similar to the dietitian survey, with addi- as a percentage.
tional questions on the process of scoring the items and Double data entry was undertaken prior to analysis to
providing feedback to dietitians. The survey included assess for errors and enhance the quality of the data. No
Likert scale response options and optional free text as changes were required. Continuous variables were
answers. The survey questions were piloted within the assessed for normality using the Shapiro-Wilk test. Nor-
research team for feedback. The results from the surveys mally distributed data were presented as the mean with
guided modification and improvement to the peer review standard deviation (SD) and assessed between groups
process as required. The results of the surveys, the DIET- using an independent t-test. Non-normally distributed
328 NOTARAS ET AL.
data were presented as the median with interquartile times = 28%), checks understanding of medical condi-
range (IQR) and assessed using the Mann-Whitney U test tion (10 times = 56%), offers information on how food
and Wilcoxon Signed Ranks Test where appropriate. relates to the condition (8 times = 44%), works in part-
Inter-rater reliability was assessed using the intra-class nership with client to identify possible dietary changes
correlation coefficient one way random effects model.13 and explores possible difficulties (9 times = 50%), checks
Values of <.5 were indicative of poor reliability, 0.5 to understanding and agreement on patient determined
0.75 = moderate reliability, 0.76 to 0.9 = good reliability, goals and develops a plan prioritising key goals (9
and >0.9 = excellent reliability.13 The data were analysed times = 50%), and acknowledges patient's views and
using Statistical Package for the Social Sciences (SPSS) feelings (5 times = 28%). There were no disagreements
(Version 26; IBM Corp, Armonk, New York). A P-value in the global assessment score of whether a dietitian
<.05 was considered statistically significant. Frequency had demonstrated the skills and clinical reasoning
counting was used to describe categorical variables required during a consultation.
(Likert scale answers) and content analysis was used to All dietitians (including assessors) completed the
analyse the free text answers from the surveys.14 dietitian evaluation survey (n = 18) (Table 2) and all the
assessors completed an additional survey (n = 8)
(Table 3). All dietitians and assessors reported the time-
3 | R E SUL T S frame from the introduction and training of the tool to
its implementation as satisfactory. All dietitians and
Eighteen dietitians completed the first round of peer assessors strongly agreed or agreed that the DIET-
review and 17 dietitians completed the second round as COMMS training package was adequate, the DIET-
one dietitian was on extended leave. Two other dietitians COMMS tool was easy to understand, applicable to die-
were on extended leave and not able to participate in the tetic practice and measured the NCP adequately. All of
program. The median score in the first round was 87% the assessors and 87% of dietitians strongly agreed or
(IQR = 78-94) for 18 dietitians and 85% (IQR = 77-93) for agreed that the DIET-COMMS tool improved the peer
the 17 dietitians that completed both rounds. The intra- review process. All dietitians strongly agreed or agreed
class correlation coefficient was 0.86 (95% confidence that the descriptor document provided clear guidelines
interval = 0.64-0.95), indicating good to excellent inter- on expectations of performance.
rater reliability for the tool based on the first round of Dietitians reported what they liked most about the
peer reviews undertaken. For this reason, the second tool was the structure, detail and enhanced focus on com-
round proceeded with one assessor. The average score in munication skills throughout the entire NCP. They also
the second round was 98% (IQR = 94-100). Dietitians had felt this could improve their communication skills
significantly higher scores in the second round compared through targeted advice as the tool provided a standard
to the first (98% vs 85%; P-value<0.001). In the first round, for the provision of assessor feedback that may have pre-
13 of 17 dietitians scored 76% to 100% on the DIET- viously been assessor dependent.
COMMS tool compared to 17 of 17 in the second round.
There were 79 times (22%) that the scoring of items “Beneficial in providing a benchmark for
differed between two assessors in the first round. The dif- supervisors to help ensure the quality of
ference was discussed and resolved 76 of 79 times (96%) feedback is more similar” (Dietitian 1)
before feedback was provided to dietitians. Assistance
from the lead researcher was required in two peer “Without this emphasis in the department,
reviews and the differences in scores were resolved by easy to fall into less effective communication
referring to the item descriptor document. Differences in styles” (Dietitian 2)
scores were not resolved in 3 of 79 times (4%). These
three instances were from one dietitian's peer review, The assessors reported being somewhat confident
where the lead researcher was off site and unavailable (50%) and very or extremely confident (50%) with provid-
to assist with resolving the difference. In this case, the ing feedback using the new tool. When prompted to
rating of the primary assessor took precedence (usually report on strategies that could improve their confidence,
the clinical supervisor of the dietitian being assessed). more practice was the most common response (four of
The DIET-COMMS tool items that had the highest five responses). All assessors reported a challenge with
amount of differing scores (in ≥5 of the 18 peer reviews) interpreting whether an item was not attempted vs not
between the assessors were: outlines what to expect applicable in a consultation. This was a particular chal-
from the visit (8 times = 44%), listening to and demon- lenge in inpatient consultations as dietitians reported a
strating understanding of the patient's story (5 difficulty with achieving some the items on the tool when
NOTARAS ET AL. 329
T A B L E 2 Data from the dietitians' surveys on the use of T A B L E 3 Data from the assessors' survey on the use of DIET-
DIET-COMMS in the peer review program COMMS in the peer review program
“Difficult to achieve all aspects of NCP in an The debriefing with the second assessor was reported
in-patient setting” (Assessor 2) to mitigate part of the challenge in addition to using the
330 NOTARAS ET AL.
descriptor document. However, only 50% of assessors rater reliability. Thus, training should be thorough and
reported the descriptor document “always” or “usually” include assessors becoming familiar with the descrip-
being used to discuss scores with the second assessor. tors, practising using the tool, and discussing their
However, it was used by 75% of the assessors “always,” scores to ensure a consistent approach between asses-
“usually” or “sometimes” to assist with reaching agree- sors. Discrepancy in scoring between the two assessors
ment between differing scores. in round one of this study highlighted the variation in
dietitian opinions on how to achieve certain DIET-
“It was my haste in trying to complete the COMMS items and reduced objectivity of the tool, in
assessment that prevented me from referring particular, when assessors did not refer to the descrip-
to them more often” (Assessor 3) tor document to complete the assessment. This issue
was also raised in the initial DIET-COMMS validation
study, emphasising the importance of assessors using
4 | DISCUSSION the tool and the descriptor document as instructed to
increase scoring consistency and minimise subjectivity.7
This study presented the successful implementation of Organisational support for consistent and correct pro-
the DIET-COMMS tool in a workplace peer review pro- cesses to be followed could assist with improving the
gram to assess dietitians' communication skills in patient tool's objectivity.
consultations. The peer review program using the DIET- Although discrepancy in scoring occurred in 22% of
COMMS tool had a good to excellent inter-rater reli- the item scores, an agreement was reached between the
ability and was well accepted by dietitians based on assessors 96% of the time, usually with the assistance of
the tool's applicability to dietetic practice and ability the descriptors. Importantly, the differences in scores
to adequately assess the NCP. Further, dietitians' were minor and did not affect the tool's reliability or
scores were found to be significantly higher in the sec- the global assessment of the consultation. Instead, the
ond round compared to the first, with all dietitians differences emphasised consultation elements requiring
scoring within the highest category following the more training for dietitians to consistently perform
completion of the program. This is the first study to patient-centred care. This may be applicable to the
implement peer review within a dietetic workplace wider dietetics community, as variations in dietitians'
with the use of a validated tool focused on the assess- practice and communication skills were found in
ment of dietitians' communication and nutrition the initial DIET-COMMS validation study and also
counselling skills. experienced by students on placement.7,16 Thus, a
Dietitians often report lower confidence in their com- standardised peer review program using DIET-COMMS
munication and nutrition counselling skills and a prefer- and the extensive training package available, will assist
ence for workplace learning opportunities focused on the identification of challenging areas and differences
communication skills acquisition.1,4-6 However, most in practice. This will enable the enhancement of com-
workplace learning opportunities in dietetics are usually munication skills for the current and future dietetic
focused on clinical skills acquisition.1 Thus, the opportu- workforce to improve the quality of patient-centred
nity for the growth of dietitians' communication skills care provided to their communities.
post-graduation is dependent on an individual dietitian's This was a single centre study with a small number of
interests and/or manager's preference for evaluating and participants that may limit the generalisability of the
nurturing performance. Workplace assessment through results. The inter-rater reliability was only assessed in
peer review allows dietitians to focus on the development round one due to the good to excellent score established.
of clinical and communication skills, both vital to quality Whilst another round of inter-reliability testing may have
patient care.7,10 It is through such an assessment that produced insightful data into the process, staffing
constructive feedback can be given to dietitians to target resources were limited. Social desirability bias was possi-
areas for improvement in their consulting skills and over- ble with some assessors working closely with the
all practice.7 Long-term skill maintenance is reliant on dietitians being reviewed.17 The tool's good to excellent
regular application and reflective practice.15 It is there- inter-rater reliability was expected to have mitigated the
fore recommended that periodic peer review programs be influence of this bias on the results. Dietitians, including
undertaken to sustain communication and nutrition assessors, identified inpatient consultations as challeng-
counselling skills. ing to achieve the DIET-COMMS tool items due to time
Adequate training in the DIET-COMMS tool for and acute patient factors. This raised concern regarding a
assessors and dietitians was vital to the success of the lack of understanding on the appropriate items needing
peer review program and the good to excellent inter- to be assessed in these consultations and perceived
NOTARAS ET AL. 331
13. Koo TK, Li MY. A guideline of selecting and reporting patient experiences of dietetic consultations. J Hum Nutr Diet.
Intraclass correlation coefficients for reliability research. 2012;25(3):275-284.
J Chiropr Med. 2016;15(2):155-163. 22. Notaras S, Conti J. A qualitative analysis of eating behaviour
14. Erlingsson C, Brysiewicz P. A hands-on guide to doing content change for patients with diabetes and non-dialysis dependent
analysis. Afr J Emerg Med. 2017;7(3):93-99. chronic kidney disease stages 4–5. J Ren Care. 2018;44(4):
15. Roach RR, Pichert JW, Stetson BA, Lorenz RA, Boswell EJ, 219-228.
Schlundt DG. Improving dietitians' teaching skills. J Am Diet 23. Sladdin I, Chaboyer W, Ball L. Patients' perceptions and experi-
Assoc. 1992;92(12):1466-1470. ences of patient-centred care in dietetic consultations. J Hum
16. Palermo C, Dart J, Begley A, et al. Dietetics students' construc- Nutr Diet. 2018;31(2):188-196.
tion of competence through assessment and placement experi-
ences. Nutr Diet. 2018;75(3):307-315. SU PP O R TI N G I N F O RMA TI O N
17. De Vellis RF. Scale Development: Theory and Applications. 2nd Additional supporting information may be found online
ed. Thousand Oaks, CA: Sage Publications; 2003. in the Supporting Information section at the end of this
18. Levey R, Ball L, Chaboyer W, Sladdin I. Dietitians' perspectives article.
of the barriers and enablers to delivering patient-centred care.
J Hum Nutr Diet. 2020;33(1):106-114.
19. Yang WY, Fu Y. Level of empathy among dietitians: a pilot How to cite this article: Notaras S, Smythe K,
study. Nutr Diet. 2018;75(4):411-417. Mak M, Whitehead K. Implementation of a peer
20. Nagy A, McMahon A, Tapsell L, Deane F. Developing mean-
review program using the validated DIET-COMMS
ingful client-dietitian relationships in the chronic disease con-
tool to assess dietitians' communication skills in
text: an exploration of dietitians' perspectives. Nutr Diet. 2019:
1-13. the workplace. Nutrition & Dietetics. 2021;78:
21. Hancock RE, Bonner G, Hollingdale R, Madden AM. 'If you lis- 324–332. https://doi.org/10.1111/1747-0080.12635
ten to me properly, I feel good': a qualitative examination of
Copyright of Nutrition & Dietetics is the property of Wiley-Blackwell and its content may not
be copied or emailed to multiple sites or posted to a listserv without the copyright holder's
express written permission. However, users may print, download, or email articles for
individual use.