See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/12011398

OSCE: Scope of the OSCE in the assessment of clinical skills in dentistry

Article  in  British dental journal · April 2001

DOI: 10.1038/sj.bdj.4800961a · Source: PubMed

CITATIONS READS

60 1,427

3 authors, including:

Peter Anthony Mossey

University of Dundee
178 PUBLICATIONS   5,413 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

The Modell Global Database of Congenital Disorders (MGDb) View project

I am employing genetics, genomics and exposomics tools to study the etiology of ororfacial clefts in the Ghanaian population View project

All content following this page was uploaded by Peter Anthony Mossey on 05 January 2015.

The user has requested enhancement of the downloaded file.

EDUCATION
OSCE

Scope of the OSCE in the assessment of

clinical skills in dentistry
P. A. Mossey1, J. P. Newton2 and D. R. Stirrups3

Introduction The objective structured clinical examination (OSCE) is now an are that, (a) different students are given dif-
accepted tool in the assessment of clinical skills in dentistry. There are however no ferent patients with different presenting
strict or limiting guidelines on the types of scenario that are used in the OSCE problems, and (b) examiner subjectivity
examinations and experience and experimentation will inevitably result in the resulting in inter-examiner variation in the
refinement of the OSCE as a tool for assessment. assessment of the same performance. It was
Aim The aim of this study was to compare and contrast different types of clinical primarily for these reasons that Harden et
operative skills scenarios in multi-station OSCE examinations. al2 introduced an alternative approach,
Methodology Student feedback was obtained immediately after the sitting of an arguing that there was a need to remove
OSCE examination on two different occasions (and two different cohorts of students). patient and examiner variation.
The same questionnaire was used to elicit the responses. They devised the Objective Structure
Results The questionnaire feedback was analysed qualitatively with particular regard Clinical Examination (OSCE) primarily for
to student perception of the usefulness and validity of the two different kinds of OSCE use in undergraduate medical assessment.
scenarios. This examination was structured in that the
Conclusions OSCE scenarios which involve phantom heads are perceived to lack questions had a well defined marking sys-
clinical authenticity, and are inappropriate for the assessment of certain clinical opera- tem with predetermined answers and
tive skills. While the OSCE is useful in the examination of diagnostic, interpretation pass/fail criteria. It was also structured in
and treatment planning skills, it has apparent limitations in the examination of inva- that it comprised of a series of consecutive
sive operative procedures. timed stations, and was clinical in that these
stations comprised scenarios to test specific
clinical skills including diagnosis, interpre-
he dental schools in the UK are cur- tance of the attainment and maintenance of tation and treatment planning. 3
T rently considering their curricula in the
light of the recent General Dental Council
high clinical standards in professional den-
tal practice. It is incumbent on dental prac-
The gradual evolution and development
of this methodology has led to its wide-
(GDC) publication entitled, ‘The First Five titioners and specialists in dentistry to attain spread use as an assessment tool in under-
Years — The Undergraduate Dental Cur- the necessary competencies and by continu- graduate and postgraduate medical and
riculum’,1 which was released in March ing education to keep up-to-date with the dental assessment.4,5,6,7 It has also been
1997. This document states that it ‘must be latest developments. This is important for used for the assessment of certain clinical
possible to demonstrate the presence of the provision of best treatment and for skills, communication skills on simulated
essential elements’ (of the undergraduate maintaining high standards of care for patients and even clinical decision making
curriculum) and so implies the need to patients. skills. 8, 9, 10, 11,12
define the essential knowledge, skills and It is equally important that appropriate One of the main strengths of the OSCE
attitudes to be achieved by the end of the methods of assessment of clinical compe- examination is its inherent objectivity
undergraduate dental course. tencies are developed so that it is possible to whereby the aim is to remove patient and
Further initiatives by the General Dental detect a fall in standards below an accept- examiner variation so that the only variable
Council with regards to continuing profes- able level. This paper examines some of the being examined is the ability of the candi-
sional development, reaccreditation and more recent methods being developed for date. Other advantages of the OSCE system
recertification reflect the increasing impor- the assessment of a range of clinical and include the flexibility and versatility made
operative skills in undergraduate dentistry, possible by the multiple station design. This
but which may also be applicable for post- means that it is possible to examine a range
graduate / GPT assessment. of skills and disciplines and even to incorpo-
1.Senior Lecturer in Orthodontics, Unit of Dental and
Oral Health, University of Dundee, Scotland, UK rate more than one skill or discipline simul-
2. Senior Lecturer in Integrated Oral Care, Unit of Objective Structured Clinical taneously in the design of a particular
Comprehensive Restorative Care, University of Examination (OSCE) station.
Dundee, Scotland, UK
3. Professor of Orthodontics & Sub-Dean of Teaching,
Over the years, the traditional clinical exam- Examples of generic skills applicable to a
Unit of Dental and Oral Health, University of ination in dentistry (as in medicine) has range of disciplines include communication
Dundee, Scotland, UK been the ‘long case’ — a patient presenting skills, dental charting, aspects of history tak-
Address for editorial correspondence: Dr Peter Mossey with a relevant clinical problem or condi- ing, impression taking and cross infection
Unit of Dental and Oral Health
University of Dundee Dental School, Park Place, tion, and the student is instructed to do a control. The application of the Objective
Dundee, DD1 4HR. Tel: 01382 425761. Fax 01382 diagnosis and treatment plan under exami- Structure Clinical Examination to the
206321. E-mail: [email protected] nation conditions and present the findings, assessment of undergraduate dental skills
REFEREED PAPER
Received 23.02.00; Accepted 13.07.00
usually verbally to an examiner. has recently been described by Mossey et
© British Dental Journal 2000; 189: 323–326 The two major drawbacks of this system al13 and Davenport et al.14

323 BRITISH DENTAL JOURNAL VOLUME 190. NO. 6 MARCH 24 2001

 EDUCATION
OSCE

Student Perception of The Objective

Structure Clinical Examination
Table 1 Using the OSCE for the assessment of clinical skills including operative
(OSCE) procedures
While the OSCE has been used in under-
graduate medical assessment for almost 20 Circuit A OSCE Station
years, it is only in the past four to five years
that the OSCE has become an assessment Station 1 Pocket charting using special periodontal model in silicon and acrylic
tool in undergraduate dental education. Its Station 2 Full mouth radiographs - diagnosis and interpretation
Station 3 Impression taking on maxilllary element of mannequin head
interpretation and application in the assess-
Station 4 Microbiological report
ment of dental competencies varies quite Station 5 Demonstration of LA administration on a mannequin
widely, and it was perceived that the OSCEs Station 6 Prescription writing - name of drug supplied
would be used for clinical operative skills Station 7 Use of labeled instruments - demonstration of application on a study model
testing in the early days when they were Station 8 Wax-up of 3+3 on a study model
Station 9 Orthodontic appliance design
introduced to dentistry. There is however a
fundamental difference between medicine
and dentistry in the conferring of clinic Circuit B OSCE Station
skills. It is sufficient for the medical gradu-
ate to have attained an adequate level of Station 1 Case scenario - behavioural sciences health belief model
Station 2 Colour photographs and radiographs of children's dentistry scenarios interpretation
competence in examination, diagnosis and
and treatment planning
interpretation in the clinical situation, but Station 3 Case history, pictures radiographs and photomicrographs - diagnosis and treatment
the dental graduate must in addition be a planning - ameloblastoma
competent surgeon. Station 4 Articulator with PJC - occlusal problem - diagnosis and treatment planning
Station 5 Case history - drug prescription
Aim of the study Station 6 Edentulous impression - identification of problems
Station 7 Orthodontic space analysis on a set of study models
The primary aim of the study was to criti- Station 8 Periodontal chart - interpretation and treatment planning
cally analyse the scope of the OSCE in den- Station 9 Prescription of partial denture connector
tistry and to answer the following questions: Station 10 Appropriate choice and application of local anaesthesia with given case scenario
is it possible to use the OSCE for the assess-
ment of all clinical skills including operative
procedures? If so, how would these assess- different cohorts of 4th year students. The and the four point response scale was as fol-
ments be designed, and if not, what alterna- dates of these were May 1994 (circuit A) and lows: Strongly Agree (SA), Agree (A), Dis-
tive methods of assessment would be used? December 1996 (circuit B). For each circuit, agree (D), Strongly Disagree (SD).
In the experience of the authors, various feedback questionnaires (Table 1) were In addition to circling what they felt was
OSCE scenarios had been tried and tested in administered immediately after the exami- the appropriate response to each question,
the preceeding years, and in considering the nation resulting in a 100% response rate; 56 the students were encouraged to record
above questions, it was agreed that a retro- students for circuit A comprising 29 female comments or observations.
spective examination of the student feed- and 27 male students and 45 students for One circuit described as circuit A in
back from different diets of the OSCE circuit B comprising 25 female and 20 male Table 1 was mainly phantom head orien-
examination could be carried out. Two sce- students. tated with an emphasis on the assessment of
narios where there were perceived philo- There was therefore a similar male to clinical operative skills, whereas circuit B
sophical differences in the designs of the female ratio in both groups and there was deliberately avoided the use of phantom
OSCE circuits were chosen for comparison, no reason to suspect that there were any dif- heads and concentrated more on assess-
one (OSCE circuit A) with mainly ‘opera- ferences in the academic profile (back- ment of diagnostic and interpretation skills
tive’ scenarios and the other (OSCE circuit ground training or knowledge) of the two in clinical dental scenarios.
B) with ‘diagnostic’ scenarios. cohorts. In addition, there was no difference Other differences were that circuit A con-
in the students familiarity with the OSCE in sisted of 10 six-minute stations, four of
Methodology the two scenarios as the two cohorts of stu- which were phantom head stations and the
Student feedback on two Objective Struc- dents in 1994 and 1996 were being intro- students were advised that among the aims
tured Clinical Examinations (OSCEs) duced to clinical OSCEs for the first time and objectives of the examination was the
obtained by means of identical question- when the assessment was done. The ques- assessment of ‘operative’ skills. Circuit B
naires was compared. These were run as tions were worded exactly the same in the contained ten 10-minute stations consisting
Class examinations at different times with feedback questionnaires in both scenarios, of clinical dental scenarios without phan-

BRITISH DENTAL JOURNAL, VOLUME 190, NO. 6, MARCH 24 2001 324

EDUCATION
OSCE

tom heads and the students were advised

that certain clinical skills were being Table 2 Student feedback questionnaire - Percentage responses
assessed, but there was no mention of oper-
ative skills being an objective of this OSCE
examination. SA (Strongly Agree) A (Agree) D (Disagree) SD (Strongly Disagree)
The objective of the study was to investi- SA A D SD
gate whether OSCEs with or without phan- I understand the aims and objectives of the OSCE
tom head scenarios were equally acceptable Circuit A (12) 21% (37) 66% (5) 9% (2) 4% (n=56)
Circuit B (26) 59% (19) 41% 0% 0% (n=45)
to students, and whether they are perceived
to be equally acceptable in the assessment of The OSCE tested my diagnostic clinical skills
both diagnostic and operative clinical skills Circuit A (7) 12% (16) 29% (22) 39% (11) 20% (n=56)
in dentistry. Circuit B (17) 38% (23) 51% (4) 13% (1) 2% (n=45)

The OSCE tested my operative clinical skills

Results
Circuit A (3) 5% (16) 29% (24) 43% (13) 23% (n=56)
Table 2 outlines the results of the two ques- Circuit B (3) 7% (6) 13% (31) 69% (5) 11% (n=45)
tionnaires in response to questions 1-6. This
reveals a strong consensus that the aims and The OSCE covered a wide range of skills and dental disciplines
objectives were understood for both circuits Circuit A (9) 15% (35) 63% (9) 16% (3) 5% (n=56)
Circuit B (18) 39% (25) 55% (2) 5% 0% (n=45)
and that a wide range of skills and dental
disciplines were tested. Circuit A was how- The OSCE indicated areas where I need to learn more
ever considered by the students to be much Circuit A (3) 5% (28) 50% (20) 36% (5) 9% (n=56)
less useful, being perceived to be a relatively Circuit B (12) 26% (28) 62% (5) 12% 0% (n-45)
poor test of clinical diagnostic skills (only
The OSCE was a useful educational exercise
41% agreed) and not a good indicator of
Circuit A (3) 5% (26) 46% (18) 32% (9) 16% (n=56)
which areas needed to be improved or Circuit B (23) 50% (23) 50% 0% 0% (n=45)
revised.
In addition, it was interesting to note
where the students were advised to expect
that their clinical operative skills would be nized the lack of clinical skills testing. Sam- interests as future clinicians to get as much
tested, 66% of them still disagreed (D and ples of comments included ‘this does not experience in doing these as possible so that
SD) that this had actually occurred. In cir- really test manual dexterity at all,’ ‘should be we can deliver concise, accurate, well pre-
cuit B where there was no attempt to test more effort to assess clinical operative skills’, sented care for our patients.’
clinical operative skills and no scenario set ‘better to examine communication skills on
up with that in mind, 80% disagreed as the clinics.’ Discussion
expected. However 89% agreed that circuit The conclusion from this qualitative It is deemed to be important that in under-
B had tested their clinical diagnostic skills analysis lends support to the notion that graduate dentistry methods are developed
compared with 41% in circuit A. The stu- clinical operative skills in dentistry cannot for the assessment of operative clinical
dents therefore generally disagreed that be validly assessed using unrealistic phan- skills, and the OSCE has certain advantages,
their operative skills were being validly tom head scenarios. Other student feed- being standardised and reasonably objec-
tested in either of the two OSCE circuits. back comments which reflect perceived tive. Attempts to set up clinical scenarios
Some qualitative feedback comments problems in the testing of operative clinical using ‘phantom heads’ and acrylic models
recorded on the questionnaires would lend skills using OSCEs are that ‘exam pressure in an effort to simulate the chairside clinical
support to this impression. Examples of definitely affects performance for what are situation have been monitored with the help
comments from the circuit A questionnaires normally simple clinical tasks — your mind of staff and student feedback. In a debriefing
included ‘phantom head stations too far can go blank’ and ‘the best place to examine meeting set up to elicit feedback from staff
removed from clinical reality’, ‘found it dif- clinical skills is on the clinic’. after the examination and with the help of
ficult to take phantom head stations seri- One comment from circuit B however the feedback elicited from the student ques-
ously,’ and ‘are we expected to wash our encompasses the positive side of OSCEs and tionnaires the following conclusions
hands and wear gloves when dealing with their perceived usefulness for assessment of emerged. The main drawbacks identified in
phantom head procedures?’ diagnostic and treatment planning skills: the use of these scenarios included the fol-
Some of the comments in circuit B how- ‘An excellent idea to enhance and expand lowing:
ever also indicate that the students recog- clinical knowledge. It would be in our best (a) Lack of clinical authenticity. Unreal-

325 BRITISH DENTAL JOURNAL, VOLUME 190, NO. 6, MARCH 24 2001

 EDUCATION
OSCE

istic compared to authentic clinical situa- communication problems, provided the 3. Cushieri A, Gleeson F A, Harden R M, Wood
R A. A new approach to a final examination in
tions in the completion of routine clinical scenarios are ‘non-invasive’ procedures. surgery, use of the objective structured clinical
tasks such as soft tissue manipulation (e.g. There is convincing evidence justifying examination. Ann Royal College of Surgeons of
cheek and tongue retraction), moisture the place of OSCEs in the armamentarium England 1979; 61: 400-405.
control, bleeding or crevicular fluids man- for assessment of clinical skills, and in medi- 4. Thomson D M. The objective structured
clinical examination for general practice;
agement problems. Also unrealistic scenar- cine it is reported that OSCEs have a benefi- design validity and reliability. J R Coll Gen Pract
ios with regard to assessment of clinical cial effect on student learning by 1987; 37: 149-153.
effectiveness such as administration of LA encouraging an orientation towards clinical 5. Walker R, Walker B. Use of the OSCE for
assessment of vocational trainees for general
(e.g. was the LA administered effectively aspects.
practice. J R Coll Gen Pract 1987; 37: 123-124.
and did it achieve adequate anaesthesia?) In the curricula where OSCEs are part of 6. Jewell D. Learning through examinations: use
and cross-infection procedures unrealistic. the summative assessment, students spend of an objective structured clinical examination
(b) Lack of communication skills testing. more time learning in the clinic as opposed as a teaching method in general practice. J R
Coll Gen Pract 1988; 38: 506-508.
No opportunity for testing of communica- to the library, they concentrate more on 7. Sloan D A, Donnelly M B, Johnson S B,
tion skills — not only the routine commu- clinical skills and a greater degree of motiva- Schwartz R W, Strodel W E. Use of an Objective
nication skills in ‘interviewing’ such as tion for clinical work is reported.15 This is Structured Clinical Examination (OSCE) to
opening, closing etc, but also those associ- testament to the formative element of this measure improvement in clinical competence
during the surgical internship. Surgery 1993.
ated with clinical tasks. Prior explanation of type of examination. 114: 343-50.
a clinical procedure or treatment options, 8. Gerritsma J G, Smal J A. An interactive patient
establishment of rapport with a patient, Conclusion simulation for the study of medical decision-
making. Med Educ, 1988; 22: 118-23.
motivating patients and use of appropriate While there is no doubt that OSCEs are a 9. Gordon J, Sanson-Fischer R, Saunders N A,
jargon when dealing with children, valuable and versatile method for of assess- Identification of simulated patients by interns
teenagers and adults before and during clin- ment in clinical disciplines, it is apparent in a casualty setting. Med Educ, 1988; 22: 533-
ical procedures. that they are best suited to the assessment of 538.
10. McAvoy B. Teaching clinical skills to medical
(c) Lack of patient management / behav- diagnostic, interpretation and treatment students: the use of simulated patients and
ioural problems. There are certain ‘routine’ planning scenarios and have limitations in videotaping in general practice. Med Educ,
patient management problems in dentistry the assessment of clinical operative proce- 1988; 22: 193-9.
11. Davies M. The way ahead: teaching with
such as dealing with apprehension, restless- dures. Furthermore students are sensitive to
simulated patients. Med Teach 1989; 11:
ness and anxiety, or managing a gagging these limitations. 315-20.
reflex during the taking of an impression or In dentistry, therefore, there is a need to 12. Clinical decision making - an art or a science?
making a complete denture. It is not possi- develop and evaluate objective methods for Part III: To treat or not to treat? Br Dent J 1995;
178: 153-155.
ble to set up OSCEs for the observation of assessment of invasive clinical operative 13. Mossey P A. Clinical Skills Assessment in
such clinical scenarios. procedures. A method of assessment Dentistry. Guide to Assessment of Students
This highlights the shortcomings of designed to address this need is being devel- Progress and Achievements, eds. Godfrey and
phantom head scenarios, which are inade- oped and a detailed analysis will be pre- Heylings. 1997: 78-81.
14. Davenport E S, Davis E C, Cushing A M,
quate not only from the operative view- sented in a sister publication. Holsgrove G J. An innovation in the assessment
point, but the lack of authenticity from an of future dentists. Br Dent J 1998; 184: 192-194.
1. General Dental Council, The First Five Years. 15. Feickert J A, Harris I B, Anderson B C,
interpersonal skills, behaviour management The undergraduate curriculum. March 1997. Bland C J, Allen S, Poland G A, Satran L, Miller
and contingency management viewpoint 2. Harden R M, Stevenson M, Downie W W, W J. Senior medical students as simulated
are also apparent. Simulated patients may Wilson G M. Assessment of clinical patients in an objective structured clinical
help to offset some of the aforementioned competencies using objective structured examination: motivation and benefits. Med
examination. Br Med J 1975 1: 447- 451. Teach 1992; 14: 167-77.

BRITISH DENTAL JOURNAL, VOLUME 190, NO. 6, MARCH 24 2001 326

View publication stats