THE ABC of PERFORMANCE - BASED ASSESSMENT

OSCE

By Berhanu Wale Yirdaw (BSc and MSc.)

Teda Health Science College
Gondar, Ethiopia, 2024
The ABC of Performance (OSCE) Based Assessment

By Berhanu Wale Yirdaw (BSc and MSc)

Address of the author:

 [email protected]

 https://orcid.org/0000-0003-0489-9634

 https://t.me/ethionurse

 +251 9 24 52 62 43 or 66 82 02 50
Letters to the user

Dear user,

Welcome to this the ABC of performance-based assessment manual. The

assessment of candidate’s performance is mainly determined by the choice of
proper measurement tool. OSCE is the powerful tool that increases the validity
and reliability of performance assessment with the chief principles of objectivity,
structure and standardization. OSCE assesses the candidate’s performance in a
simulated environment, at the ‘shows how’ level of Miller’s assessment pyramid.
Well-understood and designed OSCE can strive learning and have a positive
educational influence. In this manual, you will learn the basic OSCE (the concept,
the key elements, the key principles and features, the use, the types of stations,
the format or design, planning and implementation, the examinee’s performance
and examination process evaluation and feedback provision to the examinee).
This manual is necessary, but not sufficient for the full understanding of
student’s performance assessment (OSCE).

In preparing this manual, I have tried to make it as short, clear and brief
as possible. Even though I tried my best to be as accurate as possible, I bear all
the responsibilities for mistakes this manual may have. Your comments with
regard to this manual are very much welcome and you can send them to the
following address:

Berhanu Wale Yirdaw

[email protected]

+251 9 24 52 62 43 or 66 82 02 50

By Berhanu Wale Yirdaw (BSc and MSc.) Page - I

Acknowledgments

Above all I would like to express my gratitude to the Lord Jesus Christ and
his mother the Virgin St. Marry who carries all my burdens and shepherded my
health.

Secondly, my special gratitude is going to all teachers who taught me from

the primary education to the postgraduate program for their fatherly guidance,
encouragement and patience during my life.

Thirdly, my gratitude also extends to Teda Health Science College higher

administratives to their valuable effort in the facilitation of internet access: I
really appreciate their commitment and sincerely thank so much.

At last (but not least), I am also grateful to all my families and friends
who encouraged and support me throughout my work.

Berhanu Wale Yirdaw

By Berhanu Wale Yirdaw (BSc and MSc.) Page - II

 Table of content

Contents Page

Letters to the user ......................................................................................... I

Acknowledgments .......................................................................................... II

Introduction to OSCE ..................................................................................... 1

Key element in OSCE ...................................................................................... 2

Key principles and features of OSCE ............................................................. 5

The use of OSCE............................................................................................. 6

Types of OSCE stations ................................................................................ 7

OSCE format or design ................................................................................ 9

Planning OSCE ............................................................................................. 10

Implementing OSCE ..................................................................................... 11

Evaluating the examinee’s performance and the OSCE process ................. 13

Providing feedback to the examinee ........................................................... 14

References ................................................................................................... 15

Template for OSCE ...................................................................................... 16

Sample OSCE................................................................................................ 18

By Berhanu Wale Yirdaw (BSc and MSc.) Page - III

Introduction to OSCE

Assessment of the learner is a key element of education, but the assessment

of health science student’s competence is the most challenging task facing the
instructors and academic institutions. The reliability (consistency), validity
(measures what we need to measure) and feasibility or practicality remains the
key criteria on which an assessment tool can be judged.

In the past, more emphasis had been given on the assessment of knowledge
domain. Nowadays, the importance and the need of skill assessment is globally
recognized and various assessment tools are designed. Among the acceptable
and practical tools, OSCE is the one that can assess whether the learner can ‘do
procedures (skills)’ rather than simply remember, talk and write about them.

 Objective Structured Clinical Examination (OSCE) is a performance-based

examination in which examinees are observed and scored as they rotate
around a series of clinical encounters (called stations) according to a set plan.
 OSCE was introduced in the 1970s and it has been recognized and widely
applied in health science academic institutions throughout the world over
the last 30 decades.
o It was first conducted in Dundee, Scotland in 1972.
▪ By 18 testing and 2 rest stations.
▪ Each station was 4.5 minute long with a 30 second break
between the stations.
o And also, OSCE was first described in the literature in 1975.
 OSCE assess the learner’s performance in a simulated environment at the
‘shows how’ level of Miller’s assessment pyramid (figure 2).
 OSCE chiefly gives attention to the objectivity and standardization.
 And also, subjective bias is removed or minimized as much as possible.
 It is possible for the OSCE to assess all the three learning domains.
o But, a more appropriate method to assess the psychomotor domain.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 1

Key element in OSCE

 In OSCE there are three variables:

1. The examinees
2. The examiner
3. The patient
o Patients can be represented through:
▪ Real patient
▪ Standardized or simulated patient
▪ Models or manikins
▪ Video recording of a patient
▪ Investigation and imaging result
▪ Case record, …etc.

Examinee

OSCE

Examiner Patient

Figure - 1: the three variables in a OSCE or practical examination.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 2

Examiners and their role:

 In addition to teachers (academic staffs),

o Health professionals (clinicians) and real or simulated patients can
serve as examiners in OSCE.
o The role of examiners must be clearly defined and made explicit in
written instructions.
o Training and full briefing of the examiner is essential.
 Examiners can have the following roles:
 Check resources at the station for which the examiner is
responsible.
 Greet the examinee and check his or her name and/or ID number.
 Observe the examinee and complete the checklist and/or the rating
scale.
 Provide comments on the score sheet with regard to the examinee’s
performance which will serve later as feedback to the examinee.
 Ensure that examinees move to the next station on the time signal.
 Keep a record of any problems that arise in the examination.
 Decide the outcome for each examinee on the basis of agreed
standard.
 Provide feedback to examinees individually or in a group.
 Evaluate the stations and the examination process with a view to
determine whether any changes are required on a future occasion.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 3

Hierarchy of assessment approaches

 Miller’s Pyramid
• Describes the hierarchy of assessment approaches with increased
authenticity:
 Knows (the base of the pyramid).
 Knows how to apply the knowledge in practice.
 Shows how i.e., able to demonstrate.
 Does (the top of the pyramid).

Does: on the job

assessment.

Shows how: OSCE

assessement

Knows how: written

assessment

Knows: written
assessment

Figure - 2: Miller’s assessment pyramid with related approaches.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 4

Key principles and features of OSCE

 The three chief principles of OSCE: objectivity, structure and standardization.

 The examinees should be assessed over a number of stations (10 to 20).
o Sometimes called a multistation clinical examination.
o As the number of stations increases, the reliability of examination
increases.
o A collection of stations is called circuit.
 Uniform examination i.e., all examinees are assessed on the same set of
stations.
o Stations should be identical in terms of the tasks expected and the
scoring sheets.
 The time allocated to each station is uniform (usually, 5 to 15 minutes).
o To allow a smooth rotation of examinees around the stations.
 Stations should be arranged sequentially.
 At each station one or more element of competence can be assessed.
 What is assessed at each station should be clearly specified.
 Grid maps or blueprint should be prepared.
 Should be structured i.e., carefully planned & agreed upon before implemented.
 Specification of standards required.
o The minimum requirements for a pass should be set.
▪ Excellence can also be recognized.
 The tool should be reliable, valid and feasible.
o Reliable: consistent with occupational standard and curriculum.
▪ The reliability of the OSCE correlates with the number of
stations.
▪ The greater the number of stations, the greater the reliability.
o Valid: measure what should be measured and have educational
impact.
o Feasible: practical (cost effective), fair (free from bias) and flexible.
 Should be supported with feedback to a learner about their clinical competence.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 5

The use of OSCE

• Certify a student’s or trainee’s clinical competence (summative

assessment).

• Can contribute to a teaching and learning process.

• Many competencies and learning outcomes can be assessed.

• Increase the learner’s breadth of competency and proficiency.

• Evaluate the curriculum and/or its content.

• Evaluate course delivery.

• Evaluate the teacher.

• Help faculty development.

• Can be used on a small and/or large scale.

o Small scale: at the level of an individual course or department.

o Larger scale: as a national qualifying or exit examination.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 6

Types of OSCE stations

 Basically, there are two types of stations in the OSCE.

1. Procedure stations:
o Where the student performs a procedure.
o Examiners are there and stations are sometimes called manned
stations.
o Examinees are watched and assessed as they carry out a practical
procedure.
o The student’s history taking techniques, the questions they ask and
their approach to the patient are assessed also assessed on history
taking station.
2. Question station:
o Where the student is to answer questions.
o No examiner is present and student responses are on paper or on
computer.
▪ The stations are sometimes called unmanned stations.
o The examinee’s response can be in the form of:
▪ Multiple-choice question (MCQ).
▪ Short constructed response to a question.
▪ Post-encounter notes i.e., notes about the patient they have
seen (about their findings at the previous station and the
conclusions they have reached based on their findings).
▪ Oral report to an examiner.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 7

Occasionally, double station may be organized.

 When the time allocated to the station is too short and a longer period of time
is required to assess one aspect of competence, in this case a double station
may be organized.
 One station is duplicated with the ‘A’ and ‘B’ versions each having their own
examiner.
 When the first-time signal is given, the student does not move on to the next
station.

1
2
8

3A 3B 7

4 6
5

Figure - 3: organization of one double stations (station 3) with alternate

students passing through each version of the station.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 8

OSCE format or design

• The choice of OSCE format or design depends on the following factors:

o The number of examinees.
o The type and number of stations.
o The length of time allocated for each station.
▪ The commonly used time for a station is 5 to 15 minutes.
• But, up to 25 minutes.
▪ The total time available for an OSCE is relatively fixed.
▪ There is an inverse relationship between the number of
stations and the time allocated to each station.
• OSCEs with more stations allow less time at each
station.
• OSCEs with a smaller number of stations allow more
time at each station.
▪ Example: a 120-minute examination will have three options:
• 8 stations, each of 15 minutes or
• 12 stations, each of 10 minutes or
• 24 stations, each of 5 minutes.
o The purpose and breadth of the examination.
o The learning outcomes to be assessed.
o The resources available.
o The context of the local situation.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 9

Planning OSCE

 To deliver a successful examination, careful planning is essential.

 Actions required to advance OSCE:
o Organize (identify and agree) committee including the key
stakeholders.
o Confirm the purpose of the examination and the areas to be
assessed.
o Set timeline for the work to be undertaken in preparing for &
delivering OSCE.
o Decide the number and duration of stations as well as circuits.
o Arrange a suitable venue to accommodate the OSCE.
o Prepare an OSCE grid or blueprint.
o Set up stations and organize resources (furniture’s and
equipment’s).
o Develop checklists including the rating scale and clear instructions.
o Assign examiners, ascertain their availability and commitment.
▪ Reserve examiner should be considered.
o Prepare examinee waiting venue until their turn.
o Inform students regard to OSCE format, when and where it will be
held.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 10

Implementing OSCE

 OSCE proceed smoothly on the day of the examination if planning has

been carefully undertaken.
 The success of OSCE also depends on the organization on the day.
 OSCE lead has a coordinating role on the day of the examination.
o Manage the overall process and deal with any problems as they
arise.
o Ensure that everything is in place, the signposts are clear, the
examinees and examiners are briefed.
o Where there are multiple circuits, appoint a coordinator for each
circuit.
▪ Responsible to the OSCE lead.
 Activities performed on the day of examination:
o Stakeholders should be arrived early (30 minutes before
examination start).
▪ Stakeholders: OSCE lead, examiner, examinee & committee
members.
o Final brief by who delegated for this responsibility.
o Examiners handed their station packet with instructions and
scoring sheets.
o Examinee should be, debriefed:
▪ The range of competencies to be assessed.
▪ The circuit they assigned and the time allocated for each
station.
▪ To wear a name badge.
▪ Read carefully the instructions provided at each station and
if you are in doubt, ask the examiner for clarification.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 11

 ▪ To start and/or should move to the next station when bell or
another signal heard (a 1-minute warning signal may be
given).
o Refreshments should be made available for the examiners.
o Collect examiner score sheets and candidate response sheets.
o At the end of the examination, thank all concerned.
o Organize examinee feedback and organize debriefing if needed.
o Document any problems that have faced.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 12

Evaluating the examinee’s performance and the OSCE process

 The extensive effort undertaken during planning and implementation of

OSCE will be wasted unless careful attention is paid to:
o The assemble of evidence during OSCE that reflects examinee
performance.
o The utilization of the evidence to make decisions about the
examinee.
o The provision of meaningful feedback to the examinee.
 All of the stakeholders should be engaged with the evaluation.
 Evaluation of the OSCE can lead to better understanding of the OSCE and
to improved future practice.

Collection of evidence about the examinee performance

Checklist Red flags

Information processing

Decision about the examinee

Pass or fail Feedback

Figure - 4: evaluation of examinee in OSCE.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 13

Providing feedback to the examinee

 Feedback implies the specific information about the comparison between

examinees observed performance and a standard, given with the intent to
improve the examinee’s performance.
 The provision of feedback about performance to the examinee has been
described as one of the four key principles in making the teaching –
learning process effective.
o Feedback together with Activity, Individualization and Relevance
makes up the FAIR principle (Harden and Laidlaw, 2012).
 Telling the examinee that they have passed or failed an OSCE, that their
score was 80% or that they got a ‘C’ grade is not useful feedback.
o To be useful, the feedback to the examinee in OSCE should focus to
their performance in individual stations and to their overall
performance in relation to outcome domains.
 Feedback to be relevant, it should be:
o Timely: during the OSCE at a station or immediately.
o Individualized.
o Specific, positive and clear.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 14

References

 Healthcare simulation at a glance, 2019.

 The definitive guide to the OSCE, 2016.
 Macleod's Clinical OSCEs, 2016.
 Succeed in OSCEs & Practical Exams: An Essential Guide for Nurses, 2010.
 OSCEs at a glance / Adrian Blundell, Richard Harrison, 2013
 Kamran Z. Khan, Sankaranarayanan Ramachandran, Kathryn Gaunt &
Piyush Pushkar: The Objective Structured Clinical Examination (OSCE),
2013: https://doi.org/10.3109/0142159X.2013.818634
 Newble D. Techniques for measuring clinical competence: objective
structured clinical examinations, 2004: https://doi/10.1111/j.1365-
2923.2004.01755.x.
 Carraccio C, Englander R.: The Objective Structured Clinical Examination.
A step in the direction of competency-based evaluation, 2000.
 R. M. HARDEN: What is an OSCE?1988.
 HARDEN, R.M. & GLEESON, F.A.: Assessment of clinical competence using
an objective structured clinical examination (OSCE), 1979.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 15

Template for OSCE

◊ Name of the academic institution

◊ Objective Structured Clinical Examination (OSCE)
◊ Title: ……………………………………………………………………
◊ Program: ………………………………………………………………
◊ Course: …………………………………………………………………
◊ Name of procedure: …………………………………………………
◊ Necessary equipment’s: …………………………………………….
◊ Description:
o This OSCE is designed and used to assess ……………………………
◊ Role of examination:
o …………………………………………………………
◊ Venue: ……………………………………
◊ Total number of candidates: ……………………
◊ Circuit number: …………………… Station number: ………………
◊ Examiners: …………………………………………………
◊ Patient: ……………………………
◊ Criteria for pass or fail decision: ………………………………………
◊ Instructions for students:
 Context and essential information’s:
………………………………………………………………………………………………
………………………………………………………………………………………………
………………………………………………………………………………………………
………………………………………………………
 Time allowed: ……………………………
◊ Instructions for examiner:
o …………………………………………………………………………………
o …………………………………………………………………………………
o …………………………………………………………………………………
o …………………………………………………………………………………

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 16

 Marking checklist and criteria

Student name: ………………………………………… ID: ……… Signature: …………

Tasks or steps 2 1 0

1. ………………………………

2. ………………………………

3. …………………………………

4. …………………………………

5. …………………………………………

6. ………………………………………

7. ………………………………………

Total mark

Questions to probe the student’s underpinning knowledge (if any):

1. ………………………………………………… (their value and answer).
2. …………………………………………………… (their value and answer).
3. …………………………………………………… (their value and answer).

Name of examiner: …………………………… Signature: …………… Date: ……………

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 17

Sample OSCE

Teda Health Science College Department of Nursing

Objective Structured Clinical Examination (OSCE)

◊ Title: institutional practice assessment

◊ Program: nursing (level III)
◊ Course: basic first aid and emergency response
◊ Name of procedure: cardiopulmonary resuscitation (CPR)
◊ Necessary equipment: CPR doesn’t need special equipment’s (but pocket mask & glove).
◊ Description:
o This OSCE is designed and used to assess the students’ knowledge and skill
in relation to cardiopulmonary resuscitation (CPR).
◊ Role of examination:
o Summative contributing to in-school assessment and students should be
required to pass the examination before proceeding to the next phase.
◊ Venue: nursing simulation (skill lab) room.
◊ Total number of candidates: 162 students
◊ Circuit number: 01 Station number: 01
◊ Examiners: Teda health science college nursing instructors.
◊ Patient: resuscitation manikin.
◊ Criteria for pass or fail decision: standard referenced, decided by the national TVET
grading criteria and system (i.e., 32% out of 40%).
◊ Instructions for students:
 Context and essential information’s: you are conducting home to home visit with a
member of community nursing team. You and your mentor went to a local
restaurant to take a break. The waitress is in the kitchen making food for you and
your mentor. You heard a loud bang from the kitchen and you go to see what has
happened. When you enter the kitchen, you see that the waitress is motionless on
the floor. You are required to assess the victim and perform CPR.
 Time allowed: 5 minutes (including knowledge questions).
◊ Instructions for examiner:
o Greet the student and read the above context and essential information’s.
o Ensure the student’s full name and ID is on the marking sheet.
o Observe the students’ performance and complete the marking sheet
according to the marking schedule (performed completely = 2, performed
partially = 1 and not performed = 0).
o Total the student marks.
o Sign the marking sheet.
o Do not interact with the student during and after completion of the task.

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 18

 Marking checklist and criteria

Student name: ………………………………………………… ID: ………… Signature: …………

Tasks or steps 2 1 0
1. Check the scene is safe
2. Check victim’s response: shake the victims’ shoulder and ask
“are you okay?”.
3. Shout for help and/or call for ambulance.
4. Check pulse (circulation) and breathing simultaneously:
a. Look, listen and feel for breathing.
b. Check carotid pulse for circulation.
5. If no pulse and breathing, start CPR:
a) Lay the victim on his/her back & kneel beside the victim.
b) Exposes chest & find the lower half of sternum.
c) Position your hand (interlock fingers) with arms straight.
d) Depress and release the chest rhythmically.
e) Following 30 compressions, give ventilation:
i. Open the airway.
ii. Pinch the victim’s nose.
iii. Give 2 ventilations using a pocket mask.
f) Continue compressions & ventilation with 30:2 ratio.
6. After 5 cycles, check spontaneous breathing and circulation.
7. Place the victim in the recovery position.
Total mark
Questions to probe the student’s underpinning knowledge:
1. What is the clinical indication for CPR? Cardiac and respiratory arrest. (2 point)
2. What are the characteristics of high-quality CPR? (2 point)
a. Push hard (5 cm) and fast (100 – 120 compressions per minute).
b. Allow complete chest recoil.
c. Minimize interruptions in compression.
d. Avoid excessive ventilation.
e. Compression to ventilation ratio is 30:2.
3. When should CPR stopped? (6 point)
a. Signs of return of spontaneous circulation.
b. AED is ready to analyze the patient’s heart rhythm.
c. Other trained rescuers take over the resuscitation responsibilities.
d. If valid do not resuscitate (DNR) order.
e. If alone and too exhausted to continue.
f. If the scene becomes unsafe.

Name of examiner: ………………………………………… Signature: …………… Date: ……………

By Berhanu Wale Yirdaw (BSc and MSc.) Page - 19