14 Assessment of Practical Skills

Learning Objectives
 Explain the basis rationale of objective structured clinical examination (OSCE).
 Write OSCE stations for different practical settings.
 Conduct an OSCE in actual clinical setting.
 Use a mini CEX encounter to provide formative feedback.
 Evaluate a case presentation objectively.

One of the most important aspects of training of a doctor is acquisition of

practical skills - after all, patients do not come to quiz their doctor on the
differential diagnosis or management of their problem. Yet, objective
assessment of practical (or psychomotor, as they are called) skills poses a
formidable challenge for an examiner. Have you ever wondered during a
practical examination regarding points to discriminate between good and
Concept not - so - good students? Often, the evaluation is so subjective that different
examiners grade the students on their own criteria. For a minute, recall
one of the basic purposes of evaluation—yes, you are right- providing
feedback to the student. Unfortunately, our conventional examinations
do not provide any feedback except saying pass or fail and thus do not
provide any opportunity to the student to improve.
There have been many innovations to overcome this problem but the
one that we are going to discuss with you is called OSCE. Sounds like a
French name ? Well, OSCE stands for Objective Structured Clinical
Examination and has been designed for objective assessment of bedside
clinical competence. Let you may have doubts about this mode of
examination, let us weigh OSCE in the desirable qualities of an assessment
instrument viz. validity, reliability and feasibility. Later, we will also
discuss another tool called mini CEX.

Validity, you will recall is the ability to measure, what is intended to be

measured. A practical examination should evaluate the ability of a
Attributes of candidate to obtain relevant history, perform a physical examination,
OSCE reach a probable diagnosis, interpret laboratory reports and recommend
a management protocol. As we shall discuss with you a little later, all
these can be assessed by OSCE. The conventional examination, on the
 Assessment of Practical Skills 83

other hand focusses on reporting of abnormal findings only, ignoring the

‘doing’ part of it.
Reliability as you have learnt, refers to consistency of measurement.
In an OSCE, all students examine the same patient and are marked on
predetermined ‘checklists’ with the results that inter- observer variation
is reduced to a minimum. Thus, any difference in marks is more directly
attributable to the ability of the students, rather than to extraneous factors.

By now you must be wondering about what exactly OSCE is. Let us now
describe it for you. As the name indicates, it is a form of practical
examination which is objective and which owes its objectivity to a
structured marking scheme. Let us elaborate further. Suppose a patient
has an enlarged liver, 4 cm below the costal margin. In a conventional
examination, the student will tell the examiner ‘liver is 4 cm’ and get
credit for it, although he may have palpated it standing on the left side of
Process
the patient ! On the other hand, if he is given a mark for each of the
following points - makes the patient comfortable, warms hands, stands
on the right side of the patient, palpates gently and so on, then the
assessment is likely to be not only objective but also more valid.
This is the key concept of OSCE viz. to break a procedure into its
component skills and assess them individually. If we represent the whole
process, it will appear like this :

Clinical Competence

History taking Physical examination Investigations

Present General Urine
Past Systemic Stool
Birth CNS Blood
Family Heart X-ray
Nutrition Lungs
Development Abdomen

Each task assigned to the student is called a station. Thus, depending on

the situation, a number of stations are set up and students rotate on them,
spending a specified time on each one of them (usually between 3-5 min).
It means that the task presented at each a station should be within this
Stations time range. If it is felt that longer time is required, then, the task should be
subdivided into 2 stations (e.g. Superficial reflexes are elicited on one
station while DTR on next).

Don’t be afraid of designing the stations - they are easy to design if you
Design understand the basics of it. Generally stations are of two types— procedure
station and question station. As the name implies, at procedure station
84 Principles of Medical Education

can be further subdivided into observed and unobserved. An example of

each of these type of stations will make the difference clear.
Unobserved procedure station :
Record the weight of the infant.

Observed procedure station :

Record the weight of the infant.

Checklist for examiner :

Checks Zero level of weighing machine 2
Removes extra clothing from baby 2
Checklist
Handles the baby gently 2
Removes parallax while taking reading 2
Records the weight to an error of ±50 gm. 2

You have rightly noted that on observed procedure station, the student is
being observed by an observer on the basis of a checklist and it would be
no exaggeration to say that checklists are the ‘heart’ of OSCE. To prepare
a checklist, you have to list all the acts that go into making a complete
procedure. Once you are ready with this list, a differential weightage is
assigned to each of the acts. Thus, in the above example, if checking the
zero - level is considered to be more important than the others, then it is
given 2 marks while the rest of the points are given 1 each so that a student
who performs all the acts correctly is given 6 marks for this station.

It is obvious from the above that any range of competencies can be tested
Versatile
by OSCE. Competencies like history taking, physical examination or bed-
side lab procedures require an observed procedure station while

Example of a 10 station OSCE

 Assessment of Practical Skills 85

interpretation of lab reports, X-rays, ECGs, pictures etc. can be done by

unobserved procedure stations.
You should not go with the idea that OSCE tests only the ‘doing’ part
but ignores the ‘what’ part of clinical competence. The question stations
are meant exactly for this purpose - to test the results arrived at the previous
stations. The following examples will make it clear.

Procedure station:
Perform the general physical examination of this child.

Checklist for observer:

Looks for :
Ant. Fontanel Teeth
Pallor Hair
Example
Jaundice Lymph nodes
Throat Respiratory rate
Skin

Question station:
Regarding the case that you have just examined, write True or False.
• The child has minimal jaundice T/F
• The child has axillary lymphadenopathy T/F
• The child has 20 teeth T/F
• There is no pallor T/F

Procedure station:

Read the given chest X- ray. (Do not write anything)

Question station:
In the X- ray you have just seen -
• Trachea is shifted to right Yes / No
Example
• There is minimal fluid in pleural cavity. Yes / No
• Bones show early changes of rickets. Yes / No
• There is a primary complex. Yes / No

Both these examples would have made it clear that OSCE prompts the
student to perform the complete procedure at one go - if he has not palpated
axillary nodes or looked for evidence of rickets, he cannot go back to
review his findings.
Coming to the actual planning of OSCE, it is better to decide before
hand the competencies to be tested and weightage to be given to each.
Look at the following example :
86 Principles of Medical Education

History taking 30%

Physical examination 30%
Common procedures 20%
Interpretation of reports 20%

In effect, it could mean that there will be 3 stations on history taking (of 10
marks each), 3 on physical examination and so on. Let us make it clear
that these are only recommendations and depending on individual
requirements, a variation can be made. Thus, for junior students, more
emphasis may be laid on history taking and physical examination while
interpretation can be given more emphasis in later years. Once this decision
has been made, the whole examination will look like a circuit of stations,
Planning
though which all students rotate. For a 20 stations OSCE, the total time
required, assuming a time of 5 min per station, will be approximately 2
hours. You will agree that objectively examining 50 students in 2 hours
will never be possible by a conventional examination.
We hope, by now you are clear about what OSCE is. Can you list some
of the competencies which you can evaluate using OSCE ? Let us also do
it for you. These include :

(a) History taking: The student takes history at an observed or unobserved

station. At observed station, he is marked by an examiner while
unobserved stations are followed by a question station.
(b) Physical examination: The student is asked to perform physical
examination and is marked on a checklist. Competencies ranging from
a simple inspection (for spot diagnosis) to neurological examination
can be tested in this way. If the desired physical examination is likely
Competencies to take more than 3-4 mins, then the station should be split into two.
Evaluated For example, at one station the students tests for sensations while at
the next, he elicits the tendon reflexes.
(c) Charts and photographs: A strip of ECG, or reports of blood gas analysis
or photographs of congenital defects can be exhibited to represent
cases which may be difficult to get at examination time.
(d) Laboratory data interpretation: Hematology, biochemistry and
radiology reports can be objectively tested.
(e) Communication Skills: The student can be asked to explain to the
patient the dose of drugs, or diet to the mother of a malnourished child
or importance of immunisation and so on. By use of appropriate
checklists, these skills can be objectively evaluated in a very short
time.
(f) Instruments: The indications for use and handling of common
instruments can be evaluated.
 Assessment of Practical Skills 87

(g) Bedside lab tests: Actual procedure of urine, stool, blood examination
etc.
(h) Practical procedures like giving an injection or passing a nasogastric
tube using models.

We have given here a few more OSCE stations as illustrations. You can
also make your own stations, using them as guidelines.

Observed procedure station: Explain the method of preparing home - made

ORS to the mother
Checklist for examiner: This station is for assessment of communication
skills and attitudes of the student
Score
Example
Introduces himself 1
Intrudes the topic 1
Talks slowly and clearly 2
Invites clarifications 2
Remains patient and calm 1
Confirms that mother has understood 3
Observed procedure station: Determine the immunisation status of the
child.
Checklist examiner: Enquires specifically regarding following
vaccinations
Score
BCG 0.5
Oral Polio - Zero, I, II, III, Booster doses 1.5
DPT - I, II, III booster doses 1
Measles 0.5
MMR 0.5
Typhoid 0.5
Example Hepatitis B I, II, III 0.5
Any other vaccination 0.5
Asks regarding
Place of vaccination 0.5
Time schedule of vaccination 0.5
Any documentation or card 0.5
Looks for BCG scar 2

Interpretation station: A 3-year-old child has the following anthropometric

measurements.
Weight 11 kg
Height 76 cm
Head circumference 49 cm
88 Principles of Medical Education

Mid-arm circumference 13 cm
US : LS ratio 1.6 : 1
Use the percentile charts and write True or False for the following
statements :
• The child is a dwarf. T/ F
• The head circumference is normal. T/ F
• The measurements signify chronic malnutrition. T/ F
• The weight is at 50th percentile. T/ F

Unobserved procedure station Perform the abdominal examination of this

child.
Question question Write true or false regarding the child you have just
examined.
Example
• The upper border of the liver is in the 6th intercostal space. T/ F
• The liver is firm in consistency T/ F
• The spleen tip is palpable T/ F
• Free fluid is present T/ F
• There is tenderness in the right iliac fossa T/ F

Note:
1. Negative marking is mandatory in T / F questions.
2. In situations where the examination may be uncomfortable, multiple
alternative cases may be used for the station depending on the number
of students. But you should be careful that the question station pertains
to the particular case used.

Observed procedure station Take the natal history of this baby. Take the
natal history of this baby.
Checklist for examiner
Enquires regarding : Score
Place of delivery 1
Example
Type of delivery and indication 2
Duration of rupture of membranes 1
Meconium staining of liquor 1
Duration of labour 2
Time of cry 2
Method of cutting of umbilical cord 1

From the above, it would have become clear to you that OSCE makes
Feedback practical examination not only more valid but also more reliable. It can be
used for a large number of students in a relatively lesser time. Moreover,
 Assessment of Practical Skills 89

by analysing the checklists, feedback can be provided to teachers as well

as the students regarding efficiency of teaching. For example, if in the
palpation of liver station, it is found that most of the students are not
percussing the upper border of liver, then in subsequent teachings, this
point can be made more explicit.
It is not that OSCE does not have its critics and the major criticism is
that it tends to segregate the patient’s problems into components rather
than testing him as a whole. This may be true to some extent but most
often, it is the design of stations that is at fault rather than the examination
itself. This drawback can also be overcome by combining OSCE with a
traditional clinical case presentation.

Suggested format of UG practical examination

in final professional examination

Case OSCE Viva voce

1. Medicine and Long case Medicine, dermatology ±

allied subjects (one in medicine) and psychiatry
2. Surgery and Long case (one in Surgery, orthopaedics, ±
allied subjects surgery) ophthalmology, ENT
3. Obstetrics and Short case Obstetrics and ±
gynaecology (one in obstetrics) gynaecology
4. Pediatrics Short case (one in Neonatology, emer- ±
general pediatrics) gencies, procedures, etc.

Mini-CEX: Let us take you through the basic principles of yet another
useful method of assessing the clinical competence. This is called mini
clinical examination, often abbreviated as mini–CEX. It is called mini,
because it takes comparatively less time as compared to conventional
case presentation. However, the bigger advantage with mini-CEX is the
structured feedback that it provides to the students as well as the faculty,
thus helping them to make better decisions.
Mini-CEX is a 15 minute snapshot of doctor/patient interaction. It is
designed to assess the clinical skills, attitudes and behaviors of students
essential to providing high quality care. Students are asked to undertake
four to six observed encounters during the year with a different observer
for each encounter. Each of these encounters represent a different clinical
problem and trainees should sample from each of the core problem groups
identified as important (for example, history taking, physical examination,
diagnosis, communication, counseling etc.). However, not all elements
need to be assessed at each encounter. Each encounter takes about 20
90 Principles of Medical Education

minutes, with first 15 minutes for the encounter and last 5 minutes for
feedback. Immediate feedback is provided after each encounter by the
person assessing the performance. Strengths, areas for development and
agreed action points should be identified following each mini-CEX
encounter. You must have noticed that this form of examination is more
suitable for postgraduates, although with modifications, even under-
graduates can benefit from it.
Different Universities and Institutions have different types of recording
forms for use with mini CEX-most of them however; have a component of
essential skills from the curriculum built into them. Here is a generic form
which is most commonly used.

Mini-Clinical Evaluation Exercise (CEX)

Evaluator:___________________________________Date:____________________

Resident:_____________________________ O R-1 O R-2 O R-3

Patient Problem/Dx:__________________________________________

Setting: O Ambulatory O In-patient O ED O Other_______

Patient: Age:______ Sex:_______ O New O Follow-up

Complexity: O Low O Moderate O High

Focus: O Data Gathering O Diagnosis O Therapy O Counseling

1. Medical Interviewing Skills (O Not Observed)

1 2 3 4 5 6 7 8 9
Unsatisfactory Satisfactory Superior

2. Physical Examination Skills (O Not Observed)

1 2 3 4 5 6 7 8 9
Unsatisfactory Satisfactory Superior

3. Humanistic Qualities/Professionalism

1 2 3 4 5 6 7 8 9
Unsatisfactory Satisfactory Superior

4. Clinical Judgment (O Not Observed)

1 2 3 4 5 6 7 8 9
Unsatisfactory Satisfactory Superior

Contd...
 Assessment of Practical Skills 91

Contd...
5. Counseling Skills (O Not Observed)

1 2 3 4 5 6 7 8 9
Unsatisfactory Satisfactory Superior

6. Organization/Efficiency (O Not Observed )

1 2 3 4 5 6 7 8 9
Unsatisfactory Satisfactory Superior

7. Overall Clinical Competence (O Not Observed)

1 2 3 4 5 6 7 8 9
Unsatisfactory Satisfactory Superior

Mini-CEX Time: Observing ________Mins Providing Feedback:

___________Mins

Evaluator Satisfaction with Mini-CEX

Low 1 2 3 4 5 6 7 8 9 HIGH
Resident Satisfaction with Mini-CEX
Low 1 2 3 4 5 6 7 8 9 HIGH

Comments:

____________________________ ____________________________
Resident Signature Evaluator Signature

DESCRIPTORS OF COMPETENCIES DEMONSTRATED

DURING THE MINI-CEX

Medical Interviewing Skills: Facilitates patient’s telling of story; effectively

uses questions/directions to obtain accurate, adequate information
needed; responds appropriately to affected, non-verbal cues.
Physical Examination Skills: Follows efficient, logical sequence; balances
screening/diagnostic steps for problem; informs patient; sensitive to
patient’s comfort, modesty.
Humanistic Qualities/Professionalism: Shows respect, compassion,
empathy, establishes trust; attends to patient’s needs of comfort, modesty,
confidentiality, information.
92 Principles of Medical Education

Clinical Judgment: Selectively orders/performs appropriate diagnostic

studies, considers risks, benefits.
Counseling Skills: Explains rationale for test/treatment, obtains patient’s
consent, educates/counsels regarding management.
Organization/Efficiency: Prioritizes; is timely; succinct.
Overall Clinical Competence: Demonstrates judgement, synthesis, caring,
effectiveness, efficiency.
From: Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (Clinical
Evaluation Exercise): A preliminary investigation. Ann Intern Med 1995;
123:795-9.
Mini-CEX is considered a very useful way of providing feedback and
therefore improving the clinical skills of the students.
You can download a sample clip of a mini CEX session from http://
www.mmc.nhs.uk/pages/assessment/minicex.

Clinical case presentation is liable to be influenced by a number of factors

including language, dress, sex, poise and confidence of the student. Also,
often more time happens to be given to the students at the begining of the
examination as compared to the latter part of it.

Could clinical case presentation be modified to make it less subjective? It

is difficult to conceive a great deal of objectivity within the traditional
system, but attempts can certainly be made. One method would be to
make a checklist of important desired points in history and examination
including accuracy of clinical findings. The desired weightage for each
Traditional component would of course have to be predetermined. Certainly it would
Long Case
be an exhaustive exercise and needs a thorough study of the case prior to
the actual evaluation. While some senior and experienced examiners may
frown at the idea, the fact remains that many examinations follow a
haphazard pattern like sailing an uncharted sea. Would it not be fair to
the student that evaluation is systematic and organised? Look at the
guidelines given below , being followed by University of Limburg. You
will appreciate that by using such a checklist, the objectivity of case
presentation can be significantly improved.
 Assessment of Practical Skills 93

Objective Structured Case Record

History taking:
Pace/Clarity
Communication Process
Systematic presentation
Correct facts established
Physical examination:
Systematic
Techniques
Correct findings established
Management:
Appropriate investigations
Logical sequence
Appropriate management
Clinical acumen:
Problem identification
Problem solving

If we put it little more bluntly, it can be said that whether it is evaluation of

clinical skills or evaluation of clinical acumen, the onus of being objective
rests on you. If you are clear and consistent on how you are going to mark.
You will be fair to the students and also help them in learning better.
Remember the age old dictum that justice should not only be done but it
should also appear to be done. It is only by using these kind of tools that
you can rebuild the faith in the examination system.
Before we conclude this discussion, let us remind you that any
innovation in education requires time and effort. The time involved in
setting an OSCE is definitely more than that required for a traditional
examination - however, the increased reliability more than compensates
for it. Further, once a bank of OSCE stations is built up, subsequent
examinations become much easier to conduct and take considerably less
time.