1

Chapter 4: Using Essay Tests

C
Chhaapptteerr 44::
U
USSIIN
NGGE
ESSS
SAAY
Y TTE
ESSTTS
S

Upon completion of this chapter, you should be able to:

 Identify the attributes of an essay question
 Explain the purpose of using the essay test
 List the advantages and limitations of essay questions
 Identify those learning outcomes that are appropriately assessed using essay
questions
 Construct well-written essay questions that assess given learning objectives

CHAPTER OVERVIEW

4.0 Introduction Chapter 1: Introduction

4.1 What is an essay question? Chapter 2: What to Assess
4.2 Types of essay Chapter 3: Using Objective Tests
4.3 Why are essay questions used? Chapter 4: Using Essay Tests
4.4 Deciding whether to use essay Chapter 5: Using Projects and Practicals
questions or objective questions Chapter 6: Using Observations, Oral Tests,
4.5 Limitation of essay questions and Portfolios
4.6 Some misconceptions about essays Chapter 7: Reliability and Validity of
4.7 Guidelines for constructing essay Assessment Methods
questions Chapter 8: Item Analysis
4.8 Common verbs used in essay Chapter 9: Analysis of Test Scores and
questions Reporting
4.9 Marking or scoring essay questions
4.10 Summary
Key Terms
Readings

In Chapter 3, we discussed in detail the use of objective tests in assessing students.

In this chapter, we will examine the essay test. The essay test is a popular technique
for assessing learning and is used extensively at all levels of education. It is also
widely used in assessing learning outcomes in business and professional
examinations. Essay questions are used because they challenge students to create
their own responses rather than simply to select a response. Essay questions have
the potential to reveal students‟ ability to reason, create, analyse and synthesise
which may not be effectively assessed using objective tests.
 2
Chapter 4: Using Essay Tests

4.1 WHAT IS AN ESSAY QUESTION?

According to Stalnaker (1951), an

essay is “a test item which requires a response
composed by the examinee usually in the form
of one or more sentences of a nature that no
single response or pattern of responses can be
listed as correct, and the accuracy and quality
of which can be judged subjectively only by
one skilled or informed in the subject”.
Though the definition was provided a long
time ago, it is a comprehensive definition.
Elaborating on this definition, Reiner, Bothell, Figure 4.1 Learner composes an
Sudweeks and Wood (2002), argued that to answer that consists of one or more
sentences
qualify as an essay question it should meet the
following FOUR criteria:

1. The learner has to compose rather than select his or her response or answer.
In essay questions, students have to construct their own answer and decide on
what material to include in their response. Objective test questions (MCQ,
true-false, matching) require students to select the answer from a list of
possibilities.

2. The response or answer the learner provides will consist of one or more
sentences. Students do not respond with a “yes” or “no” response but instead
have to respond in the form of sentences. In theory there is no limit to the
length of the answer. However, in most cases its length is predetermined by
the demands of the questions and the time limit allotted for the question.

3. There is no one single correct response or answer. In other words, the

question should be composed so that it does not ask for one single correct
response. For example, the question, “Who killed J.W.W. Birch?” assesses
verbatim recall or memory and not the ability to think. Hence, it cannot
qualify as an essay question. You could modify the question, “Who killed
J.W.W. Birch and explain the factors that led to the killing?”. Now, it is an
essay question that assesses students‟ ability to think and give reasons for the
killing supported with relevant evidence.

4. The accuracy and quality of students‟ responses or answers to essays must be

judged subjectively by a specialist in the subject: The nature of essay
questions is such that only specialists in the subject can judge to what degree
responses (or answers) to an essay are complete, accurate and relevant. Good
essay questions encourage students to think deeply about their answers that
can only judged by someone with the appropriate experience and expertise in
the content area. Thus, content expertise is essential for both writing and
grading essay questions. For example, the question “List three reasons for the
opening of Penang by the British in 1789” requires students to recall a set list
of items. The person marking or grading the essay does not have to be a
 3
Chapter 4: Using Essay Tests

subject matter expert to know if the student has listed the three reasons
correctly as long as the list of three reasons is available as an answer key. For
the question, “To what extent is commerce the main reason for the opening of
Penang by the British in 1789?”, a subject-matter expert is needed to grade or
mark an answer to this essay question.

4.2 TYPES OF ESSAYS

Generally, there are two types of essays; the coursework essay and the examination
essay which are used in educational institutions:

WHAT IS A COURSEWORK ESSAY?

The coursework essay may consist of the following:
 Essay Outlines: These is a short summary or outline of a particular topic
(perhaps, about 500 words). It assesses the ability to organise material, to
construct coherent arguments, to select relevant information from a wide field
of study.
 Standard essays: These are full papers of anything from 1000 to 2500 words
in length. They assess the ability to describe to analyse the relationship
between ideas and events, to give a coherent account of a topic, to select and
to weigh evidence in support of an argument, to diagnose and to suggest
solutions to problems, to solve familiar types of problems, to solve familiar
types of problems, to express critical judgements and to make comparisons.
 Extended essays: These are full papers of between 2500 to 5000 words in
length. They assess the ability to solve less familiar problem and to analyse or
critically evaluate less familiar materials. Hence, requiring more extensive
preparation or research.

Coursework essays are intended to assess students‟ ability to research a topic

thoroughly and meticulously, and to handle a mass of material. It evaluates their
ability to answer at length (“Give them enough rope…..”) on a chosen topic. Of
particular concern are the points the student makes, how he or she has related them,
the way in which his or her essay is organised, the value he or she attaches to different
aspects of the topic. Coursework essays are a written response and students are
expected, for the most part, to use their own words and communicate their ideas
clearly and persuasively.

WHAT IS AN EXAMINATION ESSAY?

This is the short essay written as part of a formal examination [Surely you
have had experience with this format of assessment!]. For example, in a 2-3 hour
examination, students may be required to answer about three essay questions allotting
about 35-45 minutes per question. In practice, there is much variation in the number
of questions asked and duration of the examination. In some situations there is a
choice of questions while in other situations there is no choice. The reason for
controlling choice is to ensure that students are examined over a comparable range [In
this chapter we will focus on the „Examination Essay‟; i.e. the use of essay questions
 4
Chapter 4: Using Essay Tests

in examinations which is commonly a „close-book‟ setting. We will discuss further

about the Coursework Essay in Chapter 5 under Projects].

4.1 ACTIVITY
Select a few essay questions used as tests in you subject area
examinations. To what extent do these questions meet the criteria of an
essay defined by Stalnaker (1951) and elaborated by Reiner,
Bothell, Sudweeks and Wood (2002).

4.3 WHY ARE ESSAY QUESTIONS USED?

Essay questions are used to assess learning because of the following reasons:

 Essay questions provide an effective way of assessing complex learning

outcomes. They allow one to assess
students' ability to synthesise ideas, to
organise, and express ideas and to evaluate Price is determined
by supply and
the worth of ideas. These abilities cannot demand. When there
be effectively assessed directly with other is a shortage of…….
paper-and-pencil test items.

 Essay questions allow students to

demonstrate their reasoning. Essay
questions not only allow students to
present an answer to a question but also to
explain how they arrived at their
conclusion. This allows teachers to gain
insights into a student's way of viewing
and solving problems. With such insights,
teachers are able to detect problems
students may have with their reasoning Figure 4.1 The reasoning
process and help them overcome those process of a learner is
problems. reflected in the answer

 Essay questions provide authentic experiences. „Constructing responses‟ are

closer to real life than selecting responses (as in the case of objective tests).
Problem solving and decision-making are vital life competencies which
require the ability to construct a solution or decision rather than selecting a
solution or decision from a limited set of possibilities. In the world of work it
is unlikely that an employer will give a list of „four options‟ for the worker to
choose from when asked to solve a problem! In most cases, the worker will be
required to construct a response. Hence, essay items are closer to real life than
objective test question because in real life students typically construct
responses, not select them.
 5
Chapter 4: Using Essay Tests

4.2 ACTIVITY
Compare the following two essay questions and decide
which one assesses higher-order thinking skills.
 What are the major advantages and limitations of using
nuclear solar energy?
 Given its advantages and limitations, should governments
spend money developing nuclear energy?

4.4 DECIDING WHETHER TO USE ESSAY QUESTIONS OR

OBJECTIVE QUESTIONS

The decision whether to use essay

questions or objective questions in examinations
can be problematic for some educators. In such a
situation one has to go back to the objectives of
assessment. What kinds of learning outcomes do
you intend to assess? Essay questions are generally
suitable for the following purposes:
 To assess students' understanding of
subject-matter content
 To assess thinking skills that require
more than simple verbatim recall of
information by challenging the students to reason with their
knowledge.
However, students‟ understanding of subject-matter content and many of the
other higher-order thinking skills could also be assessed through objective items.
When in doubt as to whether to use an essay question or an objective question, just
remember that essay questions are used to assess students' ability to construct rather
than select answers.
To determine what type of test (essay or objective) to use, it is helpful that you
examine the verb(s) that best describe the desired ability to be assessed [We discussed
these verbs in Chapter 2]. These verbs indicate what students are expected to do and
how they should respond. These verbs serve to channel and focus the student
responses towards the performance of specific tasks. Some verbs clearly indicate that
students need to construct rather than select their answer (e.g. to explain). Other verbs
indicate that the intended learning outcome is focused on students‟ ability to recall
information (e.g. to list). Perhaps, recall is best assessed through objectively-scored
items. Verbs that test for understanding of subject-matter content or other forms of
higher-order thinking but do not specify whether the student is to construct or to select
the response (e.g. to interpret) can be assessed either by essay questions or by
objective items.
 6
Chapter 4: Using Essay Tests

4.3 ACTIVITY
a) Do you agree with the reasons for using essay tests in your
subject area?
b) Compare, explain, arrange, apply, state, classify, design,
illustrate, describe, name, complete, choose, defend, name.
Decide which of the above list of verbs is best assessed by
essay tests or objective tests or both objective and essay
tests.

4.5 WHAT ARE SOME LIMITATIONS OF ESSAY QUESTIONS?

While essay questions are popular because they enable the assessment of higher-order
learning outcomes, this format of evaluating students in examinations has a number of
limitations which should be kept in mind.

 One purpose of testing is to assess a student's mastery of the subject matter. In

most cases it is not possible to assess the student's mastery of the complete
subject matter domain with just a few questions. Because of the time it takes
for students to respond to essay questions and for markers to mark students'
responses, the number of essay questions that can be included in a test is
limited. Therefore, using essay questions will limit the degree to which the test
is representative of the subject-matter domain thereby reducing content
validity. For instance, a test of 80 multiple-choice questions will most likely
cover more of the content domain than a test of 3-4 essay questions

 Essay questions have limitations in reliability. While essay questions allow

students some flexibility in formulating their responses, the reliability of
marking or grading is questionable.
Different markers or graders may vary in
their marking or grading of the same or - Interscorer reliability
similar responses (interscorer reliability) is low.
and one marker can vary significantly in - Limited amount of the
his or her marking or grading consistency subject matter tested
across questions depending on many
factors (intrascorer reliability). - Time consuming
Therefore, essay answers of similar
quality may receive notably different scores. Characteristics of the learner,
length and legibility of responses, and personal preferences of the marker or
grader with regards to the content and structure of the response are some of the
factors that may lead to unreliable marking or grading.

 Essay questions require more time for marking student responses. Teachers
need to invest large amounts of time to read and mark students‟ responses to
 7
Chapter 4: Using Essay Tests

essay questions. On the other hand relatively little or no time is required of

teachers for scoring objective test items like multiple-choice items, matching
exercises, etc.

As mentioned earlier, one of the strengths of essay questions is that they provide
students with authentic experiences because students are challenged to construct
rather than to select their responses. To what extent does the short time given affect
student response? Students have relatively little time to construct their responses and
it does not allow them to give appropriate attention to the complex process of
organizing, writing, and reviewing their responses. In fact, in responding to essay
questions, students use a writing process that is quite different from the typical
process that produces excellent writing (draft, review, revise, evaluate, etc.). In
addition students usually have no resources to aid their writing when answering essay
questions (dictionary, thesaurus, etc.). This disadvantage may offset whatever
advantage accrued from the fact that responses to essay questions are more authentic
than responses to multiple-choice items.

4.4 ACTIVITY
a) Do you agree with the weaknesses of the essay
examinations. with regards to your subject areas?
b) Identify other weaknesses of the essay as an examination
test in your subject area.

4.6 SOME MISCONCEPTIONS ABOUT ESSAY QUESTIONS IN

EXAMINATIONS

Besides the limitations of essay questions, there are also some misconceptions about
this format of assessment:
 By their very nature essay questions assess higher-order thinking. Whether
or not an essay item assesses higher-order thinking depends on the design of
the question and how students‟ responses are scored. An essay question does
not automatically assess higher-order thinking skills. It is possible to write
essay questions that simply assess recall. Also, if a teacher designs an essay
question meant to assesses higher-order thinking but then scores students‟
responses in a way that only rewards recall ability, that teacher is not assessing
higher-order thinking.

 Essay questions are easy to construct. Essay questions are easier to construct
than multiple-choice items because teachers do not have to create effective
distractors. However, that does not mean that good essay questions are easy to
construct. They may be easier to construct in a relative sense, but they still
require a lot of effort and time. Essay questions that are hastily constructed
without much thought and review usually function poorly.

 The use of essay questions eliminates the problem of guessing. One of the
drawbacks of objective test items is that students sometimes get the right
 8
Chapter 4: Using Essay Tests

answer by guessing which of the presented options is correct. This problem

does not exist with essay questions because students need to generate the
answer rather than identifying it from a set of options provided. At the same
time, the use of essay questions introduces bluffing, another form of guessing.
Some students are „good‟ at using various methods of bluffing (vague
generalities, padding, name-dropping, etc.) to add credibility to an otherwise
weak answer. Thus, the use of essay questions changes the nature of the
guessing that occurs, but does not eliminate it.

 Essay questions benefit all students by placing emphasis on the importance

of written communication skills. Written communication is a life competency
that is required for effective and successful performance in many vocations.
Essay questions challenge students to organise and express subject matter and
problem solutions in their own words, thereby giving them a chance to
practice written communication skills that will be helpful to them in future
vocational responsibilities. At the same time, the focus on written
communication skills is also a serious disadvantage for students who have
marginal writing skills but know the subject-matter being assessed. To the
degree that students who are knowledgeable in the subject obtain low scores
because of their inability to write well, the validity of the test scores will be
diminished.

 Essay questions encourage students to prepare more thoroughly. Some

research seems to indicate that students are more thorough in their preparation
for essay questions that in their preparation for objective examinations like
multiple choice questions. However, after an extensive review of existing
literature and research on this topic, Crook (1988) concluded that students
prepare more based on the expectations teachers set upon them (higher-order
thinking and breadth and depth of content) than they do by the type of test
question they expect to be given.

4.5 ACTIVITY
a) To what extent do you agree with the misconceptions
people h with regards to essay questions?
b) Can you identify other common misconceptions or myths
about essay questions?

4.7 SOME GUIDELINES FOR CONSTRUCTING ESSAY

QUESTIONS

When constructing essay questions, whether it be for Coursework Assessment or

Examinations, the most important thing is to ENSURE THAT STUDENTS HAVE
A CLEAR IDEA ON WHAT THEY ARE EXPECTED TO DO AFTER THEY
HAVE READ THE QUESTION OR PROBLEM PRESENTED.
 9
Chapter 4: Using Essay Tests

Below are specific guidelines that can help you improve existing essay questions and
create new ones.

1. Clearly Define the Intended Learning Outcome to be Assessed by the

Question.
Knowing the intended learning outcome is crucial for designing essay questions. some
skill, ability, or trait other than the one intended. In specifying the intended learning
outcome teachers clarify the performance that students should be able to demonstrate
as a result of what they have learned. The intended learning outcome typically begins
with a verb that describes an observable behavior, action or outcome that students
should demonstrate. The focus is on what students should be able to do and not on the
learning or teaching process. Reviewing a list of verbs can help to clarify what ability
students should demonstrate and to clearly define the intended learning outcome to be
assessed.

Example:
Learning Outcome: To be able to differentiate the reproductive habits of birds and
amphibians

Essay Question: What are the differences in egg laying characteristics between
birds and amphibians?

Note: This learning outcome can be better assessed by an objective test

Objective Item:
Which of the following differences between birds and amphibians is correct?

Birds Amphibians
A Lay a few eggs at a time Lay many eggs at a time
B Lay eggs Gives birth
C Do not incubate eggs Incubate eggs
D Lay eggs in nest Lay eggs on land

2. Avoid Using Essay Questions for Intended Learning Outcomes that are
Better Assessed with other Kinds of Assessment.
Some types of learning outcomes can be more efficiently and more reliably assessed
with objective tests than with essay questions. Since essay questions sample a limited
range of subject-matter content, are more time consuming to score, and involve
greater subjectivity in scoring, the use of essay questions should be reserved for
learning outcomes that cannot be better assessed by some other means.
 10
Chapter 4: Using Essay Tests

3. Clarity about the Task and Scope.

Essay questions have two variable elements—the degree to which the task is
structured and the degree to which the scope of the content is focused. There is still
confusion among educators whether more structure of the task required and more
focus on the content are better than less structure and less focus. When the task is
more structured and the scope of content is more focused two problems are reduced:

 the problem of student responses containing ideas that were not meant to be
assessed,
 the problem of extreme subjectivity when scoring student answers or
responses.

Although more structure helps to avoid these problems, how much and what
kind of structure and focus to provide is dependent on the intended learning outcome
that is to be assessed by the essay question. The process of writing effective essay
questions involves defining the task and delimiting the scope of the content in an
effort to create an effective question that is aligned with the intended learning
outcome to be assessed by it (see Figure 4.1).

INFORMATION
DESIGN

LEARNING BODY OF
OUTCOMES CONTENT

LEARNING ASSESSMENT
ACTIVITIES TASKS

INTERACTION
DESIGN

Figure 4.1 Alignment between Content, Learning

Learning Activities and Assessment Tasks

[source: John A. Phillips, 2005. Instructional Design Principles in the Development of an E-

Learning Graduate Course. Paper presented at The International Conference in E-Learning.
Bangkok, Thailand]
 11
Chapter 4: Using Essay Tests

Example:
IMPROVING CLARITY OF TASK AND
SCOPE OF ESSAY QUESTIONS

Weak essay question:

Evaluate the impact of the Industrial Revolution on England.

The verb is “evaluate” which is the task the student is supposed to do. The scope of
the question is the impact of the Industrial Revolution on England. Very little
guidance is given to students about the task of evaluating and the scope of the task.
A student reading the question may ask:
 the impact on what in England? The economy? Foreign trade? A
particular group of people? [The scope is not clear].
 evaluate based on what criteria? The significance of the revolution?
The quality of life in England? Progress in technological
advancements? [The task is not clear].
 what exactly do you want me to do in my evaluation? [The task is
not clear]

Improved essay question:

Evaluate the impact of the Industrial Revolution on the quality of

family life in England. Explain whether families were able to provide
for the education of their children.

The improved question delimits the task for students by specifying a particular unit
of society in England affected by the Industrial Revolution (family). The task is
also delimited by giving students a criterion for evaluating the impact of the
Industrial Revolution (whether or not families were able to provide for their
children‟s education. Students are clearer as to what must be done to “evaluate.”
They need to explain how the family has changed and judge whether or not the
changes are an improvement for their children.

[source: adaptation: C.M. Reiner, T.W. Bothell, R.R. Sudweeks & B. Wood
Preparing Effective Essay Questions. 2002 New Forums Press

This alignment is absolutely necessary for obtaining student responses that can
be accepted as evidence that a student has achieved the intended learning outcome.
Hence, the essay question must be carefully and thoughtfully written in such a way
that it elicits student responses that provide the teacher with valid and reliable
evidence about the students‟ achievement of the intended learning outcome. Failure to
establish adequate and effective limits for student answers to the essay question may
result in students setting their own boundaries for their responses. This means that
students might provide answers that are outside the intended task or address only a
part of the intended task. If this happens then the teacher is left with unreliable and
 12
Chapter 4: Using Essay Tests

invalid information about the students‟ achievement of the intended learning outcome.
Also, there is no basis for marking or grading student‟s answers. Therefore, it is the
responsibility of the teacher to write essay questions in such a way that they provide
students with clear boundaries for their answers or responses.

SELF-CHECK 4.1
a) When would you decide to use an objective test item rather
than an essay test to assess learning?
b) What is the difference between the task and scope of an
essay question?

4. Questions that are Fair

One of the challenges that teachers face in composing essay questions is that
because of their extensive experience with the subject matter they may be tempted to
demand unreasonable content expertise on the part of the students. Hence, teachers
need to make sure that their students can “be expected to have adequate material with
which to answer the question.” (Stalnaker, 1952, p.52). In addition, teachers should
ask themselves if students can be reasonably expected to adequately perform the
thought processes which are required of them in the task. For assessment to be fair,
teachers need to provide their students with sufficient instruction and practice in the
subject matter and the thought processes to be assessed.
Another important element is to avoid using indeterminate questions. A
question is indeterminate if it is so unstructured that students can redefine the problem
and focus on some aspect of it with which they are thoroughly familiar, or if experts
in the subject matter cannot agree that one answer is better than another. One way to
avoid indeterminate questions is to stay away from vocabulary that is ambiguous. For
example, teachers should avoid using the verb “discuss” in an essay question. This
verb simply is too broad and vague. Moreover, teachers should also avoid including
vocabulary that is too advanced for students.

5. Specify the approximate time limit and marks allotted for each question
Specifying the approximate time limit helps students allocate their time in answering
several essay questions. Without such guidelines students may feel at a loss as to how
much time to spend on a question. When deciding the guidelines for how much time
should be spent on a question keep the slower students and students with certain
disabilities in mind. Also make sure that students can be realistically expected to
provide an adequate answer in the given and/or suggested time. Similarly, state the
marks allotted for each questions so that students can decide how much they should
write about a questions.

6. Use several relatively short essay questions rather than one long one.
Only a very limited number of essay questions can be included on a test because of
the time it takes for students to respond to them and the time it takes for teachers to
grade the student responses. This creates a challenge with regards to designing valid
essay questions. Shorter essay questions are better suited to assess the depth of student
learning within a subject whereas longer essay questions are better suited to assess the
breadth of student learning within a subject. Hence, there is a trade-off when choosing
 13
Chapter 4: Using Essay Tests

between several short essay questions or one long one. Focus on assessing the depth
of student learning within a subject limits the assessment of the breadth of student
learning within the same subject and focus on assessing the breadth of student
learning within a subject limits the assessment of the depth of student learning within
the same subject. When choosing between using several short essay questions or one
long one also keep in mind that short essays are generally easier to mark than long
essay questions.

7. Avoid the use of optional questions

Students should not be permitted to choose one essay question to answer from two or
more optional questions. The use of optional questions should be avoided for the
following reasons:
Students may waste time deciding on an option.
Some questions are likely to be harder which could make the
comparative assessment of students' abilities unfair.
The issue of the use of optional questions is debatable. It is often practiced, especially
in higher education and students often demand that they be given choices. The
practice is acceptable if it can be assured that the questions have equivalent difficulty
levels and the tasks as well as the scope required by the questions are equivalent.

SELF-CHECK 4.2
a) Why should you specify time to be allotted for each question?
a) b) Why should you avoid optional questions?
b) c) What do you mean that questions should be „fair‟?
c) d) What should you do before and after administering a test?
 14
Chapter 4: Using Essay Tests

Improving Essay Questions Through Preview and Review

The following steps can help you improve the essay item before and after you
administer it to your students.

PREVIEW (before handing out the essay question to the students)

 Predict student responses.

Try to respond to the question from the perspective of a typical student.
Evaluate whether students have the content knowledge and the skills
necessary to adequately respond to the question. In detecting possible
weaknesses of the essay question, repair them before handing out the exam.

 Write a model answer.

Before using a question, write model answer(s) or at least an outline of
major points that should be included in an answer. Writing the model
answer allows reflection on the clarity of the essay question. Furthermore,
the model answer(s) serve as a basis for the grading of student responses.
Once the model answer has been written compare its alignment with the
essay question and the intended learning outcome and make changes as
needed to assure that the intended learning outcome, the essay question,
and the model answer are aligned with each other.

 Ask a knowledgeable colleague to critically review the essay question,

the model answer, and the intended learning outcome for alignment.
Before using the essay question on a test, ask a person knowledgeable in
the subject to critically review the essay question, the model answer, and
the intended learning outcome to determine how well they are aligned with
each other.

REVIEW (after receiving the student responses)

 Review student responses to the essay question.
After students complete the essay questions, carefully review the range of
answers given and the manner in which students seem to have interpreted
the question. Make revisions based on the findings. Writing good essay
questions is a process that requires time and practice. Carefully studying
the student responses can help to evaluate students' understanding of the
question as well as the effectiveness of the question in assessing the
intended learning outcomes.
 15
Chapter 4: Using Essay Tests

4.8 VERBS DESCRIBING VARIOUS KINDS OF MENTAL TASKS

Using the list suggested by Moss & Holder (1988) and Anderson & Krathwohl
(2001); Reiner, Bothell, Sudweeks and Wood (2002) proposed the following list of
verbs that describe mental tasks to be performed. The definitions specify thought
processes a person must perform to complete the mental task. Note that this list is not
exhaustive and local examples have been introduced to illustrate the mental tasks
required in each essay question.

Verbs Definitions & Examples

 Analyse: Break material into its constituent parts and determine how the parts
relate to one another and to an overall structure or purpose.
Analyze the meaning of the line “He saw a dead crow, in a drain, near the
post office” in the poem The Dead Crow.
 Apply: Decide which abstractions (concepts, principles, rules, laws, theories,
generalizations) are relevant in a problem situation.
Apply the principles of supply and demand to explain why the consumer price
index(CPI) in Malaysia has increased in the last three months..
 Attribute: Determine a point of view, bias, value, or intent underlying
presented material.
Determine the point of view of the author in the article about her political
perspective.
 Classify: Determine which category belongs to something.
Classify the organisms into vertebrates and invertebrates.
 Compare: Identify and describe points of similarity.
Compare the role of the Dewan Rakyat and Dewan Negara.
 Compose: Make or form by combining things, parts, or elements.
Compose an effective plan for solving flooding in Kuala Lumpur.
 Contrast: Bring out the points of difference.
Contrast the contribution of Tun Hussein Onn and Tun Abdul Razak Hussien
to the political stability of Malaysia.
 Create: Put elements together to form a coherent or functional whole,
reorganize elements into a new pattern or structure.
Create a solution for the traffic problem in Kuala Lumpur.
 Critique: Detect consistencies and inconsistencies between a product and
relevant external criteria; detect the appropriateness of a procedure for a given
problem.
Judge which of two methods is the best way for solving a problem.
 Defend: Develop and present an argument to support a recommendation, to
maintain or revise a policy, program, or propose a course of action.
Defend the decision to raise fuel price by the government.
 Define: Give the meaning of a word or concept; place it in the class to which
it belongs and distinguish it from other items in the same class.
Define the term “chemical weathering".
 Describe: Give an account of; tell or depict in words; represent or delineate by
a word picture.
 16
Chapter 4: Using Essay Tests

Describe the contribution of Za‟ab to the development of Bahasa Melayu..

 Design: Devise a procedure for accomplishing some task.
Design an experiment to prove that 21% of air is composed of oxygen..
 Develop: Bring to a more advanced , effective, or usable state; produce.
Develop an essay question by improving upon a less effective essay question.
 Differentiate: Distinguish relevant from irrelevant parts or important from
unimportant parts of presented material.
Distinguish between supply and demand in determining price.
 Explain: Make clear the cause or reason of something; construct a cause-and-
effect model of a system; tell "how" to do; tell the meaning of.
Explain the causes of the First World War..
 Evaluate: Make judgments based on criteria and standards; determine the
significance, value, quality, or relevance of; give the good points and the bad
ones; identify and describe advantages and limitations.
Evaluate the contribution of the microchip in telecommunications .
 Generate: Come up with alternative hypotheses, examples, solutions,
proposals, etc. based on criteria.
Generate hypotheses to account for an observed phenomenon.
 Identify: Recognise as being a particular person or thing.
Identify the characteristics of the Mediterranean climate.
 Illustrate: Use a word picture, a diagram, a chart, or a concrete example to
clarify a point.
Illustrate the use of catapults in the amphibious warfare of Alexander.
 Infer: Draw a logical conclusion from presented information.
What can you infer happened in the experiment?.
 Interpret: Give the meaning of; change from one form of representation (e.g.
numerical) to another (e.g. verbal).
Interpret the poetic line, "The sound of a cobweb snapping is the noise of my
life."
 Justify: Show good reasons for; give your evidence; present facts to support
your position.
Justify the American entry into World War II.
 List: Create a series of names or other items.
List the major functions of the human heart.
 Predict: Know or tell beforehand with precision of calculation, knowledge, or
shrewd inference from facts or experience what will happen.
Predict the outcome of a chemical reaction.
 Propose: Offer for consideration, acceptance, or action; suggest.
Propose a solution for landslides along the North-South Highway..
 Recognise: Locate knowledge in long term memory that is consistent with
presented material.
Recognise the important events in the road to independence in Malaysia.
 Recall: Retrieve relevant knowledge from long-term memory.
Recall the dates of important events in Islamic history.
 Summarize: Sum up; give the main points briefly.
Summarise the ways in which man preserves food.
 Trace: Follow the course of; follow the trail of; give a description of progress.
Trace the development of television in school instruction.
 17
Chapter 4: Using Essay Tests

4.6 ACTIVITY
a) Select some essay questions in your subject area and
examine whether the verbs used are similar to the list.
Do you think the tasks required by the verbs are
appropriate?
b) Do you think students are able to differentiate between the
tasks required in the verbs listed?
c) Are teachers able to describe the tasks required by these
verbs?

CHECKLIST FOR WRITING ESSAY QUESTIONS

Could the item be better assessed with a different kind of assessment?

Is the essay question aligned with the intended learning outcome?

Is the essay question and should be split into short essay questions?

Does the essay question contain a clear task and a specified scope?

Is the question worded and structured clearly?

Do the students know the recommended time for each question?

Have you written a model answer or an outline for the answer?

Do you have a person knowledgeable in the subject to review the

question?

4.9 MARKING AN ESSAY QUESTION

Marking or Grading of essays is a notoriously unreliable activity. If we read an

essay at two different times, the chances are good that we will give the essay a
different grade each time. If two or more of us read the essay, our grades will likely
differ, often dramatically so. We all like to think
we are exceptions, but study after study of well Marking should be
meaning and conscientious teachers show that
essay grading is unreliable (Ebel, 1972;
free from bias and
McKeachie, 1986; White, 1985). Eliminating the subjectivity, and
problem is unlikely, but we can take steps to reliable
improve grading reliability. Using a scoring guide
 18
Chapter 4: Using Essay Tests

or marking scheme helps control the shifting of standards that inevitably take place as
we read a collection of essays and papers. The two most common forms of scoring
guides used are the analytic and holistic method.

1) The Analytic Method

Analytical marking is the system of most frequently used in large-scale public
examinations (such as the SPM and STPM) and also by teachers in the classroom. Its
basic characteristic is the marking scheme and the marks allocated (see Figure 4.2).
The marking scheme consists of a list of the major elements the teacher believes
students should include in the ideal answer. In the example, marks are allocated for
describing each of the five factors (3 marks for each factor) and one mark is given for
providing a relevant example. Marks are also allotted for the „introduction‟ and
„conclusion‟ which are important elements in an essay.

EXAMPLE:
Explain FIVE factors that determine the location of a manufacturing plant.

Marking Scheme (max 25 points)

 Introduction 2 points
o Sets the organisation of the essay
 Site or Land
o Description (how site influences location) 3
o Example 1
 Labour
o Description (role of workers) 3
o Example 1
 Capital (Loan from banks)
o Description (obtaining a loan) 3
o Example 1
 Transport system
o Description (access to port, population) 3
o Example 1
 Entrepreneurial skills
o Description (kind of skill) 3
o Example 1
 Conclusion (interplay of the five factors) 3

TOTAL 25 points

Figure 4.2 Sample of a Marking Scheme Using the Analytical Method

The marker reads and compares the student's answer with the model answer. If
all the necessary elements are present, the student receives the maximum number of
points. Partial credit is given based on the elements included in the answer. In order to
arrive at the overall exam score, the instructor adds the points earned on the separate
questions.
 19
Chapter 4: Using Essay Tests

Identify in advance what will be worth a point, how many points available for
each question so you can tell students; so they do not give more (or less) than
necessary and they know precisely what you are looking for. If students come up with
an unexpected but correct example give them the point immediately and add that
point to your answer key so the next student will get the point too.

2) Holistic Method (Global or Impressionistic Marking)

The holistic approach to scoring essay questions involves reading an entire response
and assigning it to a category identified by a score or grade. This method involves
considering the student's answer as a whole and judging the total quality of the answer
relative to other student responses or the total quality of the answer based on certain
criteria that you develop. Think of it as sorting into bins.You read the answer to a
particular question and assign it to the appropriate bins. The best answers go into the
“exemplary” bin, the good goes into the “good” bin, and the weak answer goes into
the “poor” bin. Then points are written on each paper appropriate to the bin it is in. It
is based on overall impression, not point by point using the rubric or marking scheme
based on a holistic approach (see Figure 4.3). You cannot be much finer than five
divisions; six at most. You have to re-read everything to ensure that all the papers in 5
piles roughly the same; all in middle pile roughly the same, and less good than top
pile.

Exemplary Good Adequate Poor No Answer

You can develop a description of the type of response that would illustrate each
category before you start, and then try out this draft version using several actual
papers. After reading and categorising all of the papers, it is a good idea to re-examine
the papers within a category to see if they are similar enough in quality to receive the
same points or grade. It may be faster to read essays holistically and provide only an
overall score or grade, but students do not receive much feedback about their
strengths and weaknesses. Some instructors who use holistic scoring also write brief
comments on each paper to point out one or two strengths and/or weaknesses so
students will have a better idea of why their responses received the scores they did.

4.6 ACTIVITY
a) Compare and analytical and holistic method of marking
essays?.
b) Which method is widely practiced in your institution?
Why do you think?
c) Do you think there would be a difference in marking
Essays using the methods? Explain
 20
Chapter 4: Using Essay Tests

EXEMPLARY • Addresses the question • Demonstrates an accurate

and complete
(10 pts) • States a relevant argument understanding of the
question
• Presents arguments in a logical
order • Uses several arguments
and backs arguments with
• Uses acceptable style and examples, data that support
grammar (no errors) the conclusion
GOOD • Combination of above traits, but • Same as above but less
less consistently represented (1-2 thorough, still accurate
(8 points) errors)
• Uses only one argument
and example that supports
conclusion
ADEQUATE • Does not address the question • Demonstrates minimal
explicitly, though does so understanding of question,
(6 pts) tangentially still accurate

• States a somewhat relevant • Uses a small subset of

argument possible ideas for support
of the argument.
• Presents some arguments in a
logical order

• Uses adequate style and grammar

(more than 2 errors)
POOR • Does not address the question • Does not demonstrate
understanding of the
(4 pts) • States no relevant arguments question, inaccurate

• Is not clearly or logically • Does not provide

organised? evidence to support
response to the question
• Fails to use acceptable style and
grammar
NO ANSWER

(0 pts)

Table 4.1 Sample of a Marking Scheme Using the Holistic Method

 21
Chapter 4: Using Essay Tests

SUGGESTIONS FOR MARKING OR SCORING ESSAYS

 Grade the papers anonymously. This will help control the influence of our
expectations about the student on the evaluation of the answer.

 Read and score the answers to one question before going on to the next
question. In other words, score all the students' responses to Question 1 before
looking at Question 2. This helps to keep one frame of reference and one set of
criteria in mind through all the papers, which results in more consistent
grading. It also prevents an impression that we form in reading one question
from carrying over to our reading of the student's next answer. If a student has
not done a good job on say the first question; we could let this impression
influence our evaluation of the student's second answer. But if other students'
papers come in between, we are less likely to be influenced by the original
impression.

 If possible, try to grade all the answers to one particular question without
interruption. Our standards might vary from morning to night, or one day to
the next.

 Shuffle the papers after each item is cored throughout all the papers. Changing
the order reduces the context effect and the possibility that a student's score is
the result of the location of the paper in relationship to other papers. If
Rakesh‟s „B‟ work is always followed Jamal‟s „A‟ work, then it might look
more like „C‟ work and his grade would be lower than if his paper were
somewhere else in the stack.

 Decide in advance how you are going to handle extraneous factors and be
consistent in applying the rule. Students should be informed about how you
treat such things as misspelled words, neatness, handwriting, grammar, and so
on.

 Be on the alert for bluffing. Some students who do not know the answer may
write a well-organized coherent essay but one containing material irrelevant to
the question. Decide how to treat irrelevant or inaccurate information
contained in students' answers. We should not give credit for irrelevant
material. It is not fair to other students who may also have preferred to write
on another topic, but instead wrote on the required question.

 Write comments on the students' answers. Teacher comments make essay tests
a good learning experience for students. They also serve to refresh your
memory of your evaluation should the student question the grade.

 Be aware of the order in which papers are marked which can have an impact
on the grades that are awarded. A marker may grow more critical (or more
lenient) after having read several papers, thus the early papers receive lower
(or higher) marks than papers of similar quality that are score later.
 22
Chapter 4: Using Essay Tests

 Also, when students are directed to take a stand on a controversial issue, the
marker must be careful to insure that the evidence and the way it is presented
is evaluated, NOT the position taken by the student. If the student takes a
position contrary to that of the marker, the marker must be aware of his or her
own possible bias in marking the essay because the student‟s position differs
from that of the marker.

SUMMARY

 An essay is a test item which requires a response composed by the examinee

usually in the form of one or more sentences of a nature that no single
response or pattern of responses can be listed as correct, and the accuracy and
quality of which can be judged subjectively only by one skilled or informed in
the subject.

 Two types of essay based on their function: coursework essay and

examination essay

 Essay questions provide an effective way of assessing complex learning

outcomes.

 Essay questions provide authentic experiences because constructing responses

are closer to real life than selecting responses.

 Not possible to assess the student's mastery of the complete subject matter
domain with just a few questions.

 Essay questions have two variable elements—the degree to which the task is
structured and the degree to which the scope of the content is focused.

 Whether or not an essay item assesses higher-order thinking depends on the

design of the question and how students‟ responses are scored.

 Specifying the approximate time limit helps students allocate their time in
answering several essay questions.

 Avoid using essay questions for intended learning outcomes that are better
assessed with other kinds of assessment.

 Analytical marking is the system of most frequently used in large-scale public

examinations and also by teachers in the classroom. Its basic characteristic is
the marking scheme and the marks allocated for elements in the answer.

 The holistic approach to scoring essay questions involves reading an entire

response and assigning it to a category identified by a score or grade.
 23
Chapter 4: Using Essay Tests

KEY TERMS
Essay Task & scope Holistic method
Complex learning Intrascorer reliability Analytical method
outcomes Mental tasks Model answer
Inscorer reliability Marking scheme Time consuming
Constructed responses Rubric Grading

REFERENCES:

 Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for Learning,

Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational
Objectives. New York: Addison Wesley Longman.

 Ebel. L. (1979). Essentials of Educational Measurement. Englewood Cliff, NJ:

Prentice-Hall.

 Gramm, B. F. (1981). Measuring Classroom Achievement. NY: Holt,

Rinehart, Winston, 1981.

 McKeachie, W. J. Teaching Tips. (8th ed.) Lexington, Mass.: Heath, 1986.

 Moss, A., & Holder, C. (1988). Improving Student Learning: A Guidebook

For Faculty in All Disciplines. Dubuque, Iowa: Kendall/Hunt.

 Reiner, C., Bothell, T., Sudweeks, R. & Wood, B. (2002). How to Prepare
Effective Essay Questions. Department of Instructional Psychology &
Technology. Brigham Young University Testing Services.

 Stalnaker, J. (1951). The Essay Type of Examination. In E.F. Lindquist (Ed).

Educational Measurement. Menasha, Wisconsin: Geroge Banta. 495-530.

 White, E.M. (1985). Teaching and Assessing Writing: Recent Advances in

Understanding,Evaluating and Improving Student Performance. San
Francisco: Jossey-Bass.