University of Basrah /College of Education for Human Sciences

Department of English /Evening Study 2020-2021

Subject :Test Design & Assessment.
Year: Fourth
Set by: Kareem A. Abed
________________________________________________________________
Lecture 1
Chapter One
Assessment and Testing
what is the difference between assessment and Test?
The two terms are used interchangeably.
1- Assessment​ refers to any means of checking what students
can and cannot do with the language,(the positive side is more
proper).
Why do we assess?
There are seven common purposes behind doing an
assessment.
1- Placement : It means to pace or classify students on a
suitable course.
2- Diagnosis: to check or discover students' particular
strengths and weaknesses.
3- Selection : to select the best candidates for a limited
number of place for a job or for higher education.
4- Evaluation : to evaluate the effectiveness of the syllabus as
well as the method of teaching.
5- progress : to discover what students have learnt from a
particular syllabus and what progress they have made.
6- prediction : to discover potential abilities and aptitude and
predict probable future successes in and outside schools.
7-Accreditation : to provide evidence that a student has
undergone a course or a study and he is qualified to practice
his profession.

2-Test is a ​method of ​measuring a person's ​ability, knowledge

or performance in a given domain. It means presenting a
students with a set of questions or tasks in order to obtain a
measure of that performance often presented by a score.
1- Method= instrument ument, set of techniques, procedures
that require performance on the part of Test-takers.
E. g. Multiple-choice questions with prescribed correct
answers. Writing prompts with a scoring rubric..
2- A test must measure ; some tests measure general ability
While others focus on very specific competencies or
objectives.
3- It measures the individual's ability, knowledge or
performance. Testers need to understand who the Test-takers
are. What is there previous experience?
University of Basrah /College of Education for Human Sciences
Department of English /Evening Study 2020-2021
Subject :Test Design & Assessment.
Year: Fourth
Set by: Kareem A. Abed
________________________________________________________________
Lecture 2
Types Assessment
1-​ ​Formal​ vs. ​Infomal​ Assessments
- Formal assessment is a large - scale test administered in a controlled
environment typically once a year and prepared either by the school
teacher or by professional testing services,(Ministerial Examinations).
One fundamental aim of such type of assessments is to assist institutions
in the selection, evaluation,for example,when students are distributed
among the Universities according to their scores.
- Informal assessment is one type of assessment that allows teachers to
check the ongoing progress of their students and to measure how well
students mastering the content.
2-​ ​Formative ​vs.​ Summative Assessments
- Formative assessment is the one that provides information to promote
teaching and learning. It means to evaluate the students in the process of
forming their competencies and skills with the goal of helping them to
continue. Thus, it helps teachers to identify the material which needs to
be clarified and re-taught.
- Summative assessment is given at the end of a course to measure the sum
total of the material covered (what a student has grasped) . It is used to
determine whether or not students have met the course goals or learning
outcomes at the end of a course.

Types of Test
1- ​Language Aptitude Test.
Although admittedly not not common, it predicts a person's success prior to
exposure to a second language, the capacity or ability to learn a foreign
language. ( Applied in the united States​, ​Modern Language Aptitude ​Test).
Learners are exposed to some tasks, for example, examinees must learn a set of
numbers through aural input and then discriminate different combinations of
those numbers ; or, they must learn a set of correspondences between speech
sounds and phonetics symbols​. For more details, see the table on p. 44 in your
textbook.

2-​ ​Proficiency Test.

The aim of such a test is to test global competence in language. It is not related
to any course or single skill in language ;rather, it test overall ability. Such tests
take many forms like: standardized multiple- choice items on grammar,
vocabulary, reading comprehension and so on.

3- ​Placement Test.
Some proficiency test can play in the role of a placement test, the purpose of
which is to place a student into a particular level or section of language
curriculum or school. Such a test usually indicates the point at which the student
will find the material neither too easy nor too difficult but appropriately
challenging,l. ​The English as a Second Language Placement (ESLP) at San
Francisco University has Three parts. (a) The students read a short article and
then write a summary essay about It. (b) students write a composition in
response to an article.(c ) This part is a multiple - choice : students read an essay
and identify the grammatical errors in it.

4- ​Achievement Test.
It is the most important type of Testing since school teachers are responsible for
testing how much a student has learnt of a learning syllabus. It is based on
detailed course syllabus (annual school examination and all public examinations
are of this type). In this perspective , ​the test is related to class room lessons and
units that it should be limited to a particular material within a particular time
frame. The aim is to see whether a course has achieved its objectives or not. The
maximum time allowed for the test is three hours.

5- ​Diagnostic Test.
This test is designed to diagnose a specific aspect of language ;what skills or
aspects of language programme a student has achieved or not achieved. ​For
example, a test in pronunciation might diagnose the phonological features of
English. A typical test of this type was developed by Clifford Prator (1972) to
accompany the manual of English pronunciation. Test-takers are directed to
read a 150-word passage while they are tape-recorded. These recordings are
going to be analyzed phonologically with reference to learner's production.
University of Basrah /College of Education for Human Sciences
Department of English /Evening Study 2020-2021
Subject :Test Design & Assessment.
Year: Fourth
Set by: Kareem A. Abed
______________________________________________________
Lecture 3
Approaches to Language Testing : A Brief History.

Having reasonably grasped some common assessment terms, we now turn to

discussing the creation and use of tests, particularly classroom tests. Historically
speaking, language-testing trends and practices have followed the shifting of
teaching methodology. For example, in the 1950s, an era of ​Behaviorism and
special attention to contrastive analysis, testing focused on specific language
elements such as the phonological, grammatical and lexical contrasts between
two languages. In the 1970s and 1980s, ​communicative theories of language
brought with them a more integrative view of testing in which specialists
claimed that the whole of the communicative event was considerably greater
than the sum of its linguistic element.

1- ​Discrete - point and Integrative Testing

This approach underscores two major approaches to language testing which are
the ​discrete- point ​and ​integrative testing. ​Discrete - ​point tests are
constructed on the assumption that language can be broken down into its
component parts and those parts can be tested successfully​. These components
are the skills of listening, speaking, reading, and writing and various units of
language (discrete points) of phonology, graphology, morphology, syntax,
lexicon and discourse. It was claimed that an overall language proficiency test
should sample all four language skills and as many linguistic discrete points as
possible.
This method was criticized by oller (1987) stating that language competence is
a unified set of interacting abilities that cannot be tested separately.

Integrative tests, ​on the other hand, involves the testing of language in context
and is concerned with meaning and the total communicative effect of discourse.
There are two examples integrative test. : ​Cloze test and ​Dictation​. A ​Cloze
test is a reading passage (perhaps 150-300 words) in which roughly every sixth
word or seventh has been deleted. The test-taker is required to supply words that
fit into those blanks​. Oller (1979) claimed that Cloze test results are good
measures of overall proficiency. According to theoretical conducts underlying
this claim, ​the ability to supply appropriate words in the blanks requires a
number of abilities that lie at the heart competence in language​. ​In Dictation,
learners listen to a passage of 100-150 words, read aloud by administrator or
audiotape and write what they have heard, using correct spelling. In fact there is
a short pause between every phase to give chance to the learners to write down
what is heard. Supporters of this way argue that dictation is an integrative test
because it taps into grammatical and discourse competencies.

2-​ Communicative Language Testing

This approach is sometimes linked to the integrative approach because both
emphasize the importance of the meaning of the utterance. ​Language in the
communicative approach is a tool for communication. There should be a
correspondence between language test performance and Language use​. So a
test must correspond to language use in non-test situations (authenticity). One
problem that assessment experts faced was that tasks intended to be artificial,
contrived, and unlikely to mirror language use in real life.

3- Performance- Based Assessment

Test Designers around the world are not tackling this new and student-centred
agenda. ​Instead of just offering paper-and-pencil selective respose tests,
performance assessment of language typically involve oral production, written
production, open-ended responses, integrated performance, group performance​.
Students are assessed as they are performing actual or simulated real-world
tasks​. Such assessments ​In fact, what is good about these assessments is the
presence of interactive tasks: the assessments involve learners in actually
performing the behavior that we want to measure. They are measured in the act
of speaking, requesting, responding or in combining ​listening and speaking, and
in integrated reading and writing.
 Assessment
Lecture 4
______________________________________________________________

Principles and Characteristics of a Good Language Assessment or Test.

In this part, we will explore the characteristics and principles of formal

assessment, but these principles also apply to assessments of all types. A
question might be raised here, ​how do we know if a test is effective? Such a
question can be answered by responding to such questions : can it be given
within appropriate administrative constrains ?Is It dependable? Does it
accurately measure what you want it to measure? These and other questions can
identify six cardinal criteria for testing or test. In other words, for a test , if it is
to adequately fulfill its purpose, should satisfy the requirements of six key
characteristics : Validity, practicality, reliability, accuracy, authenticity and
washback.

1- Validity
Validity is the degree to which a test measures what it is supposed to be
measured (a test should measure what it is intended to be measured), what
precisely does the test measure? How well does it do that?

There are in fact four kinds of validity. The first is ​Content Validity and is
concerned with what is being tested. The remaining three, ​empirical, face and
construct ​are concerned with the extent to which the measurement is
satisfactory.

a- ​Content Validity
Almost certainly the most important issues for a teacher when preparing a test
are (a) ​the extent to which a test covers the syllabus to be tested and (b) the
relative importance of each area and the number of items given to it. ​The first
refers to language area to be tested while the second is concerned with the
extent to which the questions adequately cover the language area to be tested.

b- ​Empirical Validity
This one is referred to as statistical validity. If we want to check the
effectiveness of a test and to determine how a test measures, ​we should compare
the test results with the result of some independent outside criteria that we
believe is an indicator of the ability tested. If there is a high correlation between
them then our test is empirically valid. ​Examples of independent criteria include
the score given at the end of a course, or an external examination.

c- ​face Validity
This means the extent to which students view the test (the way a test loos to the
testees) as fair, relevant and useful for improving learning​. Or it is the degree to
which a test looks right or appears to measure the knowledge or ability it claims
to measure.

d- Construct Validity
Construct validity means that the testing methods should be in harmony with the
teaching method used. ​Otherwise the teaching programme is not likely to
succeed in achieving its objective if there is no close relationship between the
course objective and the instructional material on the one hand and the teaching
and the testing methods on the other. ​For example, when a course of a study
emphasizes the communicative aspect of the language and the test is designed
according to discrete-point items. The construct validity of this test will be low.

2- ​Reliability
Reliability is the stability of test scores. In other words, if a test is given twice to
the same group of students, under the same conditions, it would give the same
results. The requisites of relaliable test are ​:(a) multiple samples which means
that a test must be long enough to provide a generous sampling of the area
tested, besides it should contains a wide variety of levels of difficulty, ​(b)
standard conditions which means that all students take the test under identical
conditions, in a listening test, for example, students must be able to hear the
items clearly​,(c) ​standard task which means all students must be given the same
items of equal difficulty, and​,(d) standard scoring which refers to the fact that
all papers must be scored in an identical manner. A teacher or scorer should
give the same or nearly the same repeatedly for the same test performance.

3- ​Practicality
Two features have to be considered to achieve practicality of a test: economy
which means cost in time, money and personnel of administrating a particular
test and ease, the degree of difficulty experienced in the administrating and
scoring a test, for example, an oral test that demands the use of a tape recorder
is not practical it it has to be administered to thousands of students.

4- ​Accuracy
Accuracy means a test should be free from grammatical, spelling, and
punctuation errors. The numbering of questions, sub-questions, and items
should be correct. The directions for each question should be accurately worded,
with marks allotted for it as well as the time allotted for the whole test. An
example for such a case is ​choose the correct option (accurate), and ​complete
the following sentences(​inaccurate​).

5- Authenticity
Bachman and Palmer (1996) define authenticity as ​the degree of
correspondence of the characteristics of a given language test task to the
features of a target language task. ​Essentially, when you make a claim for
authenticity in a test task, you are saying that this task is likely to be enacted in
the real world.
In a test, authenticity may be present in the following ways.:
- The language in the test is as natural as possible.
- Items are contextualized rather than isolated.
- Topics are meaningful (relevant, interesting) for the learner.
- Tasks represent, or closely approximate, real world task.

6- Washback
Washback refers to the effect of testing on teaching and learning. When a
teacher limits the weak and strong points of his students, this is called
washback.
Positive washback refers to expected test effects. For example, a test may
encourage students to study more or may promote a connection between
standards and instruction. Negative washback refers to the unexpected, harmful
consequences of
 Lecture 5
_______________________________________________________________________

Some practical steps to test construction

First of all, when you want to conduct a test, you should know what is the purpose of your
test. It is unlikely that you would be asked to design an aptitude test or a proficiency test, but
you, but it is important that you understand their nature.

Assessing Clear, Unambiguous Objectives

In addition to knowing the purpose of the test you are creating, you need to know as
specifically as possible what it is you want to test. Some teachers give test simply because it
is the third week of the course, and after hasty glances at the chapter(s) covered during
those three weeks, they dash off some test items so that students have something to do
during the class. ​Instead, begin by taking a careful look at everything that you think your
students should know,or be able to do based on the material that the students are
responsible for. In other word examine the objectives for the unit you are testing.

Drawing up Test Specifications

Test specifications for classroom use can be a simple and practical outline of your test. For
example what skills you will test? And what the items will look like?

Devising Test Tasks

Your oral interview comes first, and so you draft questions to conform to the accepted
pattern of oral interview. ​You begin and end with non-scored items( warm up and wind
down) designed to test students at ease, and then sandwich between them items intended to
test the objective.

Designing Multiple-Choice Test Items.

Multiple-choice items, which may appear to be the simplest kind of item to construct, are
extremely difficult to design correctly.
 Language Assessment /4th Year - Evening Study
Mr. Kareem Abed
_______________________________________________________________________

Lecture 5
Test Design and Construction

Now the discussion is shifted to the fundamentals of test construction and

administration. In fact the constructing of an educational test that is a
teacher-made includes the following steps:

1-Planning the test.​ ​Effective testing requires careful planning, and much
consideration of the balance of the test content.

2- ​Preparing the test items​. A teacher may write more items than he could
possibly need or be answered by the testees during the allotted time for the test
and so some items have to be eliminated. As a result, we might omit some of the
material which the students had mastered at a previous stage of learning. We
could thus (a) ​concentrate on the new range of activities that we taught in the
course at later time. Another important issue must be considered, in this context,
is the (b) ​time to be provided for the test​. Also​,(​c​) the test techniques are
important here which may include multiple choice items, completion, short
answer questions, essay writing and so on. (d) ​The test directions must be
simple, clear easily understood and free from possible ambiguity.

3- Reviewing the items​. Once the items are written, they should be set aside for
a day or two days before being reviewed by the teacher, when the teacher is
satisfied with the test, it should be submitted to a colleague who is experienced
in the subject. The comments of this outside reviewer have to be taken into
consideration. and the Faculty items must be corrected before preparing the
final version of the test.
Note​:If a test is for research purposes, it should be submitted to a jury of a least
ten language and Testing experts.

4- Setting the scoring scheme​. ​For the purpose of objectivity and reliability,
scoring scheme should be made. Scoring refers to the process of correcting tests
and assessing numerical scores. In objective tests each item is marked as correct
or incorrect(full mark or zero). In semi-objective tests like transformation or
completion with three or four words, a different scoring scheme is required. For
 Language Assessment /4th Year - Evening Study
Mr. Kareem Abed
_______________________________________________________________________
example, two marks are given to each correct answer, zero to incorrect answer
and one mark to a recognizable answer with some errors.

5- Reproducing the test.​ The next step is the presentation of the paper itself.
Where possible, it should be printed so as to appear neat and tidy. Nothing is
worse and more confusing to the testees than an untidy test paper, full of
spelling, omission and corrections.

6- Administrating the test​. ​This depends on perfect preparations for applying

the test on the testees. It includes selecting the testing room which should be
quiet, well lighted, large enough to permit the seating of the testees for
reasonable distancing, checking test material in which the teacher must ensure
that there is a sufficient supply of testing copies and extra copies for
emergencies.

7- ​Analyzing the results. Having given the test and collecting the papers, the
teacher's task is to calculate the number of students who have responded directly
to the test items and reach the required level for example 70 or 75 per cent of
the items have been successfully answered. Students reaching this level will be
those who have succeeded in terms of the course objectives and those who fail
to reach the lovell are at-risk students who need assistance.

8- Using the results. ​Besides measuring the ​students' achievement, this last
step helps to pinpoint the problematic items some students failed to answer
correctly, and diagnose their specific weaknesses. This helps the teacher to
provide feedback, and extra exercises.
 Test Design and Assessment
______________________________________________________________
Lecture 6

Constructing Questions

Before going deep into how to form various types of questions, like True/False
questions, writing a composition, multiple choice items questions, gap filling
questions and so on, we would like to refer to two types of tests : objective tests
and subjective tests.
Objective tests​, like True /False, Multiple-Choice items, matching, gap filling
and completion, rearrangement and odd item out, are the ones which are either
marked by a full mark or zero because they have one limited answer, i. e., there
is no flexibility to have half mark or something like that.
Subjective test include short-answer essay, extended-response essay, problem
solving and performance test items,writing a composition.

1- ​Multiple-Choice items( MCQ)

​The multiple - choice test is generally recognised as the most widely
applicable type of objective tests. There are some suggestions for constructing
Multiple-Choice items test :
1-The item should be meaningful and presents a definite problem.
Poor​ : The Nile_______
- A- flows from north to south
- B- is as long as Tigris
- C- Floods every year.
- D- Flows in the Sudan and egypt.
Better​ : The Nile which flows in the Sudan and egypt is _______km
- A- 1950 B- 4160 C- 2400 D- 3750
2- ​The items should be free from irrelevant material.
Poor: ​Pupils in school should wear a uniform. What is the color of the short
pupils wear in your school?
- A. Green, and B- Blue C-White D-Grey
Better: ​Iraqi school boys have to ______Shirts
- A. green B. blue C- white D- grey

3- ​An item should contain only one correct answer.

Poor: ​which of the following is the best sources of heat.
- A. Coal, B- Electricity C- Gas D-Oil
Better:​Which is the most expensive source of heating for home use in Iraq?
____
- A. Coal B-Electricity C- Gas D- Oil
4- ​Make sure you do not give away the right answer through grammatical
cues.
Poor: ​She needs to get up earlier so she is buying an ______ clock.
- A-time B- alarm C- watch D- bell.
Better: ​She needs to get up earlier so she is buying ______clock.
- A- A time B- an alarm C- a watch D- a bell.

5- ​The choices should contain repeated words or phrases.

Poor: ​Does this shop stay open​ ____?
- A- during saturdays -B- on saturdays C- in Saturdays D- at saturdays
Better: ​Does that shop stay open _____ Saturdays.
- A- during B- in C- at D- on
6-​The correct answer should appear in random order
7- Whenever possible, all options should be related to the same general
area of activity.

Advantages of Multiple-Choice Items (MCQ)

1- They can be answered in a short time.
2- The test can have a wide coverage.
3- The test is easy to mark and the marks are objective.
4- It can be set for any level of difficulty.

Disadvantages of Multiple-Choice Items

1- Writing a technically sound item is not easy.
2- It takes a long time to construct.
3- Finding sufficient number of incorrect but plausible distractors is difficult.
4- The test results can be distorted by pupils guessing the answers or cheating
by copying others' answers
 Assessment 7
_______________________________________________________

Preparing True/False Questions.

The True-False item consists of a declarative statement that The testee

is asked to mark as True or False, right or wrong, correct or incorrect,
yes or no, agree or disagree and the like. In each item there are only
two possible answers. The testee is asked to select one. True - False
items are used for measuring the ability to identify The correctness of
statements of facts, definitions or items, statements of principles and
the like,for example:

● Draw a circle round "T" if the statement is True and a circle

round "F" if the statement is false.

T F 1- Iraq is very hot in summer.

T F 2-It rains in summer in Iraq.
● Write "T" if the statement is true and "F" if the statement is
False.

Suggestions for Writing T/F items.

1-Avoid any source of ambiguity, for example, avoid generalizations

and words like :every, never, nothing, none, often, may.
Example:
Poor : Frosts often develop during cloudy night.
Better :Frosts develop during cloudy night.

2- Avoid the use of negative statements especially double negatives.

For example:
Poor: None of the experiment steps was unnecessary.
Better : All the steps in the experiment were necessary.
3-Avoid long complex sentences so that you may not miss the central
point of your statement.

Poor: Coal, which used to be the most important source of power for
ages, is used nowadays in most field of primary industries.
Better : Coal is used nowadays in primary industries.

4- Avoid lifting statements verbatim from the textbook . The testees

may have memorized the passages.

5- The number of the True statements and the false statements should
be approximately equal in one test.

6- Guessing can be overcome by including a request for an

explanation of the selection of a particular judgement.

Advantages
1- True-False items are easy to construct.
2- They can cover a wide sampling of the course material.
3- Scoring is easy.

Disadvantages
1- They can be violated by the testee's guessing and cheating.
2- The learning outcomes are largely limited to the knowledge area.
 Assessment 8

Preparing Matching Items

This test in its simplest form consists of two lists with instructions as how
matching is to be undertaken. The items in the first list,of which a match is
sought,are called " premises" and those in the other list from which a selection
is made are called responses.The testee's task is to identify the pairs of items
that are to be associated.

Suggestions for Constructing Matching Tests:

1- Use only homogeneous material in single matching test,for example:dates
and events,or pictures and words,etc.
2- Keep the number of premises small,but let the number of responses be larger
by about two or three.
3- Place all the items for one matching test on the same page.
4- Arrange the responses as far as possible in a logical order.This will help the
testees to scan the responses quickly. For example in a names list present them
in alphabetical order.
5- Keep the list of items to be matched brief.

Advantages
1-The first advantage is ease of construction.
2- It is possible to measure a large amount of related factual material in a
relatively short time.
For example:
pictures and words
Questions and answers.
Titles and Texts
Authors and Titles of Books
Machines and Uses
Verbs and Nouns (collocation)

Disadvantages
1- Matching test are restricted to the measurement of factual information.
2- Finding significant homogeneous material is sometimes difficult.
3- Matching test are sometimes considered a reduced multiple choice test. Once
you matched the first ones,you are left with fewer options. Eventually the
answer to the difficult item may be the only one left.
 Assessment 9

Testing Reading Comprehension

Comprehending a written text means getting the required
information from it as efficiently as possible. However ,in most
courses of English there is no agreement of the amount of the
input and output on the part of the students. You often hear that
reading comprehension is a passive skill. In fact reading is not a
passive skill? Because reading is given a further weight in the
format used for testing individual grammatical structure
,vocabulary spelling and listening comprehension.
What is more, the skill of reading cannot be considered entirely a
passive one because students need to be able to process the
information given in a number of ways. The processes involve
various important cognitive skills ;the main ones are exemplified
in the following list:

Reading Comprehension Cognitive Skills

● An ability to comprehend the whole despite of or incomplete
comprehension of some parts.
● An ability to understand implicitly and explicitly stated
information.
● An ability to give close attention and follow step by step
details.
● An ability to scan to obtain specifically required
information.
● An ability to use language cues to help in comprension.
● An ability to infer or draw correct conclusions.
● An ability to see the relationship of parts to the whole.
● An ability to understand pronoun reference
Choice of the Text
● It is important to use authentic test whenever possible so as
to get the students accustomed to reading authentic texts
from the very beginning.
● The text for a reading comprehension test should be kept
brief ;yet there should be sufficient content to yield five or
six comprehension items.
● The text should not deal with information that is universally
known; otherwise the testees will be able to answer the
questions correctly without referring to the text.
● The text should not be loaded with extremely difficult lexical
items or complex syntactical structure. The teacher can,
however, give the meaning of two or three words in the text
when needed or rephrase some sentences.

Suggestions for Item writing

1- Answering the question should involve interpretation of the
text ,not merely matching words in the items with the tex.
2- A test of reading comprehension is a test of concepts rather
than a test of grammar or vocabulary, though both are needed to
comprehend a text .
3- The questions should test the cognitive skills aforementioned.
4- The questions should be kept grammatically and lexically as
simple as possible since the main concern is the interpretation of
the text.