lesson3developingateachermadetest-121012222240-phpapp01-1
lesson3developingateachermadetest-121012222240-phpapp01-1
Dr. Carlo
Magno
1
Check for alignment between the course
objectives and the competency covered in
the task
Check if the assessment is appropriate with
Content Validity
30 21-30
10 31-35
30
100
No. of items R U Ap An E C
30 1-10 26-35 81-90
10 11-15 46-50
100 25 20 5 20 10 20
Advance
1 Organizer
The Test Blueprint
Outline of the Test Development Process
Table of Specifications
2 Designing Selected-Response
Items
Binary-choice items
Instructions in Writing Binary Type of
Items Multiple-choice items
Guidelines in Writing Multiple-choice
Items Matching items
Guidelines in Writing Multiple-choice
Items
3 Designing Constructed-
Response Items
Short-answer items
Guidelines in Writing Short Answer
Items 2
Essay items
Objectiv
•
es
1.Explain the theories and concepts
that rationalize the practice of
assessment.
• 2.Make a table of specifications of the
test items.
• 3.Design pen-and-paper tests that are
aligned to the learning intents.
• 4.Justify the advantages and
disadvantages of any pen-and-paper test.
• 5.Evaluate the test items according
to the guidelines presented. 18
Outline of Test Development Process
• 1. Specify the ultimate goals of the education
process
• 2. Derive from these the goals of the
portion of the system under study
• 3. Specify these goals in terms of expected
student behavior. If relevant, specify the
acceptance level of successful learning.
• 4. Determine the relative emphasis or
importance of various objectives, their
content, and their behaviors.
• 5. Select or develop situations that will
elicit the desired behavior in the
appropriate context or environment,
assuming the student has learned it.
• 6. Assemble a sample of such situations that
together represent accurately the emphasis on19
content and behavior previously determined.
Outline of Test Development Process
• 7. Provide for the recording of responses in a
form that will facilitate scoring but will not so
distort the nature of the behavior elicited that
it is no longer a true sample or index of the
behavior desired.
• 8. Establish scoring criteria and guides to
provide objective and unbiased judgment.
• 9. Try out the instrument in preliminary
form.
• 10. Revise the sample of situations on the
basis of tryout information.
• 11. Analyze reliability, validity, and score
distribution in accordance with the projected
use of scores.
• 12. Develop test norms and a manual, and 20
reproduce and distribute the test.
Test Length
• The test must be of sufficient length
to yield reliable scores
• The longer the test, the more the
reliable the results
• The test should be valid if it is reliable
• For the grade school, one must
consider the stamina and attention
span of the pupils
• The test should be long enough to be
adequately reliable and short enough
to 21
Test Instruction
1. Table of specifications 10
2. Test and Item characteristics 20
3. Test layout 5
4. Test instructions 5
5. Reproducing the test 5
6. Test length 5
7. Scoring the test 5
TOTAL 55 25
Table of
Two GridSpecifications
TOS
Weight
(Time
Content
Outline
Knowledge
30%
Comprehension
40%
Application
30%
No. of items
by content
Frame) area
35% 1. Table of specifications 1 4 4 9
5% 4. Test instructions 0 1 0 1
5% 6. Test length 1 0 1 2
8 10 8 26
33
Writing supply
items
10. Avoid extraneous clues to the correct answer
FAULTY: A fraction whose denominator is greater
than its
numerator is a
IMPROVED: Fractions whose denominator are
greater than their numerators are called
34
ALTERNATIVE RESPONSE ITEM
• 1. Avoid broad general statements if they are
to be judged true or false.
• 2. Avoid trivial statements.
• 3. Avoid the use of negative statements.
• 4. Avoid long complex sentences.
• 5. Avoid including two ideas in one statement
unless cause and effect relationship are being
measured.
• 6. If the opinion is being used, attribute it to
some source unless the ability to identify
opinion is being specifically measured.
• 7. True statements and false statements
should be equal in length.
• 8. The number of true and false statements
should be
approximately equal. 35
Writing TRUE-FALSE
Items
1. Avoid the use of “specific determiners”
FAULTY: No picture-no sound in a television set
may indicate a bad 5U4G.
IMPROVED: A bad 5U4G tube in a television
set will result
in no picture sound.
2. Base true-false items upon statements
that are absolutely true or false, without
qualifications or exceptions.
FAULTY: World War II was fought in Europe and
the Far East.
IMPROVED: The primary combat locations in
terms of military personnel during World
War II were Europe and the Far East.
36
Writing TRUE-FALSE
Items
3. Avoid negative stated items when
possible and eliminate all double
negatives.
FAULTY: It is not frequently observed that copper
turns green as a result of oxidation.
IMPROVED: Copper will turn green upon
oxidizing.
4. Use quantitative and precise rather than
qualitative language where possible.
FAULTY: Many people voted for Gloria Arroyo in the
2003 Presidential election.
IMPROVED: Gloria Arroyo received more than 60
percent of the popular votes cast in the
Presidential election of 2003. 37
Writing TRUE-FALSE
5.
Items
Avoid stereotypic and textbook statements.
FAULTY: From time to time efforts have been made to explode
the notion that there may be a cause-and-effect
relationship between arboreal life and primate anatomy.
IMPROVED: There is a known relationship between primate
anatomy and arboreal life.
6. Avoid making the true items consistently longer than
the false items.
7. Avoid the use of unfamiliar or esoteric language.
FAULTY: According to some peripatetic politicos, the raison
d‟etre for capital punishment is retribution.
IMPROVED: According to some politicians, justification for
the existence of capital punishment can be traced
to the Biblical statement, “An eye for an eye.”
38
Writing TRUE-FALSE
Items sentences with many
8. Avoid complex
dependent clauses.
FAULTY: Jane Austen, an American novelist born in
1790, was a prolific writer and is best known
for her novel Pride and Prejudice, which was
published in 1820.
IMPROVED: Jane Austen is best known for her novel
Pride and prejudice.
9. It is suggested that the crucial elements of an
item be placed at the end of the statement.
FAULTY: Oxygen reduction occurs more readily
because carbon monoxide combines with
hemoglobin faster than oxygen does.
IMPROVED: Carbon monoxide poisoning occurs
because
39
carbon monoxide dissolves delicate lung tissue.
Writing Matching Type
Test
1. Matching Exercises should be complete on a
single page.
2. Use response categories that are related but
mutually exclusive.
3. Keep the number of stimuli relatively small
(10-15), and let the number of possible
responses exceed the number of stimuli by
two or three.
4. The direction should clearly specify the basis
for
matching stimuli and responses.
5. Keep the statements in the response column
short and list them in some logical order
40
FAULTY: Match List A with List B. You will be given
one point for each correct match.
List A List B
a. cotton gin a. Eli Whitney
b. reaper b. Alexander Graham
c. wheel Bell
d. TU54G c. David Brinkley
tube d. Louisa May Alcott
• e. steamboat
Directions failed to specify thee. None
basis for of these
matching
• List are enumerated identically
• Responses not listed logically
• Lacks homogeneity
• Equal number of elements
41
• Use of “None of the above”
IMPROVED: Famous inventions are listed in the left-
hand
column and inventors in the right-hand column
below. Place the letter corresponding to the
inventor in the space next to the invention for
which he is famous. Each match is worth 1 point,
and “None of these” may beInventors
Inventions the correct answer.
Inventors may be used
1. steamboat a. more than Graham-
Alexander once.
2. cotton skin Bell
3. sewing b. Robert Fulton
machine c. Elias Howe
4. reaper d. Cyrus McCormick
e. Eli Whitney
f. None of these
42
Writing Multiple Choice
1. It is recommended that the stem be a direct
question.
2. The stem should pose a clear, define,
explicit, and singular problem.
FAULTY: Salvador Dali is
a. a famous Indian.
b. important in international law.
c. known for his surrealistic art.
d. the author of many avant-garde plays.
IMPROVED: With which one of the fine arts is
Salvador Dali associated?
a. surrealistic painting
b. avant-garde theatre
c. polytonal symphonic music
43
d. impressionistic poetry
Writing Multiple Choice
3. Include in the stem any words that might
otherwise be repeated in each response.
FAULTY: Milk can be pasteurized at home by
a. heating it to a temperature of 130o
b. Heating it to a temperature of 145o
c. Heating it to a temperature of 160o
d. Heating it to a temperature of 175o
IMPROVED: The minimum temperature that can
be used
to pasteurize milk at home is:
a. 130o
b. 145o
c. 160o
d. 175o 44
Writing Multiple Choice
4. Items should be stated simply and understandably,
excluding all nonfunctional words from stem and
alternatives.
FAULTY: Although the experimental research, particularly that
by Hansmocker must be considered equivocal and
assumptions viewed as too restrictive, most testing
experts would recommend as the easiest method of
significantly improving paper-and-pencil achievement test
reliability to
a. increase the size of the group being tested.
b. increase the differential weighting of items.
c. increase the objective of scoring.
d. increase the number of items.
e. increase the amount of testing time.
IMPROVED: Assume a 10-item, 10-minute paper-and-pencil
multiple choice achievement test has a reliability of .40.
The easiest way of increasing the reliability to .80 would
be to increased
a. group size
b. scoring objectivity
e. testing
c. differential item scoring weights 3
time 0
Writing Multiple Choice
5. Avoid interrelated items
6. Avoid negatively stated items
FAULTY: None of the following cities is a state
capital except
a. Bangor
b. Los Angeles
c. Denver
d. New Haven
IMPROVED: Which of the following cities is a
state capital?
a. Bangor
b. Los Angeles
c. Denver
d. New Haven 46
Writing Multiple Choice
7. Avoid making the correct alternative systematically
different from other options
8. If possible the alternatives should be presented in
some logical, numerical, or systematic order.
9. Response alternatives should be mutually exclusive.
FAULTY: Who wrote Harry Potter and the Goblet of Fire?
a. J. K. Rowling
b. Manny Paquiao
c. Lea Salonga
d. Mark Twain
IMPROVED: Who wrote Penrod?
a. J. K. Rowling
b. J. R. R. Tolkien
c. V. Hugo
d. L. Carrol
47
Writing Multiple Choice
10.Make all responses plausible and attractive to
the less knowledgeable and skillful student.
FAULTY: Which of the following statements makes clear the
meaning of the word “electron”?
a. An electronic tool
b. Neutral particles
c. Negative particles
d. A voting machine
e. The nuclei of atoms
IMPROVED: Which of the following phrases is a description of
an
“electron”?
a. Neutral particle
b. Negative particle
c. Neutralized proton
d. Radiated particle
e. Atom nucleus
48
Writing Multiple Choice
11.The response alternative “None of the above” should be
used with caution, if at all.
FAULTY: What is the area of a right triangle whose sides
adjacent to the right angle are 4 inches long
respectively?
a. 7
b. 12
c. 25
d. None of the above
IMPROVED: What is the area of a right triangle whose sides
adjacent
to the right angle are 4 inches and 3 inches
respectively?
a. 6 sq. inches
b. 7 sq. inches
c. 12 sq. inches 49
d. 25 sq. inches
Writing Multiple Choice
12.Make options grammatically parallel to each other and
consistent with the stem.
FAULTY: As compared with the American factory worker in
the early part of the 19th century, the American factory
worker at the close of the century
a. was working long hours
b. received greater social security benefits
c. was to receive lower money wages
d. was less likely to belong to a labor union.
e.became less likely to have personal contact with
employers IMPROVED: As compared with the American
factory worker in the
early part of the century, the American factory worker at
the close of the century
a. worked longer hours.
b. had more social security.
c. received lower money wages.
d. was less likely to belong to a labor union
e. had less personal contact with his employer 50
Writing Multiple Choice
13.Avoid such irrelevant cues as “common elements” and
“pat verbal associations.”
FAULTY: The “standard error of estimate‟ refer to
a. the objectivity of scoring.
b. the percentage of reduced error variance.
c. an absolute amount of possible error.
d. the amount of error in estimating criterion scores.
IMPROVED: The “standard error of estimate” is most directly
related to which of the following test characteristic?
a. Objectivity
b. Reliability
c. Validity
d. Usability
e. Specificity
51
Writing Multiple Choice
14.In testing for understanding of a term or concept, it is
generally preferable to present the term in the stem and
alternative definitions in the options.
FAULTY: What name is given to the group of complex
organic compounds that occur in small quantities in
natural foods that are essential to normal nutrition?
a. Calorie
b. Minerals
c. Nutrients
d. Vitamins
IMPROVED: Which of the following statements is the best
description of a vitamin?
15.Use objective items – items‟ whose correct answers are
agreed by experts
52
Factual Knowledge
• The Monroe Doctrine was announced about 10 years
after the
a. Revolutionary War
b. War of 1812
c. Civil War
d. Spanish-American War
Conceptual Knowledge
2. Which of the following statements of the relationship
between market price and normal price is true?
a. Over a short period of time, market price varies
directly with changes in normal price.
b. Over a long period of time, market price tends to equal
normal price.
c. Market price is usually lower than normal price.
d. Over a long period of time, market price determines
normal
price. 53
Translation from symbolic form to another form,
or vice
3. versa
Which of the graphs below best represent the
supply situation where a monopolist maintains
a uniform price regardless of the amounts
which people buy?
A B C
D
S S
S
S S
Pric
Pric
Pric
Pric
e
e
S
e
S
Quantit Quantit Quantit
y y Quantit
y y
54
Application
In the following items (4-8) you are to judge the effects of a
particular policy on the distribution of income. In each
case assume that there are no other changes in policy
that would counteract the effect of the policy
described in the item. Mark the item:
A. If the policy described would tend to reduce the
existing degree of
inequality in the distribution of income,
B. If the policy described would tend to increase the
existing degree of inequality in the distribution of
income, or
C. If the policy described would have no effect, or an
indeterminate effect, on the distribution of income.
4. Increasingly progressive income taxes.
5. Confiscation of rent on unimproved
6. Introduction of a national sales tax
7. Increasing the personal exemptions from income taxes 55
Analysis
9. An assumption basic to Lindsay‟s preference for
voluntary associations rather than government
order… is a belief
a.that government is not organized to make the
best use of experts
b.that freedom of speech, freedom of meeting,
freedom of association, and possible only under
a system of voluntary associations.
c.in the value of experiment and initiative as a
means of attaining an ever improving society
d. in the benefits of competition
56
Judgments in terms of external criteria
For items 14-16, assume that in doing research for a paper
about the English language you find a statement by Otto
Jespersen that contradicts one point of view in a
language you have always accepted. Indicate which of
the statements would be significant in determining the
value of Jespersen‟s statement. For the purpose of these
items, you may assume that these statements are
accurate. Mark each item using the following key.
A. Significant positively – that is, might lead you to
trust his statement and to revise your own
opinion.
B. Significant negatively – that is, night lead you to
distrust his statement
C. Has no significance
14. Mr. Jesperson was professor of English at
Copenhagen University
15. The statement in question was taken from the very
first article that Jespersen published
16. Mr. Jespersen‟s books are frequently referred to in
other works that you consult.
57
Essay
• Questions
1. Ask questions or set tasks that will
require the examinee to demonstrate a
command of essential knowledge.
• 2. Ask questions that are determinate, in the
sense that experts could agree that one answer
is better than another.
• 3. Define the examinee‟s task as
completely and specifically as possible
without interfering with measurements of
the achievement intended.
• 4. In general, give preference to more specific
questions that can be more answered briefly.
• 5. Avoid giving the examinee a choice among
optional questions unless special
circumstances make such option necessary.
• 6. Test the questions by writing an ideal answer
58
Types of Essays:
• General – extensiveness of
responses
•Restrictive Response – reliable
scoring Learning outcomes
measured by Essay:
• Explain cause-effect relationship
• Describe applications of principles
• Present relevant arguments
• Formulate tangible hypothesis
• Formulate valid conclusions
• State necessary assumptions
• Describe the limitations of data
• Explain methods and procedures
• Produce, organize, and express
ideas
• Integrate learnings in different
areas 59
• Create original forms
Understanding:
A. Comparison of two phenomena on a single designated basis:
Compare the writers of the English Renaissance to those
of the nineteenth century with respect to their ability
tot describe nature
B. Comparison of two phenomena in general
Compare the French and Russian Revolutions
C. Explanation of the use or exact meaning of a phrase or
statement The book of John begins “In the beginning was
the word…” From what philosophical system does this
statement derive?
D. Summary of a text or some portion of it
State the central theme of the Communist Manifesto
E. Statement of an artist’s purpose in the selection or organization of
material
Why did Hemingway describe in detail the episode
in which Gordon, lying wounded, engage the
oncoming enemy?
What was Beethoven‟s purpose in deviating from 60
the orthodox
Application:
A. Causes or effects
Why may too frequent reliance on penicillin for the
treatment of minor ailments eventually result in its
diminished effectiveness against major invasion of
body tissues by infectious bacteria?
B. Analysis
Why was Hamlet torn by conflicting desires?
C. Statement of relationship
It is said that intelligence correlates with school
achievement at about .65. Explain this relationship
D. Illustrations or examples of principles
Name three examples of uses of the lever in typical
American
homes
E. Application of rules or principles
Would you weigh more or less on the moon? On
the sun? Explain.
F. Reorganization of facts
Some writers have said that the American Revolution
was not merely a political revolution against England
but also a social revolution, within the colonies, of the 61
poor against the wealthy. Using the same evidence
Judgment:
A. Decision for or against
Should members of the Communist Party be allowed to
teach in American colleges? Why or why not?
B. Discussion
Discuss the likelihood that four-year private liberal arts
colleges
will gradually be replaced by junior colleges and state
universities.
C. Criticism of the adequacy, correctness, or relevance of
a statement
The discovery of penicillin has often been called an
accident. Comment on the adequacy of this
explanation.
D. Formulation of new questions
What should one find out in order to explain why some
students of high intelligence fail in school?
62
Designing Interpretive Exercise
• Guidelines in Writing Interpretive Exercise
• 1. Select an introductory that is in harmony
with the objectives of the course.
– Amount of emphasis of various interpretive skills is
a factor.
– Do not overload test takers with interpretive items in a
particular area.
– Selection of introductory should be guided by general
emphasis
to be given to the measurement of complex
achievement.
• 2. Select introductory material that is
appropriate to the curricular experience and
reading ability of the examinees.
63
Guidelines in Writing
Interpretive
•
Exercise
3. Select
pupils.
introductory material that is new to
65
• Ability to Recognize Warranted and
Unwarranted Generalizations
66
• Ability to Recognize Inferences
67
• Ability to Interpret Experimental Findings
68
• Ability to Apply Principles
69
• Ability to Recognize Assumptions
70
Reading comprehension
• Bem (1975) has argued that androgynous
people are “better off” than their sex- 35. What is the independent
typed counterparts because they are not variable in the study?
constrained by rigid sex- role concepts and
are freer to respond to a wider variety of
situations. Seeking to test this hypothesis,
Bem exposed masculine, feminine, and
a. Situations calling for
androgynous men and women to situations independence and
that called for independence (a masculine
attribute) or nurturance (a feminine
nurturance
attribute). The test for masculine b. Situation to make the sex type
independence assessed the subject‟s
willingness to resist social pressure by
react
refusing to agree with peers who gave c. Situations to make the
bogus judgments when rating cartoons for
funniness (for example, several peers
androgynous be flexible
might say that a very funny cartoon was d. Situations like sex type,
hilarious). Nurturance or feminine
expressiveness, was measured by
androgynous and sex role
observing the behavior of the subject when concepts
left alone for ten minutes with a 5-month
old baby. The result confirmed Bem‟s
hypothesis. Both the masculine sex-typed 36. What are the levels of the IV?
and the androgynous subjects were more
independent (less conforming) on the
„independence” test than feminine sex- e. masculine attribute and
typed individuals. Furthermore, both the
feminine and the androgynous subjects feminine attribute
were more “nurturant” than the masculine f. rating cartoons and taking
sex-typed individuals when interacting with 71
the baby. Thus, the androgynous subjects care of a baby
were quite flexible, they performed as g. independence and nurturance
Interpreting Diagrams
Instruction. Study the following illustrations and
answer the following questions.
101. Which group received the treatment?
Figure 1
a. group A b.
group B
b. c. none of the above
102. Why did group B remain stable
across the experiment?
a. there is an EV
b. there is no treatment
c. there is the occurence of ceiling effect