Scribd
Upload
95 views

Increasing The Validity of Adapted Tests

The document discusses guidelines for properly adapting tests for use in other languages and cultures. It begins by outlining reasons why tests are often adapted rather than newly created. It then debunks 5 common "myths" about test adaptation, emphasizing the importance of establishing construct equivalence and avoiding assumptions that translations alone are sufficient. The remainder of the document provides a 13-step process for rigorous test adaptation, emphasizing careful translation, pilot testing, and validation to ensure scores are comparable.

Uploaded by

IrisAnne
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
95 views

Increasing The Validity of Adapted Tests

The document discusses guidelines for properly adapting tests for use in other languages and cultures. It begins by outlining reasons why tests are often adapted rather than newly created. It then debunks 5 common "myths" about test adaptation, emphasizing the importance of establishing construct equivalence and avoiding assumptions that translations alone are sufficient. The remainder of the document provides a 13-step process for rigorous test adaptation, emphasizing careful translation, pilot testing, and validation to ensure scores are comparable.

Uploaded by

IrisAnne
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 18

"Increasing the Validity of Adapted Tests: Myths to

be Avoided and Guidelines for Improving Test


Adaptation Practices"

Ronald K. Hambleton and Liane Patsula


University of Massachusetts at Amherst

Association of Test Publishers


August, 1999
Reasons for Adapting Tests
1. Adapting a test is considerably cheaper and faster than
constructing a new test in a second language
2. When the purpose for the adapted test is cross-cultural or
cross-national assessment, an adapted test is the most effective
way to produce an equivalent test in a second language
3. There may be a lack of expertise for developing a new test in a
second language,
4. There is a sense of security that is associated with an adapted
test more so than a newly constructed test especially when the
original test is well-known
5. Fairness to examinees often results from the presence of
multiple language versions of a test
5 Myths About Adapting Tests
Myth 1. The preferable strategy is always to adapt
an existing test rather than develop a new test for
a second language group.

Focus: This depends on many factors including the


definition of the construct being measured.
Is the language in which performance is to be
demonstrated a part of the construct definition or not?
Myth 2. Anyone who knows the two languages can
produce an acceptable translation of a test.

Focus: There is considerable evidence suggesting that


test translators need to be familiar with both source
and target languages and the cultures, and they need
to be generally familiar with the construct being
assessed, and the principles of good test development
practices.
Myth 3. A well-translated test guarantees that the
test scores will be valid in a second language or
culture for cross-language comparative purposes.

Focus: Not only should the meaning of a test be


consistent across persons within a language group
and culture but, that meaning, whatever it is, must be
consistent across language groups and cultures.
Myth 4. Constructs are universal, and therefore all
tests can be translated into other languages and
cultures.

Focus: Different cultures define psychological


construct differently according to their own
experiences. i.e intelligence, speed, quality of life.
Myth 5. Translators are capable of finding flaws in a test
adaptation. Field testing is not usually necessary.

Focus: The fact is translators are not able to anticipate all of


the problems encountered by examinees taking a test in a
second language. Some problems can be differences in the
understanding of the construct, or non-equivalence between
the two languages.
Poor translation can make the test item considerably easier or
harder.

Conclusion: In summary, all of the myths can seriously


compromise the validity of a test in a second language or
cultural group, or negatively influence the validity of adapted
tests for use in cross-language comparison studies.
Steps for Adapting Tests
Step 1 – Ensure that construct equivalence exists in
the language and cultural groups of interest.

Suggestions:
Through discussions with psychologists and other knowledge persons
in each culture, determine if the construct exists, and if the same
definition applies equally well in both language and cultural groups.
You can find this out by interviewing or observing people from the
cultures of interest, researching the cultures of interest, asking others
who know about the cultures, or visiting people in the culture. Either
consider "decentering" (that is, revising the definition of the construct
to be equally equivalent in each language and cultural group) or
discontinue the project.
Step 2 - Decide whether to adapt an
existing test or develop a new test.
Suggestions:
Consider the purpose of the adapted test, and carefully
consider the advantages and disadvantages of adapting a
test versus constructing a new test.
But when cross-cultural comparisons are not of interest,
it may be easier to actually produce a new test that
meets the cultural parameters in the second language
group, than to adapt an already existing test which may
have a number of shortcomings (e.g., a less than
satisfactory definition of the construct, inappropriate
item formats, use of some cultural specific content, etc.).
Step 3 – Select well-qualified
translators.
Suggestions:
Seek out translators with language proficiency,
knowledge of the relevant cultures, and some subject
matter knowledge/knowledge of the construct of
interest.
Involve more than one translator in the process to
provide a mix of perspectives and to enable checking
to be conducted
Step 4 – Translate and adapt the test.
Suggestions:
Use a forward translation design but a backward translation design can
be useful too, but not as the only design.
Step 5 – Review the adapted version of the test and
make necessary revisions.
Suggestions:
Review and revision of the adapted test is absolutely necessary,
following the initial translation. In most cases, the adapted test is too
important to be dependent on the insights of a single translator or
group of translators.
In a forward translation design: Another set of translators examine the
adapted version of the test for any errors that may lead to differences in
meaning between the two language versions.
With a backward translation design: Translators would take the
adapted version of the test, back translate to the source language, and
then judgments would be made about the equivalence of the original
and back-translated versions of the test.
Step 6 – Conduct a small tryout of the
adapted version of the test.
Suggestions:
Conduct a pilot test to gain preliminary information about
the test, and revise accordingly.

Step 7 – Carry out a more ambitious field test.


Suggestions:
Design and carry out an ambitious field test to check out
test items, test and subtest reliabilities, and the factor
structure of the test (factor analysis or structural equating
modeling are popular for this analysis). Compare findings
to those obtained with the source language version of the
test.
Step 8 –Choose astatisticaldesign for connecting scores on the sourceand target languageversions ofthe test.

There are three popular linking designs:


1. Bilingual group design –single group of bilingual individuals
take both forms
2. Matched monolingual group design -2 different monolingual
groups that are matched based on a variable expected to
correlate highly with the construct measured.
3. Monolingual group design -2 different groups are involved, one
from each language/cultural group

Suggestions:
Choose a linking design to equate scores from the source and
target language versions of the test. Item response modeling is a
standard way to proceed. Large samples are highly desirable at this
step to produce a stable linking of scores from one test to the other
Step 9 – If cross-cultural comparisons are of interest,
ensure equivalence of the language versions of the test
Suggestions:
Conduct a Differential Item Functioning study using one or
more of the standard statistical procedures.
There are items that function differently for each group,
rewrite or retranslate, readminister, and reanalyze those
items to determine whether they function the same for both
groups.
Step 10 – Perform validation research as appropriate.
Suggestions:
Conduct empirical studies which address the equivalence of
the multilanguage versions of the test in the populations
where the test will be used. Evidence of construct
equivalence as well as the absence of method and item bias
are important
Step 11 – Document the process and prepare a manual
for the users of the adapted test.
Suggestions:
Document the full process of adapting a test. Everything
from the persons involved, and designs used, to the
findings and the nature of the changes which were made
needs to be compiled and placed in a technical manual
for future reference.

Step 12 – Train users.


Suggestions:
Train test administrators to follow the directions and to
answer any questions appropriately which may arise.
Step 13 – Ongoing monitoring of the adapted test.
Suggestions:
Document the full process of adapting a test.
Everything from the persons involved, and designs
used, to the findings and the nature of the changes
which were made needs to be compiled and placed in a
technical manual for future reference.

Conclusion:
Adapted tests will have limited value unless they are
adapted with a high degree of concern for issues of
usability, reliability and validity.
ITC Test Adaptation Guidelines
Context: Effects of cultural differences not relevant to the
purpose of the test should be minimized to the extent possible.

Test Development and Adaptation


Take into account the following:
- Linguistic and cultural differences
- Testing techniques, item formats, and procedurs
- Item Content
- Appropriate statistical techniques
- Evaluation of validity
- Equivalence of questions
Administration
Anticipate and be sensitive to:
-standardized administration procedures and instructions
-environmental influences
-minimal role of the examiner
-problems that can be expected in adapted test

Documentation/Score Interpretations
-Changes in the adapted test
-Score differences among populations
-Specific information on the ways in which the socio-
cultural and ecological contexts of the population might
affect performance

You might also like