NAME: Drishtee Singh

SECTION: A
SEMESTER: II

SOURCES OF BIASES IN PSYCHOLOGICAL TESTING

1. What is bias?
Bias can be defined as an
inclination towards or prejudice
for or against a specific person or
group of
people. It is also defined as
general pattern or tendency to
think and behave in a specific
way
In psychological testing, bias
refers to a factor that is inherent
within a test that prevents valid,
accurate, and impartial
measurement. → systematic error
in the measurement process that
differentially influences scores
for identified groups
Bias occurs if score differences
on the indicators of a particular
construct (e.g., percentage of
students
knowing that War-saw is
Poland’s capital) do not
correspond to differences in the
underlying trait or
ability (e.g., geography
knowledge).
WHAT IS BIAS?
Bias can be defined as an inclination towards or prejudice for or against a specific person or
group of people. It is also defined as general pattern or tendency to think and behave in a
specific way.
In psychological testing, bias refers to a factor that is inherent within a test that prevents
valid, accurate, and impartial measurement.
OR
Bias in psychological testing refers to systematic error in the measurement process that
differentially influences scores for identified groups.
Bias occurs if score differences on the indicators of a particular construct (e.g., percentage of
students knowing that Moscow is Russia’s capital) do not correspond to differences in the
underlying trait or ability (e.g., geography knowledge).
IMPACT OF BIAS
Bias may:
 Lead to issues in fair testing
 Influence the validity and reliability of test scores as well as the clinical interpretation
of test scores and assessment observations

Influences the overall outcome of the psychological testing/assessment procedure in a

negative and often discriminatory way, thus infringing upon ethical standards
 Diminish or exaggerate the actual association, create association when there actually
is none, or hide an association that actually exists.
WAYS TO IDENTIFY BIAS
Bias in psychological testing has been examined using several different methods. The
Trinitarian model of validity (content validity, criterion-related validity, and construct
validity) has been used to provide a framework to investigate bias in tests.
Combination of methods including statistical techniques and judgmental review can be used
to identify potential bias.
In order to determine the source of bias on tests, psychometric or technical, quantitative
approaches are used, particularly at the item or task level, to see whether predictions can be
made. Different techniques yield somewhat different findings.
ORIGINS OF THE TEST BIAS CONTROVERSY
Challenges related to test bias have surfaced due to the influence of societal values, the
inherent nature of tests and assessments, and differing perspectives on what constitutes bias.
1. Social Values and Beliefs play a crucial role in the ongoing debate about bias in
standardized tests. Many societies worldwide uphold principles of equality and
opportunity for all, historically emphasizing that these opportunities should be
accessible irrespective of background or ethnicity. These deeply ingrained societal
values shape people's perceptions of fairness and equality in assessments. When a test
is perceived as favouring or discriminating against certain groups, based on these
fundamental beliefs, it often sparks controversy. Public perceptions and concerns
about test bias are frequently influenced by these societal values, leading to criticism,
debate, and broader discussions on fairness and equal opportunity.
2. Character of Tests and Testing: The character of psychological tests, designed to
measure internal traits has been a focal point in the ongoing debate over test bias
(Reynolds & Brown, 1984a). These psychological characteristics are not directly
observable; instead, they are inferred from an individual's external behaviour or
responses. This indirect nature of measurement introduces complexities and
challenges, particularly in ensuring validity and reliability across diverse cultural or
socioeconomic backgrounds. The risk of misinterpreting or misunderstanding
behaviour due to these indirect measures can lead to biased results. Both scientists and
clinicians face similar limitations in assessing and interpreting psychological
 characteristics. If tests fail to account for the complexities of human behaviour,
diversity, and cultural contexts, they may inadvertently introduce bias, impacting
diagnoses, recommendations, and treatment plans.
3. Divergent Ideas of Bias: The controversy surrounding test bias is amplified by the
diverse and sometimes conflicting understandings of what bias entails across different
groups, including researchers, laypeople, clinicians, and professionals. This variability
in understanding can lead to disagreements, debates, and challenges in addressing and
resolving issues related to bias in testing. Such diverse views on bias influence the
design, implementation, and interpretation of tests, potentially impacting their
development, evaluation, and usage. Additionally, these differing perspectives can
erode public trust and confidence in standardized tests and assessments. The presence
of divergent views complicates efforts to effectively identify and mitigate bias in
testing, underscoring the need for a shared understanding and collaborative approach
to developing fair and equitable testing practices that are accepted and trusted by all
stakeholders.
SOURCES OF BIAS IN PSYCHOLOGICAL TESTING

Construct Validity related Bias

 A construct refers to an internal trait that cannot be directly observed but must be
inferred from consistent behaviour observed in people. Self-esteem, intelligence and
motivation are all examples of a construct.
 Construct bias occurs when a test has different meaning for two groups in terms of the
construct it measures.
 For example: The construct validity bias in a psychological test designed to measure
mathematical reasoning becomes evident when the test includes word problems based
on urban scenarios unfamiliar to those from rural backgrounds. This bias favours
individuals acquainted with urban contexts, which is unrelated to mathematical
reasoning. As a result, individuals from rural settings may score lower due to their
unfamiliarity with these urban scenarios, misrepresenting their actual mathematical
reasoning abilities. Such biased results can lead to inaccurate assessments, potentially
impacting educational and job opportunities based on the test scores.
 Construct validity bias concerns the relationship of observed scores to true scores on a
psychological test, if this relationship is systematically different for different groups,
then we might conclude that the test is biased.
 Construct bias can lead to situations in which two groups have the same average "true
score" but different averages on "observed scores" on a test of the construct.
 For example: Creativity test designed to assess both verbal and visual skills

Favours individuals with strong verbal skills over those with visual creativity

GROUP A (VERBAL SKILLS) GROUP B (VISUAL CREATIVITY)

Average "true score" = 80 Average "true score" = 80
Average "observed score" = 85 Average "observed score" = 75

Despite both groups having identical "true scores," the test's bias towards verbal tasks
skews the results, making Group A appear more creative than Group B. Such a biased
representation can lead to misleading conclusions about creativity levels, potentially
impacting educational and employment decisions based on these test scores.

Predictive Validity related Bias

 Predictive bias occurs when a test’s use has different implications for two (or more)
groups.
 It concerns the relationship between scores on two different tests.
 For example: The Australian Tertiary Admission Rank (ATAR) used to predict first
year university performance. Suppose it were discovered that the ATAR was more
 predictive of academic performance for males than for females. This would suggest
that the ATAR suffers from predictive bias.

Content Validity related Bias

 An item or subscale of a test is considered to be biased in content when it is

demonstrated to be relatively more difficult for members of one group than another
when the general ability level of the groups being compared is held constant and no
reasonable theoretical rationale exists to explain group differences on the item (or
subscale) in question.
 Bias in content validity is probably the most common criticism of those who
denounce the use of standardized tests with minorities (Helms, 1992; Hilliard, 1984;
Kwate, 2001).
 For example: In a driving theory test designed to assess knowledge of traffic rules and
regulations, content validity bias is evident.

The test heavily emphasizes urban driving scenarios, comprising 90% of the content,
while only allocating 10% to rural or suburban driving knowledge.

Favours individuals with urban driving experience and familiarity with city traffic
rules, potentially disadvantaging those from rural or suburban backgrounds.

Individuals with diverse driving experiences may be inaccurately assessed, leading to

challenges in obtaining a driver's license or facing unfair evaluations based on the
test's urban-centric content.

Bias Type Definition Key Points

Construct Validity related Occurs when a test has  Concerns the meaning or
Bias different meaning for two interpretation of a test
groups in terms of the for different groups.
construct it measures.  Focuses on the
consistency of construct
interpretation across
diverse groups.
Predictive Validity related Bias occurs when a test’s  Relates to the outcomes
Bias use has different or consequences of using
implications for two (or a test for different
more) groups. groups.
 Examines the
relationship between test
scores and external
criteria or outcomes for
diverse groups.
 Content Validity related An item or subscale of a test  Focuses on the fairness
Bias is considered to be biased in and representativeness of
content when it is test content across
demonstrated to be diverse groups.
relatively more difficult for  Highlights the
members of one group than importance of consistent
another. item difficulty and
relevance across
different populations.
 Often criticized for
perpetuating inequalities
and disadvantaging
specific groups,
particularly minorities,
due to unfair test
content.

Item bias

 Item bias is due to “poor item translation, ambiguities in the original item, low
familiarity/appropriateness of the item content in certain cultures, or influence of
culture specifics such as nuisance factors or connotations associated with the item
wording”.
 For example: An international English language proficiency test designed for non-native
English speakers worldwide  includes a vocabulary question that uses the word "trunk" in a
sentence without providing context

 The word "trunk" may be translated into another language in multiple ways,
leading to confusion or misunderstanding among test-takers from different
linguistic backgrounds
 Without context, the word "trunk" could refer to an elephant's trunk, the trunk of a
car, or a storage trunk, causing ambiguity and making it difficult for test-takers to
select the correct meaning.
 In certain cultures or regions where elephants are not native or where cars are less
common, the term "trunk" may be less familiar or relevant, affecting test-takers'
ability to understand and answer the question.
 Cultural nuances or connotations associated with the term "trunk" (e.g., elephants
are considered sacred in some cultures) may influence how test-takers interpret
and respond to the item.

Due to these various biases the question may disproportionately disadvantage certain
groups of test-takers, leading to inaccurate assessments of their English language
proficiency
 This item bias can result in unfair and biased test results, where some test-takers may
score lower not because of their actual language proficiency but due to the challenges
and biases associated with the specific vocabulary question.

Method Bias

 Method bias is a term that refers to the problems resulting from the way that an
assessment is administered.
 Factors surrounding the administration of the test that may impact the results.
 Administration Bias refers to the effects caused by varying or unexpectedly adverse
test administration conditions. These conditions could be social, physical or technical.
 For example: vague test administration instructions might prompt one exam
proctor to interpret the instructions one way for his group of test takers, while
another exam proctor might interpret the instructions in a completely different
way for her group of test takers. This could have devastating effects on different
groups of scores.

Sampling Bias

 Sampling bias occurs when the sample participating in the study is not representative
of the general population. This may be the result of purposeful selection of
participants by the researcher, but there are many other factors that can create
sampling bias.
 One example is surveys taken during a presidential election. The results of the surveys
often depend on the city, state, or area being surveyed. For example, people in cities
tend to vote one way, while people in rural environments often vote another.
 Similarly, one's geographic location (the Northeast, South, West, etc.) can have an
impact on who is being surveyed. If there is a high saturation of a given political party
in an area surveyed, then the results will be skewed in the direction of the political
party, and not be representative of the general population.

Types of sampling bias

Gender Bias Age Bias

When the researcher When the researcher
generalises findings generalises findings based Cultural Test Bias
based on one gender on one particular age to (Contd. below)
to another without another without empirical
empirical evidence evidence.

 Cultural Test Bias: Cultural bias in testing occurs if an assessment unfairly measures
skills and knowledge without considering the understanding of cultural traditions.
 - When assessments do not consider cultural differences, they fail to accurately
measure abilities and can lead to decisions based on inaccurate data.
- Cultural bias in testing can occur when the evaluator or the testing materials do
not take into account the lack of knowledge of semantics and experiences within a
particular cultural group.
- For example: if a student moves from South India to Ladakh and is given a test
asking them to describe a snowy winter, the test would be biased because the
student lacks any experience with snowy winters.
- Additionally, when measuring proficiency in language, students can be mistakenly
labelled as disabled because the test results indicate a language impairment.
However, the discrepancy in data may be due to cultural differences.
- The main characteristic of cultural test bias is the tests are made up by a
homogenous group of people who do not represent the cultural diversity of people
who will take the test.
- The test itself could be culturally biased because of the content of test items, the
formatting of the test, or the environment in which the assessment is being given.
- One effect of cultural bias in testing is maintaining racial stereotypes by unfairly
representing data as an indication of intelligence or ability.
- For example: A language proficiency test designed with a focus on American
cultural references and idiomatic expressions includes a section where test-takers
are asked to interpret and complete sentences using colloquial phrases commonly
used in American English, such as "break a leg."

May inadvertently favour individuals with exposure to American culture and

language nuances, potentially disadvantaging those from non-American
backgrounds who may find these expressions unfamiliar or challenging to
understand

Test-takers from diverse cultural backgrounds may struggle with this section not
due to their actual language proficiency but because of the cultural specificity and
assumptions embedded in the test content

This cultural bias can perpetuate stereotypes by equating familiarity with

American cultural expressions to language proficiency, impacting individuals'
opportunities based on unfair and culturally biased test content.

Selection Bias

 Selection bias happens when the comparisons in data from the sample population
have no meaning or value because the participants in the sample were not equally and
fairly selected for both the experimental and control groups.
 Both the experimental and control groups should be representative of the general
population, as well as representative of each other. One group should not show
substantially higher characteristics of a given variable than the other, as this can
distort the findings.
  For example: In a study examining the effectiveness of a new educational program for
improving math skills in elementary school students, researchers select participating
schools based on their willingness to implement the program and exclude reluctant or
unable schools

Introduces selection bias by potentially favouring schools more likely to support the
program, leading to differences in student demographics, resources, or other factors
that could influence the program's effectiveness

As a result, the study's findings may not be generalizable or accurately reflect the
program's true impact across diverse settings, compromising the validity of the results
and potentially influencing educational policies based on biased and unrepresentative
data.

Response Bias

 A response bias can be defined as the “systematic tendency to respond to a range of

questionnaire items on some basis other than the specific item content” (Paulhus,
1991).
 This broad definition of response biases includes different types of “non-test-relevant
response determinants” (Crowne & Marlowe, 1960):
- Response styles that reflect a differential use of the response options independent
of the items’ content (e.g., a tendency to agree with statements)
- Response biases that reflect a tendency to distort responses in order to align them
with contextual demands or one’s self-concept such as the tendency to give
socially desirable responses.
 Types of response biases:

Response Bias Characteristics

Acquiescence response Preference for categories

style stating agreement

Disacquiescence Preference for categories

response style stating disagreement
SELF-REPORT Careless responding Inattentive responding

Extreme response style Preference for extreme

categories

Midpoint response style Preference for the

midpoint of a rating scale
 Socially desirable Tendency to describe
responding oneself positively and in
accordance with social
norms and rules

Halo Tendency to exaggerate

coherence in judgements
OTHER-REPORT of multiple
characteristics

Leniency/severity Tendency to be
lenient/harsh in
assessments of all objects

Self-selection Bias

 Self-selection bias refers to the bias that can occur when individuals are allowed to
choose whether they want to participate in a research study.
 Because participants often differ from non-participants in ways significant to the
research, self-selection can lead to a biased sample and affects the generalizability of
results.
 For example: Survey of high school English students, wherein they are asked to rate
the books they read throughout the academic year, but participation is optional.
Because of that, students who either strongly enjoyed or hated the books are more
likely to fill in the survey. Students who didn’t feel strongly about the books are less
likely to participate in the survey. As a result, the sample will comprise mostly those
with strong opinions and will not be representative of all students.

In conclusion, biases in psychological testing encompass various factors that can influence
the validity, reliability, and fairness of test results. These biases can arise from different
sources and each type of bias has unique characteristics and implications that can impact the
interpretation and application of test scores, potentially leading to inaccurate assessments and
unfair outcomes. Recognizing and addressing these biases is essential for ensuring ethical
standards, promoting fairness, and enhancing the effectiveness of psychological testing and
assessment procedures.

What
- Note:
- Equivalence of


measures (or lack of bias) is

 a prerequisite for valid
comparisons across
- cultural populations.
-  Statements about bias
always refer to applications
of an instrument/measure
-  that psychological
assessment instruments,
despite tremendous value
for
- increasing an
understanding of human
nature, have the potential
to be "used in
- harmful, inaccurate, and
inappropriate ways due to
the impact of bias in the
- administration and
interpretation of
assessments and tests
- other factors that give rise to
bias
- - test is not
demographically or
culturally representative
- - Norm-referenced
tests (or tests designed to
compare and rank test takers
in relation to one
- another) may be biased if
the “norming process” does
not include representative
samples of
- all the tested subgroups
- - test formats may
have an inherent bias
toward some groups of
students (e.g. med, multiple-
- choice tests may favour
certain styles of thinking
more characteristic of males
than females,
- such as a willingness to risk
guessing the right answer or
questions that reflect black-
and-
- white logic rather than
nuanced logic.)
- - Choice of
language in test questions
can introduce b