Scribd
Upload
23 views

Lecture 8 Measurement Reliability and Validity

Uploaded by

rreezzzaa
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
23 views

Lecture 8 Measurement Reliability and Validity

Uploaded by

rreezzzaa
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 21

Lecture 8

Reliability, and Validity


RELIABILITY AND VALIDITY

• Reliability—tool is consistent
• Validity—tool measures “what-it-should”
• Good assessment tools ®
– Rejection of Null hypotheses
OR
– Acceptance of Research hypotheses
A CONCEPTUAL DEFINITION OF
RELIABILITY

Method Error
Observed Score = True Score + Error Score
Trait Error
A CONCEPTUAL DEFINITION OF
RELIABILITY
Method Error
Observed Score = True Score + Error Score
Trait Error

• Observed score
– Score actually observed
– Consists of two components
• True Score
• Error Score
A CONCEPTUAL DEFINITION OF
RELIABILITY
Method Error
Observed Score = True Score + Error Score
Trait Error

• True score
– Perfect reflection of true value for individual
– Theoretical score
A CONCEPTUAL DEFINITION OF
RELIABILITY
Method Error
Observed Score = True Score + Error Score
Trait Error

• Error score
– Difference between observed and true score
A CONCEPTUAL DEFINITION OF
RELIABILITY

Method Error
Observed Score = True Score + Error Score
Trait Error

• Method error is due to characteristics of the test or


testing situation
• Trait error is due to individual characteristics
• Conceptually, reliability = True Score
True Score
+ Error Score
• Reliability of the observed score becomes higher if error
is reduced!!
INCREASING RELIABILITY
® Decreasing Error

• Increase sample size


• Modify/eliminate unclear questions/problematic
items
• Standardize testing conditions
• Use both easy and difficult questions
• Minimize the effects of external events
• Standardize instructions
• Maintain consistent scoring procedures
HOW RELIABILITY IS MEASURED

• Reliability is measured using a


– Correlation coefficient
– r test1•test2
• Reliability coefficients
– Indicate how scores on one test change
relative to scores on a second test
– Can range from -1.0 to +1.0
• +1.00 = perfect reliability
• 0.00 = no reliability
TYPES OF RELIABILITY
Type of What It Is How You Do It What the
Reliability Reliability
Coefficient Looks
Like

Test-Retest A measure of Administer the same rtest1•test1


stability test/measure at two different
times to the same group of
participants

Parallel A measure of Administer two different forms rform1•form2


Forms equivalence of the same test to the same
group of participants

Inter-Rater A measure of Have two raters rate behaviors Percentage of


agreement and then determine the agreements
amount of agreement between
them

Internal A measure of how Correlate performance on each Cronbach’s alpha


Consistency consistently each item with overall performance Kuder-Richardson
item measures the across participants
same underlying
construct
VALIDITY

• A valid test does what it was designed to


do
• A valid test measures what it was
designed to measure
A CONCEPTUAL DEFINITION OF
VALIDITY

• Validity refers to the test’s results, not to


the test itself
• Validity ranges from low to high, it is not
“either/or”
• Validity must be interpreted within the
testing context
TYPES OF VALIDITY
Type of Validity What Is It? How Do You Establish It?
Content A measure of how well Ask an expert if the items assess
the items represent the what you want them to
entire universe of items

Criterion How well a measure is


related to an outcome
Concurrent A measure of how well Select a criterion and correlate
a test estimates a scores on the test with scores on
criterion the criterion in the present

Predictive A measure of how well Select a criterion and correlate


a test predicts a scores on the test with scores on
criterion the criterion in the future

Construct A measure of how well Assess the underlying construct on


(Convergent vs a test assesses some which the test is based and
Discriminant) underlying construct correlate these scores with the test
scores
HOW TO ESTABLISH CONSTRUCT
VALIDITY OF A NEW TEST

• Correlate new test with an established test


• Show that people with and without certain
traits score differently
• Determine whether tasks required on test are
consistent with theory guiding test
development
How to Improve Validity

• increase sample size


• Randomly select participants
• Improve measurement technique
• Randomly assign to groups (experimental)
• Use a control groups (experimental)
• Remove or modify problematic items
Validity and Reliability in Qualitative
Research
• validity referring to the integrity and
application of the methods undertaken
and the precision in which the findings
accurately reflect the data.
• Reliability describes consistency within the
employed analytical procedures. The
research tend to produce same results if
repeated.
Validity in Qualitative Research
• Credibility
– internal validity, believability and
trustworthiness of the findings
– depends more on the richness of the data
than on the quantity of data.
– participants feel the findings are credible and
accurate
– triangulation (combining theories, methods or
observers in a research study, can help
ensure that fundamental biases arising from
the use of a single method or a single
observer are overcome)
Validity in Qualitative Research

• Transferability
– External validity
– the results are generalizable and can be
applied to other similar setting or populations
• Confirmability
– how well the research findings are supported
by the actual data collected when examined
by other researcher.
Reliability in Qualitative Research

• Dependability
– the research could be repeated and result in
similar findings.
– documents all aspects of any changes or
unexpected occurences to further explain the
findings.
THE RELATIONSHIP BETWEEN
RELIABILITY AND VALIDITY

• A valid test must be reliable


But
• A reliable test need not be valid
THE RELATIONSHIP BETWEEN
RELIABILITY AND VALIDITY

You might also like