12-Reliability and Validity
12-Reliability and Validity
RELIABLE
Trial 1: Trial 2: Trial 3:
AND
55.0 kg 55.2 kg 55.1 kg
VALID
NOT RELIABLE
Trial 1: Trial 2: Trial 3:
AND
55.0 kg 268.1 kg 25.5 kg
NOT VALID
RELIABILITY ANALYSIS
RELIABILITY ANALYSIS
1. Test-retest reliability
Measure the scores twice with the same instrument
Comparing the scores from repeated testing of the
same participants with the same test
8 8 7
RELIABILITY ANALYSIS
3. Inter-rater reliability
evaluate agreement of how consistently different
raters can assign the same score or category to the
subjects
e.g. concurrent scoring during live observations. Or
scoring video footage
8 7 8 8
RELIABILITY ANALYSIS
4. Internal consistency reliability
assess the consistency of results across items within
a test
the extent to which a test/procedures assess the
same characteristic, skill or quality
Response
PH Cronbach's Alpha
.869
Based on
Standardized Items
.869
N of Items
10
Reliability Statistics
Cronbach's Alpha
Based on
PH Cronbach's Alpha
.869
Standardized Items
.869
N of Items
10
INTERNAL CONSISTENCY
Reliability Statistics
Cronbach's Alpha
MH Cronbach's Alpha
.658
Based on
Standardized Items
.679
N of Items
5
Squared Cronbach's
Scale Mean if Scale Variance Corrected Item- Multiple Alpha if Item
Item Deleted if Item Deleted Total Correlation Correlation Deleted
Mental Health 1 17.50 12.014 .408 .329 .610
Mental Health 2 16.97 10.430 .541 .384 .544
Mental Health 3 17.38 10.146 .474 .266 .575
Mental Health 4 17.26 11.535 .519 .387 .568
Mental Health 5 17.57 12.155 .197 .116 .719
Squared Cronbach's
Scale Mean if Scale Variance Corrected Item- Multiple Alpha if Item
Item Deleted if Item Deleted Total Correlation Correlation Deleted
Mental Health 1 17.50 12.014 .408 .329 .610
Mental Health 2 16.97 10.430 .541 .384 .544
Mental Health 3 17.38 10.146 .474 .266 .575
Mental Health 4 17.26 11.535 .519 .387 .568
Mental Health 5 17.57 12.155 .197 .116 .719
If MH5 is removed, the new alpha value will increase from 0.66 to
0.72
We may want to consider removing MH5, but we may not be able
to compare our results with other studies using the original scale
(with 5 items)
INTRACLASS CORRELATION
A reliability index in test-retest, intra-rater, and inter-
rater reliability analyses
The measured variable must be a continuous variable
Interpretation of ICC values
<0.5 : poor reliability
0.5 - 0.75 : moderate reliability
0.75 - 0.9 : good reliability
> 0.9 : excellent reliability
For categorical variable, Cohen's kappa (κ) is used
instead
INTRACLASS CORRELATION
(TEST-RETEST)
Dataset : https://goo.gl/RGC62w
df 105
Sig. .000
dimension0
8 .606 4.041 84.004
11
12
.396
.371
2.643
2.472
93.105
95.576
These four
13 .308 2.051 97.627 components explain
14 .198 1.322 98.950
a total of 65.89% of
the variance
15 .158 1.050 100.000