Scale Construction

Multiple methods, multiple problems

Psychometric Theory: A conceptual Syllabus
X1
Y1
X2 L1
Y2
X3 L4
Y3
X4
Y4
X5 L2
Y5
X6
Y6
X7 L5
Y7
X8 L3 Y8
X9
 Types of Validity: What are we measuring
X1 Face
Y1
X2 Concurrent
Y2
X3 Predictive
Y3
X4 Construct
Y4
X5 L2
Y5
X6
Convergent Y6
X7 Discriminant L5
Y7
X8 L3 Y8
X9
 Techniques of Data Reduction:
Factors and Components
X1
Y1
X2 L1
Y2
X3 L4
Y3
X4
Y4
X5 L2
Y5
X6
Y6
X7 L5
Y7
X8 L3 Y8
X9
 Methods of Scale Construction
• Empirical
– MMPI, Strong
• Rational
– CPI
• Theoretical
– NAch
• Homogeneous
EPI, 16PF, NEO
 Empirical Keying
• Ask items that discriminate known groups
– People in general versus specific group
– Choose items that are maximally independent and that
have highest validities
• Example:
– MMPI
– Strong-Campbell
– sex and ethnic differences in personality and music
• Problem:
– What is the meaning of the scale?
– Need to develop new scale for every new group
 Sex differences at item level
Item effect size
Get overwhelmed by emotions. 0.59
Sympathize with others' feelings. 0.45
Worry about things. 0.43
Feel others' emotions. 0.39
Get stressed out easily. 0.51
Have a soft heart. 0.38
Panic easily 0.50
Inquire about others' well-being. 0.41
Get upset by unpleasant thoughts that come into my mind. 0.38
Get upset easily. 0.37
Am indifferent to the feelings of others. -0.33
Am not interested in other people's problems. -0.33
Feel little concern for others. -0.35
Am not easily bothered by things -0.35
Love to help others. 0.34
Am not really interested in others. -0.32
Think of others first. 0.30
Take offense easily. 0.29
Take time out for others. 0.33 7
Gender differences in music preferences
effect size Item

0.9 Broadway Musicals (e.g. Rent, Cats, Phantom of the Opera)

0.68 Top 40/Pop Vocal Music (e.g. Kelly Clarkson, Madonna, The Black Eyed Peas)
0.65 Broadway, Movie and TV Soundtrack Music in General
0.59 Contemporary Rhythm and Blues (e. g. Whitney Houston, Usher, Alicia Keys)
0.59 Modern Country Music (e.g. Garth Brooks, Dixie Chicks, Tim McGraw)
0.37 Country Music in General
0.37 Movie Soundtracks (e.g. Starwars, Good Will Hunting, Garden State)
0.36 Top 40 Music/Pop in General
0.32 Pop Rock (e.g. Maroon 5, Counting Crows, John Mayer)
0.31 Modern Religious Music (e.g. 4Him, Casting Crowns)
0.3 Soul Rock (e.g. Stevie Wonder, Earth Wind and Fire)
-0.3 Acid Rock (e.g. Pink Floyd, The Doors, Jefferson Airplane)

-0.4 Heavy Metal (e.g. Metallica, Marilyn Manson, System of a Down) 8

Ethnic differences in music preferences
effect Item
1.26 size Acid Rock (e.g. Pink Floyd, The Doors, Jefferson Airplane)
1 Alternative (e.g. Pearl Jam, Incubus, Radiohead)
0.97 Electronic Music in General
0.91 Rock Music In General
0.87 Jam Bands (e.g. The Grateful Dead, Phish, String Cheese Incident)
0.87 Classic Rock (e.g. The Beatles, The Rolling Stones, Led Zeppelin)
0.85 Country Rock (e.g. The Allman Brothers, Lynyrd Skynyrd)
0.61 Electronic Dance Music (e.g. DJ Tiesto, Paul Van Dyk, Keoki)
0.59 Folk Music in General (e.g. Bob Dylan, Iron and Wine, Simon and Garfunkel)
0.57 Pop Rock (e.g. Maroon 5, Counting Crows, John Mayer)
0.56 Country Music in General
0.51 Bluegrass (e.g. Alison Krauss, Lester Flatt, Nickel Creek)
-0.56 Contemporary Rhythm and Blues (e. g. Whitney Houston, Usher, Alicia Keys)
-0.6 Blues in General (e.g. Ray Charles, Stevie Ray Vaughn, B.B. King)
-0.63 Instrumental Hip-Hop (e.g. DJ Hi-Tek, RJD2, Prefuse 73)
-0.64 Gospel Soul (e.g. Aretha Franklin, Solomon Burke)
-0.67 Soul in General (e.g. Otis Redding, Marvin Gaye)
-0.84 Religious Music in General
-1.04 Soul Rock (e.g. Stevie Wonder, Earth Wind and Fire)
-1.11 Rhythm and Blues in General
9
-1.43 Religious Gospel (e.g. Andre Crouch, Gospel Quartet)
 Rational Keying
• Ask items with direct content relevance
• Example: California Psychological Inventory
• Problems
– Not all items predict in obvious way
– Need evidence for validity
– Easy to fake
 Theoretical Keying
• Ask items with theoretical relevance
• Example: Jackson Personality Research
Form
• Problems:
– Theoretical circularity
– Need evidence for validity
 Homogeneous Keying
• Select items to represent single domain
– Exclude items based upon internal consistency
• Examples:
– 16PF, EPI/EPQ, NEO
• Problems
– Garbage In, Garbarge Out
– Need evidence for validity
Methods of Homogeneous Keying
• Factor Analysis
• Principal Components Analysis
• Cluster Analysis
Scale Construction

Pragmatics
 Comparison of techniques
• Empirical
• Rational
• Theoretical
• Homogeneous
• Does it make a difference?
– Hase and Goldberg: No
– Goldberg, Yes.
 Average Cross validated validity
varies as difficulty of criterion
Cross validated r

Homogeneous/rational

Empirical

Difficulty of predicting criteria ->

 Advantages and disadvantages
• Empirical
– Harder to fake
– Harder to interpret
– Requires new scale validation for every criterion
• Rational/Homogeneous
– More transparent
– Homogeneity of measure suggests single construct
3 stages of scale construction: I: Design
1. Review theory of attribute to be measured
1. Convergent measures
2. Discriminant measures
2. Write items based upon theory
1. items drawn from different facets of theory
2. items balanced for response styles
3. Screen items for readability, bias,
understandability
4. Include "hyperplane stuff”
1. possible related constructs
2. theoretically important alternatives
5. Define target population
1. Who is to be measured
2. Consider issues of homogeneity/heterogeneity
3 stages of scale construction: II: Data
1. Administer items and record responses
1. (1) Monitor for serious, engaged test taking
2. (2) Double check for data entry errors
2. Examine the distribution and search for
outliers
1. data entry errors
2. uncooperative subjects
3. Form proximity (correlation) matrix
4. Extract optimal number of factors or clusters
1. statistically (chi square and maximum likelihood)
2. psychometrically (maximize alpha, beta, VSS)
3. for interpretation (to maximize understanding)
 3 stages of scale construction: III:
Application
1. Form scales based upon these factors/ clusters
1. score salient items
2. drop non salients
2. Purify scales -- item analysis
1. high correlation with scale
2. low correlations with other scales
3. low correlations with measures of response styles
4. moderate levels of endorsement
3. Validate against other measures of same and
different constructs
1. Assess reliability (internal consistency &stability)
2. Demonstrate convergent, discriminant and
incremental validity
 Scale Construction: An example
• 4 sets of items were constructed to
represent 4 psychological domains
– Sociability, Impulsivity, Need Achievement,
Anxiety
• Surveys were given to friends of
experimenters who also peer rated their
friends
Scale Construction: Example (2)
• Items were entered into a spreadsheet and
checked for incorrect entries
– Missing values were replaced with a missing value
code (NA)
• Basic item statistics were examined
• Scales were constructed based upon original
scoring keys -- item whole correlations allowed
for some trimming of items
• Alphas were calculated for each scale
Scales were also constructed using a hierarchical
cluster algorithm for items (ICLUST)
• 1) Find similarity (correlation) matrix
• 2) Combine most similar pair of items to
form a new variable (cluster)
• 3) Find similarity of this cluster to all other
items/clusters
• 4) Repeat steps 2 and 3 until some criterion
(e.g., alpha or beta) fails to increase

23
 Item Analysis
What items load on scales?
Scale 1: Alpha = .90
0.81 0.31 0.11 -0.24 I would call myself a sociable person
0.81 0.36 -0.01 -0.22 At a part, I like to mingle and meet as many new people as I can
0.77 0.4 -0.1 -0.29 Other people consider me a social butterfly
0.76 0.23 0.12 -0.23 I am a people person
0.74 0.37 -0.06 -0.35 In a group of people, I am likely to initiate conversations
0.72 0.34 -0.05 -0.13 I am a terrific conversationalist
0.7 0.35 -0.08 -0.3 I enjoy talking to strangers
0.7 0.36 -0.08 -0.33 I can easily let myself go and enjoy a lively party
0.69 0.3 0.09 -0.22 I think of of myself as very lively.
0.64 0.24 -0.03 -0.26 I can always think of something to say
-0.64 -0.28 0.23 0.44 I feel uncomfortable in large groups
-0.58 -0.22 0.18 0.54 I generally become anxious when I meet new people
0.5 0.25 0.11 -0.04 I would rather attend a party than study
 Scale 2: alpha = .64

0.3 0.66 -0.05 0.09 I often act without thinking.

0.24 0.62 -0.19 -0.14 I often say things before thinking about how they'll make others feel.
-0.42 -0.54 0.11 0.2 I spend a lot of time thinking about what I want to say before I say it.
0.1 0.53 0.06 0.22 I have trouble concentrating on things for a long period of time.
0.11 0.52 -0.04 -0.19 I don't like to stick to a strict schedule.
0.23 0.5 -0.01 -0.13 When I want something, I'll stop what I'm doing to get it.
0.14 0.5 -0.23 0.14 I spend my paycheck right after I receive it.
0.09 0.5 0.12 0.26 I am easily distracted.
-0.13 -0.48 -0.15 -0.2 I am not easily distracted from tasks.
0.38 0.47 0 0.07 I often interrupt others when I have something I want to say.
-0.02 -0.47 0.3 0.16 I prefer to have a regular schedule.
0.37 0.44 0.18 -0.3 I enjoy the unexpected.
 Scale 3: Alpha = .79
0.02 -0.09 0.8 0.17 It is important for me to do well.
0.05 -0.1 0.71 0.13 I set high standards for myself.
0.02 0.1 -0.71 -0.3 It does not bother me when others think that I'm a failure
-0.04 -0.06 0.68 0.23 I judge myself by the way I perform.
0.07 0.08 0.67 0.15 Success after a lot of hard work is rewarding.
0.01 -0.01 0.66 0.42 I am upset when I do poorly.
-0.17 -0.27 0.63 0.46 When attempting a task, I often think about the consequences of failure.
0.22 0.04 0.62 0.33 Being recognized for doing something well is important to me.
-0.1 -0.06 0.61 0.27 I would rather pull an “all-nighter” than feel unprepared for an exam.
-0.04 -0.18 0.58 0.43 I prefer tasks I know I will succeed at.
-0.22 -0.31 0.57 0.22 I often find myself planning for the future.
0.02 -0.27 0.57 -0.03 I do everything to the absolute best of my ability.
0.03 -0.32 0.53 -0.07 I feel that I must complete a task once I begin.
-0.27 -0.44 0.52 0.17 I think about the consequences of my actions.
-0.03 -0.3 0.49 -0.02 I often do more than is required on a task.
0.18 0.05 0.49 -0.06 I tend to do better in tasks that will be evaluated.
-0.12 0 0.47 0.51 I find it hard to recover when someone criticizes me
-0.17 -0.2 0.44 0.61 I worry about things that have already happened
0.06 0.28 -0.43 -0.07 I rarely do work beyond the minimum.
 Scale 4: alpha = .67

0.27 0.09 -0.29 -0.7 I am nearly always relaxed

-0.24 -0.19 0.34 0.69 I often worry about things that others find trivial
-0.2 -0.16 0.37 0.64 I often feel stressed
-0.32 -0.11 0.4 0.6 A number of upcoming events currently have me feeling stressed
-0.11 -0.36 0.25 0.55 Change stresses me
-0.11 -0.04 0.03 0.53 I often can't go to sleep at night because I've got a lot on my mind
-0.06 0.04 0.14 0.52 Sometimes I feel like things are out of my control
-0.19 0.07 0.42 0.51 When doing a task, I often think about the consequences of failing
0.28 0.2 -0.32 -0.48 I don't worry about things I can't control
-0.13 -0.05 0.26 0.44 I have a pessimistic attitude regarding my abilities
-0.32 -0.18 0.06 0.44 I assume the worst going into a situation
-0.17 -0.35 0.4 0.42 It takes me a while to make a decision.
-0.31 0 -0.31 0.39 I am nervous right now
 Structure of Class Scales
(alphas on diagonal)

Soc Imp Nach Anx

Soc 0.90 0.46 0.00 -0.33
Imp 0.46 0.64 -0.16 -0.15
Nach 0.00 -0.16 0.79 0.33
Anx -0.33 -0.15 0.33 0.67
Structure of Self Report Scales
Class scales vs. Big 5 scales
(alphas on diagonal)
Soc Imp Nach Anx Extra Con Open Stab Agree
Soc 0.90 0.46 0.00 -0.33 0.72 -0.05 0.20 0.23 0.54
Imp 0.46 0.64 -0.16 -0.15 0.24 -0.43 0.00 -0.04 0.22
Nach 0.00 -0.16 0.79 0.33 0.21 0.58 0.48 -0.26 0.26
Anx -0.33 -0.15 0.33 0.67 -0.28 0.15 -0.05 -0.64 -0.04
Extra 0.72 0.24 0.21 -0.28 0.79 0.15 0.47 0.26 0.64
Con -0.05 -0.43 0.58 0.15 0.15 0.81 0.45 -0.01 0.25
Open 0.20 0.00 0.48 -0.05 0.47 0.45 0.70 0.04 0.44
Stab 0.23 -0.04 -0.26 -0.64 0.26 -0.01 0.04 0.82 0.15
Agree 0.54 0.22 0.26 -0.04 0.64 0.25 0.44 0.15 0.60
Scatter Plot Matrix of Peer Ratings
Scatter Plot Matrix of Self Report
 How do we validate scales?
Multi-Method-Multi Trait Matrix
• Structure of scales and structure of peer
ratings do not imply validity for either
• We need to compare
– Mono Trait - Mono Method (reliability)
– Mono Trait - Hetero Method (convergent)
– Hetero Trait - Mono Method (discriminant)
– Hetero Trait Hetero Method (discriminant)
 MultiTrait-Multi Method
Self report with class items Self report Big 5 items Peer ratings
Soc Imp Nach Anx Extra Con Open Stab Agree S I N A
Soc 0.90 0.46 0.00 -0.33 0.72 -0.05 0.20 0.23 0.54 0.59 0.28 -0.21 -0.47
Imp 0.46 0.64 -0.16 -0.15 0.24 -0.43 0.00 -0.04 0.22 0.25 0.40 -0.32 -0.25
Nach 0.00 -0.16 0.79 0.33 0.21 0.58 0.48 -0.26 0.26 -0.02 -0.19 0.39 0.17
Anx -0.33 -0.15 0.33 0.67 -0.28 0.15 -0.05 -0.64 -0.04 -0.12 -0.02 0.11 0.30
Extra 0.72 0.24 0.21 -0.28 0.79 0.15 0.47 0.26 0.64 0.44 0.03 0.06 -0.41
Con -0.05 -0.43 0.58 0.15 0.15 0.81 0.45 -0.01 0.25 0.05 -0.17 0.50 0.22
Open 0.20 0.00 0.48 -0.05 0.47 0.45 0.70 0.04 0.44 0.17 -0.10 0.26 0.06
Stab 0.23 -0.04 -0.26 -0.64 0.26 -0.01 0.04 0.82 0.15 0.10 -0.09 -0.13 -0.25
Agree 0.54 0.22 0.26 -0.04 0.64 0.25 0.44 0.15 0.60 0.38 0.04 0.05 -0.19
S 0.59 0.25 -0.02 -0.12 0.44 0.05 0.17 0.10 0.38 1.00 0.30 0.04 -0.31
I 0.28 0.40 -0.19 -0.02 0.03 -0.17 -0.10 -0.09 0.04 0.30 1.00 -0.25 -0.18
N -0.21 -0.32 0.39 0.11 0.06 0.50 0.26 -0.13 0.05 0.04 -0.25 1.00 0.37
A -0.47 -0.25 0.17 0.30 -0.41 0.22 0.06 -0.25 -0.19 -0.31 -0.18 0.37 1.00
A multi-Trait, Multi-Method Matrix (alphas on the diagonal)
 PRQ-07- Anxiety: alpha .86
q42 47 Even trivial proble 1 0.65 0.21 -0.11 -0.03 -0.28
Anxiety 2 Anxiety 1 0.62 0.08 -0.07 0.06 -0.23
q6 11 I dont handle stress 1 0.60 0.33 -0.19 0.04 -0.33
q50 55 Even in non stressf 1 0.58 0.40 -0.05 0.02 -0.16
q2 7 I get nervous very e 1 0.55 0.22 -0.23 0.06 -0.38
q18 23 I rarely feel tense 1 -0.54 0.01 -0.13 -0.08 0.19
q34 39 I have a hard time f 1 0.51 0.26 0.21 -0.08 -0.19
q26 31 I often feel anxious 1 0.50 0.24 0.19 0.18 -0.16
q10 15 I am easily bothered 1 0.48 0.18 -0.04 0.07 -0.13
q22 27 I feel stressed when 1 0.47 0.26 0.18 -0.17 -0.20
q30 35 I often feel tense, 1 0.47 0.07 -0.07 0.23 -0.18
q62 67 A small unpleasant 1 0.46 0.28 0.16 -0.02 -0.16
q66 71 I worry about what 1 0.44 0.25 -0.04 0.12 -0.04
q54 59 I feel tension in m 1 0.42 -0.27 0.08 0.12 -0.12
q70 75 I bounce back quick 1 -0.41 -0.26 0.3734 0.15 0.39
 Achievement: alpha .87
q81 86 I believe that if so 3 0.08 -0.06 0.70 -0.03 0.26
q33 38 I find myself needi 3 0.06 0.17 0.65 -0.01 0.25
q17 22 I have high standar 3 0.11 0.16 0.64 -0.23 0.13
q41 46 I always make sure 3 0.02 -0.06 0.58 -0.15 0.19
q4 9 I am thoughtful and 3 -0.09 0.00 0.57 -0.44 0.06
q25 30 If I fail, I keep t 3 -0.09 0.23 0.57 -0.08 0.30
q1 6 I love to seek out 3 -0.04 -0.08 0.56 -0.05 0.39
q77 82 I always see projec 3 0.16 0.09 0.55 -0.19 0.13
q13 18 I like to go the ex 3 0.09 0.01 0.54 -0.26 0.20
q49 54 The joy of success 3 0.03 0.05 0.54 -0.01 0.25
q61 66 I experience great 3 -0.01 0.00 0.54 -0.16 0.12
q60 65 I stay on task unti 3 0.12 0.07 0.53 -0.28 0.13
q45 50 I prefer challengin 3 -0.10 0.08 0.50 -0.06 0.15
q73 78 I set long term and 3 0.15 -0.07 0.46 -0.09 -0.01
q78 83 I tend to back away 3 0.20 0.16 -0.45350.27 -0.05
q57 62 I always reach the 3 -0.10 0.14 0.44 -0.18 0.27
 Impulsivity: alpha = .87
q24 29 I often change my p 4 0.09 -0.34 0.08 0.62 0.33
q52 57 I often get sidetra 4 0.21 -0.32 -0.16 0.61 0.18
q8 13 I say things that I 4 0.10 -0.14 -0.12 0.59 0.21
q28 33 I dislike planning 4 0.13 -0.14 -0.18 0.56 0.08
q40 45 I act on sudden urg 4 0.02 -0.30 0.07 0.55 0.24
q44 49 I often regret deci 4 0.28 -0.10 -0.14 0.55 0.26
q84 89 I am an impulsive pe 4 -0.07 -0.18 0.07 0.55 0.36
q69 74 I tend to procrasti 4 -0.03 0.03 -0.32 0.53 0.18
Impulsivity 4 Impulsivity 4 0.08 0.04 -0.24 0.51 0.24
q32 37 I indulge in my des 4 0.13 0.05 0.16 0.50 0.25
q76 81 I sometimes look ba 4 0.11 -0.11 0.07 0.46 0.31
q20 25 I plan my activitie 4 0.17 0.24 0.27 -0.44 -0.14
q68 73 I always think befo 4 -0.03 0.17 0.25 -0.44 -0.23
q55 60 Ill spend time talk 4 0.21 0.02 -0.09 0.43 0.26
q80 85 I often say the fir 4 -0.12 0.01 -0.13 0.42 0.40
 Sociability alpha=.92
q35 40 I have a large soci 5 -0.27 -0.07 0.19 0.31 0.79
q83 88 I am a very sociable 5 -0.25 -0.04 0.38 0.17 0.79
q11 16 I tend to avoid soc 5 0.30 -0.11 -0.23 -0.22 -0.70
q23 28 I make friends easi 5 -0.25 0.05 0.27 0.28 0.69
q51 56 People are more lik 5 0.19 0.13 -0.20 -0.25 -0.67
q19 24 I am good at mainta 5 -0.15 -0.04 0.28 0.11 0.65
q67 72 I am always willing 5 -0.10 -0.17 0.20 0.34 0.63
q39 44 Id rather spend tim 5 -0.05 0.06 0.15 0.25 0.62
q43 48 I am happier when I 5 -0.06 0.03 0.45 0.30 0.60
q3 8 I like to meet new 5 -0.12 0.21 0.25 0.14 0.59
q31 36 I tend to talk a lo 5 -0.33 -0.38 0.19 0.17 0.59
Sociability 3 Sociability 5 -0.19 -0.02 -0.17 0.32 0.56
q16 21 I tend to make deci 5 -0.15 -0.14 0.19 0.41 0.54
q59 64 I prefer large crow 5 -0.18 0.01 -0.08 0.24 0.52
q7 12 I can easily start 5 -0.13 0.16 0.17 0.06 0.49
 PRQ-07:
More reliable, greater validity
except for Nach
PNach PAnx PSoc PImp Nach Anx Soc Imp
PNach 1.00 0.21 -0.08 -0.30 0.20 0.10 0.00 -0.31
PAnx 0.21 1.00 -0.10 -0.03 -0.01 0.66 -0.22 0.06
PSoc -0.08 -0.10 1.00 0.29 -0.16 -0.18 0.60 0.37
PImp -0.30 -0.03 0.29 1.00 -0.25 0.16 0.22 0.53
Nach 0.18 -0.01 -0.14 -0.23 0.84 0.08 0.28 -0.23
Anx 0.09 0.60 -0.16 0.15 0.07 0.82 -0.25 0.09
Soc 0.00 -0.21 0.57 0.21 0.24 -0.22 0.89 0.44
Imp -0.29 0.05 0.35 0.50 -0.19 0.08 0.39 0.87

38
 Personality-Music-IQ
alphas on diagonal, unattenuated above
A C E O N P R H FC g math matrix iq?

A 0.90 0.35 0.44 0.27 -0.09 0.46 0.08 0.35 0.17 0.08 0.07 -0.03 0.16

C 0.31 0.89 0.21 0.11 -0.16 0.23 -0.15 0.13 0.03 0.00 0.02 -0.06 0.04

E 0.39 0.19 0.91 0.27 -0.27 0.30 0.12 0.27 0.13 -0.11 -0.09 -0.13 -0.06

O 0.24 0.09 0.24 0.86 -0.07 -0.01 0.27 0.07 0.42 0.36 0.36 0.16 0.36
N -0.09 -0.14 -0.24 -0.06 0.92 -0.01 0.03 -0.13 -0.12 -0.04 -0.06 -0.04 0.00
Pop 0.39 0.20 0.26 -0.01 -0.01 0.82 0.21 0.43 0.38 0.01 -0.02 0.01 0.04

Rock 0.06 -0.12 0.10 0.22 0.02 0.17 0.76 0.18 0.38 0.13 0.15 0.04 0.13
HipHop 0.28 0.10 0.22 0.06 -0.11 0.34 0.14 0.75 0.48 -0.07 -0.07 -0.01 -0.09
Folk.clas 0.14 0.02 0.11 0.34 -0.10 0.31 0.29 0.37 0.78 0.25 0.28 0.21 0.12
g 0.08 0.00 -0.10 0.32 -0.04 0.01 0.11 -0.06 0.21 0.89 1.05 0.76 0.97
math 0.06 0.02 -0.07 0.30 -0.05 -0.02 0.11 -0.06 0.22 0.88 0.80 0.47 0.81
iq.matrix -0.03 -0.05 -0.11 0.14 -0.04 0.01 0.03 -0.01 0.17 0.67 0.38 0.85 0.32
iq3 0.14 0.03 -0.05 0.30 0.00 0.03 0.10 -0.07 0.10 0.81 0.64 0.26 0.79
 Personality-Music
Regression models
Pop Rock HipHop Folk.classic
Agreeable 0.34 0.04 0.24 0.07
Conscientious 0.08 -0.16 0.00 -0.04
Extraversion 0.16 0.08 0.12 -0.01
Open -0.13 0.21 -0.03 0.33
Neuroticism 0.06 0.03 -0.06 -0.08
R2 0.19 0.08 0.10 0.13

40
 Personality + Demographics = Music
Pop Rock HipHop Folk.classic
Agreeable 0.28 0.09 0.21 0.06
Conscientious 0.06 -0.13 -0.02 -0.06
Extraversion 0.15 0.07 0.12 0.02
Open -0.10 0.18 0.01 0.30
Neuroticism 0.02 0.05 -0.06 -0.07
sex 0.19 -0.09 0.04 -0.01
bw 0.00 0.29 -0.28 0.00
age 0.07 -0.09 -0.02 0.23
$R2 0.23 0.17 0.18 0.18
What is a cluster?
 Clustering rules
• Distance:
– Nearest neighbor
– Farthest neighbor
– Centroid distance
• Methods
– Hierarchical
• Agglomerative
• Divisive
– non-hierarchical 43
Hierarchical Clustering

44
 More clustering
100
Original Tree Re-start from 10 clusters

100
80

80
60

60
Height

Height
40

40
20

20
0

dist(USArrests) dist(cent) 45
hclust (*, "centroid") hclust (*, "centroid")
 Height

0 50 100 150

Alabama
Georgia
Arkansas
Louisiana
Florida
Texas
Mississippi
South Carolina
Alaska
Vermont
Hawaii
Maine
Arizona
Utah
Montana
Nevada
New Mexico
Oklahoma
Delaware
Maryland
Kentucky
Washington
Missouri
West Virginia
North Carolina
Tennessee
Virginia

votes.repub
California
Oregon
Connecticut
New York

Divisive Coefficient = 0.89

New Jersey
Illinois
Ohio
Michigan
Pennsylvania
New Hampshire
Wisconsin
Iowa
Colorado
Indiana
Idaho
Wyoming
Kansas
Clusters of voting behavior

Nebraska
46

North Dakota
South Dakota
Massachusetts
Dendrogram of diana(x = votes.repub, metric = "manhattan", stand = TRUE)

Rhode Island
Minnesota
 Clustering Issues
• Cluster Objects/people
– similarities or distances?
• what distance metric
– can objects be reversed? (not usually)
• Cluster items (unusual, but see ICLUST)
– items can be reversed (-happy)
– results are similar to factor analysis
• Stopping rules for cluster
47
– number of cluster problem
 Measuring similarity
Profile Similarity
14

12

10

8
Scores

6 Z
Y

2
X

0
0 2 4 6 8 10
Tests
 Similarity and distance
Questions:

Given a set of scores on multiple tests (a subject profile), how

should we measure the similarity between different profiles? What does
it mean to have a similar profile?

What metric to use?

r
Minkowski Distances = ∑(Xi-Yi)r

r=1 city block metric ==> all distances equally important

(no diagonals)
r=2 Euclidean metric ==> diagonals are shorter than sums
r>2 non-Euclidean ==> emphasizes biggest differences
r=∞ non-Euclidean ==> distance = biggest difference
 Consider different metrics
A B Min
X Y
A B C D
A 1 7
A
C B 7 7 B 0

C 1 3
C 2 4 D 4 2 3

D D 5 1
Euclidean City block Max
A B C D A B C D A B C D

A A A

B 6 B 6 B 6

C 3.2 5.8 C 4 8 C 3 5

D 7.2 6.3 4.2 D 10 8 6 D 6 6 3

 A comparison of metrics
4 5 6 7 8 9 10 0 1 2 3 4

7
euclidean

6
0.90 0.94 0.35

5
4
10

cityblock
9

0.71 0.71
8
7
6
5
4

6.0
maximum

5.0
0.02

4.0
3.0
4

minimum
3
2
1

51
0

4 5 6 7 3.0 3.5 4.0 4.5 5.0 5.5 6.0

 Similarity and correlation
D= ∑(Xi-Yi)2

let Mx= mean X My=mean Y L=Mx-My

x=X-Mx y=Y-My

D= ∑(Xi-Yi)2 = ∑{(Xi - Mx) - (Yi - My) + L}2

D = ∑(x-y+L)2 ==>D = Varx + Vary - 2Covxy + L2

Distance is a function of differences of Level, Scatter, and Pattern

Level ==> differences of means L2 =(Mx-My)2
Scatter ==> Variances Varx + Vary
Pattern ==> Covariance 2Covxy

If variables are standardized (means set to zero and variances to 1) then

distance is a function of the correlation between the two profiles.
D2 = 2 (1- rxy)
 Similarity
Profile Similarity
14

12

10

8
Scores

6 Z
Y

2
X

0
0 2 4 6 8 10
Tests
 City blocks vs. Euclid
MATRIX OF CITY BLOCK DISTANCES
X Y Z W
X 0.000
Y 3.778 0.000
Z 5.000 5.000 0.000
W 5.000 5.000 1.000
0.000
(W and Z are most similar, followed by X and Y)

MATRIX OF NORMALIZED EUCLIDEAN DISTANCES

X Y Z W

X 0.000
Y 4.028 0.000
Z 5.000 6.420 0.000
W 5.115 5.855 1.080
0.000
(W and Z are most similar, followed by X and Y)
 Covariance and Correlation
COVARIANCE MATRIX
X Y Z W

X 5.250
Y -3.875 5.250
Z 5.250 -3.875 5.250
W 2.625 -1.938 2.625 1.313
(X and W are most similar, X is negatively related to Y)

PEARSON CORRELATION MATRIX

X Y Z W
X 1.000
Y -0.738 1.000
Z 1.000 -0.738 1.000
W 1.000 -0.738 1.000 1.000
(X is identical to W and Z, negatively related to Y)
Similarity of Profiles: Level,
scatter, pattern
Profile Similarity
14

12

10

8
Scores

6 Z
Y

2
X

0
0 2 4 6 8 10
Tests
 Sources of Data
Self Report
Direct subjective
empirical scales: MMPI/Strong-Campbell
factorial scales: EPI/16PF/NEOPI-R
rational scales: PRF
Indirect/projective (access to subconscious?)
TAT
Rorschach
Indirect/objective
Cattell objective test battery
Implicit Attitudes Test (RT measures)
Emotional “Stroop”
Indirect/other
a) Kelly Construct Repetory Grid
a) Carroll INDSCAL
George Kelly and the theory of Personal Constructs

•Man as scientist:
–"each man contemplates in his own personal
way the stream of events upon which he
finds himself so swiftly borne"

–"Man looks at his world through transparent

patterns or templates which he creates and
then attempts to fit over the realities of
which the world is composed. The fit is
not always very good. Yet without such
patterns the world appears to be such an
undifferentiated homogeneity that man is
unable to make any sense out of it. Even a
poor fit is more helpful to him than
nothing at all.
58
 George Kelly and the theory of Personal Constructs
•Fundamental postulate:
–"A person's processes are psychological
channelized by the ways in which he anticipates
events."

•Measurement:
–The role construct repertory test (REP test).

•Analysis:
–What are the fundamental constructs with which
one views the world? This can be the entire set
of constructs elicited by the REP test, or some
clustering or grouping of these constructs.
59
 Kelly Rep Test
self O O
lover O
mother O
father O
sib O
teacher O
Best friend O O
Boss O
coworker O O
construct
 REP test: complications
•Completely idiosyncratic. There is no concern with
any fundamental dimensions. However, it is
possible to apply same group space and still detect
individual construct dimensions
•But consider a similar model: individuals as having
unique distortions of shared space. The INDSCAL
and ALSCAL algorithms are available to solve for
joint and individual spaces.

61
 Multidimensional Scaling
• Application of metric or non-metric scaling
• Metric scaling:
– Find dimensional representation of observed
distances (e.g., latitude and longitude)
– Strong assumption of data and metric
• Non-metric scaling
– Scaling to minimize a criterion insensitive to
ordinal transformations
 Distances between cities
Athen Barcelona Brussels Calais Cherburg Cologne CopenhagenGeneva Gilbralter Hamburg
Barcelona 3313
Brussels 2963 1318
Calais 3175 1326 204
Cherbourg 3339 1294 583 460
Cologne 2762 1498 206 409 785
Copenhagen 3276 2218 966 1136 1545 760
Geneva 2610 803 677 747 853 1662 1418
Gibralta 4485 1172 2256 2224 2047 2436 3196 1975
Hamburg 2977 2018 597 714 1115 460 460 1118 2897
Hook of Holkand 3030 1490 172 330 731 269 269 895 2428 550

What is the best representation of these distances in a

two dimensional space?
Scaling of European Cities
 Individual Differences in MDS
INDSCAL
• Consider individual differences in MDS
– Each individual applies a unique weighting to
the MDS dimensions
• Solve for Group space as well as individual
weights to be applied to the group space
 INDSCAL
• Consider a set of points Xi with a
corresponding set of distances in K
dimensional space:
– Dij =(∑(xik-xjk)2).5 (k=1 .. K)
• Consider individuals 1 .. n who differ in the
relative importance (weight) they place on
the dimensions wk.
• Then, the distances for individuall are
– Dijl =(∑{wlk*(xik-xjk)}2).5 (k=1 .. K) 69
 Carroll IndScal model
Individual Differences in MDS
Anxious

Group Space Individual Spaces as

Sad Tense
Distortions of group space
Anxious
Sad Tense Sleepy Alert

Sleepy Alert Happy

Happy Relaxed

Relaxed
Sad Anxious
Tense
Sleepy Alert
Relaxed Happy
 Representation of Countries and
attitudes towards Vietnam
Weight space
Cuba USSR hawks

Cuba USSR

doves
Haiti USA

Haiti USA Cuba USSR

Haiti USA
 INDSCAL- Wish data of countries

72
from J.D. Carroll and M. Wish, 2002
Weight space - Wish data

73
 Sources of Data
Structured interviews (e.g., SCID)
Other ratings
Peer ratings
supervisory ratings
subordinate ratings
archival/unobtrusive measures
unobtrusive measures
historical record
GPA
Publications
Citations
Neuropsychological
a) neurometrics
b) "lie detection”
 Sources of Data
Performance tests
OSS stress tests
New faculty job talks
Clinical graduate applicant interviews
Internships
Probationary Periods
Web based instrumentation
self report
indirect (IAT)
The data box
Multiple ways of assessment
 The data box: measurement across time,
situations, items, and people

P1
P2
P3
P4
.
.
Pi
Pj
… Tn
Pn …
T2
T1
X1 X2 … Xi Xj … Xn
 Cattell’s data box
Integrating People,Variables, and Occasions

• Person x Variables
• Variables over People, fixed Occasion (R)
• People over Variables, fixed Occasion (Q)
• Person x Occasions
• Occasions over People, fixed Variable (S)
• People over Occasions, fixed Variable (T)
• Variables x Occasions
• Variables over Occasions, fixed People (O)
• Occasions over Variables, fixed People (P)
Cattell, R.B (1978) The scientific use of factor analysis. p 323
 Traditional measures
• Individuals across items
– correlations of items taken over people to
identify dimensions of items which are in turn
used to describe dimensions of individual
differences
• Ability
• Non-cognitive measures of individual differences
– stable: trait
– unstable: state

• INDSCAL type comparisons of differences

in structure of items across people
• 3 Mode Factor Analysis 79
 Other ways of measurement

• Example of measurement of the structure of

mood
– between subjects
– within subjects

80
 Introversion/Extraversion as one
dimension of affect/behavior space
• Personality trait description
– Introversion/Extraversion
– Neuroticism Stability
• Affective Space
– Positive Affect
– Negative Affect
• Behavior
– Activation and Approach
– Inhibition and Avoidance
 Personality and Emotions
• Standard model
– Dimensional model of personality
• Particularly Extraversion and Neuroticism
– Dimensional model of emotions
• Positive Affect and Negative Affect
– Dimensional congruence
• Extraversion and Positive Affectivity
• Neuroticism and Negative Affectivity
Measuring the dimensions of affect
• Motivational state questionnaire (MSQ)
– 70-72 items given as part of multiple studies on personality
and cognitive performance
– Items taken from
• Thayer’s Activation-Deactivation Adjective Checklist (ADACL)
• Watson and Clark Positive Affect Negative Affect Scale (PANAS)
• Larsen and Diener adjective circumplex
– MSQ given before and after various mood manipulations
• Structural data is from before
• Structural results based upon factor analyses of
correlation matrix to best summarize data
 2 Dimensions of Affect
1.0

FRUSTRATDISTRESS
UPSET
UNHAPPY SAD TENSE
DEPRESSED BLUE ANGRY CLUTCHED
NERVOUS
GLOOMY SORRY AFRAID
SCARED
IRRITABLASHAMED FEARFUL
ANXIOUS
GROUCHY HOSTILE GUILTY
0.5 LONELY
JITTERY INTENSE

ASTONISH

SURPRISE
DULL DETERMIN
SLUGGISH
SLEEPY INSPIRED
TIRED BORED AROUSED
STRONG VIGOROUS
EXCITED
DROWSY
ACTIVE
QUIET QUIESCEN FULL_OF_ ELATED
INACTIVE ALERT
ATTENTIV
0.0 IDLE ENERGETI
ENTHUSIA
LIVELY
WIDEAWAK
INTEREST
PROUD WAKEFUL
DELIGHTE
PLACID CHEERFUL SOCIABLE
PLEASED
STILL WARMHEAR
CONFIDEN
HAPPY
SATISFIE

TRANQUIL AT_REST
SERENE CONTENT
CALM
RELAXED
AT_EASE
-0.5
-1.0 -0.5 0.0 0.5 1.0
Energetic Arousal/Positive Affect
 1.0 2 Dimensions of Affect
DISTRESSED
FRUSTRATED
SAD TENSE

IRRITABLE ANXIOUS
0.5

TIRED
EXCITED
SLEEPY
ELATED
INACTIVE ENERGETIC LIVELY
0.0
ENTHUSIASTIC
ATTENTIVE

AT_REST
CALM
RELAXED
-0.5
-1.0 -0.5 0.0 0.5 1.0
 Representative MSQ items
(arranged by angular location)
Item EA-PA TA-NA Angle
energetic 0.8 0.0 1
elated 0.7 0.0 2
excited 0.8 0.1 6
anxious 0.2 0.6 70
tense 0.1 0.7 85
distressed 0.0 0.8 93
frustrated -0.1 0.8 98
sad -0.1 0.7 101
irritable -0.3 0.6 114
sleepy -0.5 0.1 164
tired -0.5 0.2 164
inactive -0.5 0.0 177
calm 0.2 -0.4 298
relaxed 0.4 -0.5 307
at ease 0.4 -0.5 312
attentive 0.7 0.0 357
enthusiastic 0.8 0.0 358
lively 0.9 0.0 360
 Personality and Emotions

• Standard model
– Dimensional model of Personality
• Behavioral Activation/Approach <-> Extraversion
• Behavioral Inhibition <-> Neuroticism
– Dimensional model of Emotions
• Positive Affect
• Negative Affect
• Arousal?
– Dimensional congruence
• Extraversion, Approach, and Positive Affectivity
• Neuroticism, Inhibition, and Negative Affectivity
 Personality measurement:
snapshot or movie?
• Cross sectional measurement of a person is
similar to a photograph-- a snapshot of a
person at an instant.
• Appropriate measurement requires the
integration of affect, behavior, and
cognition across time.
 Personality and affect: within
subject measurements
• High frequency sampling: the example of
body temperature

• Low frequency sampling: palm pilot

sampling of affect
 Within subject diary studies-1
• Very High Frequency (continuous)
measurements
– Physiological assays
• Cortisol
• Body temperature <--
– Core body temperature collected for ≈ 2 weeks
– Data taken by aggregating subjects from multiple studies
conducted by Eastman and Baehr on phase shifting by
light and exercise
Body Temperature as f(time of day)
(Baehr, Revelle & Eastman, 2000)

90
80
38.0
70 Sleep
Temperature (C°)

60
50 T emp.
37.0 at bed East
40 West
Latest T T emp .
North
MINS at wak e
30 (N = 11)

20 36.0
16 :00 20 :00 00 :00 04 :00 08 :00 12 :00 16 :00
10
Time (hours)
0
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
Morningness/Eveningness and BT
(Baehr, Revelle and Eastman, 2000)

100

80 3 7.5
Temperature (¡C)

M -types
East
60
3 7.0 West
40 E-types North

20 3 6.5

0
3 6.0
1st Qtr20:00 2nd00:00
16:00 Qtr 3rd Qtr 8:00 4th 12:00
4:00 Qtr 16:00

Tim e (ho u rs )
= Average Sleep
= Average T M IN
 Within subject diary studies-2
• Measures
– Check lists
– Rating scales
• High frequency sampling <--
– Multiple samples per day
• Low frequency sampling
– Once a day
– Sometimes at different times
High frequency measures of affect
• Measures taken every 3 hours during
waking day for 6-14 days
• Paper and pencil mood ratings
– Short form of the MSQ -- Visual Analog Scale
– Sampled every 3 hours
• Portable computer (Palm) mood ratings <--
– Short form of the MSQ
– Sampled every 3 hours
Palm Affect Survey
Palm affect and activity survey
 Traditional measures
• Mean level
– Energetic arousal
– Tense arousal
– Positive affect
– Negative affect
• Variability
• Correlation across measures (Synchrony)
 Phasic measures of affect
• Fit 24 hour cosine to data
– Iterative fit for best fitting cosine
– Permutation test of significance of fit
• Measure
– Fit (coherence)
– Amplitude
– Phase
 Affective rhythms can differ in phase
(simulation - double plotted to show rhythm)
Phase differences of simulated daily data
Differences in coherence (fit) simulated daily data
Phase and Coherence differences
(simulated data -- double plotted)
Multi-level analysis of patterns of
affect across time-1: Method
• Within subject estimates of basic parameters
– Level
– Scatter (variability)
– Phase
– Coherence (fit)
• Between subject measures of reliability
– Week 1/Gap/Week 2
Multi-level analyses of affect-2:
1-2 week Test-Retest Reliability
VAS-1 VAS-2 Palm
Energetic Arousal .67 .81 .82
Tense Arousal .68 .57 .81
Fit EA .55 .41 .07
Fit TA .61 .25 .17
Phase EA .69 .36 .58
Phase TA .39 .25 .36
EA -TA Synchrony .63 .48 .35
 Affective rhythms and
cognitive performance-1
• Design:High frequency diary study of affect
combined with a low frequency study of
reaction time
• Subjects: 28 NU undergraduate voluneteers
• Method:
– 1 week diary study 5 times a day
– Simple reaction time once a day at 5 different
times using a Mac program at home
 Affective rhythms and
cognitive performance-2
• Low negative correlations of RT with
concurrent measures of Energetic Arousal
• Stronger negative correlations of RT with
Cosine fitted Energetic Arousal
• => Diurnal variation in RT may be fitted by
immediate and patterns of arousal
 Behavioral variation over time
• William Fleeson and studies of personality
variability over time
• Personality traits and personality states
• Traits as aggregated states
 Behavioral Variability:
Model 1:
Frequency

Behavioral state ->

 Behavioral Variability:
Frequency

Behavioral state ->

 Stability of trait means and
variances
• Fleeson examined within and between day levels
of behaviors and affects
• Low correlations of single measurement with
other single measurements
• High correlations of means over multiple days
with similar means over different days
• High correlations of variability over multiple days
with similar estimates over different days
Extraversion and Affect
Positive Affect and acting Extraverted
 The data box: measurement across time,
situations, items, and people

P1
P2
P3
P4
.
.
Pi
Pj
… Tn
Pn …
T2
T1
X1 X2 … Xi Xj … Xn
 Cattell’s data box
Integrating People,Variables, and Occasions

• Person x Variables
• Variables over People, fixed Occasion (R)
• People over Variables, fixed Occasion (Q)
• Person x Occasions
• Occasions over People, fixed Variable (T)
• People over Occasions, fixed Variable (S)
• Variables x Occasions
• Variables over Occasions, fixed People (P)
• Occasions over Variables, fixed People (O)
Cattell, R.B. (1966), Handbook of Multivariate Experimental Psychology. p 69-70.
but see Cattell, R.B (1978) The scientific use of factor analysis. p 323 where P is swapped with O and T with S.
 Traditional measures
• Individuals across items
– correlations of items taken over people to
identify dimensions of items which are in turn
used to describe dimensions of individual
differences
• Ability
• Non-cognitive measures of individual differences
– stable: trait
– unstable: state

• INDSCAL type comparisons of differences

in structure of items across people
• 3 Mode Factor Analysis 119