“There is very little difference

between one person & another. But

what little there is, is very important. ”
William James

Selection and Assessment

Dr. David Hughes

Twitter: david_j_hughes
E-mail: [email protected]
Overview
• Me

• Selection

• Interviews and selection

– Pros and Cons of interviews for selection

• Psychometrics and selection

– Pros and cons of psychometrics for selection

• The all important predictive validity…

 Me

Failed BSc. Ψ MSc. Org. Ψ

footballer

Director MSc. Director MSc. Ph.D. Behav.

Bus. & Org Ψ Business Ψ Econ. Ψ

DJH Consulting (2011 - Present): Psychometrics; selection and assessment;

development of bespoke psychometric tools; training programme development and
evaluation; statistical analysis of company data; report writing.
 My Research

Mechanisms:
Mediators
Moderators
Workplace
Performance
Theoretical
Nature
Leader & Employee
Construct Theory Creativity &
Personality &
Innovation
Individual Differences
Measurement

Deviant &
CWB
Selection

• Finding the best candidate for the job

– Best in terms of ability, attempting to predict job performance
– Best in terms of ‘organisational fit’

• Workplace selection is all about discriminating.

– Differentiating between candidates
– We must ensure that we discriminate in a FAIR and JUSTIFIED manner.
That is, we are assessing candidates only on job relevant skills and
abilities.
– To this end, job analysis is key… it allows for the identification of
appropriate selection methods, those that best measure the core skills.
Selection

• Process must be legally defensible – e.g. tools used shown to

predict future job performance.

• Employing the ‘wrong’ person is costly! Recruitment costs 10-40%

of 1st years salary + training costs, lost productivity, knock on
effect for colleagues...
– Firing people is tough
 Evaluation criteria of selection methods

• Reliability:
• Precision of measurement, giving a consistent account

• ‘Validity’:
• Accurate measurement (i.e., measuring what is claimed to be measured)
• Appropriate levels of prediction (i.e., predicting future job performance)

• Fairness:
• Selection methods must not unfairly discriminate against any specific
subgroups
•Candidate reactions:
• Selection tools might ‘put off’ certain candidates

…as well as cost and practicality

The Selection Paradigm
Organisational Needs Analysis

Job Analysis

Job description & Person specification

Evaluation
& Ethics
Identify selection criteria & choose assessment methods
Attract
Candidates
Administer Selection Methods

Interpret Results & Make Decision

Job Analysis
Purpose is to identify the skills and behaviours needed to be
successful in the job

• Identifies tasks and responsibilities involved in the job

• Provides information on skills and characteristics that are important

for job success

• Other factors: job title, hours of work, direct report, equipment, pay
& benefits etc.
Selection Methods

Dr. David Hughes

Twitter: david_j_hughes
E-mail: [email protected]
Selection Methods
Methods Popular? Predict?
• References
• CV / Bio data
• Work samples
• Job knowledge test
• Graphology
• Assessment centres
• Interviews
• General Mental Ability / IQ
• Personality
 Interviews

Dr. David Hughes

Twitter: david_j_hughes
E-mail: [email protected]
Interviews
• Most popular method of selection (Robertson & Makin, 1986;
Wilk & Cappelli, 2003)
– 80% businesses always use
– 1% never use

• Two main types

– Unstructured (traditional)
– Structured
• Behavioural (Janz, 1989)
• Situational (Maurer et al., 1999)
Unstructured Interviews

• A conversation between employer and candidate

– Any experience?

• Subjective, questions can change, candidates have

large influence
– Narcissists tend to do rather well in interviews (Paulhus,
Westlake, Calvez & Harms, 2013)
– Anxious individuals less so
Unstructured Interviews

• Bias galore (Dipboye, 1994; Harris, 1989; McDaniel et al., 1994):

– Sex
– Race
– Attractiveness (Beards?)
– Clothes
– Non-verbal communication
– Primacy and Recency effects
– Halo / Horns effect
– Like me
Unstructured Interviews

• So, why are unstructured interviews so popular?

– Cheap, easy, quick
– Candidates like them (tend not to be legal challenges, though there
should be!).
– Managers trust their instincts/intuition: “I’ve been in this game for
30 years I know a good/bad employee when I see one”
– Gives power to senior org members: ‘Jobs for the boys’
Structured Interviews
Predictive validity increases with structure (Huffcutt & Arthur, 1994;
Hufcutt, et al., 2001):
Level of Structure Predictive
Validity
Level 1: Unstructured .20
Level 2: Topic areas pre-determined .35
Level 3: Pre-determined questions, follow-up allowed .56
Level 4: Highly structured, predetermined questions, .57
follow-up not allowed
Why?
• Standardization – less biased, fewer opportunities for candidates to ‘steer’
• Job-related knowledge is tested
• Still get a sense of person-organization fit
Structured Interviews
• Behavioural Interviews
– Ask candidates to describe previous behaviour in past, job relevant,
areas.
• (e.g. Describe dealing with angry customer).

• Situational
– Prospective… what would you do in X scenario?
• (e.g. If 25% of your staff phoned in sick due to a flu epidemic, what
would you do?)

• Answers rated against predefined scoring systems, created by ‘experts’

(work psychologists and job holders), based on job analysis.
– Points assigned to answers and score calculated. In essence, a verbal
psychometric test – situational judgement.
Structured Interviews
• You are a manager of a large supermarket (around 500
employees). One Saturday, a flu bug renders 30% of your
scheduled to work staff sick. What do you do?
1. Close the store
2. Close one department
3. Ring them all, ask them to come in anyway
4. Ring staff who are ‘off’, ask them to cover
5. Get all managers/office staff to work the ‘shop floor’
Interviews: Conclusion

• Unstructured… AVOID
– Only use if there is no other possible way to measure what you need – This is
highly unlikely.
– If you do use, use at the end of the process once you have identified a few really
good candidates and always place as little emphasis on them as possible

• Structured… Good, do use.

– Can be expensive and time consuming to produce
– Always check validity and amend, just as you would a psychometric test.
– Trained interviewers; Multiple Interviewers (cancel out biases?); Good inter-rater
reliability
Individual Differences & Psychometrics

Dr. David Hughes

Twitter: david_j_hughes
E-mail: [email protected]
Individual Differences

“No two persons are born exactly alike; but each

differs from the other in natural endowments, one
being suited for one occupation and the other for
another.”
Plato (approx. 460BC)
Individual Differences
• Evolved and learned patterns of uniqueness
• Much of psychology studies similarities – group
processes, biological underpinnings.
• In experimental settings IDs are often treated as error!!!
• IDs are characteristics and traits that we all possess but
on which we differ: e.g. personality, intelligence,
attitudes, values
“Unique constellation of traits”
Intelligence / Cognitive Ability

Dr. David Hughes

Twitter: david_j_hughes
E-mail: [email protected]
Defining Intelligence…

With your neighbours,

define Intelligence…
Definition
• For practical purposes intelligence may be defined as
the rate and accuracy of human information
processing across different sense modalities and
domains of experience.

• (Sternberg and Detterman)

 General Intelligence (g)

Broad
Fluid Crystallized General
Memory Visual
Intelligence Intelligence
Perception

Comprehension Memory Spatial

Reasoning
Vocabulary Span
Abilities
Working
Memory Know-ledge

Part of Carrol’s Hierarchical factor model (1993)

 Testing IQ
Verbal/Numerical
• 13 - - - 4
• 27 - - - 9
• 43 - - - 7
• 11 - - -?

• ZXTRNL?

• If AK=10, RV=4 and JZ=16 What does MO= ?

• If all Bloops are Razzies and all Razzies are Lazzies,

then all Bloops are definitely Lazzies?
Testing IQ
Non-Verbal
Predictive validity
• General mental ability (GMA) is one of the best
predictors of job performance (r=.51) exceeded only
by work samples.
– (Robertson & Smith, 2001).

• In Europe the validity of GMA appears to be higher

at .62.
– (Salgado, Anderson, Moscoso, Bertua & Fruyt, 2003).
Predictive validity

• (Bertua, Anderson, Salgado, 2005)

Predictive validity
• Correlations with training performance range from r≈.27 (Judge, Colbert,
& Ilies) to r ≈ 50 (Bertua, Anderson, Salgado, 2005)

(Bertua, Anderson, Salgado, 2005)

Predictive validity
• “…intuition may tell [people] that personality and other non-cognitive
traits are more important than GMA (Hunter & Schmidt, 1996).
However... Causal analyses of the determinants of job performance show
that the major effect of GMA is on the acquisition of job knowledge:
People who are higher in GMA acquire more job knowledge and acquire
it faster. The amount of job-related knowledge required on even less
complex jobs is much greater than is generally realized. Higher levels of
job knowledge lead to higher levels of job performance. Viewed
negatively, not knowing what one should be doing--or even not knowing
all that one should about what one should be doing--is detrimental to job
performance. In addition, knowing what one should be doing and how to
do it depends strongly on GMA.”

• (Schmidt & Hunter, 2004)

 Why do Intelligent People Fail?

• Lack of motivation
• Lack of impulse control
• Lack of perseverance
• Fear of failure
• Procrastination
• Inability to delay gratification
• Too little/too much self-confidence
Source: Sternberg (1986)
 Personality

Dr. David Hughes

Twitter: david_j_hughes
E-mail: [email protected]
Defining Personality…
“Characteristic patterns of behaviour,
thoughts and feelings”
Allport (1961)
Personality
 Approaches
• Psychoanalytical
– Freud, Jung, Adler

• Social-Cognitive learning theories

– Bandura; Kelly; Rotter-Mischel; Eriksen
– https://www.youtube.com/watch?v=qy_mIEnnlF4

• Humanistic
– Maslow; Rogers

• Biological
– Eysenck; Gray; Cloninger
Trait approach
• Trait / Psychometric
– Allport; Cattell; Eysenck; Costa & McCrae

The main idea behind the trait approach to personality is that there exists in
people consistent personality characteristics that can be identified, measured and
studied.

Myth: Traits are deterministic

Truth, traits are probabilistic. People do not behave the same in all situations, but
those who are higher in a trait behave accordingly more often… traits are actually
density distributions
(Fleeson & Jayawickreme, 2015)
Personality trait hierarchy

See: Costa & McCrae (1985; 1992) and DeYoung, Peterson & Quilty (2007)
FFM or Big 5
Trait Item
Neuroticism Is depressed, blue
Is relaxed, handles stress well
Extraversion Is full of energy
Is sometimes shy, inhibited
Openness Is original, comes up with new ideas
Prefers work that is routine
Agreeableness Has a forgiving nature
Tends to find fault with others
Conscientiousness Does a thorough job
Is easily distracted
Personality and Job Choice
• Do different personality traits lead us to make
different career choices?

• https
://learningonscreen.ac.uk/ondemand/index.php/clip
/89019
(BBC child of our time E2, Part 1)
Personality and work
• Is it useful? Does it predict performance?

• Morgeson, Campion, Dipboye, Hollenbeck, Murphy, &

Schmitt, 2007:
– Due to the low validity and content of some items, many
published self-report personality tests should probably not be
used for personnel selection

• Ones, Dilchert, Viswesvaran & Judge, 2007:

– Any selection decision that does not take the key personality
characteristics of job applicants into account would be deficient.
Big 5 and work outcomes
• Mount & Judge (2001), comprehensive meta-analysis

• Job Performance
– All roles: Conscientiousness (r = .23) & Neuroticism (r = -.13)
– Management: Extraversion: (r = .21)
– Training: Openness (r = .33)
– Team-working: Agreeableness (r = .34)
– CWB: Dark Triad (r = .20-.35)
Leadership/Management
• Judge, Bono, Ilies & Gerhardt (2002)

• Meta-analysed 222 correlations from 73 samples

– Extraversion = .31
– Conscientiousness = .28
– Neuroticism = -.24
– Openness to Experience = .24
– Agreeableness = .08
Bandwidth & Fidelity

Findings from Judge et al (2013), Table from Hughes & Batey, 2017
Personality: Disadvantages
• Self-reports have biases:

– Self-deception – individuals are overly optimistic in their self-

perceptions and play down their perceived negative aspects

– Impression management – cheating… people try to fake to get closer

to what they think the selectors are looking for.
 Correlations between job performance and personality as assessed
by self-ratings and other-ratings of taken from Connelly and Ones
(2010) and Oh, Wang, & Mount (2011).

Trait and rating type r1 r2 r3

Connelly & Ones (2010) Oh, et al. (2011)

Emotional Stability

Other-rating .14 .17 .37 .17 .24

Self-rating .06 .11 .12 .09 .14
Extraversion
Other-rating .08 .11 .18 .21 .29
Self-rating .06 .11 .12 .06 .09
Openness
Other-rating .18 .22 .45 .20 .29
Self-rating .03 .04 .05 .03 .05
Agreeableness
Other-rating .13 .17 .31 .23 .34
Self-rating .06 .11 .13 .07 .10
Conscientiousness
Other-rating .23 .29 .55 .31 .41
Self-rating .12 .20 .23 .15 .22

Table from Hughes & Batey (2015)

 Adaptive personality regulation?
Study 1: Experiment Study 2: Comedians
Predictor Network Solo Work Predictor Material Delivery
N -.01 -.21* N .09 .09
E .22* -.18* E .09 -.05
O -.13 -.28* O .05 -.05
A -.08 .01 A .06 .06
C .19 .04 C .10 .01
Self- -.02 .13 SM -.13 .12
Monitoring APR .49* .38*
APR .25* .29* R2 .28 .20
R2 .26 .28 ΔR2 .23 .14
ΔR2 .05 .08
 The all important…
‘Predictive Validity’

Dr. David Hughes

Twitter: david_j_hughes
E-mail: [email protected]
Criterion Validity
Selection method r Selection method r
combinations
Work samples .54 General Mental Ability
+ .64
General Mental Ability .51 Personality
Structured Interviews .51

Job knowledge .48 General Mental Ability

+ .63
Personality .40 Structured Interviews
Assessment centres .37

Unstructured Interviews .38 General Mental Ability

+ .60
Biographical data .35 Work Samples
References .26

Graphology .02

(Schmidt & Hunter, 1998)

 ‘Validity’ of selection procedures
(Arnold et al. 2005)

Selection method Criterion-related validity Applicant reactions Extent of use

Structured interviews High Moderate to positive High

Cognitive ability High Negative to moderate Moderate

Personality tests Moderate Negative to moderate Moderate

Biodata Mixed Moderate Moderate

Work sample tests High Positive Low

Assessment centres Mixed Positive Moderate

Handwriting Low Negative to moderate Low

References Low Positive High

Thank you!