Part 4

Staffing Activities:
Selection
Chapter 7:
Measurement
 Staffing Organizations Model
Organization
Mission
Goals and Objectives

Organization Strategy HR and Staffing Strategy

Staffing Policies and Programs

Support Activities Core Staffing Activities
Legal compliance Recruitment: External, internal
Selection:
Planning Measurement, external, internal
Employment:
Job analysis Decision making, final match
Staffing System and Retention Management
7-2
 Chapter Outline

• Importance and Use of Measures

• Key Concepts • Collection of Assessment Data
• Measurement • Testing Procedures
• Scores • Acquisition
Manuals
of Tests and Test
• Correlation Between Scores • Professional Standards
• Quality of Measures
• Legal Issues
• Reliability of Measures • Determining Adverse Impact
• Validity of Measures • Standardization
• Validation
Staffing
of Measures in
• Best Practices
• Validity Generalization
• Staffing Metrics and Benchmarks
7-3
 Learning Objectives for This Chapter

• Define measurement and understand its use and importance in

staffing decisions
• Understand the concept of reliability and review the different ways
reliability of measures can be assessed
• Define validity and consider the relationship between reliability and
validity
• Compare and contrast the two types of validation studies typically
conducted
• Consider how validity generalization affects and informs validation of
measures in staffing
• Review the primary ways assessment data can be collected

7-4
Discussion Questions for This Chapter
• Imagine and describe a staffing system for a job in which there are no measures used.
• Describe how you might go about determining scores for applicants’ responses to (a)
interview questions, (b) letters of recommendation, and (c) questions about previous
work experience.
• Give examples of when you would want the following for a written job knowledge test
• a low coefficient alpha (e.g., α = .35)
• a low test–retest reliability.
• Assume you gave a general ability test, measuring both verbal and computational
skills, to a group of applicants for a specific job. Also assume that because of severe
hiring pressures, you hired all of the applicants, regardless of their test scores.
• How would you investigate the criterion-related validity of the test?
• How would you go about investigating the content validity of the test?
• What information does a selection decision maker need to collect in making staffing
decisions? What are the ways in which this information can be collected?

7-5
 Key Concepts
• Measurement
• the process of assigning numbers to objects to
represent quantities of an attribute of the
objects
• Scores
• the amount of the attribute being assessed
• Correlation between scores
• a statistical measure of the relation between
the two sets of scores 7-6
 Importance and Use of Measures

• Measures
• Methods or techniques for describing and assessing attributes of
objects
• Examples
• Tests of applicant KSAOs
• Job performance ratings
of employees
• Applicants’ ratings of their
preferences for various types
of job rewards
7-
7
 Importance and Use of Measures
(continued)

• Summary of measurement process

• (a) Choose an attribute of interest
• (b) Develop operational definition of attribute
• (c) Construct a measure of attribute as operationally
defined
• (d) Use measure to actually gauge attribute
• Results of measurement process
• Scores become indicators of attribute
• Initial attribute and its operational definition are transformed
into a numerical expression of attribute
7-
8
 Measurement: Definition

• Process of assigning numbers to objects to represent

quantities of an attribute of the objects
• Attribute/Construct - Knowledge of mechanical principles
• Objects - Job applicants

7-
9
Ex. 7.1 Use of Measures in Staffing

7-10
 Measurement: Standardization
• Involves
• Controlling influence of extraneous factors
on scores generated by a measure and
• Ensuring scores obtained reflect the attribute
measured
• Properties of a standardized measure
• Content is identical for all objects measured
• Administration of measure is identical for all objects
• Rules for assigning numbers are clearly specified and
agreed on in advance
7-11
 Measurement: Levels
• Nominal • Interval
• A given attribute is • Objects are rank-ordered
categorized and numbers • Differences between
are assigned to categories adjacent points on
• No order or level implied measurement scale are equal
in terms of attribute
among categories
• Ordinal
• Ratio
• Similar to interval scales -
• Objects are rank-ordered equal differences between
according to how much of scale points for attribute
attribute they possess being measured
• Represents relative • Have a logical or absolute
differences among objects zero point

7-12
 Measurement: Differences in
Objective and Subjective Measures
• Objective measures
• Rules used to assign numbers to attribute are
predetermined, communicated, and applied
through a system
• Subjective measures
• Scoring system is more elusive, often involving a rater
who assigns the numbers
• Research shows these may not be strongly related, but
purely objective measures can miss important parts of job
performance 7-13
 Scores
• Definition
• Measures provide scores to represent
amount of attribute being assessed
• Scores are the numerical indicator of attribute
• Central tendency and variability
• Exh. 7.2: Central Tendency and Variability: Summary
Statistics
• Percentiles
• Percentage of people scoring below an individual in a
distribution of scores
• Standard scores
7-14
 Correlation Between Scores
• Scatter diagrams
• Used to plot the joint distribution of the two sets of scores
• Exh. 7.3: Scatter Diagrams and Corresponding Correlations
• Correlation coefficient
• Value of r summarizes both
• Strength of relationship between two sets of scores and
• Direction of relationship
• Values can range from r = -1.0 to r = 1.0
• Interpretation - Correlation between two variables does not
imply causation between them
• Exh. 7.4: Calculation of Product-Movement Correlation
Coefficient
7-15
Exh. 7.3: Scatter Diagrams and
Corresponding Correlations

7-16
Exh. 7.3: Scatter Diagrams and
Corresponding Correlations

7-17
Exh. 7.3: Scatter Diagrams and
Corresponding Correlations

7-18
 Significance of the Correlation
Coefficient
• Practical significance
• Refers to size of correlation coefficient
• The greater the degree of common variation
between two variables, the more one variable
can be used to understand another variable
• Statistical significance
• Refers to likelihood a correlation exists in a population, based on
knowledge of the actual value of r in a sample from that
population
• Significance level is expressed as p < value
• Interpretation -- If p < .05, there are fewer than 5 chances in
100 of concluding there is a relationship in the population
when, in fact, there is not
7-19
 Quality of Measures
• Reliability of measures

• Validity of measures

• Validity of measures in staffing

• Validity generalization
7-20
 Quality of Measures: Reliability

• Definition: Consistency of measurement of an attribute

• A measure is reliable to the extent it provides a
consistent set of scores to represent an attribute
• Reliability of measurement is of concern
• Both within a single time period and between time
periods
• For both objective and subjective measures
• Exh. 7.6: Summary of Types of Reliability
7-21
Ex. 7.6: Summary of Types of Reliability

7-22
 Quality of Measures: Reliability
• Measurement error
• Actual score = true score + error
• Deficiency error
• Failure to measure some aspect of attribute
assessed
• Contamination error
• Occurrence of unwanted or undesirable
influence on the measure and on individuals
being measured
7-23
Ex. 7.7 - Sources of Contamination Error
and Suggestions for Control

7-24
 Quality of Measures:
Reliability
• Procedures to calculate reliability estimates
• Coefficient alpha
• Should be least .80 for a measure to have an acceptable degree of
reliability
• Interrater agreement
• Minimum level of interrater agreement - 75% or higher
• Test-Retest reliability
• Concerned with stability of measurement
• Level of r should range between r = .50 to r = .90
• Intrarater agreement
• For short time intervals between measures, a fairly high
relationship is expected - r = .80 or 90%
7-25
 Quality of Measures: Reliability
• Implications of reliability
• Standard error of measurement
• Since only one score is obtained from an applicant, the
critical issue is how accurate the score is as an indicator of an
applicant’s true level of knowledge
• Relationship to validity
• Reliability of a measure places an upper limit on the possible
validity of a measure
• A highly reliable measure is not necessarily valid
• Reliability does not guarantee validity - it only makes it
possible
7-26
 Quality of Measures: Validity

• Definition: Degree to which a measure truly

measures the attribute it is intended to measure
• Accuracy of measurement
• Exh. 7.9: Accuracy of Measurement
• Accuracy of prediction
• Exh. 7.10: Accuracy of Prediction
7-27
Ex. 7.9: Accuracy of Measurement

7-28
Exh. 7.12: Accuracy of Prediction

7-29
Exh. 7.12: Accuracy of Prediction

7-30
 Validity of Measures in Staffing

• Importance of validity to staffing process

• Predictors must be accurate representations of
KSAOs to be measured
• Predictors must be accurate in predicting job success
• Validity of predictors explored through validation studies
• Two types of validation studies
• Criterion-related validation
• Content validation
7-31
 Ex. 7.13: Criterion-
Related Validation
•Criterion Measures: measures of
performance on tasks and task
dimensions
•Predictor Measure: it taps into
one or more of the KSAOs
identified in job analysis
•Predictor–Criterion Scores: must
be gathered from a sample of
current employees or job
applicants
•Predictor–Criterion Relationship:
the correlation must be
7-32
calculated.
Ex. 7.14: Concurrent and Predictive
Validation Designs

7-33
Ex. 7.14: Concurrent and Predictive
Validation Designs

7-34
 Content Validation
• Content validation involves
• Demonstrating the questions/problems (predictor
scores) are a representative sample of the kinds of
situations occurring on the job
• Criterion measures are not used
• A judgment is made about the probable correlation
between predictors and criterion measures
• Used in two situations
• When there are too few people to form a sample for
criterion-related validation
• When criterion measures are not available
• Exh. 7.16: Content Validation
7-35
 Validity Generalization
• Degree to which validity can be extended to other
contexts
• Contexts include different situations, samples of
people and time periods
• Situation-specific validity vs. validity generalization
• Exh. 7.18: Hypothetical Validity Generalization
Example
• Distinction is important because
• Validity generalization allows greater latitude
than situation specificity
• More convenient and less costly not to have to
conduct a separate validation study for every
situation 7-36
Exhibit 7.18 Hypothetical Validity
Generalization Example

7-37
 Staffing Metrics and Benchmarks
• Metrics
• quantifiable measures that demonstrate the
effectiveness (or ineffectiveness) of a particular
practice or procedure
• Staffing metrics
• job analysis
• validation
• Measurement
• Benchmarking as a means of developing metrics
7-38
 Collection of Assessment Data

• Testing procedures
• Paper and pencil measures
• PC- and Web-based approaches
• Applicant reactions
• Acquisition of tests and test manuals
• Paper and pencil measures
• PC- and Web-based approaches
• Professional standards
7-39
 Legal Issues
• Disparate impact statistics
• Applicant flow statistics
• Applicant stock statistics
• Standardization
• Lack of consistency in treatment of applicants is
a major factor contributing to discrimination
• Example: Gathering different types of background
information from protected vs. non-protected groups
• Example: Different evaluations of information for protected
vs. non-protected groups
• Validation
• If adverse impact exists, a company must either eliminate it or
justify it exists for job-related reasons (validity evidence)
7-40
 Ethical Issues

• Issue 1
• Do individuals making staffing decisions have an ethical
responsibility to know measurement issues? Why or why not?
• Issue 2
• Is it unethical for an employer to use a selection measure that
has high empirical validity but lacks content validity? Explain.

7-41