Week 6 – Predictive Analytics: Using Data to Predict Employee

Performance
Summary Transcripts

Staffing Cycle
A fundamental part of managing people is staffing. The three different aspects of staffing are:
1. Hiring: Making sure we have the right people in the job
2. Internal Mobility and Career Development: Ensuring that people are in the right job and can
be moved through different jobs over time
3. Attrition: Making sure that people in the organization stay in their jobs

Staffing Cycle: Hiring

From an analytics standpoint, hiring is about predicting performance. To screen applicants, companies
use:

• Job knowledge tests

• Cognitive ability tests

• Personality tests
• Reference checks

• Structured interviews
• Unstructured interviews
• Work samples

• Integrity tests

Hiring: Predicting Performance

Correlation is a measure of how closely two variables move together. As you can see in the chart
below, work samples have the highest correlation with future performance, while reference checks
have the lowest correlation.
Correlation with subsequent performance on a scale of 0-1 is as follows:
Screening Techniques Correlation

Work samples 0.54

Cognitive ability tests 0.51

Structured interviews 0.51

Business Analytics: From Data to Insights Page 1 of 12

 Job knowledge tests 0.48

Integrity tests 0.41

Unstructured interviews 0..31

Personality tests (conscientiousness) 0.31

Reference checks 0.26

Benefits of People Analytics

People analytics allows us to estimate what makes people effective. For example, performance
evaluations (how do people do at their jobs?) use a variety of performance measures (who is likely to
get rated as high versus low?).

Objective information:

Objective information is available for:

• Sales
• Productivity
• Absenteeism
• Quality assurance
• Customer satisfaction
• Attrition
• Rate of promotion

Predictors for Hiring People

Focus on attributes that drive performance, e.g., by looking at patterns in data sets. The predictors for
hiring are:

Background/Experience Test Scores Interview Performance

• Resume • Intelligence tests • Unstructured

• Academic records • Personality tests interviews
(e.g. GPA) • Job knowledge tests • Specific questions
• Characteristics aimed to test
attributes the job
requires

JetBlue’s Story
• JetBlue wanted to hire flight attendants.
• They went to customers to find out if customers rated them higher when the flight
attendant was friendly? Or when they were helpful?

Business Analytics: From Data to Insights Page 2 of 12

 • Results suggested that helpfulness of attendants was more important, which helped them
fine-tune their hiring.

Hints on Predicting Performance

The idea is to take predictors and look at them for well-performing employees in the organization, i.e.,
which of these predictors say something about people's performance.

Compare people who have the same characteristics, such as work, location, manager/unit, level, and
time in the job.

Approaches to Predicting Performance

The approaches to predicting performance are classified into Okay, Good, and Better below:

Okay:

• Compare characteristics of the best and worst performers

• Test for statistical significance

Better:

• Compare characteristics of the best and worst performers in the same cohort and job

Good:

• Use multivariate regression to separate out influences of different characteristics

Multivariate Regression
Multivariate regression finds the best-fit line through the data, summarizing the relationship between
two variables (e.g. GPA and performance in the chart on the left).
Characteristics of multivariate regression:
• Quantifies the effect of a variable (with an equation)
• Gives a level of statistical significance to the data
• Allows comparison of multiple variables, enabling us to identify those that strongly affect
performance

Internal Mobility: Beyond the Peter Principle

Analytics can help us optimize the process (to get the right people into the right jobs), where people
move from one job to another within the organization. We have to go beyond the Peter Principle to
analyze promotability.

The Peter Principle

In time, every post is occupied by an employee who is incompetent to carry out its duties. People who
do well on their jobs, eventually get promoted. People will keep getting promoted until they hit a job
that they can't do.

Internal Mobility: Whom to Promote?

Your goal is to identify dimensions of their current performance to predict whether they're going to
be a high performer in a higher-level job.

Business Analytics: From Data to Insights Page 3 of 12

 Performance across multiple dimensions:

• Output measures
• Competence
• Assessments

Optimizing Movement Within an Organization

An organization fills empty jobs in three ways: hiring from outside, promoting from within, and moving
laterally (sideways).

Which will be more effective?

• Promotional internal development and promotional mobility

• Focusing efforts towards effective recruitment

Staffing: Hiring vs. Promoting

Points taken into consideration when comparing hiring and promotions:

• Compare people in the same job

• Observe outcomes based on how they entered the job
• Evaluate jobs that can be entered into in multiple ways

From this study, it was found that people who were hired performed worse but were paid more than
people who were promoted.

Comparing Staffing Within Firms

Out of internal postings and promotions, which method should be encouraged? Which method works
better?

Promotions enable people to do different things as they move along their career paths. Significantly,
performance was higher after people had been moved through postings compared to when they were
promoted.

Internal Postings vs. Promotions

Internal postings are more successful because they are more structured than promotions. The
advantages of internal postings are:

• Ability to choose from a wider pool of applicants

• People who had been in the same group and were promoted through internal postings
performed better than those promoted through sponsorships

People Analytics and Causality

People analytics is performed when we want to bring about a change using data. However, correlation
is not causation. For instance, the inferences in the examples below may not always be correct.

Findings from data Inferences

People who enter jobs through formal posting perform Should we discourage posting?
worse

Business Analytics: From Data to Insights Page 4 of 12

 People who have been in the job longest have lower Should we move people around
performance more?
People who have taken a training program perform better Should we invest more in training?

People who have undergone training show improved Does that training program work?
performance

Types of Causality Problems:

There are two types of causality problems: omitted variable bias, and reverse causality.

Omitted variable bias:

If X is correlated to Y, that doesn’t mean X causes Y. There may be a third variable, O, that causes both
X and Y.

Example: Ashenfelter dip phenomena

Reverse causality:

Reverse causality is when, rather than X causing Y, Y causes X.

Examples:

• Peter Principle
• People who have been in the job for some time have not been promoted out.
• If the best performers are sent to training, it doesn’t necessarily mean training improved their
performance.

Approaches to Address Causality

To address causality, analyze what leads to differences in the main predictor variable. The different
approaches that people use to address causality are:

1. Measure and control for omitted variables

2. Look for evidence to rule out alternatives
3. Exploit natural sources of randomization
4. Conduct an experiment

Measure and Control for Omitted Variables

• Collect data on possible omitted variables

o Include in regressions
o Create matched pairs with similar values
• Examine changes in performance of a single individual

A lot of things cannot be measured with precision.

Look for Evidence to Rule out Alternatives

• What would be the implications of alternative explanations?

• Can you find evidence for or against those explanations in the data?

Business Analytics: From Data to Insights Page 5 of 12

 Performance evaluation can be objective or subjective. It’s objective when evaluations are tied to
results, and subjective when evaluations are based on competence.

Exploit Natural Sources of Randomization

Randomly assign people to a condition and have controls associated with it.

Natural experiments are places where the experiment has already been done. People were randomly
assigned to the treatment.

For example, going to the university in the US during Vietnam war.

Business Analytics: From Data to Insights Page 6 of 12

 Conduct an Experiment

• Randomly assign training to people in jobs and determine its impact

• Determine the treatment and control groups (ensuring balanced characteristics of each
group)
• Test whether the results in two groups are different

Understanding and Managing Attrition

Turnover leads to increased hiring costs, training costs, loss of unique knowledge, and impact on
customer relationships.

To reduce turnover:

• Implement an informed hiring strategy

• Make targeted interventions
o Address unmet needs
o Improve work conditions
o Train managers
o Focus retention efforts

Understanding Attrition

People leave their jobs because there is something else they would rather be doing. Two things that
determine if people will stay in their current jobs are:

1. Attractiveness of outside opportunities

a. Demand for skills
b. Industry/regional growth
2. Satisfaction with current job situation
a. Perception of future opportunities/trajectory in the organization

Why People Move: Basic Predictors

Here’s the inverse correlation with respect to turnover:

Attributes Correlation

Supervisor relationship 0.25

Job satisfaction 0.22

Role conflict 0.22

Promotion opportunities 0.16

Stress 0.13

Co-worker satisfaction 0.13

Pay 0.11

Business Analytics: From Data to Insights Page 7 of 12

 Process Perspectives on Turnover: Reasons for Quitting

• Compare your current job to an alternative

o Direct contact from recruiter
o Awareness of attractiveness alternatives
• Plan a move at a certain age or after a certain level of experience
• Quit without any specific reason

The Process of Leaving: Search

Assumptions:

• A different job will better suit our preferences

• A new job will be a good fit

We learn over time what is that we are good at and what is that we enjoy.

Assumptions:

We can only assess fit once we are actually in the job.

Implications:

• Rate of turnover falls as workers get older

o More likely to know what we like and what we are good at

Predicting Attrition
To predict attrition, we must analyze:

• Why people leave jobs?

• Who is leaving?
• Where are they leaving from?
• What we want to do about it?

Analyzing how people are hired:

• How do we hire people?

• Are we hiring people who are likely to leave?
• Do we want to look for slightly different attributes?

Approaches to Predicting Attrition

Approaches to predicting attrition can be classified into okay, good, better, and best.

Okay:

• Comparisons of % attrition across time and across units

• Test for statistical significance

Good:

• What proportion of people last three months, six months, or a year leave?

Better:

Business Analytics: From Data to Insights Page 8 of 12

 • Use multivariate regression to predict who reaches each milestone

Business Analytics: From Data to Insights Page 9 of 12

 Best:

• Use of survival or hazard rate models to test which factors accelerate the risk of exit

Survival Analysis
Questions involved in survival models:

• How long does it take them to die?

• Who survives?
• For how long?

Survival analysis tries to fit this kind of time function:

• What is the likelihood of their leaving?

• How does that change?

Ordinary Least Squares (OLS) Regression

The Regression Equation

Y = a + b1X1 + b2X2 + b3X3 + b4X4 + … + e

where,

• Y is outcome variable
• Xi are predictor variables
• a is the intercept or constant
• bi is the coefficient or slope
• e is the random error

OLS estimates values of a and b that minimize the sum of (Ypredicted - Yactual)2

Correlations Among Variables

OLS works well when the predictor variables have low correlations with one another. As the
correlations become higher, co-linearity problems may occur, leading to:

• Less precise estimates because it’s not clear which of the two similar variables create the
outcomes

• Estimates becoming more influenced by a few observations

There is no clear threshold for co-linearity problems, but correlations of 0.5-0.7 are used as a general
rule depending on the size of the data set and variables. If there are highly correlated variables,
consider dropping some of them.

X and Y Variables for Predicting Turnover

In a table with data predicting turnover, data about bachelor’s degree, prior experience,
organizational tenure, department, time in job, performance rating, manager performance rating,
pay, applied to internal job last year, becomes the X variables, whereas column with quit this year?
becomes the Y variable.

Business Analytics: From Data to Insights Page 10 of 12

 Only numerical data can be used for regression analysis. For textual data, identify the indicator
variable that is closely related to attrition and use that to convert textual data into numerical data.

For example, customer representatives serving the financial industry may be more likely to quit, hence
we replace the column about department with identifying person being in finance department.

Running the Regression

• Click on the Data Analysis button under the Data tab in Excel.

• Choose Regression from the list of Analysis Tools.

• Select column “Quit this Year” as an input for Y Range.

• Select columns from “Bachelor's Degree” to “Applied Internal Job Last Year” as an input for X
Range.

• Fill out the details and click OK.

• Regression analysis results are added in a new sheet.

Diagnostics

Here is a snapshot of the regression statistics:

Regression Statistics
Multiple R 0.613481549
R Square 0.376359611
Adjusted R Square 0.336268443
Standard Error 0.39930447
Observations 150

• Multiple R is the correlation between the predicted and true values of Y.

• R Square (square of multiple R) is a percentage of the variance in the outcome variable that
can be explained by a predictor variable. It takes a value between 0 to 1. If R square is 1, the
predictor variables perfectly explain Y.
• Adjusted R Square accounts for the number of predictors used in the analysis.
• Standard Error explains the noise in the estimates.

Interpretation

Here is the regression analysis summary:

Coefficients Standard Error t Stat P-value

Intercept 3.182319394 0.680341657 4.6775313 6.76839E-06
Bachelor's degree 0.048670393 0.06848225 0.710700847 0.478451929
Prior Experience -0.005116514 0.005934675 -0.862138869 0.390085452
Organizational Tenure -0.017146831 0.007162267 -2.394050822 0.017988111
Financial 0.142877069 0.094769434 1.507628179 0.133902983
Time in Job 0.04623104 0.013164087 3.511906341 0.000599248
Performance Rating -0.046859639 0.035512048 -1.319542007 0.189141809

Business Analytics: From Data to Insights Page 11 of 12

 Manager Performance rating 0.043250763 0.033248157 1.300846945 0.195447888
Pay -0.000149395 3.19252E-05 -4.679534273 6.71149E-06
Applied to Internal Job Last
Year? 0.497512908 0.072495065 6.862714163 2.0104E-10
In the table above,

• The first column represents X variables.

• Intercept is the constant a in the regression equation.
• Coefficients represent the variable b in the regression equation.
• Standard error is possibility of difference between true coefficient and derived coefficient
• p-value is probability of the coefficient value being different from zero

Interpretation based on the regression summary:

• An employee with a Bachelor’s Degree (coefficient of 0.05) has 5% higher probability of

quitting.
• An employee with more experience (coefficient of -0.005), i.e. for every additional year
probability of quitting falls by 0.5%.
• p-value for Bachelor’s Degree is ~0.478, which explains that there is a 48% chance that the
Bachelor’s Degree is different from zero, hence we have less confidence of considering
Bachelor’s Degree as predictor of turnover.
• According to the thumb rule, variables having p-value less than 0.05 are usually considered
as statistically significant. In this case it would be Organizational Tenure, Time in Job, Pay,
Applied to Internal Job Last Year?

Reasons for turnover based on perception

• Experience
• Tenure
• Performance
• Pay scale
• Internal movements

Reasons for turnover based on regression

• Tenure
• Time in job
• Pay scale
• Internal movements

Regression enables us to:

• Quantify the relationship between variables

• Understand the association between data
• Test ideas

Business Analytics: From Data to Insights Page 12 of 12