REPRINT H036ON

PUBLISHED ON HBR.ORG
OCTOBER 17, 2016

ARTICLE
ASSESSING PERFORMANCE
How to Hire with
Algorithms
by Oren Danieli, Andrew Hillis and Michael Luca

This document is authorized for use only in Yasmeen Makarem's MHRM 301_Fall 2022_Yasmeen Makarem at American University of Beirut from Aug 2022 to Feb 2023.
 ASSESSING PERFORMANCE

How to Hire with

Algorithms
by Oren Danieli, Andrew Hillis and Michael Luca
OCTOBER 17, 2016

HBR STAFF

Choosing the right person for a job can be challenging. The sheer number of resumes can be
overwhelming. But even for organizations patient enough to review each application, poor choices
can arise from psychological biases ranging from racial discrimination to narrow bracketing (in which
people overemphasize subsets — rather than the universe — of choices, for example, choosing the
best candidate interviewed that day rather than the best candidate interviewed over the course of the
search). For tech-savvy organizations, recent applications of machine learning coupled with
increased access to data raise the possibility of improving hiring decisions with the help of
algorithms.

COPYRIGHT © 2016 HARVARD BUSINESS SCHOOL PUBLISHING CORPORATION. ALL RIGHTS RESERVED. 2

This document is authorized for use only in Yasmeen Makarem's MHRM 301_Fall 2022_Yasmeen Makarem at American University of Beirut from Aug 2022 to Feb 2023.
 To see the close relationship between algorithms and hiring, consider the simple fact that hiring is
essentially a prediction problem. When a manager reads through resumes of job applicants, she is
implicitly trying to predict which applicants will perform well in the job and which won’t. Sales
organizations are trying to predict which sales associates will successfully close deals. School districts
are trying to predict which teachers will bring a classroom to life. Police departments are trying to
predict which officers will keep a neighborhood safe.

Statistical algorithms are built for prediction problems. They can be helpful in improving human
decision-making in contexts ranging from judges gauging who will skip bail to health inspectors
identifying restaurants with health code violations. Similarly, algorithms have the potential to
improve hiring and promotion decisions in areas ranging from sales teams to teachers to police.

We explored that potential in a recent study on selecting teachers and policemen. We used machine
learning algorithms to transform data about teacher and police characteristics – for example,
educational background, surveys, and test performance – into predictions about their likely
performance in the future. Our results demonstrate that students and communities alike could
benefit from a more data-driven selection process. Algorithms can help with some of the nation’s
most challenging personnel issues. For example, the data suggest that police departments can
predict, at the time of hire, which officers are most likely to be involved in a shooting or accused of
abuse.

Of course such an exercise creates plenty of controversy in organizations entrenched in existing

practices. Skeptics will rightly want to know more about the sort of data that are used. Our point is
not that any one algorithm will yield the perfect answer and replace all human interaction in the
hiring process. Our point is much simpler – algorithms are an important decision aid for managers
looking to make hiring and promotion decisions. Algorithms have strengths and limitations, and
need to be managed.

The question is not whether to use algorithms for hiring, but how to get the most out of them. That
is, what sorts of decision rules should be used to select the candidate most likely to succeed? For
organizations interested in the promise of data, we offer five principles for using statistical
algorithms to aid the personnel selection process:

1. Pick the right performance metric. Algorithms are ruthless in pursuing the objective you give
them – they will optimize for that, and nothing else. This means you need to be very clear about
how you define success. Often the right metric will be a combination of characteristics. For
example, a manager hiring a salesperson may wish to balance their likelihood of turnover,
projected close rate, and impact on relationships with clients.

COPYRIGHT © 2016 HARVARD BUSINESS SCHOOL PUBLISHING CORPORATION. ALL RIGHTS RESERVED. 3

This document is authorized for use only in Yasmeen Makarem's MHRM 301_Fall 2022_Yasmeen Makarem at American University of Beirut from Aug 2022 to Feb 2023.
 2. Collect the right variables. Organizations and applicants alike often use heuristics to determine
what characteristics of a resume matter most (College GPA? Previous Job Title? Other Interests?).
Effective algorithms require human intuition, experimentation, and iteration to decide which
characteristics to measure about an applicant to help predict the performance metric you care
about.

3. Gather many data points. After each person is hired, track his performance and keep records of his
application data. Algorithms will use these data points to help guide future hiring. But algorithms
are also data hungry: the more data points you keep, the better the prediction will be. There is a
competitive advantage to scale here. Companies with more employees can learn more.

4. Compare apples to apples. A common mistake when measuring previous performance is to

overlook differences in the difficulty of tasks assigned to different employees. For example, if the
best salesperson took on the hardest clients, they might have the lowest closing rates. The right
performance metric will need to adjust for the underlying difficulty of the task.

5. Anticipate incentives. Applicants or employees may be incentivized to “game” the metric by

acting in ways that superficially increase the score, but do not accurately reflect their talent.
Salespeople incentivized to close deals at any costs may score well on a performance metric, but
provide little of value to the company. Anticipating this will allow you to create sufficiently broad
metrics, and to take strategic behavior into account.

While there are limits to statistical algorithms, there are also limitations to human judgment. Within
larger organizations, it’s hard to imagine a world in which using both human and machine
intelligence for hiring is not the preferred route. In other words, when trying to choose the right
person for the job, it’s best to draw on the strengths of both.

Oren Danieli is a PhD candidate in the Business Economics program at Harvard Business School.

Andrew Hillis is a PhD student in Business Economics, a joint program through Harvard’s Department of Economics and
the Business School.

Michael Luca is the Lee J. Styslinger III Associate Professor of Business Administration at Harvard Business School and a
coauthor (with Max H. Bazerman) of The Power of Experiments: Decision Making in a Data-Driven World (forthcoming
from MIT Press).

COPYRIGHT © 2016 HARVARD BUSINESS SCHOOL PUBLISHING CORPORATION. ALL RIGHTS RESERVED. 4

This document is authorized for use only in Yasmeen Makarem's MHRM 301_Fall 2022_Yasmeen Makarem at American University of Beirut from Aug 2022 to Feb 2023.