EMPLOYEE TRAINING AND DEVELOPMENT:

How To Measure
Effectiveness and Impact

AN EBOOK
Brought to you by:
DELIVERING MEANINGFUL LEARNING RESOURCES TO YOUR EMPLOYEES

The ASTD 2013 State of the Industry report shows the average cost per learning hour
used is $89, and the cost per learning hour available is $1,772.

Both of these metrics increased over the previous year and have been on an
upward trend since 2004. The cost per learning hour used increased by 4% in 2012
compared with 2011, and the cost per learning hour available increased by 20% in
$1,195
the same time period.

Overall spending on employee training in U.S.

COMPANY SIZE (# of Employees)
organizations is $164 billion.
Smaller organizations
typically spend more
per employee than
How can we measure the effectiveness of our training initiatives? larger organizations.

How can we make sure the investment of time and resources is making a positive $700
business impact? $964
$1,800
In this eBook we'll review:
Training metrics - where we've been and where we're going
How to determine goals and key indicators COST PER EMPLOYEE PER YEAR

How to create a measurement plan for your program

BIZLIBRARY.COM
KEY PERFORMANCE INDICATORS AND BENCHMARKS

To get started, here are a few things to do:

How do you currently measure success? Do not worry about creating new metrics. Well spend more time on this in
later pages, but metrics created for the sole purpose of evaluating the effectiveness of training usually dont have
enough history or depth to be meaningful. Use your existing key performance indicators at the organizational AND
employee level.
Use your existing data to set benchmarks. Its impossible to know how much progress you will make unless you know
clearly where to start. You have to make some smart choices, but decide which key business areas your employee
training will target, and get benchmark data to understand the current level.

HR and Training professionals can help managers and supervisors in developing effective performance
goals by taking these steps:

1. Designing the goal-setting process.

2. Developing forms to be used in connection with goal-setting meetings.
3. Providing education and training about the following:
The importance of clear and measurable goals.
The elements of effective written goals.
Ways to create a safe and productive environment for goal-setting.
Ways to engage in goal-setting dialogue with employees.
4. Establishing mechanisms to ensure that managers set, discuss, and document goals with employees
early in the appraisal cycle.
SOURCE: SHRM 2013, Developing Individual Performance Goals

BIZLIBRARY.COM
MEASUREMENT AND BUSINESS

In the 1950s, two great business thinkers and visionaries articulated some
foundational business principles that remain influential today.

W. Edwards Deming began working on the post war reconstruction of

the Japanese economy based upon persistent use of statistics to seek
continuous improvement. His work was so valued by the Japanese
government that The Deming Prize was inaugurated in 1951, and It's not enough to do your best;
remains the most prestigious business award in Japan. Deming was the
you must know what to do and
foremost thought leader on the power of analytics to drive
improvement, and his work influenced an entire generation of writers then do your best.
and academics all around the world. W. Edwards Deming

Read more about Deming

The leaders who work most
effectively, it seems to me, never
Peter Drucker is another great business thought leader whose work
remains relevant today. Drucker introduced the concept of modern say I. And thats not because
business management to corporate America. In 1954, he wrote one of they have trained themselves not
the greatest books on management ever written The Business of
to say I . They dont think I. They
Management. He first coined the term knowledge worker in 1959
and helped pioneer knowledge work productivity. think we.; they think team. They
understand their job to be to
Drucker was an advocate for rational management based upon
make the team function. They
motivating people and a commitment to continuous learning. Like
Deming, he believed strongly in the power of data and information to accept responsibility and dont
inform management decisions, when the data was rational, gathered sidestep it, but we gets the
well and made sense.
creditThis is what creates trusts,
Read more about Drucker what enables you to get the task
done.
Peter Drucker

BIZLIBRARY.COM
MEASUREMENT AND BUSINESS

In 1959, Donald Kirkpatrick introduced the world to a rational

methodology for measuring the impact and ROI of employee training.
He published four articles called, Techniques for Evaluating Training
Programs. The Kirkpatrick Institute continues Mr. Kirkpatricks work
today, and this four-step method remains the gold standard for most
organizational efforts to measure the impact of employee training
programs 55 years since Kirkpatrick introduced it to the market.
THE KIRKPATRICK PHILLIPS
Read more about Kirkpatrick MODEL

In the 1970s, Dr. Jack Phillips built upon the foundation of work begun LEVEL 5
by Donald Kirkpatrick and added a 5th level to the methodology.
RETURN ON INVESTMENT
Phillips added a rigorous data-driven 5th layer to the technique
designed to illustrate business benefits derived from employee training
efforts. Phillips remains at the head of his ROI Institute and is one of the
LEVEL 4
top thought leaders in this space.
RESULTS business impact
Read more about Phillips

LEVEL 3
And that pretty well sums up the state of leadership on measuring the effectiveness BEHAVIOR - application
of employee training. Not a lot has changed other than refinements to the
Kirkpatrick/Phillips method, in over 40 years.
LEVEL 2
Which leads to an important question. Has the business environment changed in
LEARNING knowledge, skills and
those 40 years? If so, have the changes been profound enough that maybe we
should reexamine how we look at and measure the effectiveness of employee attitudes
training efforts?

LEVEL 1
REACTION participant
satisfaction

BIZLIBRARY.COM
THEN AND NOW

2013 3D PRINTING
THE AVERAGE COST OF A
NEW HOUSE

THE COST OF ONE

GALLON OF GAS
2013: $271,600 FIRST FLOPPY
2013: $3.61 1970
DISK

1970: $0.36

1959: $0.25

SPUTNIK
1970: $23,500 launch of first
1959 manmade
satellite
1959: $12,400
THEN AND NOW

Obviously, the world has changed in the last 40 years. And that begs an important
question. Why havent we changed the way we look at measuring the
effectiveness of employee training?

One of the best things about baseball is that we know how to measure success.
Baseball has been measuring success since the introduction of the box score in the
mid-19th century. Pitchers have earned run average. Hitters have batting average
and on-base percentage. There are a number of ways to measure success on a Its not the 95% that's right that
baseball field. And we can see the effect of training in these very measures, too. If a makes something work; it's the 5%
pitcher learns a new pitch and its a good pitch, well see his ERA improve. If a hitter
learns how to hit a curve ball better, we will see his batting average improve. that's wrong that messes
everything up.
If only corporate employee training were as easy! We train and train but can we
really tell if improvements in performance are the result of the training? The
challenge of isolating the impact of training on improved organizational Will Kaydos,
performance remains one of the most imposing challenges facing the profession Measuring, Managing and
today.
Maximizing Performance.
What can we do? Walk away? Or do we find new ways to approach the
challenge?

We already know that few organizations measure the impact of employee training
at the business impact or more sophisticated levels of the prevailing methodology.
But what if we could find a simpler way to get at the actual business impact of our
employee training efforts?

BIZLIBRARY.COM
RETURN ON INVESTMENT

Beyond the fact that so few organizations get beyond the first two levels of ROI
evaluation (learner reaction, the smile sheets) and learning (usually measured by
an assessment at the end of a course), the vocabulary of ROI focuses almost
exclusively on training inputs. This is clearly true of these first two levels.

Even if organizations move beyond the first two levels, the conventional methods
do little to help us isolate the effects or improvement we can attribute to the What makes measurement so
employee training initiative or program. Even at the top levels, the conventional
potent is its capacity to instigate
approach requires a broad focus on organizational results alone. Such a focus isnt
all negative and is an important element of measuring the effectiveness of informed action to provide the
employee training efforts. Without a stronger foundation than we get from smile opportunity for people to engage
sheets and simple assessments after a class, we still cannot really link improved
business results to the improved performance of employees gained through a in the right behavior at the right
learning program. time.

Isolation Techniques
There are three traditional techniques that can begin to tell you an important story Dean Spitzer, PHD,
about the impact of employee learning and training on organizational Transforming Performance
performance. All three methods have merit and can generate important, Measurement: Rethinking the Way We
meaningful and very useful data. The illustration below helps explain the challenge Measure and Drive Organizational
we are trying to overcome. There might be any number of factors that explain why
Success.
an organization improved performance.

EXTERNAL FACTORS
MANAGER SUPPORT/ATTENTION
TOTAL
IMPROVEMENT INCENTIVE/MOTIVATION
AFTER SYSTEM/PROCEDURE CHANGES
PROGRAM
EFFECT OF
LEARNING PROGRAM LEARNING ON
IMPROVEMENT

BIZLIBRARY.COM
RETURN ON INVESTMENT

External factors such as market conditions or competition may have an impact, however neither of these factors have
anything to do with your employee training. Both factors could influence organizational performance. Management
might be paying more attention to the area you are trying to improve, and this focus might have a positive impact.
You might have systems or process improvements that helped improve outcomes, as well. Any of these factors may
have been as influential on your employee training. Conversely, your employee training may have been the single
most critical factor in the improved performance, but you wont know unless you can isolate its impact from the
impact or influence of other factors.

Trend Line Analysis

This is a great way to get a big picture perspective on how your organization is doing before, during and after training
initiatives. The key to an effective trend line analysis is to examine performance along a long enough time frame to
provide a complete picture of success and gaps both before and after training initiatives start.

Start your trend line analysis at a meaningful time interval before you launch your training effort. In order to do this, you
need to be committed to measuring the business impact of training using existing key performance indicators.

Here is a specific example of how this might work.

Average Days
Lets assume that we are working to reduce the time it takes us to fill job
80 71 69 68 69 70 71 68 vacancies in the organization.
70 65 62
58 55 Training offered An HR recruiting team improved skills on sourcing
60 52
candidates, screening candidates and basic interviewing. The
50 Average organization also offered basic interviewing skills training to all hiring
40 Days managers with openings to fill.
30
20 Linear The training program launched in Q2 of 2012. The objective was to
(Average reduce the average time to fill open positions from the 70 day range to
10
Days) less than 55 (three full business weeks).
0
Q1 2011
Q2 2011
Q3 2011
Q4 2011
Q1 2012
Q2 2012
Q3 2012
Q4 2012
Q1 2013
Q2 2014
Q3 2014
Q4 2014

The trend line illustrates that the time to fill open positions showed a
steady improvement from the poor level of performance before the
organization initiated the training program with the organization hitting its
stated goal six quarters after launching the training program.

BIZLIBRARY.COM
RETURN ON INVESTMENT

While the trend line analysis tells us a good story, what is missing? What we cannot
understand clearly is whether the training program is the sole cause of the improved
performance, or whether there are outside factors that might also be influencing
the improved time to fill vacant positions. But from a high level, this trend line
method is a great starting place. We can gain further value from this method if
weve established an actual BUSINESS value to this improved time to fill vacancies.
For instance, this analysis would be far more powerful if we knew that for each day BUSINESS NEEDS AND
saved in filled a vacancy, we gained a set amount of value, we would be able to
QUANTIFYING QUESTIONS TO ASK:
make a strong business case for the impact of the training program.

NEED: Increase Revenue

Average Days QUESTION: By how much? In which
80 areas?
71 69 71
68 69 70 68
70 65
62
58 NEED: Improved Quality
60 55
52 QUESTION: Improved how?
50

40 NEED: Reduction in Turnover

Average Days
30 QUESTION: What percentage in
Linear (Average Days)
reduction?
20

10 NEED: Improved Morale

0 QUESTION: In what way?

BIZLIBRARY.COM
RETURN ON INVESTMENT Other methods:
Control Group
Participant And Manager Estimation Of Impact. This
Setting up a control group is an excellent way to truly
method can provide some useful information that is
isolate the effects and influence of your training
close to the point of performance. The issue is the
program on improved performance. In our example
reliability of the data generated. Without training
where we are trying to reduce the amount of days
and clear evaluation guidelines, the estimations will
required to fill vacant positions, a control group might
only be as good as the employees ability to
include a team of recruiters and hiring managers who
provide the information.
do not participate in the training.
Senior Management Estimation. This is another
Comparing the performance of this group to the
method that can be filled with data problems due
performance of recruiters and hiring managers who
to bias and simple human error. The more removed
participate would help illustrate exactly how the training
you are from the employees actually affected by
influenced improved performance. This works well,
the training, the less reliable these sort of estimates
because both groups would be influenced by the same
become.
external factors, so the difference in improvement
should be tied pretty directly to the training.
Success Case Method. This method comes from the
book, Telling Trainings Story by Robert O.
The drawback to this method is that you will have some
Brinkerhoff. Assuming that one single person was
population of your workforce that you deliberately hold
the focus of our inquiry, we thought we would really
back from improvement to prove a point. While this
only need to have answers to three pretty simple
may sound good in theory (and it really does), the
and fundamental questions to make the case for a
reality of implementing control groups can be messy.
training success:
You have to be prepared for some management and
communication challenges if you choose this method.
What, if anything, did this person learn that was
new?
Recruiters Recruiters How, if at all, did this person use the new learning
and Hiring and Hiring in some sort of job-specific behavior?
Mangers Mangers Did the usage of the learning help to produce
Group 1 Group 2 any sort of worthwhile outcome?

BIZLIBRARY.COM
REALITY - NOT TO BE IGNORED
In an organizational setting, its more complex. For
One of the challenges training professionals have instance, we know that training programs work best
faced, and generally failed to overcome, in evaluating when managers provide a high level of direct
training effectiveness is isolating the influence of training support both before and after specific events. We
on improved performance. The simple truth is that it may also know that employees learn most of what they
be virtually impossible to ever truly isolate the need to perform successfully outside of formal
influence of training in improved performance. Human learning.
behavior may just be too complex for such simple if a
then b analysis. So should we ignore those factors that great
learning organizations understand and leverage to
Lets return to our baseball analogy improve employee learning? The answer is a
resounding, NO!
One long-held hitting statistic used to measure a hitters
success is runs batted in (the number of base runners The best way to truly measure the effectiveness of
that score as the result of the batters action a hit, training is to identify ALL of the factors that influence
sacrifice fly, etc.). RBIs have long been the benchmark successful employees to reach higher levels of
statistic for current star players like Albert Pujols or David performance.
Ortiz. But RBIs are NOT influenced solely by the
contributions of the hitter. Another consideration should the opposite end of
the performance spectrum those who have
Before a run can be batted in, another player has to be participated in your training program that didnt
on base. So even in the world of baseball where succeed. Compared to the successful participants,
everything really, everything is measured and what factors or influences are missing?
analyzed, its nearly impossible to truly isolate the
performance in many areas of one player from his The objective then becomes finding ways to spread
teammates. the factors that have the greatest positive influence
on successful performance when combined with
training to as many employees as possible.

Training programs work best when managers

provide a high level of direct support both before
and after specific events.
Chicago Tribune Illustration by Rick Tuma

BIZLIBRARY.COM
TRAINING IMPACT Step 2: Create a Model for Success
This step requires you to articulate
Common factors that promote amplified training impact include:
the following:
Manager support before and after training programs start.
Opportunities to apply learning on the job during the course of the
Organizational goals you are
training program.
supporting with employee
Peer support and opportunities to ask and answer questions.
training.
On-demand access to online learning and performance support
resources.
Business unit goals that support
Visible and active senior management involvement in the program.
specific organizational goals
(cascade goals down to
With these factors in mind, you can begin to build a program to prove
business unit or functional areas).
the effectiveness of your total employee development program one
employee at a time using a relatively simple five-step process.
The business unit results you
expect to deliver or impact with
1. Focus and plan the evaluation. improved employee
2. Create an Impact Model that defines potential results and benefits. performance.
3. Design and conduct a survey to gauge overall success versus non-
success rates. The behaviors or actions
4. Conduct in-depth interviews of selected success and non-success employees need to take to
instances. deliver the desired results.
5. Formulate conclusions and recommendations, value, and return-on-
investment. The skills and knowledge
employees need to perform the
actions or effectively engage in
Step 1: Focus and Plan the Evaluation.
desired behaviors at the
Its hard to be successful without a clear plan, and thats required if you
required level of proficiency.
want to be able to prove your employee training is effective. At its core,
your plan should cover four things:
You will need to model success for
each area of your organization for
1. Identify the specific performance areas you seek to improve.
methodology to work, and thats a
2. Engage all of the key stakeholders in identifying performance needs.
key reason you need the
3. Clarify and define what a successful training effort will look like.
engagement of key stakeholders
4. Establish the data points and metrics that will help clarify success.
from Step 1.

BIZLIBRARY.COM
TRAINING IMPACT

Step 3: Design and conduct a survey to gauge overall success versus non-success rates.
There are two important aspects to building an effective survey to develop useful information. You need to ask good
questions, and you need to time the survey well so you can gather information at an appropriate time interval after
the initiation of the training effort.

As for the questions, here is an example of a question that will get you started:

Which statement below best describes your experience since participating in the Performance
Management training?

I learned something new, I have used it, and it has led to some very worthwhile results.

I learned and tried some new things but cant point to any very worthwhile results yet.

While I may have learned something new, I have not been able to use it yet.

I already knew about and was doing the things this training taught.

I dont think I can really use what I learned in the training.

An important point to make here is you do NOT need more than two or three well-crafted questions to get at what
employees learned and applied from the training content and program.

The timing of the survey is likewise important, and you need to plan this carefully based upon the objectives of the
training effort. For some specific skill or knowledge improvement goals, a 90-day interval might be perfect. For the
improvement of more sophisticated management, communications or other soft skills, you might consider an interval
more like six months. Soft skills improvements take longer to demonstrate, and are harder to master. So the way you
will measure the effectiveness of those types of training efforts will by necessity be different.

BIZLIBRARY.COM
TRAINING IMPACT

Step 4: Conduct in-depth interviews with employees whove been BOTH successful
and unsuccessful.
You have to work at both ends of the spectrum so you can begin to understand all
of the factors that influenced these outcomes. The goal of these interviews is to
identify both positive and negative influences. Once youve identified these
influences, you work hard to maximize and enhance the factors that exert a positive
influence on training outcomes. Conversely, you work just as hard to reduce or MANAGER SUPPORT:
eliminate the influence of factors that negative ly influence training results.
Managers can give great insight
Step 5: Develop conclusions, write action plans and implement needed changes. into what skills would make a
As youve built your case for the effectiveness of your training through the difference for the employee, while
information youve gathered and planning in the first four steps, youre now ready
to dive into the data. Here are the pieces of data you are looking to report: also providing coaching and
feedback as the person develops
Specific areas of increased performance
his or her capabilities. Be sure to
The actual VALUE of this increased performance
The costs of the overall effort to deliver this value continue providing guidance and
Recommended improvements to deliver more value coaching throughout the skill
development process.
The example on the following page shows of how this might work using our former
example of a reduction in the time to fill vacant openings.

RECOMMENDED RESOURCE

Course Title:
Developing Others - Connect
every employee's individual
goals to the organization's vision
and strategy

BIZLIBRARY.COM
TRAINING IMPACT

REPORT: VALUE OF TRAINING ON INTERVIEWING SKILLS FOR SALES MANAGERS

Goal:
Reduce time to fill open positions from over 70 days to less than 55.

$435/day $32,000 Value:

The expected value to be gained is
First 90 days. In six months measured by using the average
The average rep closes revenue generated by new sales reps in
two more deals at an their first 90 days of work on a per day
average value of basis, and the added value realized in
$16,000 each when the first 6 months of employment for
they started in 60 days each day we can get them onboard
after position opened. and trained more quickly.

Training Costs: Value Added From Improved Performance:

Human Resources: $45,000 7 reps X 10 days each = 70 days. 70 days saved were worth
Sales: $15,000 $30,450. 4 of 7 reps started in 60 days or less after open position.
Total: $60,000 $128,000 added in new sales.
Impact:
Seven reps were hired with an TOTAL VALUE GAINED:
average reduction of 10 days saved $158,450 on a $60,000 program.
per rep in the six month time frame of
the study.
Other Factors:
Better communication and cooperation between sales managers and HR recruiters meant the
candidates being interviewed fit a success profile for sales reps more clearly than before the program
started. We also learned that social media sourced candidates tended to be more successful in the first
six months than blind applicants.

BIZLIBRARY.COM
TRAINING IMPACT IN CONCLUSION
These reports and results occur when each One of the biggest disconnects we continue to see between
of the steps are followed. learning and development, employee training and the c-suite in
organizations is at the strategic level. We see too many employee
Step One engaged sales managers in the training efforts that dont directly address organizational strategic
training effort and helped establish the initiatives, and as a result these same programs cannot deliver the
goals and metrics used to track progress. type of business results crucial to success for the organization or
employees.
Step Two allowed to us to create a model
for success, so we knew that with better This same disconnect continues to plague most training programs
interviewing skills, our entire HR and sales in efforts to prove the effectiveness of the employee training
management team could move more being delivered.
quickly to identify and hire qualified
candidates more efficiently. It doesnt have to be that way. By shifting our focus away from just
organizational metrics and looking at the success of employees,
Step Three provided information from we can develop a far more accurate picture of the effectiveness
surveys of the participants so we knew of our training efforts.
what worked and didnt work in the
training. We also learned about additional Organizational perspectives and metrics are very helpful. For
factors that helped influence the instance, trend line analysis provides indicators of success, but the
improved performance. traditional methodologies that remain virtually unchanged in over
40 years dont really help us understand what is happening at the
In step four, we learned that two sales employee level. And its at the employee level where we are
managers were not well aligned with HR, trying to improve performance and change behaviors.
and thats where the four reps started that
fell outside the 60-day window for early Its logical, therefore, to evaluate the effectiveness of our training
sales success. These managers were also programs by focusing our efforts on the very segment of our
not active in social media and did not organization who benefits and participates most directly in our
value candidates sourced from social programs.
media. Weve taken corrective steps to
ensure better alignment and more
effective use of social media for sourcing
candidates.

BIZLIBRARY.COM
5,000+ Courses. 25 Topic Areas.
Unlimited Access.
Improve your employees' performance with the
largest and fastest-growing library of on-demand
training videos and eLearning courses today!

For a free 30-day trial of BizLibrary, click here.

http://www.bizlibrary.com/free-trial

BIZLIBRARY.COM