CORPORATE EDUCATION AND TRAINING

Chapter 6: Training Evaluation

Tong Qian Ying, IES17613

INDEX

01 Introduction

02 Outcomes used in Evaluating Training Programs

03 Determining appropriateness
Chapter 6: Training Evaluation
Introduction

TERMINOLOGY

Training Effectiveness
Benefits that the company and the trainees experience as a result of training. Trainee Benefits include
learning new knowledge, skills, and behaviors. Potential Company benefits increased sales, quality and
satisfied customers.

Training Outcome
Measures that the trainer and the company use to evaluate training programs.

Training Evaluation
Process of collecting data regarding outcomes needed to determine if training objectives were met.

Evaluation Design
From whom, what, when & how information is collected to determine the effectiveness of the training
program.
Chapter 6: Training Evaluation
Introduction

REASONS FOR EVALUATING TRAINING

As companies made large dollar investments in training and and view training as a strategy to be
successful, they expect the outcomes/benefits to be measurable.
Carry out

Training Evaluation
provides the data needed to demonstrate that training does provide benefits to the company.

F o r m a t i v e E v a l ua t i o n Summative Evaluation

Conducted to improve training process: Ensure that training is Conducted to determine the extent to
well-organized, run smoothly and that trainees are learning which trainees have improved or acquired
and satisfied with the training. knowledge, skills attitudes, behaviors, or
Provides information about how to make the program better other outcomes specified in the learning
Training content may be changed to be more accurate, objectives, as a result of the training.
easier to understand, or more appealing. The training
method can be adjusted to improve learning. May also measure the return on
Example: Pilot testing: previewing a training program with investment (ROI) that the company
potential trainees and their managers, or other customers receives from the training program
Chapter 6: Training Evaluation
Introduction

REASONS FOR EVALUATING TRAINING

Formative Summative

Aims of Evaluation

Identifying programs strengths and weaknesses, including whether the program is meeting the learning
objectives, the quality of the learning environment, and if transfer of training back to the job is occurring.

Assessing whether various features of the training context and content contribute to learning and the transfer of
learning back to the job.

To identify which trainees benefited most of least from the program and why.

To gather information, such as trainees testimonials, to use for marketing training programs.

To determine financial benefits and costs of the program.

To compare the costs and benefits of training versus other human resource investments.

To compare the costs and benefits of various training programs in order to choose the most effective programs.
Chapter 6: Training Evaluation
Introduction

OVERVIEW OF EVALUATION PROCESS

Conduct Needs Analysis

Develop Measurable Learning Outcomes

Develop Outcome Measures

Choose Evaluation Strategy

Plan and Execute Evaluation

Chapter 6: Training Evaluation
Outcomes used in evaluating training programs

Kirkpatricks Four-Level Model

suggests training can be evaluated on the following levels:
Level Criteria Focus
4 Results Business results achieved by trainees
3 Behavior Improvement of behavior on the job
2 Learning Acquisition of knowledge, skills attitudes, behaviour

1 Reactions Trainee Satisfaction

Insights Criticisms

Hierarchical nature suggests that higher level Research hasnt found that each level is

outcomes should not be measured unless positive caused by the preceding level, nor does

changes occur in lower level outcomes. evidence suggest that the levels differ in
importance.

Framework implies that changes at a higher level are

more beneficial than changes at a lower level. Does not take into account the purpose of
the evaluation.
Chapter 6: Training Evaluation
Outcomes used in evaluating training programs

Comprehensive Models of Training Criteria

classified into six major categories

Reaction Cognitive Skill-Based Affective Return on

Outcomes Results
Outcomes Outcomes Outcomes Investment

Measures: Measures: Measures: Measures: Measures: Measures:

Learners Techniques Technical skills Attitudes and Payoffs for Comparison of
Satisfaction acquired by acquired Motivation company benefits with costs
learners
Example: Example: Example: Example: Example:
Comfortable Example: Listening Customer Production, $ value of
Training room. Safety Rules Service Quality productivity
Method:
Method: Method: Observations Method: Method: Method:
Surveys Test and Ratings Focus Group Observation Economic Value
Chapter 6: Training Evaluation
Outcomes used in evaluating training programs

Comprehensive Models of Training Criteria

Reaction Outcomes
trainees perceptions of the training experience, including the content, the facilities,
the trainer and the methods of delivery.

These perceptions are typically obtained at the

end of the training session via a questionnaire
completed by trainees, but usually are only
weakly related to learning or transfer. It is
collected at the programs conclusion.

Useful for identifying what trainees thought was

successful or what inhibited learning.

Findings:
Reactions are significantly related to changes in declarative and procedural knowledge.
For courses such as diversity training, reactions are crucial as they affect learners receptivity to attitude change.
Reactions have the strongest relationship with post-training motivation, trainee self-efficacy, and declarative
knowledge when technology is used for instructional delivery for online methods, it is crucial to ensure easy
accessibility for trainees and meaningful training content
Chapter 6: Training Evaluation
Outcomes used in evaluating training programs

Comprehensive Models of Training Criteria

Cognitive Outcomes
demonstrate the extent to which trainees are familiar with information, including principles, facts,
techniques, procedures, and processes, covered in the training program

Measure what knowledge trainees learned in the

program
Typically, pencil-and paper tests are used to
assess cognitive outcomes.
Help to measure whether a trainee knows how to
execute the process he or she would use.
They do not help to determine if the trainee will
actually use skills on the job.
Chapter 6: Training Evaluation
Outcomes used in evaluating training programs

Comprehensive Models of Training Criteria

Skill-Based Outcomes
assess the level of technical or motor skills and behaviors acquired or mastered. This incorporates
both the learning of skills and the application of them (i.e., transfer).

The extent to which trainees have learned skills can be

evaluated by observing their performance in work
samples such as simulators.

Skill transfer is usually determined by observation.

It is recommended that skill or behavior ratings be

collected from multiple perspectives (e.g., managers
and subordinates or peers).
Chapter 6: Training Evaluation
Outcomes used in evaluating training programs

Comprehensive Models of Training Criteria

Affective Outcomes
Affective outcomes include attitudes and motivation

Affective outcomes that might be collected in an evaluation

include tolerance for diversity, motivation to learn, safety
attitudes, and customer service orientation.
Affective outcomes can be measured using surveys.
Chapter 6: Training Evaluation
Outcomes used in evaluating training programs

Comprehensive Models of Training Criteria

Results
Outcomes used to determine the benefits of the training program to the company.

Results determine the training programs payoff for the

company.

Examples include reduced costs related to employee

turnover or accidents, increased production, and
improved quality or customer service.
Chapter 6: Training Evaluation
Outcomes used in evaluating training programs

Comprehensive Models of Training Criteria

Return on Investment
involves comparing the training programs benefits in monetary terms to the programs costs, both
direct and indirect

Cost of the training

Direct costs include salaries and benefits of trainees, trainers, consultants,

and any others involved in the training; program materials and supplies;
equipment and facilities; and travel costs.
Indirect costs include office supplies, facilities, equipment and related
expenses not directly related to the training program; travel and expenses
not billed to one particular program; and training department management
and staff salaries not related to a single program.

Benefits
Benefits are the gains the company receives from the training.
Chapter 6: Training Evaluation
Determining whether outcomes are appropriate

Determining Appropriateness

Criteria The extent to which training outcomes are related to the

Relevance learned capabilities emphasized in the training program.
Criterion contamination - the extent that training
outcomes measure inappropriate capabilities or are
affected by extraneous conditions.
Criterion deficiency - the failure to measure training
outcomes that were emphasized in the training
objectives.
Reliability The degree to which outcomes can be measured
consistently over time.
Discrimination The degree to which trainees performance on the
outcome actually reflects true differences in performance.

Practicality The ease with which the outcome measures can be

collected.
Chapter 6: Training Evaluation
Determining whether outcomes are appropriate

Outcomes
Outcomes Identified by
Outcomes Needs
Measured in Related to
Training Assessment and
Evaluation Included in
Objectives
Training
Objectives

Contamination Relevance Deficiency

Chapter 6: Training Evaluation
Personal Reflection

I believe that training is like making a soup, and participants responses are like food critics review of the soup.
Hence, evaluation is critical to ensure the success of the training program. At the previous company that I had
interned at, the trainer made use of a simple exercise at the end of every training session I by asking the
participants to reflect on what they learned and how they will apply it. Feedback on which training content
helped learning and areas of improvement were also asked. I believe that this is a form of formative evaluation.
In order to encourage both positive and constructive criticisms, the trainer made used of a technique called
Plus/Delta. Examples of questions that the learners were asked to written down before discussing in small groups
are What part of the workshop was most useful to your work? and What really struck you as interesting, new,
provocative, or meaningful during this workshop? Another method is to handout index cards and ask people to
write positives on one side and please change on the other. Also, one can ask people to share it
anonymously. Sometimes participants are asked them to include their names and I do a drawing for a free book
during the learning culmination exercise. After the session, the trainer spend a few hours doing an analysis of what
people wrote on the cards and use it to review with my lesson plan.
Chapter 6: Training Evaluation
Personal Reflection

I believe that documentation is also important as well. The organisation that I had interned at (Sofitel) implements
a process documentation strategy of what and how the trainer will document. Usually, the photo
documentation and social media is used to document training. Also, the trainer collects evidence of learning
such as notes, drawings or any other items that students has created during the workshop. These items are then
shared back with participants as training resources by digitizing it and putting it on a shared platform system. I
believe that participants can play a part in helping photo documentation. However, the trainer will need to
make sure that specific instructions are given on what to photograph and there should be an organised system
for collecting the documents.