Implementation of Lessons Learned Programs

Prepared by the Construction Industry Institute

Effective Management Practices and Technologies for Lessons Learned Programs Research Team

Implementation Resource 230-2

July 2007
© 2007 Construction Industry Institute™.

The University of Texas at Austin.

CII members may reproduce and distribute this work internally in any medium at no cost
to internal recipients. CII members are permitted to revise and adapt this work for their
internal use provided an informational copy is furnished to CII.

Available to non-members by purchase; however, no copies may be made or distributed

and no modifications made without prior written permission from CII. Contact CII at
http://construction-institute.org/catalog.htm to purchase copies. Volume discounts may
be available.

All CII members, current students, and faculty at a college or university are eligible to
purchase CII products at member prices. Faculty and students at a college or university
may reproduce and distribute this work without modification for educational use.

Printed in the United States of America.

Contents

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2. Self-Assessment Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3. Jump Start Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4. Transactional Work Flow Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5. Path Forward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

6. Special Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Appendix A: Key Characteristics of an LLP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Appendix B: Maturity Model Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Appendix C: Self-Assessment Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Appendix D: Jump Start Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Appendix E: Checklist of Typical Data Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Appendix F: Transactional Work Flow Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

iii
Executive Summary

Organizations in the engineering and construction industry cannot afford to make repetitive
mistakes on major projects. Conversely, great benefits come from repeating positive project
experiences. This need for institutional memory is amplified by the reality that in the course of normal
turnover and retirement, people with years of experience leave their organizations.

An effective lessons learned program is a critical element in the management of institutional

knowledge; it will facilitate the continuous improvement of processes and procedures and provide a
direct advantage in an increasingly competitive industry. For organizations that have already developed
a lessons learned program, this publication provides a Maturity Model Matrix and Self-Assessment
Questionnaire to determine its effectiveness with respect to seven key program characteristics. For
organizations without a current program or those wanting to re-examine their existing one, it offers a
Jump Start Guide with recommended steps for the development of a program. In addition, a sample
Transactional Work Flow Diagram is provided and serves as a roadmap for typical lessons learned
transactions. Together, the tools form a framework for organizations to integrate a lessons learned
program into their current work processes. When organizations are able to transfer knowledge
through a lessons learned program, they can increase project efficiency—an important capacity in
the fast-paced engineering and construction industry.

v
Chapter 1: Introduction

Organizations that implement effective lessons learned programs (LLPs) as part of their larger
knowledge management systems demonstrate a commitment to continuously improving their
service to stakeholders and customers (Figure 1). Competencies, skills, and experience acquired by
members of project teams must remain within an organization, even as employees move on or retire.
An effective lessons learned program allows organizations to document project experiences so an
organization as a whole can learn from one person’s experiences.

Best Practices

Career Ladders Work Processes

W

Experience Champion T
Training
KNOWLEDGE
MANAGEMENT
Document Repositories Mentoring

Communities of Practice Knowledge Maps

Lessons Learned

Figure 1. Lessons Learned Context Diagram

Definition of a Lesson Learned

A Lesson Learned (LL) is defined as knowledge gained from experience, successful or otherwise,
for the purpose of improving future performance. Examples include:

• A lesson that is incorporated into a work process.

• A tip to enhance future performance.
• A solution to a problem or a corrective action.
• A lesson that is incorporated into a policy or a guideline.
• An adverse situation to avoid.

Some lessons learned should be incorporated into organizations’ work processes, while others
should be shared to improve organizational performance. It is important to develop a definition of
lessons learned to fit the needs of each organization. Regardless of the definition used, lessons
learned should be used to help an organization achieve its business needs and goals.

1
 A lessons learned program fosters the accumulation of wisdom within an organization. Many
individuals associate lessons learned with mistakes that produced undesirable results in the past.
Naturally, organizations want to avoid repeating such missteps. However, it is equally important for
organizations to recognize lessons learned derived from positive experiences. These are the practices
that organizations should promote in order to repeat past successes.

Definition of Lessons Learned Program

A lessons learned program (LLP) involves the people, processes, and tools that support an
organization’s collection, analysis, and implementation of validated lessons learned. People are the
source of organizational knowledge, and their support and involvement determine the program’s
success. Processes within the organization must be structured to allow people to easily collect,
analyze, and share knowledge. Finally, program tools must effectively allow knowledge transfer
between individuals. Organizations must strive to focus equally on all three components to properly
balance their program.

The Lessons Learned Process

The lessons learned process includes three key steps: collection, analysis, and implementation.
These actions are illustrated in Figure 2.

Employees/
Communities Co
of Practice Identify
on
t ati Document
lle
ct
en Publish Reuse ion
m

Change Submit
ple

W
Work
Im

Communicate Process
Implementation

LL Organizational
Gatekeeper
Repository Improvement

Screen
Cost/
Benefit
An

Review
is
lys
V
Validate
aly

n a
sis

T
Technical
Review Subject Matter A
Expert

Figure 2. Lessons Learned Flow Diagram

2
 Collection entails gathering data and generating information on the experiences of individuals
and teams in the organization. Individuals may submit lessons by electronic means or by sharing
them in formalized workshops. Collection can be conducted at various stages of project execution
by different members of the organization; however, after lessons are collected, they must be analyzed
and validated before they are disseminated through the organization.

Analysis can be performed by a team or by one nominated individual who is regarded as a

subject matter expert (SME). This crucial portion of the lessons learned process guarantees that the
information shared throughout the organization is relevant, correct, and easily understood.

Finally, the lessons must be implemented. Lessons can only help an organization if they are put
directly into action. Implementation can take many forms, ranging from publication of lessons in an
electronic database to changes in practices and procedures that are derived from lessons learned.

Key Characteristics of a Lessons Learned Program

Seven key characteristics, described in detail in Appendix A, affect the success of lessons
learned programs:

• Leadership
• Lesson Collection
• Lesson Analysis
• Lesson Implementation
• Resources
• Maintenance and Improvement
• Culture

Each of the seven characteristics can be assessed in terms of four levels of maturity within an
organization. Each maturity level characterizes how developed an organization’s lessons learned
program is for each of the bulleted items above. The higher the maturity level, the more fully the lessons
learned program has been embedded into the organization’s culture and daily work processes. For
example, Level 1 maturity is the lowest rating and indicates an immature program characteristic, and
Level 4 is the highest rating, indicative of a mature characteristic.

3
Chapter 2: Self-Assessment Process

The first step in analyzing or developing an effective lessons learned program is self-assessment
based on the seven key characteristics. The Maturity Model Matrix functions as a high level self-
grading system, and the Self-Assessment Questionnaire rates an organization’s maturity based on
pre-defined questions.

Maturity Model Matrix

The Maturity Model Matrix (Appendix B) is designed to enable an organization to assess the
level of maturity or the stage of development of their lessons learned program categorically. The
Maturity Model Matrix examines the seven key program characteristics that are fundamental to
the successful implementation of an effective lessons learned program. A simplified version of the
Maturity Model Matrix for lessons learned is shown in Figure 3. In the full matrix, each cell of the chart
includes a description of how the maturity of each characteristic is defined at each level.

MATURITY LEVELS
CHARACTERISTICS
Level 1 Level 2 Level 3 Level 4
Leadership
Lesson Collection
Lesson Analysis
Lesson Implementation
Resources
Maintenance and Improvement
Culture

Figure 3. Lessons Learned Maturity Model Configuration

The Maturity Model Matrix provides a means for an organization to quickly assess its lessons
learned program’s maturity by aligning the program’s characteristics to those described in the
model matrix. To accomplish this, an organization only needs to read the criteria of each level for
each characteristic and rate their organization accordingly. After completing this task for all seven
characteristics, the organization gets an overarching view of its lessons learned program. Assessment
using the Maturity Model Matrix provides a subjective evaluation of an organization’s LLP.

Self-Assessment Questionnaire
In addition to the subjective assessment elicited by the descriptions in the maturity model, an
organization can use the Self-Assessment Questionnaire and accompanying score sheet found in
Appendix C to determine quantitatively the maturity level of each characteristic of the lessons learned
program. The responses to the questionnaire, when scored and averaged, can be categorized into
one of the four maturity model levels. For example, consider the following hypothetical answers to
the culture portion of the Self-Assessment Questionnaire, as shown in Figure 4 (next page).

5
 CULTURE
32. Individuals participate in your LL program because they understand the value of the system.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☒ Agree d) ☐ Strongly Agree

33. Communities of practice encourage their members to use your LL program to avoid/solve
project problems or enhance performance.
a) ☐ Strongly Disagree b) ☒ Disagree c) ☐ Agree d) ☐ Strongly Agree

34. The Lessons learned process is an ingrained part of day-to-day activities for all individuals.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☒ Strongly Agree

Figure 4. Example Answers to Culture Questions for Self-Assessment

Once all questions are answered, the responses are assigned a numeric value, as shown in
Figure 5.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

1 pt. 2 pts. 3 pts. 4 pts.

Figure 5. Numerical Scores for Self Assessment

After the assessor has scored each question, each characteristic score can be calculated as the
mean of the numerical scores in the category, as shown in Figure 6. For instance, since the culture
score is equal to the mean of the numerical score generated in questions 32 through 34, the culture
score equals 3.0. That is:
3+2+4
= 3.0
3

Question # Response Points

32. c 3
33. b 2
34. d 4
Total Sum: 9
LLP Culture Score (divide sum
3.0
by # of questions in category):

Figure 6. Numerical Scores for Self Assessment Example

An organization can calculate a score for each characteristic in the same manner. The assessor
can then determine each category’s corresponding level of maturity using the calculated categorical
score and the levels provided in Table 1. The maturity levels are divided into four, evenly-split numerical
ranges as seen in Table 1.

6
 Table 1. Numerical Ranges for Evenly Split Maturity Levels

Maturity Numerical Range

Level 1 1.00–1.74
Level 2 1.75–2.49
Level 3 2.5–3.24
Level 4 3.25–4.00

When using this method to determine the numerical range of each level, the example answers
would indicate that the culture characteristic is at a Level 3 in the organization.

What the Results Mean

Table 2 illustrates a research sample. More than half of the organizations in the research study
were assessed at Level 2 for the leadership characteristic of their lessons learned program. For an
organization aiming to be a top performer, this table can be used to benchmark an organization and
focus in on areas that need improvement. See Chapter 6 for a post-assessment process for program
levels desired and the steps necessary to reach those levels.

Table 2. Current State of Surveyed Organizations

Percentage of Organizations per Maturity Level

Characteristic
Level 1 Level 2 Level 3 Level 4
Leadership 0% 56% 33% 11%
Lesson Collection 3% 33% 36% 28%
Lesson Analysis 8% 36% 31% 25%
Lesson Implementation 3% 31% 44% 22%
Resources 11% 47% 34% 8%
Maintenance and Improvement 25% 39% 19% 17%
Culture 8% 36% 42% 14%

7
Chapter 3: Jump Start Guide

The Jump Start Guide is a tool developed for organizations that want to start up an LLP or re-
examine the approach and framework of their existing system. The guide consists of step-by-step
recommendations and actions to help an organization develop, implement, and maintain a lessons
learned program. A portion of the Jump Start Guide is given in Figure 7. It appears in its entirety in
Appendix D.

Step Description Action Item

• Review Transactional Work Flow Diagram
7 Develop a process flow map • Adopt or adjust to meet the organization’s
needs
Agree on a roll-out / • Roll out and deploy in accordance with the
8
deployment plan organization’s change management process
• Prepare value proposition or business case
Garner buy-in from the for the use of an LLP
9
organization • Incorporate value proposition into roll-out /
deployment plan
• Identify local organizational entity for LLP
Develop & implement a pilot test
10
testing plan
• Develop pilot test plan

Figure 7. Excerpt from Jump Start Guide

The descriptions of the steps in the Jump Start Guide are complemented by a series of typical
action items, which should be considered at each step; note that these lists are not exhaustive and
that an organization using the guide should consider all aspects involved in each step. For example,
Step 7: Develop a process flow map will have many action items associated with it that are not listed
in the Jump Start Guide. Typically, the design of the process has many options associated with it.
One of the main decisions that must be made is: At what point in time does the organization think it
is better to collect lessons? Progressively as they happen or at pre-defined stages of projects; or in
fact, both? To answer this and numerous other questions, this Implementation Resource should be
used as a tool to determine what best suits the organization (in the manner in which it operates, with
the available resources) for the development and proper functioning of the lessons learned program.
The most important thing to remember is to use the Jump Start Guide correctly for the organization
concerned. It is also important to realize that many of the action items in the Jump Start Guide relate
to the leadership characteristic.

The key to achieving an effective and sustainable lessons learned program is the degree of
continuing commitment and leadership from the organization’s top management. Any initiative to
establish a program will ultimately fail without top level direction and support. The process and
mechanics of lessons learned programs are secondary to the critical issue of leadership. Consequently,
many of the steps in the Jump Start Guide relate directly to the leadership characteristic of the

9
program.

Strong and lasting leadership will eventually help drive the culture within the organization to
sustain the program and ensure that it is maintained and improved over time. Steps 1 through 6 must
not be ignored. Serious and earnest effort during this initial phase will ensure that the organization
gets off on the right track and stays on it.

10
Chapter 4: Transactional Work Flow Diagram

A typical transactional process diagram is offered as a roadmap highlighting the basic lesson-
related tasks performed in a lessons learned program. The Transactional Work Flow Diagram
(see Appendix F) is divided into three phases: Collection, Analysis, and Implementation (as applied
to the processing of individual lessons). Maintenance and Improvement is a recurring activity that is
initiated after the three initial work phases. Activity functions involved in the work flow process are
identified and displayed vertically along the left edge of the diagram, forming “swim lanes” as they
relate to and “swim” across each phase. The activity function’s roles and responsibilities include:

• Contributors – any individual, body of individuals, community of practice (COP), or process

that provides potential lessons learned.
• Gatekeepers – personnel who perform a screening function to verify that the lesson input
satisfies the minimum requirements for admission into the lessons learned process.
• Evaluators/Legal/Subject Matter Experts (SMEs) – acknowledged experts representing
the collective wisdom of the organization who are engaged as needed. Their role is first to
apply specific knowledge and guidance to the process to ensure that the screened lesson
meets the LLP criteria. They also ensure that any legal implications are considered and the
lesson input is applicable to a work process or a lesson to share. Root cause analysis may
also be assigned to the SMEs.
• Implementers – individuals who make certain that input provided for incorporation into
a work process is designed and crafted appropriately; also may share and provide any
necessary training.
• Process Owners – persons responsible for approving new or revised work processes
within his or her domain; also responsible for periodically assessing the effectiveness of the
work processes.
• Lessons Learned Champions – appointed personnel who monitor the effectiveness of the
overall LLP, including the lessons learned process and the value it brings to the organization.
• Database Management – professionals who provide IT knowledge and resources to review
the entire lessons learned database, remove lessons that are no longer applicable, and
revise the technology behind the program to meet the organization’s requirements.

The transactional lessons learned process, as it progresses through the three phases (Collection,
Analysis, and Implementation) as well as Maintenance and Improvement, are defined as follows:

• Lesson Collection – Lessons can come from many different sources. Some input may
come from project review post-mortem evaluations; other data may be derived from
corrective action processes; and another source may be in the form of ad hoc suggestions
by anyone inside or outside of the organization. Appointing an employee in a “gatekeeper”
role is recommended to ensure consistent initial review of all lessons, including judgment of
the input’s value to the organization. Input that passes the gatekeeper’s initial evaluation is
then loaded into the lessons learned database to facilitate further efforts in the process.
• Lesson Analysis – Lesson input deemed worthy of further evaluation is assessed on its
overall impact to the organization including legal implications. Input evaluated as having a
significant impact may dictate its formal incorporation into a work process. Input of lesser
impact that is nevertheless useful could be treated as a lesson to share with others.

11
• Lesson Implementation – Having been streamed as either a work process change or
a lesson to share, the input is then crafted into a new or existing work process or into a
lesson to share. Approval by the affected process owner should be obtained at this point.
Having received the requisite approval, the work process or lesson to share is finalized and
entered into the lessons learned database. It is then released for incorporation within the
organization and any necessary training for the new process is established. Additionally,
giving appropriate recognition to the parties who provided the input will encourage future
input submission.
• Maintenance and Improvement – The data contained within the lessons learned database
should be reviewed periodically for continued applicability. Data that requires updating
should re-enter the process at the Implementation phase for revising. The organization
must decide if purging obsolete data is desired or if it should be retained, archived, or
re-classified to a “sunset” status. Lessons learned champions will periodically review the
effectiveness of the overall program and make adjustments as needed.

12
Chapter 5: Path Forward

An assessment provides information on the level of maturity for each characteristic of an

organization’s LLP, and indicates the relative maturity of the program in relation to similar organizations
in the industry. Armed with this information, senior management must determine where they want
their program to be and identify the steps necessary to reach their vision. Some questions that might
be asked include the following:

• Is improvement needed? If the organization’s program is at a Level 3 or 4 in all aspects of

the maturity model, perhaps no changes are necessary. It is possible that the organization
is satisfied with the quantity and quality of the feedback from the existing process; in such
cases, the organization may not feel that incremental improvements will justify the increases
in resources that are required.
• What characteristics of the process require improvement? The maturity model is especially
helpful in highlighting the areas where improvement is possible.
• Which characteristics of the process, if improved, will provide the greatest benefit to the
organization? Organizations have finite resources and must apply them judiciously. Senior
managers may find that the most important resources for improving the process are
leadership and attention, resources that only they can provide.
• How will the improvement be implemented? The Jump Start Guide is especially helpful
here, regardless of the maturity level of the LLP. Allocation of resources and responsibilities,
a milestone schedule, and periodic senior management reviews to assess progress and
provide additional guidance are essential to ensuring successful implementation.
• How long will it take to see the results? Managers and program champions must realistically
assess anticipated results, accurately define criteria for success, and develop useful
metrics. A change in the process may make the program more efficient but may not result in
a highly visible payoff.

Results can be viewed in two ways: 1) the level of usage and input to the system (actions and
behaviors of the users); or 2) the effects of lesson implementation on the performance and reliability
of the process (quantified benefits to the organization). However, both require some method of
reliable measurement. Generally, few methods are available for directly measuring the value of a
lesson to a process improvement. However, some already established measurement systems like key
performance indicators and cost of quality may be useful. System or process metrics, conversely,
are widely used—albeit to varying degrees—within the industry and are intuitively much easier to
design and construct.

Ways to Improve
While it is natural to want to improve a lessons learned program, making improvements is not
a simple process. The Maturity Model Matrix and the Transactional Work Flow Diagram assist by
identifying the characteristics of successful programs. The Jump Start Guide can systematically
isolate weak areas in the implementation process.

The Maturity Model Matrix describes how each characteristic of an LLP might look at the next
level of maturity. Once an organization identifies the maturity level of a particular characteristic of the

13
program and makes a decision to improve, it can refer to the maturity model for guidance on getting
to the next level. For instance, an organization may assess itself as operating at Level 2 in the area of
Analysis. Often, project managers screen lessons learned in an informal way before distributing them
to other members of the organization. To improve in this area, the organization should establish a
formal review process involving subject matter experts. Specific criteria for incorporating a proposed
lesson into the database should first be established. Individuals within the organization should then
be designated to review proposed lessons in a particular subject area. Finally, the changes in the
Analysis process would be publicized by the organization through newsletters and changes in the
LLP documentation.

Level 4 Characteristics and Examples of Level 4 Programs

This section provides insight and details on characteristics from actual Level 4 organizations.
Specific examples of what organizations have done with respect to the development of their
successful lessons learned programs are also included in this section.

Leadership – For the most mature of the organizations surveyed, LLPs were found to be an
integral part of daily work processes. Senior leadership supports the lessons learned program with
appropriate resources and management attention. Leaders advocate the use of the LLP and ensure
that the system is well advertised (e.g., newsletters, posters) and easily accessible to its users (i.e.,
via direct link on the organization’s intranet homepage). Awareness campaigns and presentations
can be successful in promoting programs and educating employees on system use. Motivating users
has been identified as a barrier to successful LLP implementation; however, incentives and reward
systems linked to lessons learned programs can help remedy this issue. Some Level 4 organizations
have made lessons learned a best practice that is required at key stages in project development
processes; others add lessons learned participation as a component of their employees’ annual
performance appraisals. Also, mid-level supervisors in communities of practice (COPs) share senior
management’s vision of the LLP, willingly participating and suggesting improvements.

Lesson Collection – The preferred collection method is overwhelmingly electronic, followed by

meetings or interviews and paper forms. Overall, the most common lesson types collected for Level
4 organizations refer to quality or inspection, construction, procurement, engineering or design, and
safety. Many organizations also collect lessons on technology, controls, and environmental issues.
The person who collects lessons is typically a lessons learned initiator or a lessons learned program
coordinator/team. For most organizations, lessons are collected at the time of occurrence and at the
end of a project. For instance, some organizations collect lessons during a “lookback process,” which
takes place during project closeout. Additionally, some organizations choose to collect lessons near
project completion, for example, at the end of facilitated workshops or during peer reviews.

Lesson Analysis – Lessons are usually analyzed at group meetings or by an SME. Most
organizations do not seek assistance from outside consultants for lessons learned analysis, but
instead delegate the task to an internal team or SME. Often, the internal team is comprised of
people in the same knowledge community as the Lesson Initiator. The types of analyses used for
system validation usually include a study on applicability to new projects and a subjective cost/
benefit analysis. Level 4 organizations tend to favor a continuous analysis approach as lessons are

14
submitted. In some organizations, Lesson Reviewers are responsible for identifying all potential legal
implications of a lesson. A quality check is performed on the lessons and only the most valuable
and applicable are validated. Level 4 organizations also have an ongoing program to review both the
content of the lessons learned database and the LLP itself.

Lesson Implementation – Lessons are typically implemented by means of changes to a work

process or through inclusion in an electronic database. In addition to electronic databases, lessons
are implemented mostly as changes to ongoing projects and at project kickoffs. Most organizations
also identify the entire organization or the Lesson Contributor’s project or organization as the party
responsible for implementing lessons learned. Most organizations implement lessons during project
planning or at another time of their choosing. Some benefits experienced from the implementation of
an LLP are in cost savings, service to customers, application of best practices, safety improvements,
and business process refinement. Methods for implementing lessons learned vary in complexity, but
some Level 4 organizations have found success with implementation workshops held during front-
end planning. Here, users are introduced to the lessons learned system and learn how to review
lessons. For successful implementation, all members of the organization must be trained and given
the time and resources to be able to contribute to the program. Some organizations have identified
the goals of including lessons in company standards or work procedures and ultimately of archiving
the lessons in the system. Some systems automatically archive lessons after a set period of time has
passed. One interesting feature is the classification of lessons by subject matter. If a project manager
or process leader specifically requests lessons learned on a particular work process, the information
can be obtained easily through a query function in an electronic database or through a request to the
database manager.

Resources – Most organizations dedicate full-time personnel during the initial development of a
lessons learned program. After the program achieves a higher maturity level, the management and
participation in the lessons learned program tends to become distributed across the organization.
Nearly all Level 4 organizations use web-based databases to support their lessons learned program.
Some organizations have adopted an e-mail notification system that alerts a user when a lesson
learned in their area(s) of interest has been published. A checklist of the most common data fields
on lessons learned collection forms is available in Appendix E. Lastly, the use of training modules is
common for Level 4 organizations.

Maintenance and Improvement – Even though performance metrics are desirable, most Level
4 organizations only use process metrics for their LLPs. One common process metric employed is hit
counters, which track the popularity of lessons and other components of a lessons learned database.
User comments and ratings are also solicited. Some Level 4 organizations use their lessons learned
system to produce reports that track process information about lessons; this information conveys
how many lessons are in review, how many have been approved, where the lesson originated, and
how many have been incorporated into procedures. These reports can shed light on many issues.
For example, a low rate of lessons submitted from a particular location may indicate issues with
communications, training, leadership, and culture that could require further evaluation. The timeliness
of the process can also be tracked and analyzed. In addition, some organizations hope to develop a
metric for the cost savings resulting from lessons learned implementation. Process metrics should

15
be assessed on an ongoing basis. Even though performance metrics are desired, they are not
common.

Culture – In many organizations, running an LLP requires an ongoing, deliberate effort for the
program to operate consistently at its highest level. Embedding the lessons learned culture globally
in an organization takes a considerable amount of time and resources. Typically, some employees
are more willing and able to play an active role in lessons learned programs than others. The key to
embedding lessons learned throughout an organization lies in the hands of the organization’s leaders.
Organizations that have observed a disparity between a user’s appreciation of the system and the
user’s consistent utilization of that system have made efforts to increase the role of leadership in
promoting the LLP. Overall, successful LLPs consist of individuals who participate in the program as
a matter of course in their daily work processes.

16
Chapter 6: Special Considerations

The approach each organization takes in implementing a lessons learned program is dependent
on the nature of the organization and the environment in which it operates. Whether refining or
expanding an existing program or establishing a new program, organizations should make a number
of special considerations. This chapter formally explores these considerations and also offers
sidebars on real-life applications.

Ownership and Leadership of Lessons Learned Program

Each organization needs to determine the ownership of the program in each phase of its lessons
learned process. The resources dedicated to the program will affect the breadth and depth of the
program and its ownership as well. The following questions need to be answered:

• What is the Lesson Originator’s role in the

The Human Factor — process?
Leadership, Champions,
• Who has responsibility for completing the
Resources, and Rewards
analysis?
➢ Don’t overlook leadership’s role. • Are the lessons learned managed centrally
Without it, the LLP will fail! throughout the process or are they de-centralized
➢ Champions must assume during analysis?
responsibility and authority.
• How are broad, cross-cutting issues addressed?
➢ Adequate resources must be
allocated to implement and • Who has responsibility for implementation?
sustain the program. • What level of quality assurance is applied to the
lessons learned program itself?

Regardless of the working level assignments, without senior managers’ ownership, the lessons
learned program tends to lose effectiveness over time.

Similarly, the organization needs to identify and promote lessons learned champions. Whether
program managers, lessons learned coordinators, or SMEs, these program proponents must have
the support of top management, be self-motivated, have superior technical skills and knowledge,
and be articulate and persuasive. Top management should give the program visibility, support, and
encouragement. Their vision, guidance, and commitment of resources in support of a lessons learned
program are critical to creating the environment for success.

Line of Sight – Communicate, Communicate, Communicate

➢ The key to success in real estate is “location, location, location.” For knowledge
management and information systems it’s “communicate, communicate, communicate.”
LLPs are no exception.
➢ Line of sight must be maintained from lesson collection to success. Don’t let the program
become the organization’s best kept secret. Continually promote the LLP and keep
awareness up.

17
Legal Aspects
Some organizations are reluctant to embrace lessons
Don’t Show Up on the
learned programs due to their uncertain legal impact. It
Five O’Clock News
is unclear if organizations would be held accountable for
lessons learned programs, regardless of their structure,
➢ Don’t underestimate the role
of legal counsel in setting
content, or intent. For example, a program may identify a up an LLP. Stay in close
lesson learned that is not implemented on a subsequent communication!
project; as a result the organization may be held liable for ➢ Make sure sensitive issues get
not responding to an identified lesson. Important legal proper and timely review.
questions include:

• Would the courts react differently if the program were formal rather than informal?
• How would the courts respond to the proactive nature of having a lessons learned program?
• Since many lessons learned programs include documented improvements to business
processes, should they be discouraged due to the uncertainty of the courts’ position on
them?

Each organization needs to ascertain all applicable legal issues and seek ways to overcome or
mitigate these legal concerns. This may take the form of crafting a lessons learned program that
includes specific steps to address these concerns. One technique is to collect a lesson learned using
generic descriptions or hypothetical scenarios, thus removing it from the actual circumstance that
led to its discovery. Another practice is the submission of lessons learned to the legal department
for review and approval. This technique would remove any legal conclusions or admissions against
interest. Another approach is to incorporate a limit on the discoverability of a lessons learned
program into contract actions or other legal documents. However, this approach would not protect
an organization from discovery between non-contracting parties. Ultimately, each organization must
establish its own comfort level and should seek expert legal advice to chart a course for their own
lessons learned program.

IT Resources
No commercial IT applications can manage lessons learned without some form of in-house
support. Most CII organizations that already have a lessons learned program use a searchable, web-
based database (on the internet or an intranet) with some degree of security.

Web-based programs that permit lesson collection without regard to locale provide the greatest
flexibility, but require a considerable commitment to establish, operate, and maintain the database.
If a web-based solution is unworkable, stand-alone applications provide simplicity and may be well
suited to starting a lessons learned program where resources are limited. Regardless, the information
that IT tools make available provides an ideal platform for managing lessons learned programs.

18
 Help Desk – Where Are You? Automated analyses, artificial intelligence,
and other hands-off features are beginning to
➢ Don’t expect to find an off-the-shelf software
that will meet all your needs. find their way into lessons learned processes,
but have not been widely adopted. Similarly,
➢ Invest in time, talent, and treasure to fully
develop and maintain the IT tools for an LLP. subscription service and shopping cart tools
➢ The LLP interface must be easy to access are beginning to be used to distribute and
and use. Leverage the web to provide global retrieve lessons learned for use on upcoming
accessibility to the entire organization. projects.

Corporate Culture
Implementing a lessons learned program can require both
cultural change and cultural awareness. Corporate leadership needs All in the Family
to establish a vision and be able to overcome resistance from those ➢ Culture changes
who are reluctant to embrace change. The apprehensions of people over time. Keep it
who fear that lessons learned programs only highlight past mistakes positive and growing
in the right direction.
or failures should be addressed. Lessons learned programs are
about organizing information for the purposes of improving business ➢ For a positive culture
practices and products. Organizations should take a positive to develop, the
entire organization
approach and accept that “better” may mean fewer mistakes, higher needs line of sight
quality, greater efficiency, or any other permutation where the value from experiences to
of an action or product is improved. lessons to success.

Globalization
For multi-national organizations, cultural awareness
Don’t Get Lost in Translation!
is important. Some cultures are receptive to an open
dialogue on past efforts as a mechanism to support ➢ Be sensitive to cultural
differences within the
future changes. Other cultures take a more reserved, organization. What’s offensive to
personal approach to such discussions. Still others see some may be typical for others.
such discourse as negative or embarrassing. Similarly, ➢ Global organizations are most
the presentation of information, especially in written prone to cultural differences. To
form, can require considerable care to ensure that leverage differences to the LLP’s
advantage, champions should
it is interpreted correctly and consistently across an
represent a cross-section of the
audience with diverse backgrounds, skill sets, education organization.
levels, positions, and languages.

19
Metrics
Most organizations seek to quantify results and are thus
Measure Your Success
interested in the quantification of the benefits of a lessons learned
➢ Measure and program. Process metrics simply document whether an action was
display the
results of your taken and when. These are easily quantifiable measures. Accurate,
LLP to gauge its meaningful performance metrics, however, are difficult to ascertain.
effectiveness. The economic benefits of lessons learned are difficult to identify for
Start by measuring several reasons. First, when a lesson is learned and implemented,
the actions and
behaviors of it is hard to project how far downstream the benefit will extend:
the organization One project? Five projects? Three years? Second, sometimes no
through the LLP economic baseline exists from which to measure the lesson. While
process. most activities can be evaluated and a cost can be developed for a
➢ When the LLP is work activity, some tend to simply make the improvement without
established and quantifying the benefit. People intrinsically know they are saving,
measurable benefits
which is often enough reason to make the change. Therefore,
are realized,
measure benefits to when an organization tries to quantify the overall benefits across a
the organization. lessons learned program, numerous gaps may appear in economic
➢ Don’t keep it a analyses. Organizations are willing to establish and maintain
secret! Prominently lessons learned programs despite the lack of quantifiable economic
display the results data, indicating that most organizations inherently believe lessons
for all to see.
learned programs are worth the effort.

Raising Awareness
If sufficiently advertised, lessons learned programs become a central repository for useful ideas
and information. Facilitating this process, however, requires awareness. Most organizations link
lessons learned databases to homepages to encourage employees to browse the system and use the
lessons. Broadcast messages, newsletters, annual reports, brochures, posters, online forums, and
other media can also be used to launch, expand, operate, and maintain lessons learned programs.
The message can be particularly effective when the promotional medium shows real results, cites
examples, presents metrics, or shows how lessons learned have been implemented in other ways
that may affect the readers personally or organizationally.

Resources
One of the most difficult decisions to make when establishing a lessons learned program involves
determining the level of resources that should be dedicated. Factors such as scale, scope, schedule,
and budget all factor into this decision.

One of the general characteristics of most formal programs is the assignment of a lessons
learned coordinator or some other individual responsible for managing the program. The level of
involvement by others in the organization may vary depending on the nature of the lesson and the
authority level of the lessons learned program participants. Another factor is the breadth and depth
of the program. The number of lessons learned (solicited or offered) and the scale and lifecycle

20
of the lesson should all be considered when establishing the scope of the program and resource
allocations. Finally, maintenance of the program itself needs to be considered. As with any process,
product, or equipment, proper care and maintenance are essential to an LLP’s long-term benefit and
it is important that its preservation is not neglected.

Timing
Organizations differ with respect to when
Is Dinner Ready Yet?
lessons are collected for evaluation. While some
are dynamic (i.e., collected at the time of discovery), ➢ Timing for collecting ideas to feed
the LLP is dictated by the feeding
others use end-of-project or end-of-year milestones system developed. These can
to catalyze a review of the successes and failures range from ad-hoc suggestions to
of the preceding project or year. Each organization formalized project reviews to output
needs to establish its own collection phase timing. from a corrective action process.
The optimum timeframe can be linked to other ➢ Allow a wide variety of input to gain
the most from the organization’s
similar activities, such as project closeout or regional
collective experience.
meetings or conferences.
The earlier the lesson is collected after discovery, the better. For obvious reasons, more
information surrounding the lesson is readily available and can be documented more completely at
that time. Similarly, some organizations continuously process lessons learned while others collect
and process them in a batch mode. Each organization should look for opportunities to incorporate
lessons learned programs that are least disruptive to existing practices.

Maintaining Program/Enthusiasm
If a lessons learned program is seen as effective (i.e., contributors see results), the program
has a greater chance of succeeding. For many participants, this is intuitive, for they see value in the
program on its own merits. However, this may not mean that the process is ingrained personally or
at the corporate level. To get to that point, some internal and external promotion to attract attention
and maintain enthusiasm for the program may be necessary. Incentives can range from personal
recognition to organizational rewards to publicity through outreach tools. Beyond incentives, tangible
results (through process metrics or publicized anecdotes) are likely to motivate individuals within the
organization.

Owners and Contractors

While owners and contractors both benefit from LLPs, differences do exist regarding benefits.
For example, the lessons learned programs for contractors may have to incorporate requirements
from clients on specific projects and they are also more sensitive to the legal issues surrounding
discoverability as discussed previously. Owners see inherent benefits on their future business
operations even without formally or quantifiably justifying their current programs. Also, owners tend
to derive much of the value they realize at the beginning of projects (e.g., during planning), whereas
contractors typically derive more value during the project execution phase.

21
Appendix A: Key Characteristics of an LLP

— The essence of leadership is to create an environment that fosters success within an

organization. Leaders provide vision and guidance by their words and actions. Consistent and
aligned leadership actions and communication throughout an organization are key elements
in creating an environment for success.

— Collection is a fundamental step in any knowledge management process. Systems for

collection can range from simple to complex; however, the important element is that the
system is appropriately suited to the needs of the lessons learned program.

— Analysis is what turns data into information. Analysis provides data consistency and a
method of prioritizing data to provide added value to the organization. Experience is a
valuable factor in analysis.

— Implementation is a crucial part of lessons learned programs. The key to implementation

is to include lessons learned as a required step in an organization’s mainstream business
process. Seamless communication between the lessons learned database and business
processes will greatly facilitate implementation.

— Prudent allocation of required resources (e.g., human, monetary, material, technological,

etc.) is necessary for program success. Resources are the glue that holds the entire program
together. Commitment to the lessons learned program will be reflected in the attention that
the process gets in terms of allocation of resources and the training provided to users.

— Because neither the world nor technology is static, any LLP requires constant attention to
ensure applicability and functionality in the ever-changing global environment. Lessons
learned programs must be maintained and improved to eliminate waste, improve efficiency,
and add value. This tool will fall into disuse if neglected.

— Culture develops when actions and behaviors result in positive outcomes. An organization
must develop a “learning and teaching” culture to embrace and effectively utilize an LLP.
Development of such a culture within an organization is a long-term process, and actions and
behaviors supporting a learning and teaching culture need to be ingrained in the organization.

23
 Appendix B: Maturity Model Matrix
Maturity Levels
Characteristics
Level 1 Level 2 Level 3 Level 4
Upper management has limited Upper management stresses the Upper management strongly supports,
Upper management mentions the use
focus on the use of LL. importance of LL and recognizes the need promotes, and expects the use of the LLP.
of LL. Some communities of practice
Leadership Communities of practice do not
advocate the use of LL, but others do
for a LL program. Most communities of Communities of practice willingly participate
stress the importance of using practice advocate the use of LLs, but a few and advocate the use of LLs and share
not actively participate.
LL. groups do not actively participate. management’s vision of the LLP.
There is a consistent, organized LL collection and submittal is part of the
LL are collected, but methodology is
methodology for collection of LL in some organization’s recognized work processes.
Lesson not consistent or organized. No
Minimal collection of LL. parts of the organization. Some members of There is a consistent, organized
Collection specific member of the organization is
the organization are responsible for LL methodology for collection throughout the
responsible for LL collection.
collection. entire organization.
Most LL are analyzed before being LL are always analyzed by subject matter
Some LL are analyzed before, being
LL are seldom analyzed before communicated and there is a documented, experts before being communicated. There
Lesson communicated, but there is no
they are distributed in the consistent methodology for analysis. is a documented, consistent methodology
Analysis organization.
documented, consistent methodology
Sometimes subject matter experts are used for analysis that all members of the
for analysis.
to analyze LL. organization are familiar with.
A distinct methodology exists to make LL
There is a distinct methodology for storage,
available throughout the organization.
LL are sometimes used by members of retrieval, and use of LL. Implementation
25

Methodology for storage, retrieval, and use

Lesson LL are rarely used by the the organization, but there is no occurs consistently in some communities of
of LL is consistently followed. Lessons are
Implementation organization. consistent methodology to facilitate, practice but is not organization-wide.
stored and shared throughout the
communication and implementation. Lessons are sometimes incorporated into
organization. Lessons are incorporated into
the organization’s way of doing business.
the organization’s way of doing business.
A champion is assigned to maintain and
Adequate resources and personnel are
implement an organization-wide LL
allocated to effectively sustain and
Some resources are allocated and program. All individuals are trained in how to
Minimal resources and/or coordinate the LL program. Individuals are
some personnel are assigned, but use the program and are given the time and
Resources personnel are associated with LL
resources are inadequate to effectively
given training and adequate time to
resources needed to use and contribute to
efforts. participate in the LL program. Some
sustain a coordinated LL program. the program. The LL program is integrated
information technology resources are in
and distributed across the organization
place to support the LL program.
using appropriate information technology.
Maintenance of the LL program is consistent
The LL program is maintained, and there is a
and ongoing. Process and performance
Maintenance The LL program undergoes some defined process for continuous
No maintenance and/or metrics are used to evaluate the
and improvement takes place.
maintenance, but continuous improvement of the program. Some metrics
effectiveness of the LL program. The
Improvement improvement is not a priority. are used to evaluate the effectiveness of the
program is constantly being improved
LL program.
incorporating feedback from users.
Some individuals and/or communities Most individuals and/or communities of Individuals and communities of practice
Use of LL is not part of the of practice use LL, but it is not practice use the LL program to enhance across the organization understand the
Culture organizational culture. generally a part of the organizational their work. However, the program is not value of the program and willingly
culture. embraced across the organization. participate in the LL program.
Appendix C: Self-Assessment Questionnaire

Instructions: First mark your answers on this questionnaire and then transfer your responses to the
Self-Assessment Score Sheet to calculate your organization’s maturity level for each key characteristic
of a lessons learned program.

Organization: ___________________________________ Date: ___________ Name: _________________

_________________________________________________________________________________________

I. LEADERSHIP
1. Upper management encourages the use of your LL program.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

2. Project teams consistently participate in your LL program.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

3. Managers communicate your LL program to staff and employees.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

4. Individuals understand their role in your LL program.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

5. Upper management promotes and supports your LL program by providing encouragement.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

6. Upper management promotes and supports your LL program by providing rewards.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

7. Upper management has a shared vision of your LL program that involves the entire
organization.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree
_________________________________________________________________________________________

II. LESSON COLLECTION

8. A well-defined work process for submitting or collecting LL exists within your organization.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

9. The work process for submitting/collecting LL is consistently followed within your organization.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

10. Your LL submission/collection process is effective.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree
_________________________________________________________________________________________

III. LESSON ANALYSIS

11. Submitted LL undergo an initial screening before they are analyzed and inputted into the
system.
a) ☐ Never b) ☐ Sometimes c) ☐ Most of the time d) ☐ Always

27
12. Submitted LL are analyzed before they are shared within your organization.
a) ☐ Never b) ☐ Sometimes c) ☐ Most of the time d) ☐ Always

13. Qualified personnel analyze LL.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

14. There is a defined work process for analyzing lessons learned in your organization.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

15. Members of your organization are aware of the analysis procedure for LL.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

16. Individuals submitting LL are given feedback.

a) ☐ Never b) ☐ Sometimes c) ☐ Most of the time d) ☐ Always

17. Your LL analysis process is effective.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree
_________________________________________________________________________________________

IV. LESSON IMPLEMENTATION

18. There is a defined work process for making LL available within your organization.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

19. There is continuous (24/7) access to LL in your organization.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

20. Some LL in the system may be removed/retired after a certain amount of time or after
incorporated into a defined work process/procedure.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

21. Individuals understand how to retrieve and apply LL.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

22. There is a defined work process which requires the retrieval and application of LL.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

23. Your LL implementation process is effective.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree
_________________________________________________________________________________________

V. RESOURCES
24. The IT resources used in your organization enhance the ability of the lessons learned program.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

25. Your LL program IT system is integrated with other IT systems.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

26. Your LL program has adequate human resources to manage/administer the process.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

28
27. Individuals are trained to use the LL program effectively.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

28. Individuals are given the time and resources needed to use and contribute to your LL program.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree
_________________________________________________________________________________________

VI. MAINTENANCE AND IMPROVEMENT

29. Maintenance of your LL program is constant and ongoing.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

30. Feedback from individuals is solicited to improve your LL program.

a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

31. What types of metrics are used to evaluate the performance of your LL program?
a) ☐ No metrics are used b) ☐ Qualitative metrics c) ☐ Quantitative metrics
d) ☐ Both qualitative and quantitative metrics
_________________________________________________________________________________________

VII. CULTURE
32. Individuals participate in your LL program because they understand the value of the system.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

33. Communities of practice encourage their members to use your LL program to avoid/solve
project problems or enhance performance.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree

34. The lessons learned process is an ingrained part of day-to-day activities for all individuals.
a) ☐ Strongly Disagree b) ☐ Disagree c) ☐ Agree d) ☐ Strongly Agree
_________________________________________________________________________________________

29
 Self-Assessment Scoresheet

Directions:
1. Transfer your answers from the Self-Assessment Questionnaire to the “Your Response”
column of the scoresheet.
2. Allocate points based on the scale: a=1 pt, b=2 pts, c=3 pts, d=4 pts.
3. Add up all of the points in each category and record this value as the “Sum of Points for
Category.”
4. Divide by the number of questions in the category to obtain your “Categorical Scoring.”
5. Using Table 1, Numerical Ranges for Evenly Split Maturity Levels, record your
organization’s categorical level.

EXAMPLE:

Your Response
(a=1 pt, b=2pts, Points Categorical Scoring
c=3pts, d=4 pts)

I. Leadership
1. b 2
2. d 4
3. c 3
4. a 1
5. c 3
6. c 3 Leadership Leadership
7. b 2 Score Level
Sum of Points ÷
for Category: 18 by 7 2.57 LEVEL 3

Table 1. Numerical Ranges for Evenly Split Maturity Levels

Maturity Numerical Range

Level 1 1.00–1.74
Level 2 1.75–2.49
Level 3 2.5–3.24
Level 4 3.25–4.00

30
 Self-Assessment Scoresheet

Your Response
(a=1 pt, b=2pts, Points Categorical Scoring
c=3pts, d=4 pts)

I. Leadership
1.
2.
3.
4.
5.
6. Leadership Leadership
7. Score Level
Sum of Points ÷
for Category: by 7

II. Collection
8.
9. Collection Collection
10. Score Level
Sum of Points ÷
for Category: by 3

III. Analysis
11.
12.
13.
14.
15.
16. Analysis Analysis
17. Score Level
Sum of Points ÷
for Category: by 7

IV. Implementation
18.
19.
20.
21.
22. Implementation Implementation
23. Score Level
Sum of Points ÷
for Category: by 6

31
Your Response
(a=1 pt, b=2pts, Points Categorical Scoring
c=3pts, d=4 pts)

V. Resources
24.
25.
26.
27. Resources Resources
28. Score Level
Sum of Points ÷
for Category: by 5

VI. Maintenance & Improvement

29. Maintenance & Maintenance &
30. Improvement Improvement
31. Score Level
Sum of Points ÷
for Category: by 3

VII. Culture
32.
33. Culture Culture
34. Score Level
Sum of Points ÷
for Category: by 3

32
 Appendix D: Jump Start Guide

Step Description Action Items

1 Review and assess current status • Use of Maturity Model Matrix and/or Self-Assessment Questionnaire
• Assess organization vs. matrix
• Compile results

2 Set out improvement initiative needed • Review each matrix category for current level of maturity
• Determine levels of maturity needed to initiate an effective LLP

3 Involve Senior Leadership • Prepare results from steps 1 and 2 in presentation format
• Review presentation with Senior Leadership
• Obtain Senior Leadership commitment to proceed in establishment of an LLP

4 Define the vision, objectives, and • Establish Senior Leadership vision for LLP (What will success look like?)
priorities • Define near term (6 months) and long term (2–5 year) objectives in establishing LLP
33

• Prioritize characteristics to improve upon and obtain Senior Management agreement

5 Assign and engage champions • Define resource requirements to meet near term and long term objectives
• Identify potential process champion(s) and present to Senior Management
• Obtain Senior Management agreement on resources and process champions including
incentives
• Assign and engage champions in the development of the LLP

6 Define the program scope • Establish parameters for employment of the LLP
• What type of input goes into the process?
• What are the sources of input?

7 Develop a process flow map • Review Transactional Work Flow Diagram

• Adopt or adjust to meet the organizational needs

8 Agree on a roll-out / deployment plan • Roll out and deploy in accordance with the organization’s change management process

9 Garner buy-in from the organization • Prepare value proposition for the use of an LLP
• Incorporate value proposition into roll-out / deployment plan
 10 Develop and implement a testing plan • Identify local organizational entity for LLP pilot test
• Develop pilot test plan

11 Plan and execute a pilot effort • Test process feasibility with local organizational entity per the pilot test plan
• Identify gaps in process and develop action plans to close gaps

12 Establish training plan for Users and • Determine training requirements based on pilot test results
Subject Matter Experts (SME’s) • Develop training plan and training material
• Conduct training per training plan

13 Revise the system as required • Evaluate LLP effectiveness after six months vs. short-term objectives
• Use of Maturity Model Matrix and/or Self-Assessment Questionnaire to gage
improvements
• Adjust the program as needed to meet objectives

14 Organize full deployment and user • Develop organization wide roll out / deployment plan
support • Develop and implement organization wide training plan

15 Plan for user recognition & reward and • Engage Human Resources in development of a reward program
34

implementation • Base reward on value added to the organization and to external clients
• Publicize results in company wide communiqués

16 Establish a continuous improvement • Establish a Users Group to provide internal feedback on the effectiveness of the LLP
plan • Establish a Steering Committee to prioritize User Group input for allocation of resources
(time, talent, and treasure) to implement recommendations
• Establish a process to define roles and responsibilities of the Users Group and Steering
Committee

17 Schedule periodic management review • Review LLP annually for achievement of long-term objectives.
• Adjust process as required to achieve long-term objectives
Appendix E: Checklist of Typical Data Fields

Percentage of
Checklist
Responses
Date 88% ☐
Suggestion/recommendation response or action 88% ☐
Occurrence that caused the lesson to be learned 88% ☐
Result of the occurrence 75% ☐
Employee name 75% ☐
Geographic location/region 75% ☐
Search keywords 75% ☐
Type of construction 75% ☐
Project name 63% ☐
Lesson learned statement (summary) 63% ☐
Status of lesson 63% ☐
Generic project info. 50% ☐
Phase of project 50% ☐
Employee contact information 50% ☐
Situation 50% ☐
Client 50% ☐
Estimated cost savings 50% ☐
Originating action 38% ☐
Type of contract 38% ☐
Technology rated 38% ☐
Vendor 25% ☐
Other 25% ☐
Future tasks 13% ☐
Priority descriptor 13% ☐

35
Appendix F: Transactional Work Flow Diagram

Collection Analysis Implementation Maintenance and Improvement

Contributors

1. Input from:
• Corrective Action 15. Advise
Contributor/
• Project Review End
Award &
• Suggestion Recognition
• Other

No
Gatekeeper

3. Further
2. Input Is Review 7. Closed Out/
Screened Candidate? Archived

Yes
SME Resources

No
Evaluators/

5. Evaluate v. 8. Work Work

Legal/

Lessons Learned 6. Meets LL Yes Process or Process

B
Criteria including Criteria Lesson to
Legal Review Share?

Lesson
to Share

13. Issue &

Implement
Implementers

Work Process
9. Prepare
10. Notify Yes
Work Process 16. Training 17. Provide
Process Owner
Format & Required? Training
for Approval
Content
14. Implement No
Lesson to
Share A A A
Process
Owner

11. Process 18. Periodic

No
Owner Review of
Approval? Implemented
Work Process

Yes
Champions

Periodic Review
Lessons
Learned

19. Measure
LL Program
Effectiveness
Management
Database

20. Periodic
4. Initial Entry into 12. Finalize Entry Database 21. Update End
Lessons Learned into Lessons Review Database? No
Database Learned Database
Yes

B
References

Construction Industry Institute (CII). Effective Management Practices and Technologies for
Lessons Learned. Research Summary 230-1, Austin: CII, 2007.

Construction Industry Institute (CII). “An Analysis of Lessons Learned Programs in the
Construction Industry.” Research Report 230-11, Austin: CII, 2007.

39
CII Member Companies

3M ABB Lummus Global

Abbott AMEC
Air Products and Chemicals AZCO
Alcoa Adolfson & Peterson Construction
Amgen Aker Kværner
Anheuser-Busch Alstom Power
Aramco Services Company Atkins Faithful & Gould
BHP Billiton Autodesk
BP America BE&K
Biogen Idec Baker Concrete Construction
CITGO Petroleum Corporation Bechtel Group
Cargill Black & Veatch
Chevron Corporation Burns & McDonnell
Codelco - Chile CB&I
ConocoPhillips CCC Group
The Dow Chemical Company CDI Engineering Solutions
DuPont CH2M HILL
Eastman Chemical Company CSA Group
ExxonMobil Corporation Day & Zimmermann International
General Motors Corporation Dick Corporation
GlaxoSmithKline Dresser-Rand Corporation
Intel Corporation Emerson Process Management
International Paper Fluor Corporation
Kraft Foods Fru-Con Construction Corporation
Eli Lilly and Company GS Engineering & Construction Corporation
Marathon Oil Corporation Grinaker-LTA
Merck Harper Industries
NASA Hatch
NOVA Chemicals Corporation Hill International
Naval Facilities Engineering Command Hilti Corporation
Ontario Power Generation Hyundai Engineering & Construction
Petroleo Brasileiro S/A - Petrobras JMJ Associates
Praxair Jacobs
The Procter & Gamble Company KBR
Progress Energy Kiewit Construction Group
Rohm and Haas Company J. Ray McDermott
Sasol Technology M. A. Mortenson Company
Shell Oil Company Mustang Engineering
Smithsonian Institution R. J. Mycka
Solutia The Nielsen-Wurster Group
Southern Company Parsons
Sunoco Pathfinder LLC
Tennessee Valley Authority Perot Systems Corporation
U.S. Architect of the Capitol Primavera Systems
U.S. Army Corps of Engineers S&B Engineers and Constructors
U.S. Bureau of Reclamation SNC-Lavalin
U.S. Department of Commerce/NIST/ The Shaw Group
Building and Fire Research Laboratory Skire
U.S. Department of Energy Technip
U.S. Department of Health and Human Services Victaulic Company
U.S. Department of State Walbridge Aldinger Company
U.S. General Services Administration Washington Group International
U.S. Steel WorleyParsons Limited
Weyerhaeuser Company Yates Construction
Zachry Construction Corporation
Zurich
Effective Management Practices and Technologies for Lessons Learned Programs
Research Team
William C. Beck, WorleyParsons Limited, Co-Chairman
* Carlos H. Caldas, The University of Texas at Austin
Mark R. Fox, ABB Lummus Global, Inc.
George Frondorf, CITGO Petroleum Corporation
* G. Edward Gibson, Jr., University of Alabama
Randy R. Hendricks, Southern Company
Gary W. House, U.S. Army Corps of Engineers
Mauricio M. Lamartine, Petrobras
Cindy Richartz, Abbott
Barry E. Rittberg, Fluor
Carol Ritz, Bechtel
Brian L. Schmuecker, U.S. Department of State
Brian J. Smith, J. Ray McDermott, Inc.
Samuel S. Steinmiller, Aker Kværner Songer, Inc.
Russell M. Svendsen, Dresser-Rand Company
Carl E. Vorhees, E. I. du Pont de Nemours & Co., Inc.
Paul T. Walls, Rohm and Haas Company
Runi Weerasooriya, The University of Texas at Austin
Angela M. Yohe, The University of Texas at Austin
James Yuengert, Smithsonian Institution, Co-Chairman

* Principal authors

Editor: Rusty Haggard

 ®

Construction Industry Institute®

The University of Texas at Austin
3925 W. Braker Lane (R4500)
Austin, Texas 78759

(512) 232-3000
FAX (512) 499-8101