Lessons
Learned: A Practical Approach
Ofer Pharhi, PMP
ABSTRACT:
The
purpose of this paper is is to give a practical
outline of the Lessons Learned process: How it should be applied and when; what
should be included in the LL process; organizational and personal issues, and
the rules of storing and retrieving the lessons that were learned in a
practical and “use it” manner
Keywords: Lessons learned, Process
application, Issues, Storage, Retrieval
1. Introduction
The
study of human knowledge has always fascinated curious amateurs and
professional researchers alike. The significance of knowledge has been
emphasized by professionals in many a field, from classical philosophy, to the
exact sciences world, to pedagogy and education.
In
recent years, a new field of research has emerged around improving our human
knowledge in respect to repetitive missions, projects that we undertake:
Knowledge Management. Organizations and corporations are beginning to realize
that managing their information is a key factor to success in all dimensions –
financial, business, operational and organizationally. Thus arose a variety of
methods, systems and curricula regarding this emerging topic, and many organizations
are making every effort to transform themselves into a learning entity that
uses its knowledge efficiently.
In
project-oriented organizations, knowledge management consists mainly of taking
information from past projects and putting it to use in upcoming projects. In
the methodology of project management, this procedure is defined as the
“Lessons Learned” process (LL). This is done by extracting the relevant
information (or epistemologically
speaking – turning hidden knowledge to available knowledge) and storing
it for future use and analysis.
The
purpose of this paper is not merely academic. Its goal is not to give a
theoretical view of the process, but to give a practical outline of the lessons
learned process: How it should be applied and when; what should be included in
the LL process; organizational and personal issues, and the rules of storing
and retrieving the lessons that were learned in a practical and “use it”
manner.
While
the context of this paper is project management, it should be noted that the
process described here can be applied to other fields as well. At times, it
will be productive to give the process a different name (such as "event
analysis" or "guided learning"). But as we shall see later,
“lessons learned” is the most appropriate term when dealing with project
management.
2. Lessons Learned As A
Concept
As a
concept lesson learned might well be defined as learning gained from the
process of performing a project; however, some additional clarifications should
be added in order to really understand what the lessons learning process is all
about:
¨
It should be noted that the learning process
covers both positive events (a milestone completed, products delivered to
customers etc.), and negative events (delays in schedule, failing to meet
product specifications etc.).
¨
Every project has end-products which define
the nature of the project. However, projects also have “internal products”
which are defined by the performing organization itself, such as project documentation,
process optimization, historical data etc. An organization whose managers are
committed to learning and improvement will also have the lessons learned
summary document as an internal product of the project.
¨
A formal and orderly LL process is an
essential part of releasing the team when the project is completed. Those who
take part in such a process, often see it as the final wrap-up of their work on
the project.
¨
Projects which terminate prior to completion
are not exempt from the LL process. Indeed, in such projects the LL process may
be even more important than in “regular” ones.
¨
The responsibility for the LL process lies on
the project manager. However, the commitment of the organization's management
is a critical and predominant factor to this process and to ensuring that it
will lead to effective results.
3. Lessons Learned As A
Process
The
main goal of the LL process is to provide information that will enhance the
efficiency of future projects. The information gathered during the project is
held, in bits and pieces, in the hands of the project's team members. These
people, as individuals, cannot transform the information they have into the
desired useful knowledge needed by the process. In many cases, especially in
the software development world, the team members leave the organization upon
the completion of the project, leading to possible loss of critical information
when the project is complete.
It is
known that people tend to feel uneasy towards the LL process. The terminology
is also a factor here; for example, the term “Investigation” would make people
even more uneasy in being part of the LL process. Here, the organization should
pick a term which is well accepted organization-wide.
Never-the-less,
and regardless of its name, there are several factors which contribute to this
uneasy feeling:
¨
Psychological reasons: People don't want to be
a part of the LL process, because they fear of being blamed for mistakes and
errors which occurred during the work on the project.
¨
Political reasons: People have a tendency to
avoid situations which might damage their position (or their team's) in the
organization. Nobody wants to have his weaknesses brought up to the spotlight,
freely shared and recorded for all to see, as this may damage the person's reputation
and impair the ability to deal with the political reality within the
organization.
¨
Cultural reasons: There are certain cultures
which heed to the motto of “what's done is done”. Such people are not
accustomed to speak of past failures or to learn from them.
An
organization which systematizes the LL process and realizes that the focus of
the process is learning rather than blaming, may reduce this resistance
by educating and preparing all involved, to include full support and commitment
of management. It's particularly important to explain that mistakes are an
opportunity to learn, and that the LL process is not about pointing a finger to
one person or another for these mistakes.
It is
very important to formalize the LL process by creating an orderly
organizational methodology, appointing the people who'll supervise the process,
tracking recommendations and their application, and ensuring the commitment of
the management to the process. An informal and disorderly process will result
in an inefficient use of the organization's learning ability and future use of
the knowledge. The process itself, as a process, may improve and empower the
organization's general ability to learn.
4. The Lessons Learned Process
As
stated earlier, the goal of this paper is to provide a practical outline for
the structure of the LL process. Such a process is built upon the following
steps:
1.
Planning the LL process
2.
Creating and distributing LL questionnaires
3.
Collecting the LL questionnaires and preparing
a lessons learned meeting
4.
The lessons learned meeting
5.
The lessons learned summary document
6.
Monitoring
4.1. Planning The
Lessons Learned Process
The LL
process is one of the roles and responsibilities of the project manager.
Planning the LL process should include setting goals for the process,
appointing a moderator, notifying the team members, and allocating tasks for
the next steps of the LL process into the project's master schedule.
The
goals of the LL process will be usually organization-wide in nature. Such goals
may include statements as:
¨
Documenting the duration/resources that was
needed to accomplish various tasks in the project.
¨
Analyzing events that occurred during the
project.
¨
Providing knowledge tools for future projects
in the organization
¨
Finding the cause of errors and setbacks that
occurred during the project.
¨
etc.
Choosing
a moderator from outside the project team is recommended for the purpose of
leading the LL meeting. Such a moderator is someone with no personal interest
in the outcomes of the project and should be uninvolved with the project's team
in order to provide an unbiased point-of-view that will help the team get the
most from the process. It is common to appoint another project's manager (from
the same organization) as the moderator. Another common choice for moderator is
a PMO representative, when there is such office. The project manager will build
a process frame for the moderator, and will provide information regarding the
process' goals and the schedule set for next steps.
In the
opening meeting of the project team, typically at project kickoff, the project
manager will inform the team about the LL process, and will explain the
importance and goals of this process. The project manager will ask everyone to
carefully document – while the project is running – any event which they deem
useful as input to the LL process.
When
planning the project's schedule, the project manager will allocate time for
filling in the questionnaires, planning and executing the lessons learned meeting,
and creating the summary document for the process.
4.2. Creating And
Distributing LL Questionnaire
At the
heart of the lessons learned meeting, is the analysis of questionnaires filled
by the project's team members. These questionnaires will cover all the phases
in the life cycle of the project (initiation, planning, execution, monitoring
and closure), as well as the knowledge areas of the project (integration,
scope, time, risks, communications, etc.). There should also be questions
regarding the technical details of the specific project at hand.
For an
organization which is already well-trained in the LL process, there should be
templates of such questionnaires. At the very least, there will be example
questionnaires from previous projects. Constructing the questionnaire is the
responsibility of the project manager, although it is perfectly acceptable to
assign other team members to assist with this task. The final version of the
questionnaire will be distributed only after the moderator approves its content.
The
project manager will distribute the questionnaires during the late phases of
the project. It should be done before the official closure phase, yet after
most of the project deliverables are completed, with the memory of the major
project events still fresh in the team's mind. The project manager will ask his
team to completed the questionnaires, and emphasize that they are to be filled
anonymously. This should decrease the natural reluctance most people have
towards the LL process. Finally, a definite deadline for handing in the
questionnaires should be set. Without such a deadline, it is likely that most
team members simply won't fill them at all.
A
complete example of such a questionnaire is outside the scope of this paper.
However, example questions and a proposed structure for such a questionnaire
are provided in Figure 1 below:
Figure
1: Example Of Questionnaire
4.3. Collecting The Questionnaires And
Preparing The Meeting
It is the
project manager's responsibility to schedule the date and time of the lessons
learned meeting. Before this meeting is held, the project manager will make
sure that all team members had adequately completed the questionnaires, and
will start the process of gathering the data into a coherent workable document.
Most
of the task of analyzing the completed questionnaires should be the
responsibility of the project manager. In some organizations, however, it is
acceptable for the LL moderator to help the project manager.
The
task of assembling and sorting the data from the questionnaires includes:
¨
Creating a list of the most common issues that
arose from the questionnaires
¨
A primary analysis of the reasons for these
issues, according to the questionnaires
¨
Extracting additional discussion topics from
the answers given to open questions in the questionnaires
¨
Gathering all the information which may help
the evaluation process of future projects
¨
Looking for bottlenecks in the execution
processes of the project
¨
Gathering data which may give clues to proper
risk mitigation strategies in future projects
After
gathering the relevant information the project manager will create the schedule
for the meeting itself. For example, it may be desirable to allocate time for:
¨
Discussing how the project products were
accepted by the customer
¨
Analyzing progress indicators (the most common
one being the EV)
¨
Analyzing the project management methodology
and the actual execution of the project
¨
Discussing interpersonal, team and organizational
issuesAny other topic which the project manager or
the moderator finds worthy of discussing
The LL
meeting should occur after the customer officially accepted the project's final
product. This will allow for the reaction of the customer, as well as the
closing processes of the project, to be taken into account. All team members
should be invited to this meeting and notified when it will take place. The
time scheduled for the meeting, of-course, should be allocated beforehand, as
an inseparable part of the project's schedule.
The
project manager and the moderator may wish to (and usually do) invite people
outside the core project team to the LL meeting. It is common practice to
invite people such as the project's sponsor, managers of other projects within
the organization, technical support staff and PMO representatives. Such
participants can have the status of passive observers, but they are usually
asked to fulfill a more active role and state their opinions about the project
events discussed in the meeting.
There
are three main reasons, cultural and psychological in nature, which create a
reluctance to set up lessons learned meetings:
1.
The feeling that it's all about finding
someone to blame.
2.
The feeling of “what's done is done” which is
especially strong when the project is close to completion.
3.
The feeling that such a meeting would be a
waste of time, and that nobody is going to bother with retaining or
implementing the learned lessons in the future.
There
are ways to deal with these factors and reduce the resistance, but nothing can
come in place of a true commitment to the process by the organization's
managers: A clear statement that the LL process is a learning process rather
then a finger-pointing act; embedding the process as an integral part of the
project, rather than treating it like some “appendix” attached to the end of
the project; setting up guidelines for storage, retrieval and proper use of the
lessons learned. These actions will reduce the resistance to the process and
will help the team to become more actively involved in it.
If the
project was terminated before completion, the LL meeting should be held
immediately after the decision to terminate the project prematurely. Such a
meeting will obviously be more psychologically difficult then a regular lessons learned meeting, but it is of crucial
importance, especially if the project was aborted due to factors which the
organization has some control over.
4.4. The Lessons Learned Meeting
At the
LL Meeting the moderator will present the agenda of the meeting, as planned by
the project manager. It is usually a good idea to start the meeting with the
project manager (or an appointed representative) congratulating the team
members for their achievements during the project. This is true for projects
that were terminated before completion as well. Even when the project as a
whole has failed, people still put effort in their work and they deserve to be
commended for it.
At the
core of the meeting, will be the discussion of project events which were most
important or most frequent, as shown by the questionnaires. These are, usually,
central events in the project, such as milestone completion, product quality,
and communication between shareholders. The moderator will lead the discussion
towards the goal of finding what factors caused the said events, rather then
focusing on the event itself (or the person responsible for it). It should be
clear that this procedure applies equally to positive and negative events:
Finding the factors leading to success is no less important then finding those
who lead to failure. The goal of this part of the meeting is to produce a list
of causes and their effects, which will help in preventing failures, which will
help in ensuring repetitive success, in decision making at future projects.
Other
topics which may be relevant to the lessons learned meeting are:
¨
Discussing the assumptions on which the
project was based on. In retrospect, were they accurate? Was the process of
making these assumptions reasonable? Were there any important assumptions
left-out? If so, what are they?
¨
Project change management analysis: Were the
changes in the project managed properly? What was the method of authorizing
changes during the project?
¨
Did the project products meet the desired
specifications? Discuss the feedback received from the customer after every
milestone, and the feedback received at the end of the project.
¨
Discussing progress indicators: Was the
progress adequately measured? Were the chosen indicators a good choice? What
was the effect, if any, of these measurements on the execution of the project?
¨
Analyzing the effect of QA and QC components
on project events.
¨
Were the management and execution
methodologies consistent with the type of the project?
¨
Were lessons learned from previous projects
put to use, were they put to use in a successful and operational manner in the
planning phase of the current project? If so, how?
It is
imperative to avoid mentioning names of people involved in the events
discussed, unless they've voluntarily taken the responsibility and clarify
their position about what happened. This, of-course, does not apply to positive
events.
It is
also imperative that the discussion will be held candidly and openly, and the participants
will discuss the topic at hand without hiding anything. An open and honest
discussion is the only way to efficiently utilize the knowledge and information
held by the participants.
4.5. The Lessons Learned Summary Document
The
primary product of the LL meeting is a document which will be distributed among
the people attending the meeting and other shareholders, including the
organization's management. This document will be included in the official list
of project documents, and its content will be stored in an organizational
database.
Creating
this document is the responsibility of the project manager, but the moderator
or other team members may also take part with this task.
The
document will open with the goals of the LL process (as defined in section 1
above) and a description of the method used to manage the process before the
meeting. The document will then detail the main points which were reached
during the meeting itself (as demonstrated by the points in section 4 above).
An
important part of every analysis of every event is providing an organizational
context which will serve as a reference for future projects. For example, it
may prove useful to list the name of the person or the department who can
provide more detailed information regarding the event in question. It is also
important to write down the final conclusions drawn (in terms of causing
factors and consequences for future projects) for every event being discussed.
The
next part of the document will be a list of practical recommendations for
future projects. Examples for such recommendations are:
¨
Increasing the estimation range on certain
specific tasks
¨
Appointing a single person to communicate with
shareholders
¨
Appointing a single person to be in charge of
managing project changes
¨
Involving the management whenever there's an
escalating conflict with a customer
¨
Managing the LL process itself: Was it done
properly? In what ways can the process be improved?
¨
etc.
Once
the first draft of this document is complete, the project manager will
distribute it to all the participants of the lessons learned meeting, for their
approval. This is especially important when analyzing failure events, due to
the personal and emotional sensitivities that tend to accompany such events.
Once
the document is approved by all participants, the project manager will make
sure that the document summary is stored in an organizational database. The
nature of this database varies from organization to organization. It can be an
application built specifically for lessons learned, a query-based database, or
even a simple spreadsheet. Regardless of its technical form, appropriate steps
must be taken to ensure that the database contains all the relevant lessons
learned information, and that this information is retrievable.
The
final version of this document will be distributed among the participants of
the lessons learned meeting as well as all other stakeholders (excluding the
customer) and the organization's management.
4.6. Monitoring
The distribution
of the Lessons Learned Summary Document marks the end of the last phase of the
project. Therefore the job of monitoring the implementation of the lessons
learned recommendations does not rest on the shoulders of the project manager.
Instead, it becomes a responsibility of the PMO office (if it exists) or the
organization's management.
The
monitoring stage includes three main components:
1.
The Summary Document should include practical
recommendations to those who are managing other projects in the organization.
It is therefore necessary to check that these recommendations are actually
followed through.
2.
It is necessary to check that all the
conclusions of the LL process are properly stored in the organizational
database, and that the organization's employees were notified about the
availability of this new information.
3.
It is necessary to check that the information
in question can be retrieved efficiently (keywords, categories, queries,
technical classifications etc.). It is also important to ensure that other
project managers in the organizations are applying the lessons learned when
they begin planning their own projects.
5. Conclusion
Knowledge
management is an important tool for success in the modern business world,
especially in project-oriented organizations. The ability to utilize previous
knowledge is especially useful in organizations in which the projects are
similar in nature to one another, where this process of learning from past
mistakes and past successes can help save resources, boost efficiency and
mitigate risks.
However,
it should be noted that such ability does not come on its own. Without the
necessary commitment from the management, the data gathered from previous
projects will remain in its unprocessed form. Only a conscious and formal
implementation of the lessons learned process, will serve the organization's
future success.