Available online at www.sciencedirect.

com

International Journal of Project Management 30 (2012) 252 – 263

www.elsevier.com/locate/ijproman

Measuring the efficiency of project control using

fictitious and empirical project data
Mario Vanhoucke ⁎
Ghent University, Tweekerkenstraat 2, 9000 Gent, Belgium
Vlerick Leuven Gent Management School, Reep 1, 9000 Gent Belgium
University College London, Gower Street, London WC1E 6BT, United Kingdom

Received 4 January 2011; received in revised form 14 March 2011; accepted 31 May 2011

Abstract

Dynamic scheduling refers to the integration of three important phases in the life cycle of a project: baseline scheduling, schedule risk analysis
and project control. In this paper, the efficiency of controlling a project is measured and evaluated using a Monte-Carlo simulation study on
fictitious and empirical project data. In the study, the construction of a project baseline schedule acts as a point-of-reference for the schedule risk
analysis and project control phases. The sensitivity information obtained by the schedule risk analyses (SRA) and the earned value management
(EVM) information obtained during project control serve as early warning control parameters that trigger corrective actions to bring projects back
on track in case of problems. The focus in this paper lies on the time performance of a project, and not on the prediction and controlling of the
project costs.
The contribution of this paper is twofold. First, this paper summarizes the main conclusions of various experiments performed in a large
simulation study on the efficiency of project control techniques and the ability to trigger corrective actions in case of project problems. The main
purpose of these simulation experiments is to understand why EVM and/or SRA work so well in some projects and fail so miserably in others.
This study has been awarded by the International Project Management Association in 2008 on the IPMA world congress in Rome (Italy).
Secondly, the paper compares the results obtained on fictitious project data with additional tests performed on a set of real-life data from 8 Belgian
companies from various sectors.
© 2011 Elsevier Ltd. APM and IPMA. All rights reserved.

Keywords: Project management; Project control; Simulation study

1. Introduction techniques. From this moment on, a substantial amount of

research has been carried out covering various areas of project
Project scheduling began as a research track within the scheduling (e.g. time scheduling, resource scheduling, cost
mathematical field of Operations Research in order to scheduling). Today the project scheduling research continues to
mathematically determine start and finish times of project grow in the variety of its theoretical models, in its magnitude
activities subject to precedence and resource constraints while and in its applications.
optimizing a certain project objective (such as lead-time While the research has expanded over the last decennia,
minimization, cash-flow optimization, etc.). The initial research leading to project scheduling models with deterministic and
done in the late 1950s mainly focused on network based stochastic characteristics, single- and multi-mode execution
techniques such as CPM (Critical Path Method) and PERT activities, single and multiple objectives, and a wide variety of
(Programme Evaluation and Review Technique) which are still resource assumptions, the practitioners and software tools
widely recognized as important project management tools and mainly stick to the often basic project scheduling principles.
This can probably be explained by the limited capability of a
⁎ Tel.: +32 92643569; fax: +32 92644286. project schedule to cope with the uncertainty that characterizes
E-mail address: [email protected]. the real life execution of the project. Indeed, the benefits of a
0263-7863/$ - see front matter © 2011 Elsevier Ltd. APM and IPMA. All rights reserved.
doi:10.1016/j.ijproman.2011.05.006
 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263 253

resource-constrained project schedule have been questioned by project information of 8 Belgian companies. Section 4 draws
many practitioners, and the effort someone puts into the general conclusions and highlights paths for future research.
development of a project schedule is often not in line with the
benefits. Moreover, “a project schedule will change anyway due 2. Project control and corrective actions
to circumstances” is often a widely used excuse to skip this
important step in the project life cycle. This section starts with a brief review of the three dimensions of
Nevertheless, project scheduling plays a crucial role in the dynamic scheduling with references to the main research efforts
life cycle of a project and can play a key role in the failure or done in the past. In the following subsections, more details are
success of a project. In order to appreciate the importance of a given on their use in the simulation experiments of this paper. In
project schedule, it should be generally accepted that the Section 2.1, four sensitivity parameters of a schedule risk analysis
usability of a project schedule is rather limited and only acts as a are reviewed with references to previous research studies where
point of reference in the project life cycle. Consequently, a these parameters are used. Section 2.2 briefly reviews the main
project schedule should especially be considered as nothing parameters used in earned value management and displays refer-
more than a predictive model that can be used for resource ences to the recent literature on time performance measurement.
efficiency calculations, time and cost risk analysis, project Section 2.3 explains how and why these two sources of information
control and performance measurement, and so on. In this paper, (EVM and SRA) can be used in a project control environment, and
the project baseline schedule is assumed to be given and plays a how they can trigger actions in case a project is in danger. In a final
central reference point in the analysis of a project schedule's Section 2.4, three research questions are given that will be tested in
risk and during the project performance measurement and the computational experiment discussed later in this paper.
control phase. This triangular relation between the project Dynamic scheduling is used to refer to a dynamic project
schedule, the risk analysis and the project control phase is called management process to control projects using information from a
dynamic scheduling throughout this paper. project's baseline schedule and its corresponding risk. More
The purpose and contribution of this paper is twofold. First, precisely, it refers to three integrative phases of the project life cycle
it aims at reviewing previous research efforts on project in order to trigger and steer the corrective action decision making
management control using schedule risk analysis and earned process to bring projects back on track in case of problems. These
value management. Although various papers have described three crucial dimensions of dynamic scheduling can be briefly
these two techniques, little effort has been done to put these outlined along the following lines and are summarized in Fig. 1.
techniques in a dynamic project control environment. To that
purpose, a review on research results will be given in order to • Baseline scheduling: construct a timetable that provides a
filter out practical rules-of-thumb interesting for project start and end date for each project activity, taking activity
managers. Secondly, it lies in the intention of this paper to relations, resource constraints and other project characteris-
illustrate that both fictitious and empirical data are used to tics into account, and aiming to reach a certain scheduling
validate the obtained results. It should be noted that this paper objective. Literature on project baseline scheduling can be
takes a strict time focus on the project and hence little or no found in the overview papers written by Brucker et al.
attention is paid to the dynamic nature of cost performance (1999), Herroelen et al. (1998), Icmeli et al. (1993), Kolisch
measurements and predictions. and Padman (2001) and Özdamar and Ulusoy (1995). More
The following sections of this paper are structured as recently, Kolisch and Hartmann (2006) have given an update
follows: Section 2 gives an overview of the three dimensions of on experimental results on project scheduling with heuristics,
dynamic project scheduling and refers to important research while Hartmann and Briskorn (2010) have given a survey of
sources in literature. In a first subsection, an overview is given variants and extensions on the traditional resource-con-
of the main measures used in a schedule risk analysis (SRA) strained project scheduling problem.
study and references to recent studies are highlighted. A second • Risk analysis: analyze the strengths and weaknesses of a
subsection deals with the project control dimension which project baseline schedule in order to obtain information about
makes use of earned value management (EVM) parameters to the schedule sensitivity and the possible changes that
measure a project's time performance. The recent and renewed undoubtedly occur during project progress. Two excellent
attention on EVM for measuring and controlling the duration of overviews on schedule risk analysis are given by Williams
a project is briefly summarized in this section. In a third (1995) and Hulett (1996) and in the Practice Standard for risk
subsection, the use of the SRA and EVM information as early management (PMI, 2009).
warning control parameters to trigger corrective actions are • Project control: measure the (time and cost) performance of a
explained in detail. In a final subsection, three research project during its progress and use the information obtained
hypotheses are formulated that will be tested in the computa- during the baseline scheduling and risk analysis steps to
tional experiment of this paper. Section 3 describes the input monitor and update the project and to take corrective actions
parameters of the computational experiment using Monte-Carlo in case of problems. In this paper, project control is mainly
simulations and gives an overview of the main conclusions that discussed from an earned value management point-of-view.
can be drawn on the efficiency of controlling projects and their More information on this control technique can be found in
corrective actions. These results are obtained on both fictitious Fleming and Koppelman (2005) and in the Practice Standard
project data as well as on an empirical data set containing for earned value management (PMI, 2005).
254 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263

Baseline Schedule
The baseline schedule acts as a The baseline schedule acts as a
point-of-reference point-of-reference
for schedule risk analysis. for project performance
Sensitivity information of activities measurement. Time and cost
is obtained by simulating performance is measured as a
deviations from this baseline comparison between EV (project
schedule during fictitious project progress) and PV (baseline
progress using Monte-Carlo runs Dynamic schedule) and AC (actual cost)

Scheduling
Schedule Risk Project
P Control
Both the information from a Schedule Risk Analysis
(SRA) and the performance measurement of an Earned
Value Management (EVM) approach can be used to
improve the corrective actions that need to be taken
when projects are in trouble

Fig. 1. Dynamic scheduling: baseline scheduling, risk analysis and project control.

The focus of the current paper is on the efficiency of the critical project parts. In doing so, a clear focus should reduce a
project control phase and the ability to respond (i.e. corrective project manager's effort while still guaranteeing a correct and
action decision making process) in case projects are in danger timely response during project control to bring the project back
through the integration of the schedule risk and project control on track in case of problems.
component. The literature on project performance measurement A schedule risk analysis study consists of four main steps, as
and risk analysis is rich and widespread. Recent example con- given in Fig. 2. The initial step requires the construction of a
tributions are given by Ahsan and Gunawan (2010), Görög deterministic baseline schedule given the project network data. In
(2009) and Collyer and Warren (2009) in the International a second step, the deterministic data (activity duration, activity
Journal of Project Management. cost, precedence relations, etc.) in this baseline schedule are put
In the remaining subsections, it is assumed that the project into question by adding stochastic elements on it. Pre-defined
baseline schedule has been constructed and is used as a point of distributions are assigned to the deterministic data from which
reference for a schedule risk analysis (SRA) and an earned value numbers will be drawn in the third Monte-Carlo simulation step.
management (EVM) approach. These two techniques and In this step, the fictitious execution of the project is simulated and
relevant research topics will be discussed in Sections 2.1 and 2.2. its performance is compared to the deterministic baseline
schedule point-of-reference. In a final step, sensitivity informa-
2.1. Schedule risk analysis tion is automatically reported as measured during the different
simulation runs.
Schedule risk analysis is a simulation technique to reveal the SRA studies have been described, praised as well as
critical components of a project that likely have the biggest criticized in literature (see e.g. the studies done by Williams
impact on the project objective. The need for a quantitative (1992, 1995, 1999)) and the use of Monte-Carlo simulations on
measure for the relative sensitivity of the project activities to a project's baseline schedule often is a standard option in a
assess their impact on the project objective stems from the need commercial project management software tool. Vanhoucke
a project manager has to focus his/her attention on only the (2010b) has compared four known sensitivity metrics on its

Baseline schedule Uncertainty Monte-Carlo simulation Simulation output

Uniform
Project network distribution Distributions
Triangular
distribution Triangular
2 4 distribution
(with skewness)

1 6

3 5

Project schedule

Etc...

Fig. 2. Schedule risk analysis (SRA): measuring a project activity's sensitivity.

 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263 255

ability to act as reliable triggers to predict project problems. Lipke (2003) criticized the use of the classic SV and SPI
More precisely, he has done various simulation experiments on metrics since they give false and unreliable time forecasts near
a fictitious project dataset using four activity duration sensitivity the end of the project. Instead, he provided a time-based
metrics from Williams (1992) and PMBOK (2004), as given measure to overcome this unreliable behavior of the SV and SPI
along the following lines: indicators. This earned schedule method relies on similar
principles than the earned value method, but translates the
• Criticality Index (CI): measures the probability that an monetary metrics into a time dimension. He reformulated the
activity lies on the critical path. time performance measures SV and SPI to SV(t) and SPI(t)
• Significance Index (SI): measures the relative importance of (where the t between brackets is used to distinguish with the

 traditional performance measures) and has shown that they have
an activity as SI = E di
⋅ EðPD
PD
with E(x) the expected
di + f i a reliable behavior along the whole life of the project.
value of x, di the duration of activity i, fi its activity float and Fig. 3 shows the two versions of the schedule performance
PD the total project duration. index (SPI and SPI(t)) along the life of the project expressed by
• Schedule sensitivity index (SSI): measures the relative its percentage completed using data of an example project that
importance of an activity taking the CI into account as ends later than originally planned. While the SPI(t) can be
"sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi#
σ 2 ð di Þ considered as a reliable indicator for the real project duration
⋅CI with σ 2(x) the variance of variable x. performance, the SPI indicator shows an unreliable trend from
σ2 ð PDÞ
the 75% percentage project completion on and finally goes to
• Cruciality Index (CRI): measures the correlation between the one, although the project ends late.
activity duration and the total project duration, in three Since its introduction, a continuous stream of publications on
different ways: the project time performance measurement and/or the earned
– CRI(r): Pearson's product-moment correlation coefficient, schedule concept has been published in the popular as well as in
– CRI(ρ): Spearman's rank correlation coefficient, the academic literature. A comparison between three methods to
– CRI(τ): Kendall's tau rank correlation coefficient. measure the time performance of a project can be found by
Anbari (2003), Rujirayanyong (2009) and Vandevoorde and
In the study, it has been shown that the information given by Vanhoucke (2006). Simulation experiments to validate the power
the schedule sensitivity index has the highest potential to of the three time performance methods can be found in
dynamically steer and improve the schedule performance of a Vanhoucke and Vandevoorde (2007). Statistical validation of
project. the time performance indicators and stability studies on empirical
data can be found by Lipke et al. (2009). Other interesting
2.2. Earned value management contributions can be found on www.earnedschedule.com.

Earned value management is a methodology used since the 2.3. Corrective action thresholds
1960s, when the USA department of defense proposed a standard
method to measure a project's performance. The system relies on Performance measurement systems are made to trigger
a set of often straightforward metrics to measure and evaluate the corrective actions and hence serve as early warning systems to
general health of a project. These metrics serve as early warning generate signals when the desired performance drops below a
signals to timely detect project problems or to exploit project certain predefined management threshold. To that respect, these
opportunities. The basic principles and the use in practice have systems and corresponding thresholds replace the continuous
been comprehensively described in many sources (for an attention of the project manager and can therefore be seen as
overview, see e.g. Anbari, 2003 or Fleming and Koppelman, systems that reduce the effort the project manager has to spend
2005). Although EVM has been set up to follow up both time and on the daily control of the various activities of the project. A
cost, the majority of the research has been focused on the cost crucial choice during project control is the level of detail and the
aspect (see e.g. the paper written by Fleming and Koppelman resulting management effort needed to effectively manage the
(2003) who discuss EVM from a price tag point of view). project control process, by setting the thresholds to acceptable
Similar to a SRA study, EVM requires a fixed point-of- levels. The use and interpretation of action thresholds depend on
reference, given by the project baseline schedule, in order to the project control approach and underlying project perfor-
periodically measure the project performance along the life of mance measurement system in use. Vanhoucke (2011) has
the project. Project performance, both in terms of time and made a black-and-white distinction between top–down project
costs, is determined by comparing the three key EVM control and bottom–up project control, which will be briefly
parameters, planned value (PV), actual costs (AC) and earned reviewed here.
value (EV), resulting in the time performance measures known
as schedule variance (SV = EV − PV) and schedule performance 2.3.1. Top–down project control
index (SPI = EV PV) and the cost performance measures known as Project control using earned value management should not
cost variance (CV = EV − AC) and cost performance index be considered as an alternative to the well-known critical path
EV
(CPI = AC ). In the remainder of this paper, a focus lies on the based scheduling and control tools. Instead, the EVM
time performance indices. methodology offers the project manager a tool to calculate a
256 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263

Start of unreliable SPI trend

(75% complete)

Earned Value Management

1.05 SPI: ends at one: wrong!

1
0.95
0.9
0.85
SPI(t): correct from start till end
0.8
of the project
0.75
0.7
0%
5%

%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
0%
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
10
Percentage Completion (PC)

Fig. 3. Earned value management (EVM): measuring a project's performance.

quick and easy sanity check on the control account level or even 2.4. Research questions
higher levels of the work breakdown structure (WBS). To that
respect, we do not follow the advice by Book (2006a,b), Jacob The main purpose of the simulation experiments is to
(2006) and Jacob and Kane (2004) who argue that the well- understand why EVM and SRA work so well in some projects
known performance measures are true indicators for project and fail so miserably in others. Since both techniques generate
performance as long as they are used on the activity level, and information of the project, they should be used in a project control
not on the control account level or higher WBS levels. Instead, environment to improve the accuracy of performance measure-
we follow the approach taken by Vanhoucke (2010a) who ment and to trigger actions in case the project is in danger. Three
recognizes that, at higher WBS levels, effects (delays) of non- research questions will be tested in the computational experiment
performing activities can be neutralized by well performing of the next section, and can be given along the following lines:
activities (ahead of schedule), which might result in masking
potential problems, but nevertheless argues that the project 2.4.1. RQ1
based approach is the only approach that can be taken by Is the topological structure of a project network a main driver
practitioners as a trigger to drill down to lower WBS levels in for the accuracy of EVM and SRA information?
case of problems. Lipke et al. (2009) also note that a detailed In this research question, it is measured whether static
schedule analysis is a burdensome activity and if performed information of a project in terms of the distribution of activities
often can have disrupting effects on the project team. EVM in the project network and the density of the network given by
offers calculation methods yielding reliable results on higher the amount of precedence relations between activities is a driver
WBS levels, which greatly simplify final duration and for the accuracy of EVM and SRA information.
completion date forecasting. These early warning signals, if
analyzed properly, define the need to eventually drill down into 2.4.2. RQ2
lower WBS levels when action thresholds are exceeded. In Can EVM and SRA information trigger early warning actions
conjunction with the project schedule, it allows taking for project control?
corrective actions on those activities which are in trouble In this research question, it is measured whether EVM and
(especially those tasks which are on the critical path). The top of SRA information are able to generate reliable parameters in a
Fig. 4 illustrates the use of dynamic action thresholds on the dynamic project environment to support corrective actions during
schedule performance index (SPI(t)). project progress.

2.3.2. Bottom–up project control 2.4.3. RQ3

A bottom–up control approach works exactly the opposite of Are the simulation results on fictitious project data in line
the top–down approach, and relies on a subset of activities to with results obtained on empirical data?
trigger the corrective action process. Using schedule risk A comparison between the fictitious data results and the
analysis for the detection of activity sensitivity information is results obtained on the empirical data is made as a validation of
crucial to steer a project manager's attention towards a subset of the quality and validity of the simulation experiments.
the project activities that has a high expected impact on the
overall project performance. These highly sensitive activities 3. Research results
are the subject to intensive control, while others require less or
no attention during project execution. The bottom of Fig. 4 This section is structured as follows. In Section 3.1, the two
illustrates the use of action thresholds using the schedule sensitivity sources of project data are described in detail. Section 3.2 gives
index (SSI). an overview of the methodology used. In Section 3.3, the
 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263 257

Earned Value Management

1.05 Increasing threshold

No action zone
(= higher attention)
1
0.95
0.9
SPI (t)
Decreasing threshold
0.85 (= lower attention)
Action zone
0.8
0.75
0.7
0%
5%

%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
0%
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
10
Percentage Completion (PC)
Schedule Risk Analysis

No action zone Action zone

19
17
15
Activity List

13
11
9
7
5
3
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Schedule Sensitivity Index

Decreasing threshold Increasing threshold

(= higher attention) (= lower attention)

Fig. 4. Corrective action threshold during project control (source: Vanhoucke, 2011).

simulation approach is briefly illustrated on a real project and algorithm needs to schedule a project (see e.g. Elmaghraby and
in Section 3.4 the main conclusions of the computational Herroelen (1980), among many others who have done research on
experiments are drawn. this topic). For an overview of project network topology indicators,
the reader is referred to Demeulemeester et al. (2003).
3.1. Project data In the current study, the I2 indicator is used to control the
topological structure of a project network since it has been
3.1.1. Fictitious project data shown by Vanhoucke and Vandevoorde (2007) that this
Several experiments have been performed in a large research indicator has a major influence on the accuracy of project
study on a fictitious dataset containing 4100 project network in- duration predictions using EVM. In this study, the authors have
stances generated by a project generator RanGen (Demeulemeester called this indicator the serial/parallel (SP) indicator, since it
et al., 2003; Vanhoucke et al., 2008). This network generator measures the closeness of a project network to a serial (S) or
generates project networks under a controlled design taking the parallel (P) network. The formula is based on the progressive
density of the project network into account. More precisely, the and regressive level calculations of activities in a project
topological structure of each project network is set as an input network as defined by Elmaghraby (1977). In two other studies
parameter. The topological structure of a project network describes (Vanhoucke, 2010b, 2011), it has been shown that the
the relation between the project activities and its corresponding efficiency of project control and risk analysis both depend on
precedence relations, and can be measured in various ways, such as the SP indicator: the project network topologies for which
the coefficient of network complexity, defined by Pascoe (1966) schedule risk analysis is highly efficient (i.e. the bottom–up
and redefined by Davies (1974) and Kaimann (1974, 1975), the approach) show a weak performance for earned value man-
order strength (Mastor, 1970), the density (Kao and Queyranne, agement control (i.e. the top–down approach), and vice versa.
1982), the restrictiveness (Thesen, 1977), the complexity index CI The I2 or SP indicator has been varied from values of 0.1
(Bein et al., 1992) and four newly developed indicators (known as (almost 100% parallel) to 0.9 (almost 100% serial) in steps of
I2, I3, I4 and I5) initially proposed by Tavares et al. (1999, 2002) and 0.1. For each value, various different project networks have
redefined by Vanhoucke et al. (2008). In literature, it has been been generated, leading to 4100 project networks in detail. In
shown that the topological structure of a project network plays an doing so, it is believed that the fictitious dataset captures the full
important role in the quality of a constructed schedule (Patterson, range of complexity with respect to the topological structure.
1976), the risk for delays (Tavares et al., 1999), the criticality of a More detailed information on this dataset can be found in the
network (Tavares et al., 2004) or the computational effort an book of Vanhoucke, (2010a).
258 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263

3.1.2. Empirical project data run, the activity data distribution curves described in the
Extensive results on the fictitious datasets have been previous section are used as inputs to generate deviations
published in the book of Vanhoucke (2010a) with only little between the project baseline schedule and the simulated project
feedback to real-life project data. In order to confirm, reject or progress. An overview of these runs can be summarized along
put the results of this simulation study into the right perspective, the following lines.
similar experiments have been done on empirical data after the
publication of the book. To that purpose, the empirical data 3.2.1. Static project information
have been collected during the period 2007–2010 from students A baseline schedule is constructed for each project and SRA
at Ghent University and from various consultancy projects in information is obtained using a first set of Monte-Carlo
collaboration with eight companies in Belgium. Based on simulation runs. In doing so, the highly sensitive activities
interviews and detailed analyses of various project schedules, can be discovered for each project that serve as action threshold
template project networks and real life activity data have been during project control.
constructed, as summarized along the following lines:

3.2.2. Dynamic project information

• Project network: activity-on-the-node project networks with
In order to simulate deviations between the project baseline
precedence relations between the activities are constructed
schedule and the real project performance, project progress is
based on the project data. Since the network structures of
simulated using a second set of Monte-Carlo simulation runs in
projects within a company were relatively similar, project
order to generate EVM data.
templates have been defined in collaboration with the
company members that were used as inputs in the simulation
study. For each project, the serial/parallel indicator has been 3.2.3. Project control
calculated, leading to SP values ranging between 0.13 and Both the static SRA information and the dynamic EVM
0.88. information are used as action thresholds to trigger corrective
• Activity data: activity durations and costs were estimated actions in case the project duration is in danger. When
based on historical data. Using curve fitting techniques, each thresholds are exceeded, corrective actions are necessary. A
activity in the project template gets a time and cost database of potential actions have been constructed that imitate
distribution that reflects the deviations on activity durations the real project control approach of the companies under study.
and costs compared to their baseline estimates. In case corrective actions must be taken due to threshold
exceedance (see Section 2.3), the simulator can choose between
a range of options that are often very project specific. To that
This study has resulted in 13 classes of project templates
purpose, a wide variety of corrective actions have been discussed
from 8 different companies, as a result of 48 projects in total.
with the project managers of the companies, and translated into
Fig. 5 shows the 13 classes of projects with their corresponding
three different classes:
SP values on the x-axis, ranging from projects with low SP
values (close to 100% parallel) to projects close to a complete
serial network (SP values high). The bubbles above the • Reducing activity durations, often resulting in a cost increase.
horizontal line show from which company sectors the projects This is known in literature as activity crashing.
have been obtained (8 companies in total). • Parallel execution of precedence related activities. This is
known in literature as fast tracking.
3.2. Methodology • Re-baselining parts of the original baseline schedule in case
of significant changes in scope (i.e. changes in the project
The methodology used in the computational experiment network templates).
consists of a double set simulation runs in order to generate
static and dynamic project information in order to measure the
Each time an action is taken by the simulator, the action is
efficiency of the project control process. During each simulation
selected from one of the three classes based on a pre-defined
probability distribution that has been assigned to projects based
on the discussions with the project managers. Consequently,
although this approach might be rather subjective, it improves
Airport the credibility of the automatic actions taken by the program
Water Production Infrastructure
Centre since it reflects reality to a certain degree. The purpose of these
Bridge Construction actions is to bring the expected project duration back in line
Maintenance
IT with the planned project duration. Consequently, the study aims
Research Satellite
Tunnel at measuring how efficient the corrective actions were taken
during the control phase based on EVM and SRA thresholds.
100% Parallel 100% Serial The efficiency is measured as the contribution of corrective
actions taken during the project execution divided by the total
Fig. 5. Real life data according to the serial/parallel indicator. effort a project manager has spent on controlling the project
 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263 259

performance. The two dimensions of efficiency are calculated display the results on the empirical project data, with a circle dot
as follows: the results for the SRA approach and a square dot the results for
an EVM approach. Since each project network template has
• Effort: number of times that project activities need to be been simulated 25 times, each time drawing random numbers
evaluated for possible actions to bring the project back on for the activity durations given the fitted curves, an indication
track. These evaluations are triggered by the exceeded has been given of the variability in the results (between a
thresholds and are a measure of the total effort a project minimum and maximum efficiency) by means of the vertical
manager has to spend on the daily control of the project lines. It should be noted that the bottom–up control approach
during its execution. using SRA is based on results reported for the schedule
• Action contribution: the total number of days the project has sensitivity index (SSI) which has been proven to outperform
decreased in comparison with a similar project performance other sensitivity measures in literature (Vanhoucke, 2010b).
(i.e. identical durations for each activity during a simulation The top–down approach using EVM thresholds is measured
run) without actions taken. In doing so, the contribution of using the SPI(t) indicator as an action threshold since it is more
the actions is measured as the difference in the project reliable than the SPI towards the end of a project (Lipke, 2003).
duration between simulation runs with and without actions The following conclusions for the three research questions
triggered by the predefined thresholds. can be drawn:

3.3. Simulation example • RQ1: the results show that differences in variability and
efficiency are mainly determined by the SP indicator. The
Fig. 6 illustrates a simulation experiment on a biogas variability in the results is remarkably higher for more parallel
installation maintenance project with a total project duration of oriented project networks compared to project networks with a
somewhat more than 5 months and a budget at completion more serial structure. It is conjectured that this higher
BAC = €737,212.38. The baseline schedule information is used variability comes from the interplay between critical and
as a project template as shown in the Gantt chart. The simulations non-critical (or near-critical) activities, which will significantly
are done with a software tool ProTrack 1 linked to a database of bias the results of both the SRA and EVM simulations. Indeed,
activity time and cost distributions (picture to the left of Monte- in a SRA, near-critical activities may cause delays in non-
Carlo simulation) and a set of actions (picture to the right of critical paths and may potentially delay the whole project. This
Monte-Carlo simulation). A first simulation run is done without is known in literature as (merge bias) and may be the cause of
any actions by taking only the activity distributions into account. the higher variability compared to more serial networks. The
In doing so, the total project duration can be determined, which is EVM control efficiency variability results from the fact that
in the example equal to 04/10/2009, which acts as a reference changes in activity durations of serial networks always have a
point to calculate the “action contribution” in the numerator of the real effect on the total project duration and hence the SPI(t) is
project efficiency calculations. In a second simulation run, the more reliable and a more true indicator of project delays than
project is linked to the activity distributions and actions database for projects with more parallel activities.
and corrective actions are taken in case thresholds are exceeded. • RQ2: the results also show that the efficiency of EVM and
In the figure, two actions are taken (circles in the bottom right SRA depends on the SP indicator, and can therefore be used
picture) based on a SPI(t) value which drops below the threshold to trigger corrective actions.
(curved line in the bottom right picture). These actions lead to an – EVM project control provides reliable results in case the
increase in the project manager's effort (denominator of the project network has a rather serial structure. In these
project efficiency calculations) which results in a total project cases, the project contains many critical activities, and
duration ending at 30/08/2009. Consequently, the “action hence, the probability to mask errors by measuring the
contribution” is equal to the difference between 04/10/2009(no project performance on high WBS levels decreases.
actions) and 30/08/2009 (with actions) while the “effort” is equal Consequently, in these cases, there is no need for a
to two interventions. After the two simulations, the project detailed activity based CPM project control, but instead,
efficiency is calculated and put in a graph with many other project rough project based EVM performance measures can
simulations, which will be discussed in the next section. serve as reliable triggers to drill down to lower WBS
levels (down to the activity level) to look for possible
3.4. Main results problems to take corrective actions.
– CPM project control combined with SRA provides
Fig. 7 shows the simulated results on the empirical and reliable results in case the project has many parallel
fictitious project data. The x-axis shows the SP values from 0.1 activities. Moreover, the sensitivity information
(almost completely parallel) to 0.9 (almost completely serial) in obtained through a risk analysis enables the project
steps of 0.1 while the y-axis measures the efficiency of the manager to reduce the effort of CPM project control to
dynamic control phase including the corrective actions that have those activities with a high expected impact on the
been taken when action thresholds are exceeded. The dots project performance. Hence, a clear focus on only
highly sensitive activities in order to reduce the control
1
www.protrack.be. effort still results in a relatively high control efficiency.
260 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263

Fig. 6. Example of simulation runs to measure project control efficiency.

 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263 261

50%

40%

Control Efficiency
30%

20%

10%

0%
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Parallel networks Serial networks

Legend

EVM with SPI(t) (empirical data, with an indication of minimum and maximum value)
SRA with SSI (empirical data, with an indication of minimum and maximum value)

Fig. 7. Efficiency of project control.

– The thick line on top of the picture shows the results for a methodology, originally developed for DoD projects, has
combined control approach, performed on both the received academic attention since its development and has
fictitious as well as the empirical data. The combination received, with the development of some recent extensions such
has been set up such that the total effort is approximately as Earned Schedule (Lipke, 2003), an increasing attention the
the same as for the separate scenarios in order to have a last seven years. Schedule risk analysis using Monte-Carlo
fair comparison. Various degrees of SRA/EVM combi- simulations has been used widely in academic as well as
nations have been tested by varying the SRA and EVM practical environments and can be used to refine the black-and-
thresholds and the best option has been retained and white view on the critical path by measuring the sensitivity of
reported in the graph. The results show that a careful project activities and their potential impact on the project
combination of bottom–up and top–down project control objective.
leads to a significant increase in the efficiency of the This paper gives the construction of a baseline schedule for a
project control phase. project a central place for the integration of a schedule risk
• RQ3: the deviations between the fictitious and empirical tests analysis with an earned value management control approach in
lie close to each other, which leads to the conclusion that the order to efficiently control projects and to accurately trigger
accuracy of the simulated fictitious results is high and can be corrective actions in case the project performance drops below a
confirmed by similar tests on empirical data. It should be predefined threshold. This paper gives an integrative review of
noted that the empirical results are based on similar different studies on the efficiency of project management
simulations as the fictitious results and hence care should control. Moreover, results from an IPMA award winning
be taken to draw general conclusions. However, the simulation study on fictitious project data are compared with
empirical data consist of other project networks (the project empirical data obtained from 8 Belgian companies.
templates) and the corrective actions are in line with actions The simulation experiments have clearly shown that
normally taken by the companies from which we have information obtained during the scheduling step (baseline
obtained the data. The graphs also show that the empirical plan) as well as sensitivity information obtained by SRA and
average control efficiency lies a little bit lower than the project performance evaluation obtained through EVM should
simulated fictitious efficiency for the top–down control allow the project manager to improve the project control process
approach, while the opposite holds for the bottom–up control and the corrective action decision making process. The main
approach. We were not able to find out what the real reasons conclusions of the computational experiments can be summa-
are for this observation. rized as follows. First, it has been shown that the topological
structure of a project network is a main driver for the variability
4. Conclusions and efficiency of the project control phase. This topological
information can be easily and automatically calculated by any
Project management in general and project control and software tool, and based on this information, the software tool
performance measurement in particular have a long history in can automatically propose suggestions to the user. An example
the academic research literature. The earned value management is given in Vanhoucke and Shtub (2011) where the project
262 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263

management tool ProTrack is extended by an assistant that Brucker, P., Drexl, A., Möhring, R., Neumann, K., Pesch, E., 1999. Resource-
generates tips and hints to users based on topological project constrained project scheduling: notation, classification, models, and
methods. European Journal of Operational Research 112, 3–41.
information. Secondly, it has been shown that the efficiency of Collyer, S., Warren, C., 2009. Project management approaches for dynamic
project control can be significantly improved by combining a environments. International Journal of Project Management 27, 355–364.
top–down and a bottom–up project control approach. The Davies, E., 1974. An experimental investigation of resource allocation in
efficiency is measured as a combination of effort a project multiactivity projects. Operational Research Quarterly 24, 587–591.
Demeulemeester, E., Vanhoucke, M., Herroelen, W., 2003. A random network
manager has to put in the control process and the return the
generator for activity-on-the-node networks. Journal of Scheduling 6,
project manager gets when taking corrective actions. Clearly, 13–34.
the results show that both a top–down and bottom–up approach Elmaghraby, S., 1977. Activity Networks: Project Planning and Control by
allow the project manager to set action threshold, and hence to Network Models. John Wiley and Sons, Inc., New York.
reduce effort, while improving the positive return of his/her Elmaghraby, S., Herroelen, W., 1980. On the measurement of complexity in
actions. Finally, thanks to the observation that the results for activity networks. European Journal of Operational Research 5, 223–234.
Fleming, Q., Koppelman, J., 2003. What's your project's real price tag? Harvard
simulation experiments on a large and controlled fictitious data Business Review 81, 20–21.
set are confirmed by results on a small set of empirical data, we Fleming, Q., Koppelman, J., 2005. Earned Value Project Management, 3 rd
believe that this increases the credibility of this research study. Edition. Project Management Institute, Newtowns Square, PA.
Given the importance to understand and improve the Görög, M., 2009. A comprehensive model for planning and controlling contractor
corrective action decision making process in a dynamic project cash-flow. International Journal of Project Management 27, 481–492.
Hartmann, S., Briskorn, D., 2010. A survey of variants and extensions of the
control environment, future research lines should be set up. It resource-constrained project scheduling problem. European Journal of
lies in our intentions to focus on two funded research lines. Operational Research 207, 1–15.
First, a deeper knowledge on the impact of variability on the Herroelen, W., De Reyck, B., Demeulemeester, E., 1998. Resource-constrained
project objective is necessary to further enhance our under- project scheduling: a survey of recent developments. Computers and
standing on activity sensitivity information. Secondly, further Operations Research 25, 279–302.
Hulett, D., 1996. Schedule Risk Analysis Simplified. Project Management
research should extend the current body of knowledge on Network. July.
project performance measurements with statistical techniques Icmeli, O., Erenguc, S., Zappe, C., 1993. Project scheduling problems: a survey.
borrowed from the process engineering industry and based on International Journal of Operations & Production Management 13, 80–91.
principles taken from Statistical Process Control (SPC). The Jacob, D., 2006. Is “earned schedule” an unreliable indicator? The Measurable
increasing computer power and automated measurement News 15–21 Fall.
Jacob, D., Kane, M., 2004. Forecasting schedule completion using earned value
techniques have led to the development of statistical control metrics? Revisited. The Measurable News 1, 11–17 Summer.
techniques able to cope with a large amount of variables within Kaimann, R., 1974. Coefficient of network complexity. Management Science
a reasonable time, possibly leading to an increase in the overall 21, 172–177.
project management control efficiency. Kaimann, R., 1975. Coefficient of network complexity: erratum. Management
Science 21, 1211–1212.
Kao, E., Queyranne, M., 1982. On dynamic programming methods for assembly
Acknowledgments line balancing. Operations Research 30, 375–390.
Kolisch, R., Hartmann, S., 2006. Experimental investigation of heuristics for
We acknowledge the support by the Research collaboration resource-constrained project scheduling: an update. European Journal of
Operational Research 174, 23–37.
fund of PMI Belgium received in Brussels in 2007 at the
Kolisch, R., Padman, R., 2001. An integrated survey of deterministic project
Belgian Chapter meeting, the support for the research project scheduling. Omega The International Journal of Management Science 29
funding by the Flemish Government (2008), Belgium as well as (3), 249–272.
the support given by the Fonds voor Wetenschappelijk Lipke, W., 2003. Schedule is different. The Measurable News 31–34 Summer.
Onderzoek (FWO), Vlaanderen, Belgium under contract Lipke, W., Zwikael, O., Henderson, K., Anbari, F., 2009. Prediction of project
outcome: the application of statistical methods to earned value management
number G.0463.04. This research is part of the IPMA Research
and earned schedule performance indexes. International Journal of Project
award 2008 project by Mario Vanhoucke who was awarded at Management 27, 400–407.
the 22 nd world congress in Rome (Italy) for his study Mastor, A., 1970. An experimental and comparative evaluation of production
“Measuring Time — An Earned Value Simulation Study”. line balancing techniques. Management Science 16, 728–746.
Özdamar, L., Ulusoy, G., 1995. A survey on the resource-constrained project
scheduling problem. IIE Transactions 27, 574–586.
References Pascoe, T., 1966. Allocation of resources — CPM. Revue Française de
Recherche Opérationnelle 38, 31–38.
Ahsan, K., Gunawan, I., 2010. Analysis of cost and schedule performance of Patterson, J., 1976. Project scheduling: the effects of problem structure on
international development projects. International Journal of Project heuristic scheduling. Naval Research Logistics 23, 95–123.
Management 28, 68–78. PMBOK, 2004. A Guide to the Project Management Body of Knowledge, Third
Anbari, F., 2003. Earned value project management method and extensions. Edition. Project Management Institute, Inc., Newtown Square, Pa.
Project Management Journal 34 (4), 12–23. PMI, 2005. The Practice Standard for Earned Value Management. Project
Bein, W., Kamburowski, J., Stallmann, M., 1992. Optimal reduction of two- Management Institute, Inc., Newtown Square, Pa.
terminal directed acyclic graphs. SIAM Journal on Computing 21, 1112–1129. PMI, 2009. The Practice Standard for Project Risk Management. Project
Book, S., 2006a. Correction note: “earned schedule” and its possible Management Institute, Inc., Newtown Square, Pa.
unreliability as an indicator. The Measurable News 22–24 Fall. Rujirayanyong, T., 2009. A comparison of three completion date predicting
Book, S., 2006b. “Earned schedule” and its possible unreliability as an indicator. methods for construction projects. Journal of Research in Engineering and
The Measurable News 24–30 Spring. Technology 6, 305–318.
 M. Vanhoucke / International Journal of Project Management 30 (2012) 252–263 263

Tavares, L., Ferreira, J., Coelho, J., 1999. The risk of delay of a project in terms Vanhoucke, M., 2011. On the dynamic use of project performance and schedule
of the morphology of its network. European Journal of Operational Research risk information during project tracking. Omega The International Journal of
119, 510–537. Management Science 39, 416–426.
Tavares, L., Ferreira, J., Coelho, J., 2002. A comparative morphologic analysis Vanhoucke, M., Coelho, J., Debels, D., Maenhout, B., Tavares, L., 2008. An
of benchmark sets of project networks. International Journal of Project evaluation of the adequacy of project network generators with systematically
Management 20, 475–485. sampled networks. European Journal of Operational Research 187,
Tavares, L., Ferreira, J., Coelho, J., 2004. A surrogate indicator of criticality for 511–524.
stochastic networks. International Transactions in Operational Research 11, Vanhoucke, M., Shtub, A., 2011. Adding value to earned value analysis. PM
193–202. World Today XIII (1).
Thesen, A., 1977. Measures of the restrictiveness of project networks. Networks Vanhoucke, M., Vandevoorde, S., 2007. A simulation and evaluation of earned
7, 193–208. value metrics to forecast the project duration. Journal of the Operational
Vandevoorde, S., Vanhoucke, M., 2006. A comparison of different project Research Society 58, 1361–1374.
duration forecasting methods using earned value metrics. International Williams, T., 1992. Criticality in stochastic networks. Journal of the Operational
Journal of Project Management 24, 289–302. Research Society 43, 353–357.
Vanhoucke, M., 2010a. Measuring time — improving project performance Williams, T., 1995. A classified bibliography of recent research relating to
using earned value management. : International Series in Operations project risk management. European Journal of Operational Research 85,
Research and Management Science, vol. 136. Springer. 18–38.
Vanhoucke, M., 2010b. Using activity sensitivity and network topology Williams, T., 1999. Towards realism in network simulation. Omega The
information to monitor project time performance. Omega The International International Journal of Management Science 27, 305–314.
Journal of Management Science 38, 359–370.