PREDICTION OF PROJECT PERFORMANCE

DEVELOPMENT OF A CONCEPTUAL MODEL FOR PREDICTING FUTURE

PERFORMANCE OF AN OG&C PROJECT IN EPC ENVIRONMENT.

Naresh K. Kaushik
Delft University of technology

PREDICTION OF
PROJECT
PERFORMANCE

Development of a prediction model for

predicting future performance of an O&C
project in EPC environment
Thesis report
Public version

Naresh Kaushik
Student Number 4141555

Master of Science thesis

System Engineering Policy Analysis and Management
Faculty of Technology, Policy and Management

PROJECT DETAILS
Author: Naresh K. Kaushik
Student Number: 4141555
Email: naresh_kasuhik@hotmail.com
This report is for thesis graduation project for:

Study program: System, engineering policy analysis and management (SEPAM)

Graduation section: System Engineering
Faculty of Technology, Policy and management
Delft University of technology

Graduation Date:

5th of April, 2013

This research is performed in collaboration with

FlUOR B.V, Haarlem

Department: Project controls

Graduation committee:

Chair: Prof. dr. ir. Alexander Verbraeck

Section: Systems Engineering, Faculty of Technology, Policy, and Management

First supervisor: Dr. Mamadou D. Seck

Section: Systems Engineering, Faculty of Technology, Policy, and Management

Second supervisor: Dr. W.W. Veeneman

Section: Policy, Organization, Law and Gaming Faculty of Technology, Policy, and Management

External supervisor: Robert V. Velzen

E&C Global Leader for Project Controls/Estimating Fluor Corporation

External supervisor: Erik J. Groeneweg

Project controls manager Fluor Corporation

PREFACE
This thesis report on development of performance prediction model is the result of my
graduation thesis for master program System engineering policy analysis and management
at Delft University of technology. I performed this thesis research as graduation intern at
Fluor Corporation at their Haarlem Office.
The past 7 months of my master thesis has been a great learning experience academically,
professionally and personally. The research topic turned to be quite complex and resulted
into lot a large scope for research. However, I enjoyed every bit of this research.
At the conclusion of my research, I convey my warm thanks to my supervisors at TU
delft: Mamadou Seck, Wijnand veeneman and Alexander Verbraeck for their continuous
support and encouragement. My special thanks to Mamadou seck for extra support in form
of frequent meetings and discussion that help my research.
I would like to thank Robert V. Velzen for providing me this opportunity to conduct this
research at Fluor and his invaluable role as my supervisor. In addition, I would like to
thank Erik for his continuous guidance and feedback on my research. Furthermore, I
would like to thank everybody at Fluor Haarlem that contributed to my research in form of
semi-structured interviews and informal discussions.
Finally, I would like to thank Kees Berends, Professor Hans Bakker from shell and Ted
Ong from Exxon for providing their useful insights during interviews.
I hope you all enjoy reading the results
Naresh Kaushik
Delft, March 2013

EXECUTIVE SUMMARY
The projects in oil and chemical (O&C) industry often experience problems during their
execution, because of those problems, some of the project ends with large cost and
schedule overruns. The poor performance of projects not only affects the strategic objective
of projects owner but also poses a dual threat to engineering and construction (E&C)
companies. They negatively affect their profit margins and their business objectives. Given
the strict budget constraints imposed by the present global economic situation, owners and
stakeholders expect their projects to be delivered cost effectively and efficiently. Therefore,
it is important for E&C companies to strive for improvement in their project management
practices.
The current thesis research is a step in direction to introduce a new concept for
improvement in performance management practices. For that purpose, the research
introduces early detection of project problems as the main instrument and uses the
quantitative information from past project to develop a body of knowledge and first
conceptual model to predict the future performance of projects at their early stages.
The research is conducted in five phases, the first phase of the research explores O&C
project and their performance management practices. Based on the gathered knowledge via
literature study and available information, the main research question is formulated as
How can future problems and performance of a current O&C project be predicted at early
stages using knowledge and experience from past projects in an EPC environment?
Thereafter, a series of sub questions were formulated aimed to answer the above-mentioned
research question. The later part of the first phase developed a structured research approach
and research methods.
In the second part of the research, efforts were directed to find the so-called early
warnings of problems. To identify the early warnings, two main sources were explored,
literature and experts from O&C project industry. Each investigation into respective
sources resulted into number of early warnings. Each identified early warning was
evaluated on selection criteria with three selection parameters. After the careful evaluation,
the following ten early warnings were selected.
ID
LES
PTE
COC
NCO
CCO
FED
PH
PS
CE

Early warning indicator

Lack of understanding of project execution strategy among project team
Project team lacks experience required for the project
Conflicts between owner and E&C contractor
Numbers of change orders
Cost impact of changes
Percentage of missing information in FEED package
Growth in process man-hours
Delay in process engineering
Change in concurrency level between process and piping engineering

DPO Delay in issuance of purchase orders

The selected early warnings were carried to the third phase of the research, in which four
detailed case studies were performed to have observatory evidence. The case studies in this
phase consisted of four project with different performance levels. The difference in
performance levels of case projects set the contrast in which the predictive capability of
early warnings could be observed. The case study investigation found that there is a
relationship between early warnings, project problems and project performance.
After obtaining the observatory evidence, the fourth phase of the research adopted a purely
quantitative approach and studied the behavior of early warnings in a relatively larger set of
past projects. Subsequently correlation analysis was performed to find correlations between
early warnings and final project outcomes (which collectively asses the project
performance). The quantitative analysis did present interesting and encouraging results.
The main results are mentioned as follows:
I. Early warnings do behave differently in case of poor and good performance
projects, few in terms of their absolute value and few in their incremental changes.
II. Correlations do exist between EWI and project outcomes, however not all the EWI
found to be correlated with all project outcomes.
III. The EWI indicators does show a dynamic quantitative relationship with project
outcomes over engineering duration of the project
Using the results from quantitative analysis, an attempt is made in the last phase of this
research for the development of prediction model, which can predict the future
performance of projects. The results of pilot prediction model were analyzed and compared
with forecasts made via traditional forecasting methods. The comparison of forecasts found
that prediction model does make prediction that is more accurate. However, there are errors
with-in prediction models. In addition, the external validation of model suggested limited
reliability and accuracy of pilot model.
The dataset used for quantitative analysis and building of prediction model is relatively
small and limit the generalization of findings. Therefore, to have a more accurate prediction
in good projects, a dataset is required which contains a balance of Successful and less than
successful performance projects. Despite the smaller dataset, the findings and approaches
presented in this research can be used to build a useful model and subsequently applied in
O&C project industry. A set of insights and recommendations (short term and long term)
has been made for Fluor to implement the findings of this research to develop an
operational performance prediction system.
The research possibly has following main contributions to scientific and industry.
Contribution to scientific community
I. A shift from reactive project management to proactive project management

II. A new and constructive role of past projects

Contribution to O&C project industry
I. An approach, which facilitate the early detection of future potential problems
II. An approach to capitalize on past projects to improve project performance
management
Note: The confidentially apply to the part of attachments, therefore are not attached with
this report.

TABLE OF CONTENTS
1

Introduction .....................................................................................................................12

Research description .......................................................................................................16

2.1
Summary ................................................................................................................16
2.2
Overview of oil & chemical project execution.......................................................16
2.3
Research problem ..................................................................................................21
2.4
Research questions ................................................................................................25
2.5
Research goals and deliverables ...........................................................................26
2.6
Relevance...............................................................................................................26

Research design ...............................................................................................................28

3.1
Summary ................................................................................................................28
3.2
Research scope ......................................................................................................28
3.3
Fundamental approach..........................................................................................28
3.4
Research methods ..................................................................................................32

Literature study................................................................................................................34
4.1
Summary ................................................................................................................34
4.2
Author affiliations..................................................................................................34
4.3
Concept of project success.....................................................................................35
4.4
Concept of early warnings.....................................................................................40
4.5
Conclusions and discussions .................................................................................45

Early warnings in projects...............................................................................................47

5.1
Summary ................................................................................................................47
5.2
identification of early warnings.............................................................................47
5.3
Selection criteria of early warnings ......................................................................48
5.4
Selection of Early warnings...................................................................................49
5.5
Early warnings from literature..............................................................................50
5.6
Early warnings from experts .................................................................................53
5.7
Early warning indicators.......................................................................................58
5.8
Discussion and conclusion ....................................................................................62

6
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
7

Case Studies.....................................................................................................................63
Summary ................................................................................................................63
Case study design ..................................................................................................63
Case study selection...............................................................................................64
Case 1 (Less than successful project) ....................................................................65
Case 2 (successful project) ....................................................................................68
Case 3 (successful project) ....................................................................................71
Case 4 (Less than successful project) ....................................................................73
Cross case analysis................................................................................................77
Discussion and conclusion ....................................................................................79
Quantitative analysis .......................................................................................................81

Summary 81
7.1
Analysis approach .................................................................................................81
7.2
Exploratory data analysis......................................................................................82
7.3
Quantitative analysis .............................................................................................85
7.4
Correlations over engineering duration ................................................................89
7.5
Discussion and conclusions...................................................................................93
8

Development of prediction model ....................................................................................97

8.1
Requirements and guidelines for performance prediction model..........................97
8.2
Selection of prediction Methodology .....................................................................98
8.3
Prediction model development approach ..............................................................99
8.4
Development of pilot Prediction model ...............................................................102
8.5
Model evaluation methods...................................................................................104
8.6
Analysis of predictions.........................................................................................105
8.7
External validation ..............................................................................................110
8.8
Final evaluation...................................................................................................114
8.9
Integration of project problems with prediction model .......................................115
8.10
Discussion and conclusion ..................................................................................117

Insights and recommendations for implementation......................................................119

9.1
Insights and recommendations ............................................................................119
9.2
Recommendations for implementation ................................................................120

10

Conclusions and reflections...........................................................................................125

10.1
Revisiting research questions ..............................................................................125
10.2
Answer to the main RESEARCH question ...........................................................129
10.3
Discussion on research goals and deliverables...................................................130
10.4
Contribution to scientific community...................................................................130
10.5
Contribution to O&C project industry ................................................................131
10.6
Final reflections...................................................................................................132
10.7
Future research opportunities .............................................................................134

11

References......................................................................................................................135

LIST OF FIGURES
Figure 1 Success and failure of O&C projects........................................................................................... 13
Figure 2 Phases of OG&C projects............................................................................................................ 17
Figure 3: The generic control cycle............................................................................................................ 19
Figure 4: Conceptual procedure for controlling of projects ..................................................................... 21
Figure 5: Cost of reactive approach........................................................................................................... 23
Figure 6: Existing knowledge gaps............................................................................................................. 24
Figure 7: The wheel of science (Wallace, 1971) ........................................................................................ 29
Figure 8: Fundamental research approach................................................................................................ 31
Figure 9: Iron triangle of projects.............................................................................................................. 36
Figure 10: Project success criteria............................................................................................................. 37
Figure 11: 95 % engineering completion milestone .................................................................................. 39
Figure 12 Potential benefits of EWI ........................................................................................................... 45
Figure 13: Early warning selection criteria............................................................................................... 48
Figure 14 Classification of early warnings from literature by source ...................................................... 51
Figure 15 Early warnings from literature by sub-category ....................................................................... 52
Figure 16: Early warning from experts by sub-category........................................................................... 56
Figure 17 Early warning mentioned by numbers of experts ...................................................................... 57
Figure 18 Framework for mapping the relationship between early warnings, project problems, and
project outcomes.......................................................................................................................................... 64
19-27 Confidential
Figure 28: Quantitative analysis approach................................................................................................ 82
28-37 Confidential
Figure 38 Significant correlations between EWI and project outcomes at each prediction moment...... 91
Figure 39 Significant correlations of project outcomes with EWI over engineering duration................. 92
Figure 40 : Step approach for development of prediction model ............................................................ 101
Figure 41 Predictive capability comparison of traditional method and developed prediction tool....... 105
Figure 42 : Errors in prediction of final TIC ........................................................................................... 108
Figure 43 Errors in prediction of ESI....................................................................................................... 108
Figure 44 Errors in prediction of MHI..................................................................................................... 109
Figure 45 Errors in prediction of MCI ..................................................................................................... 110
Figure 46: Model validation: prediction of TIC....................................................................................... 111
Figure 47 Model validation: prediction of MCI....................................................................................... 112
Figure 48: Model validation- prediction of ESI ....................................................................................... 113
Figure 49: Usability of prediction Model................................................................................................. 121

Figure 50 future project problems and project outcomes associated with NCO .................................... 128
Figure 51: synthesis of answer to main research question ...................................................................... 129
Figure 52: Data collection moments ........................................................................................................ 153

10

KEY ABBREVIATIONS
O&C

Oil and Chemicals

EPC

Engineering, Procurement and Construction

E&C

Engineering and Construction

E&P

Energy and petroleum

BOD

Basis of Design

BDP

Basic Design Package

CII

Construction Industry Institute

FEED

Front End Engineering Design

IPA

Independent Project Analysis

PEP

Project Execution plan

EVM

Earned Value Management

EWI

Early Warning indicator

ESI

Engineering Schedule Index

TIC

Total Installed Cost

MCI

Mechanical Completion index

COP

Cost of Problems

11

1 INTRODUCTION
The oil and chemical owner companies rely heavily on engineering and construction (E&C)
companies to meet their strategic objectives such as building of a new assets, expansion
and performance improvement of existing assets. Moreover, given the strict budget
constraints imposed by the present global economic situation, owners and stakeholders
expect their projects to be delivered cost effectively and efficiently. E&C companies are
working hard to match these expectations by changing their project management methods,
tools and the way they execute projects.
However, there are sufficient examples of projects, where E&C companies face problems
in meeting their as sold cost estimates, agreed upon schedules and desired quality
requirements. The number of project, that fail to meet their stated objectives vary
significantly per industry, mainly due to the difference in complexity, the industrys market
dynamics, the type of stakeholders and their influence levels. Many researchers
investigated the reasons for poor performance of projects (Flyvbjerg & Bruzelius, 2003;
Morris & Hough, 1987; Turner, 1999; Thamhain & Wilemon, 1986).
For example, handbook of project-based management by Turner mentions several reasons
for projects poor performance such as poor project establishment in terms of priorities, bad
initial planning, inefficient control procedures and many more (Turner, 1999). Flyvbjer and
Bruzelius (2003) suggested that in projects decision-making, planning and management are
typically multi-actor processes with conflicting interests and therefore, projects are often
faced with mistrust, violation of good project governance practices, ambiguity and poor
collective decision-making (Flyvbjerg & Bruzelius, 2003). The above-mentioned behaviors
of stakeholders penetrate through the permeable boundaries of project plans and can lead a
project to high cost and schedule overruns.
In this respect, projects in the oil and chemical (O&C) industry are no exception. Although
the performance of O&C projects seems to be better than that of civil or mining projects,
there are still ample examples of poor performing projects. Mckenna, Wilczynski and
Vandersee (2006) estimated that about 30-40 % of capital project in O&C industry suffer
from a budget and/or schedule overrun larger than 10%.
Figure 1 shows the result of a study conducted by Independent Project Analysis (IPA). The
study includes 318 projects across the O&C industry. Out of those projects, only 50% can
be categorized as successful. The other 50% incurred either 33% cost overrun and/or
schedule overrun of more than 30% (Merrow, 2012). Two third of the projects even failed
to meet the production schedule or targets, thus affecting the profitability of its investors.
The above results definitely are of serious concern for both the E&C and the owner
companies.

12

Figure 1 Success and failure of O&C projects

(Source: IPA, 2012)

From the above discussion, it can be concluded that O&C projects had experienced
problems in the past and might encounter problems and challenges in future. There have
been multiple attempts by the academic as well as industry experts to explore potential
areas of improvement such as risk management, stakeholder management, benchmarking
practices, and project control practices to improve the situation. Accordingly, there have
been achievements such as development of value improvement practices (VIP), industrys
best practices, and front end loading (FEL) to mention a few. However, the majority of the
research has been focused in the frond end phase of the projects.
Prominently, the importance of the FEED phase for improving project performance is
suggested over the years (Artto, Lehtonen, & Saranen, 2000; Thamhain & Wilemon, 1986)
and little focus has been given to the execution phase of the project, where the problems
actually surface and affect the project performance. The control mechanism of project
execution phase (EPC) has seen little advancement and is still relying on the principles
defined in 50-60s such as principle of deviation management (Vanhoucke, 2011;
Nikander, 2002).
Why the deviation based traditional control mechanisms would not be suitable for
successful control of project execution? There are two major problems with traditional
deviation based control methods. First, the deviations are reported on aggregated level
therefore, the poor performance in one part of the project is masked by good performance
in other part of the project (Vanhoucke, 2011; Nikander, 2002). Secondly, even if the
localized deviations are observed, they are seen in limited manner. The cascading effect of
localized deviations on other activities is neither reported nor anticipated by these methods.
Therefore, the accuracy of future forecasts of project performance based on the deviations
is somewhat debatable.
As a consequence to above mentioned fallacies in deviation management principles, often
problems in a project are not visible until they are already manifested and degraded the
project performance. The corrective strategy to manifested problems can be termed as

13

reactive approach, as the mangers act to correct what has already gone wrong. This reactive
approach brings additional schedule requirements and incurs substantial cost, thus add into
the cost and schedule overrun of the projects.
In addition, forecasting of future project performance based on traditional methods is
vulnerable to optimism bias. The mangers are often seen as very optimistic against the
localized deviations and do not consider them as potential risk to future activities.
Interestingly, forecasters never mention optimism bias as a main cause of inaccurate
forecasts.
Then the question arises, where to look then for the improvement in the project control
management? A guided investigation of poor and good performance of past O&C project
could provide us an answer to this question. If problems could not be eliminated from
projects, can we predict problems, allowing longer correction time at lesser cost? This
capability will allow for their pro-active management with considerably less cost and
schedule impact.
The aim of this thesis is to take a first step in creating a scientific understanding of
prediction of problems via early warnings. Using this understanding, an attempt is made
within this research to build a quantitative model to predict the future performance of
project based on selected identified early warnings.
Looking at the different chapters that build this thesis, chapter 2 provides the background
for conducting this research by defining important concepts and delineating the main
research problem. The problem delineation guides the formulation of research questions.
Furthermore, research goals are introduced, the relevance of these goals is explained and
the main deliverables are defined.
In chapter 3, the design of this research is presented. The fundamental approach is
described with logical sequence of research phases. Subsequently, the employed research
methods and tools are explained and coupled with the goals set in chapter 2.
A literature study regarding project performance of O&C projects is provided in chapter 4.
The adopted measures of O&C project performance are presented. In this literature study,
the concept of early warnings is explored and relevant literature is reviewed. The chapter
also highlights the potential benefits of operationalizing early warnings in projects. Chapter
5 describes the identification of the early warnings from literature and from experts from
O&C project industry. Furthermore, the selection criteria for selecting key early warnings
are formulated, by focusing on the main objective of research. Each early warning is
evaluated on selection criteria and few were selected for further analysis.
In chapter 6, in-depth case studies are performed with an objective to have the preliminary
evidence of relation between early warning, problems and their relation with project
performance. In addition, Individual case conclusion and cross case analysis is performed
and presented.
In chapter 7, quantitative analysis is performed to 1) analyze the dynamic behavior of EWI
over engineering duration of the project to map the behavior with successful or less than

14

successful projects. 2) Correlation analysis of EWI with specific project outcomes and 3)
Longitudinal correlation analysis to find suitable EWIs for development of prediction
model at each prediction moment
In chapter 8, based on the past project data, early warnings are assigned quantitative
indicators and an effort is been made to build a performance prediction model. The results
of developed pilot model are analyzed and external validation is performed.
Insights and general recommendations are provided in chapter 9. In addition, a short term
and long term implementation strategy is presented in chapter 9.
Finally, chapter 10 concludes the research by revisiting the research questions and their
answers and evaluating the contribution to scientific and O&C project industry.
Reflections have been made towards research approach, adopted methods, and results of
the research.

15

2 RESEARCH DESCRIPTION
2.1 SUMMARY
The main objective of the current chapter is to understand the research problem, its context
and the research questions, which needs to be answered in order to find a solution to the
main research problem. In addition, the scientific and social relevance of research had been
provided.
This objective has been achieved sequentially by understanding the 1) execution of O&C
projects 2) their controlling mechanisms.
With the understanding of context, a critical review of current practices enabled the
delineation of research problem and existed knowledge gaps. The problem delineation
helped in forming the main research question. Furthermore, the main research question has
been broken into sub questions that need to be answered to obtain the solution to main
research problem.
The section 2.3.1 provides the overview of oil and chemical projects. Section 2.2.2
provides information on the subject of controlling mechanism of projects. Section 2.2.3,
integrates the above two sections and shift the attention specifically on current project
controlling mechanism employed in O&C projects.
Section 2.3 provides a critical overview of the current controlling mechanism and describes
the research problem. Based on the defined research problem, research questions are
formulated in section 2.4. Section 2.5 describes the research goals and main research
deliverables followed by relevance of research in scientific, social and business domains
(section 2.6).

2.2 OVERVIEW OF OIL & CHEMICAL PROJECT EXECUTION

2.2.1

Oil & Chemical projects

O&C plants are also addressed as process plants, mainly due the fact that they have
chemical processes at their heart. The chemical process convert the input (Crude oil,
chemicals) into other chemicals with higher economic value (Fuels, industrial chemicals)
by means of mechanical equipments, auxiliary facilities and the infrastructure to support
the whole plant.
The process plants are strategic assets of major petrochemical companies and are
fundamental to their business. Furthermore, the O&C chemical projects should not be seen
just as economical assets, they do contribute significantly in meeting the rising demands for
energy of society as a whole. Although an increase in production of renewable energy is
expected, experts still believe that the O&C industry will play an important role as energy
producer, at least in near future.

16

O&C projects are capital intensive and do require systematic economic and project
planning to deliver their intended results. Therefore, almost every (O&C) project is
executed in systematic phases and its project life cycle encompasses the total time between
identification of the project need to its completion.
The different phases in project life cycle are (sub) projects in themselves and are separated
by gates or decision points. The gated project lifecycle means that at certain points in the
life cycle of project, the evolving design or plant concept and associated parameters (e.g.
cost, schedule, and environmental impact) must pass through certain decision/review gates.
The gated process allows for the evaluation of options based on the intended objectives of
its stakeholders and consequently the selection of optimal option. In this sense, the gated
process for a project allows for the structured way of decision-making. In addition, due to
the comprehensive reviews, the project stakeholders are more informed about the
deficiencies and/or risks in the project at a certain gate.
The figure 2 shows the stage gated project life cycle of a typical O&C project from scope
definition phase to its completion. Harpum, in his article in book titled: The Wiley guide
to managing projects defined the basic rules for a project to pass through these gates
(Harpum, 2004). However, the specific rules and passing requirements differ according to
the individual companys procedures and criticality of a project.

Figure 2 Phases of OG&C projects

(Adapted from: The Wiley guide to project management, 2004)

17

Furthermore, in practice, the strictness of these gates also depends upon many other
factors such as capital expenditure, urgency of project, contracting philosophy of owner
to name a few. The following paragraphs describe each phase of an O&C project in brief
manner.
At scope definition level, the requirements of owner are identified and what has to be
done is defined on a broader level. At the conceptual phase, basic functional
characteristics of a project are described as a system in terms of input(s), throughput(s),
outputs and major equipments required to achieve the desired production. In addition, the
major interconnections between subsystems of a project are determined based on the
process philosophy of the project (CII, 2004).
Subsequently, the preliminary design is performed to provide basic design information i.e.
process flow sheets, general design specifications, preliminary equipment specifications
and their arrangements, preliminary plot plans, preliminary estimates and preliminary
project execution strategy. In oil and chemical industry, conceptual and preliminary
engineering phase together are called front-end engineering design (FEED) and a key
deliverable at the end of FEED phase is the basic design package (BDP) (CII, 2004).
In the detailed engineering design phase, the BDP is detailed further as engineering
disciplines initiates detailed engineering in their respective domains. The main deliverables
of this phase are technical, procurement and construction documents. Table 1 shows the
main engineering disciplines typically involved in typical O&C project and their associated
main deliverables.
Table 1 Main engineering deliverables of detailed engineering phase
Engineering disciplines

Key deliverables in detailed design

Process engineering

Process and instrument diagrams (P&ID), equipment

and Instrument requirement list, control and relief valve
specs

Mechanical

Equipment data sheets and equipment bid evaluation

Piping engineering

Plot plan, Piping design, stress calculations, Iso metrics

and plant 3D model

Civil,
structural
Architectural

and

Electrical and control systems

Foundations drawings, Structural steel drawings

Power system design, instrument data sheets, DCS
specification

In procurement phase, the buying process is initiated based on the design specifications of
equipment, instruments and materials. Later the contracts for civil works and installation of
mechanical, electrical, instruments and piping materials are awarded.

18

During the construction phase, the facility is constructed according to the drawings and
specifications prepared during detailed design phase using material and equipment obtained
via procurement. In the start-up phase equipment are subjected to testing and inspection,
both individually and in combination to validate the proper functioning of the facility. The
phases detailed engineering, procurement and construction phase together are commonly
known as EPC phase of project (CII, 2004).

2.2.2

Controlling of projects

Controlling is the measurement and correction of performance in order to make sure that enterprise
objectives and the plans devised to attain them are accomplished.
- Harold Koontz (1909-1984)

By definition, the control in project execution is exercised by measurement and comparing

of what was planned with what is being done i.e. finding the deviation between the
planned (known as baseline) and the actual. Figure 3: The generic control cycle Shows the
generic control cycle employed in a project.
The deviations could be caused by internal sub optimal performance and/or by influences
from external environment penetrating the permeable boundaries of project.
Fundamentally, control tries to make sure that the project stays on course to meet its predefined objectives and goals. By definition, good monitoring and control mechanism
provides a better performance management over a project.

Figure 3: The generic control cycle

(Source: Brandon, 2004)

In the control cycle, What to measure varies with the type of project and the perspective
of the organization managing the project. The same is true for how to measure.

19

Corrective actions are management prerogatives that are available to project manager based
on the type of organization and authorities of the project manager. Taking action to
improve the performance refers to the corrective action necessary to bring deviation to a
minimum level. Various examples of corrective action employed in project are fast
tracking, adding additional resources, scope reduction, trade-offs, increasing risks and
disciplinary actions and so on. Moreover, a specific corrective action is depending on the
type of problem causing the deviation.

2.2.3

Controlling of O&C projects:

Having defined the control mechanisms, the project execution control of O&C projects
could be seen in similar manner except the variables to be measured and tools could vary in
accordance with O&C projects.
The section 2.2.2 implies that for controlling, the first requirement is to establish a baseline
against which we could measure the deviation and actual performance of project. To
establish a project baseline for an O&C project, the following project information should be
in available.
I. Overall cost estimates (-10%/+20% variation)
II. Work scope (refers to activities need to be accomplished to achieve the project
objectives)
III. Cost breakdown structure (Cost associated with activities i.e. services, equipments,
overheads, contingency)
IV. Project approved schedule (Milestones dates, activity durations)
V. Comprehensive risk analysis along with accepted risks, planned mitigation
strategies and actions
VI. Commercial baseline: As sold pricing, time bound revenue and margins.
The above documents act as basis for baseline developments. The final baselines for scope,
schedule and cost are established along with control strategies and parameters to identify
the deviations from the baseline.

20

Figure 4: Conceptual procedure for controlling of projects

(Source: Fluor Corporation)

The Figure 4 above shows the applied concept of control cycle, specific to O&C projects.
As soon as the project proceeds into detailed engineering execution, progress and
performance are measured and monitored. In addition, the risks are monitored and dealt
with during the course of execution.
The progress in engineering, procurement and construction is monitored through earned
value1 (EVM) concept with visualization via progress curves (cost progress and schedule
progress). The primary instruments of project control are deviations between planned value
of work (PV), earned value of the work performed (EV), actual cost (AC).
Performance ratios are calculated at project level, phase level and discipline level,
signifying the performance at respective levels. Based on the deviations and performance
ratios, the required resources and cost for the balanced scope of work is forecasted along
with incorporation of any strategy to recover the deviations (Vanhoucke, 2011). Along with
cost and schedule performance ratios, multiple key performance indicators (KPI) such as
safety performance, quality performance are monitored.

2.3 RESEARCH PROBLEM

Earned value management is a concept, in which progress is measured via integration of scope, cost
and schedule. (For more information on EVM, please refer Christensen, 1998; Lipke et.al, 2009.)

21

2.3.1

Problem in current controlling practices

During project execution, projects are evaluated periodically using above described
parameters such as earned value, performance ratios, KPIs and variances from baseline.
Such conventional methods are based on the principle of deviation management. At a
certain moment in time, aggregated deviations reflect two aspects of project execution 1.)
How much project is deviating from its baseline 2) given the deviations, how the project is
performing i.e. performance of project?
When aggregated deviations in the project are visible and are regarded as significant, it
implies that there is/are problem(s) that has already manifested and degrading the project
performance: the problem can no longer be avoided. After identifying deviations and nondesirable performance ratios, backward analysis is performed to search for the problems
and strategies to manage the impacts of the problem(s).
It should be noted that the deviations in project are mostly seen on aggregated level and the
impacts of deviations within an area are often seen as limited that area, as their impact on
total project performance is not clear. These localized problems become more critical if
they have significant effect on downstream parts of the project. However, in current
practices, these localized problems are not seen as problems but overlooked by aggregated
performance of project might be still in acceptable limits. Furthermore, when localized
problems develop into project problems, their delayed identification leads to additional
cost.
Figure 5 below explains this current problem more explicitly. The additional cost due to
reactive approach called as cost of reactive approach which could be significant based on
the nature of the problem and the timing of problem detection.
Generally, this cost of reactive approach contributes significantly to cost overrun on
projects. In addition, if the reporting of deviations is delayed due to any reason the cost to
fix those problems will increase significantly, driving the project cost and schedule way off
the baseline. The key to manage a project with predictability and certainty is to manage the
problems before they affect the project outcomes. In other words, acting proactively based
on the symptoms of problems (termed as early warnings) rather than reactively to the
problems.

22

Figure 5: Cost of reactive approach

If these localized problems appear in the early phase of a project, given the interdependent
nature of project activities in O&C projects it is almost certain that will have negative
effect of downstream activities. However, the cascading effect of these problems at
aggregated level performance reporting is likely delayed. Therefore, the future forecasts
and project performance based on aggregated current performance is inaccurate.
The above paragraphs clearly indicate that the current controlling and performance
management practices lack the capability to detect the problems early enough and are
always somewhat late. In addition, the forecast based on these traditional methods might
not capture the change in dynamics of project due to localized problems.
Thus, rather than minimizing the cost and schedule overrun in projects they add to it by
providing inaccurate picture of project performance. However, if the localized problems
can be measured as early warnings in projects and proactive management of these early
warnings could minimize their impact and could significantly reduce the cost and schedule
overrun in projects.
In addition, having a more focused proactive approach can predict the future performance
of project with more certainty, But how can E&C companies can achieve that is still to be
discovered. Despite the vast body of literature covering the topic of project control and
project performance, there is still no clear knowledge regarding early detection of problems
and performance prediction based on the early warnings of problems (Vanhoucke, 2011;
(Nikander & Eloranta, 2001). Most of the literature either focuses on quantifying
deviations, diagnosis of deviation cause or corrective action decision making signifying a
clear knowledge gap.

23

Diagrammatically, the existing knowledge gap can be presented by Figure 6 below.

Figure 6: Existing knowledge gaps

2.3.2

Under-utilization of past projects

Although in the domain of project management, each project is considered unique, certain
types of projects are characterized by large amount of similarities, especially in types of
deliverables, work sequence and procedures.
For example, building of a O&C facility follows the same sequence of engineering design.
Engineering deliverables flow through a logical path across these disciplines thus having
standardized engineering work processes (CII, 2004; Fluor, 2012). Similarly, dependencies
exist between engineering, procurement and construction. The above dependencies are not
completely one directional rather a cycle of reviews and revisions occur before a design is
finalized. As a result, the dependencies multiply the negative effects of a local problem and
significantly influence the project performance.
A detailed analysis of past projects can provide us with valuable quantitative and
qualitative information regarding relationships between early warnings of problems and
their impact on project performance.
In the EPC phase of a project, the engineering cost is significantly lower compared to
procurement and construction costs. However, actions and decisions during engineering
influence the cost of a project much more than during procurement and construction,
therefore detection of potential problems in the engineering phase can avoid large amount
of rework/problems that ultimately cost/schedule overruns.
However, at an O&C project industry, this knowledge is not captured in practice and only
limited information is extracted from past projects. The current practices can be seen as
under-utilization of past project information. Almost none of the available literature focus
on capturing the actual quantitative relationship between early warnings and project
performance. Rather they stress on capturing more qualitative information as lesson learned

24

or limited quantitative data for improving their benchmarking database (Barber, 2004;
Williams, 2004).
This fallacy in past project analysis is another identified knowledge gap, which this
proposed research intends to fill. The fulfillment of above two identified knowledge gaps
can be seen as complement to each other towards the development of performance
prediction model.

2.4 RESEARCH QUESTIONS

Having provided a background of the topic and description of the problem that the
proposed research intends to tackle, the main research question is formulated as follows:

How can future problems and performance of a current O&C project be predicted at
early stages using knowledge and experience from past projects in an EPC
environment?

In order to find the answer to this main research question, it is necessary to proceed
systematically through a series of sub questions. The first set of sub-questions will
investigate performance assessment criteria employed in O&C projects and the concept
early warnings of potential future problems.

RQ.1 What constitutes project success and what are performance assessment criteria of
O&C projects?
RQ.2 What do we understand by early warnings of project problems?
The second set of sub questions will focus on identifying the early warnings in project
execution in general followed by identifying early warnings that are specific to O&C
projects. After having a set of early warnings the efforts will be directed to search the early
warnings that can be used in an accurate performance prediction model.
RQ.3 What early warnings can be identified in project execution?
RQ.4 Which early warnings can be operationalized to build a performance prediction
model.
The third set will use the identified early warnings in RQ.4 and investigates their detection,
problem prediction capability and their relation with project performance
RQ.5 What are the dynamics between early warnings and project performance?

25

The final set of sub questions investigates the development of prediction model for
predicting the probable future performance of O&C projects.
RQ.6 What early warnings are indicative of deviation in project performance?
RQ.7 How performance prediction model could predict the future performance of O&C
projects?

2.5 RESEARCH GOALS AND DELIVERABLES

Having described the research problem and main research question, this thesis ultimately
aims to achieve following goals
I. To provide a new scientific base for understanding and analyzing the early
detection of project problems in capital O&C projects
II. To present a new scientific approach which facilitates more constructive utilization
of knowledge from past projects and exploring the power of prediction modeling
for successful performance management of capital O&C projects
In order to meet the above-mentioned goals, this thesis intent to deliver
I. An overview of early warnings to predict future problems in projects derived from
both academic literature and industry leaders, with observatory and quantitative
evidence from real past projects.
II. A methodology for analyzing early warning indicators in projects, their associated
future project problems and project performance
III. A conceptual performance prediction model systematically derived from
quantitative information from past projects.
Apart from this thesis, a set of recommendations will be presented along with conceptual
performance prediction model to Fluor Corporation

2.6 RELEVANCE
The relevance of the research results presented in this thesis is both scientific and social.

2.6.1

Scientific relevance

This thesis will contribute to scientific knowledge on project management, with a specific
focus on project execution of O&C projects, by
I. Exploring and gathering industry specific knowledge regarding early detection of
future project problems

26

II. Providing a methodology to utilize past project information by pointing out the
early warnings and their relations to project performance
III. Exploring usefulness of performance prediction modeling techniques in project
management
The points mentioned above can act as a starting point for future research in project
management, marking a shift from traditional methods of project control to more enhanced
performance prediction. In addition, the content of this thesis will highlight the usefulness
of past project data, beyond their current use as estimation and planning benchmarks.

2.6.2

Social relevance

The insight gained from this research can be used to improve controlling practices in O&C
projects. The systematic process of early problem detection and development of prediction
model will be most important contribution, which can be applied to other industries. The
concept can be extended to other industry such as offshore facility development or civil
infrastructure.
More realistic predictions could lead to more proactive and informed decision making and
ultimately to better project performance. In a world of projects, where the capital
investments are high and efficient capital utilization is a prerequisite for development of
new projects, an improved project performance can provide strategic certainty in capital
planning.

27

3 RESEARCH DESIGN
3.1 SUMMARY
As indicated in chapter 2, the main objective of this research project is to develop a model
to predict the future performance of projects at early stages by capitalizing on the
knowledge from past projects. The main instrument argued in chapter 2 for development of
such capability is the early warnings of problems in projects.
To direct the research efforts towards the achievement of the objective, a clear research
approach has been designed and is presented in this chapter. The section 3.2 illustrates the
scope of the research together with the argumentation for its selection. Subsequently, the
chapter provides a blue print of the researchs fundamental approach (section 3.3).
The section 3.4 aims to provide an overview of the research methods, tools and data
collection methodology. The chapter concludes by discussing the possible limitations of the
adopted research approach.

3.2 RESEARCH SCOPE

To tackle the research problem efficiently, it is wise to limit the scope of research project
around relevancy of existed knowledge gaps. The present research focuses on EPC phase of
the project, which means phases between start of detailed engineering and mechanical
completion. More specifically, the research is focused on project within O&C industry,
which consists of either refineries or petrochemical processing plants and exclude offshore
projects.
In present research, the perspective of main engineering and construction (E&C) contractor
is adopted, mainly due the fact that throughout the EPC phase of the project, E&C
contractor is the main custodian and has primary responsibility to deliver the project as per
agreed term and conditions. In addition, the present research has been conducted with
significant support from Fluor Corporation, which is a renowned multinational E&
company and main stakeholder in this research.
The research is performed with in project controls department at Fluor Corporation at their
Haarlem office. Fluor Corporation is one of the largest multinational E&C contractors and
executed many O&C projects since its inception 100 years ago. The past projects executed
by Fluor Corporation are the primary sources of industrys project execution practices and
past project data.

3.3 FUNDAMENTAL APPROACH

In a research approach, two main methods of logic can be distinguished: deductive and
inductive reasoning. These are described in a well-known wheel of science (Wallace,
1971). The starting point of the present research is deductive in nature; Theory of weak
signals or early warnings is explored analogically in project management domain. This is
done by exploring the relevant scientific and professional literature. To compensate for the

28

practical deficiencies in literature on early warnings, expert interviews are conducted to

gain in depth knowledge of early warnings and project performance in O&C project
execution. Based on the literature sources and expert interviews the main hypothesis is
formed and defined as early warning indicators have a (in) direct relationship with project
performance. Consequently, the early warnings have the ability to predict project
performance. In the subsequent deductive phase, the hypothesis has been put to test via indepth case studies. Case studies based on past projects are performed to have observational
evidence of the hypothesis.
Subsequently, in the induction phase of the research, quantitative analysis is performed on
a larger set of past projects to have an empirical evidence of prediction ability of early
warning indicators. Based on the finding of quantitative analysis, a conceptual prediction
model is build and has been validated. In the last part of the research, conclusions are
formed based on results obtained from model and its validation. In the final section,
recommendations are made for implementation of conceptual model and future research
work.

Figure 7: The wheel of science (Wallace, 1971)

The fundamental approach has been illustrated in figure 8 with a detailed description in
following paragraphs along with the research processes. The research methods are
described in more detail in section 3.4.
In the first phase of the research, the concept of project performance is explored and criteria
for measurement of project performance are defined. The second part of this phase includes
exploration of early warnings concept, its application in project management and its
potential benefits during control of project execution.
In the second phase of the research, relevant literature is explored and experts are
interviewed to find out to what early warnings can be detected during execution of O&C
projects. Semi-structured interviews are held with experts in the O&C industry. The
majority of expert interviews are conducted within Fluor Corporation, along with some
experts from owner companies (to get the perspective of project owners). The second phase
is concluded by consolidating the early warnings from both literature and interviews,

29

followed by the selection of those early warnings that are used in further investigation. The
employed selection criteria are based on main objective of the research project.
Third phase of the research is focused on the in-depth explanatory case studies. The cases
are selected from projects executed by Fluor in the past. Choosing different projects within
one company reduces the variations in execution procedures of projects, as all the projects
were executed with more or less same standard of project execution processes. The selected
projects include both Successful and less than successful performance projects to set the
contrast in which the differences can be visible.
In observatory sense, this phase is used as a reality check of our hypothesis and at the same
explained the relationship between early warnings, project problems and project outcomes
(performed in subsequent sections). As a result, this phase has a more explanatory
character.
The fourth phase of the research is purely quantitative in nature and investigates the
quantitative data from past projects with an objective to establish the predictive relationship
between early warnings and project performance. The quantitative data from eight O&C
projects is collected via available project documentation such as close out reports, project
status reports and detailed monthly progress reports.
The final phase of the research explores the methodology for building the prediction model
and presents the model itself. In this phase, several quantitative prediction methods are
presented and discussed, followed by selection of stepwise multi-regression as adopted
method. The developed model has been evaluated with a new past project (different from
projects those used to develop the model).
The research is concluded at two levels,
I. Presenting a set of recommendations and implementation strategy for Fluor
Corporation to adopt the model in their project control processes
II. Discussing the results of each phase and drawing conclusions from them and
subsequently integrating the parts of research to provide answer the main research
question.

30

Figure 8: Fundamental research approach

3.3.1 Limitations
Having provided the detailed overview of research problem and adopted fundamental research
approach. It is necessary to realize the limitations of research approach and methods.
Quantitative analysis is highly depended on availability of data corresponding to early
warnings. The early warnings, for which the past data is not available, will be excluded from
quantitative analysis. This in turn, will affect the quality of research and subsequently,
development of prediction model.
Another identified and more critical limitation is that the past project data is very limited
therefore could limit the accuracy and reliability of prediction model. Furthermore, the data
is specific to Fluor Corporation. Thus, the data will likely be product of the standard and
practices of Fluor rather than O&C industry as whole.

3.4 RESEARCH METHODS

3.4.1

Bibliographic and desk research

The proposed research project consists of an evaluation of the existing knowledge on the
concepts of project management and primarily on early detection of problems in projects.
Relevant literature from scientific and professional domains was studied.
The main aim of this part is to understand the tools and procedures applied in management of
O&C projects. Project performance and success are defined based on the academic,
professional literature study and Fluors measurement standards. The concept of early
warnings was defined by the study of available literature by academicians, professional
organizations such as CII, IPA, and PMI along with expert interviews.

3.4.2

Expert interviews

Identifying early warnings relevant to O&C projects is an important task of the proposed
research. For that, the concept of early warning is defined upfront. Experts from O&C
industry were asked to provide potential early warnings based on their experience. The expert
interview is selected as suitable method because there is little or no literature is available
regarding early warnings, especially during the execution of O&C projects. Past project could
be seen as potential source of selecting early warnings. Nevertheless, the time required for
analysis of vast project data does not fit into the available timeframe, yet the past projects
played as role for observatory evidence and provider of past data to build the prediction
model.
Interview base include experienced project directors, project managers and project control
managers within Fluor Corporation and from some external owner companies. Interviewees
were asked to provide potential early warning along with their possible measurement criteria.
Furthermore, the interviewees were asked to provide additional information such as associated

future problems. A measurable quantitative attribute were attached to each identified early
warning and will be termed as EWI.

3.4.3

Data collection

To obtain the understanding of relationship between early warnings and project performances,
data had to be collected and analyzed. Fluor Corporation is the primary source of past project
data. Due to the time constrain, date from past eight projects is used for quantitative analysis,
However the each project will provide 8 data collection point, collected at 0% (baseline),
15%, 30%, 45%, 60%, 75% and 95% of actual engineering duration to normalize the project
with different durations. The primary objective of collecting multiple data within one project
is to understand the dynamic relationship between early warning and project performance and
to develop a dynamic prediction model.
Apart from quantitative analysis, part of past projects are studied as case studies understand
the relationships and to differentiate between coincidences and causality of early warning and
project performance.

3.4.4

Performance prediction model development

Exploratory data analysis was performed before establishing statistical relationship between
identified early warnings and project performances. R Project for Statistical Computing will
be used to perform the statistical analysis due to its capability of customization the statistical
techniques and graphical outputs.
Relationship of EWI with project outcomes was established through collection and analysis of
past project data via stepwise multi regression. The conceptual prediction model was validated
using past project data (which were not included in training) and current projects. The results
of the validation test will be analyzed to form recommendations and conclusions.

33

4 LITERATURE STUDY
4.1 SUMMARY
First step in collecting the available literature on the topic of project performance management
and early warning is database research. Various search phrases were used to find the relevant
literature. Google scholar was used as primary internet search tool. For all the relevant
literature that could be identified, an attempt was made to get access. Further references of
many sources were searched to get the more specific literature regarding O&C industry.
The purpose of this chapter is to investigate how project management literature treats the
detection of early warnings during project execution. The chapter first defines the project
success and project performance, followed by adaption of project performance from current
research perspective. The chapter then proceeds to define the concept of early warnings from
theoretical perspective, followed by reflecting on their benefit in project execution control.

4.2 AUTHOR AFFILIATIONS

When the preliminary literature research was performed, it seems logical to describe the
affiliations of respective authors because the different affiliations are strongly related to the
mental framework from which the literature was written. The different groups of authors, their
interests, and assumption that might underlie their respective literature are presented below:
Construction Industry Institute (CII):
Established in 1983, the construction industry institute (CII) based at the University of Texas
at Austin, is a consortium of over 100 owner, engineering-construction contractors and
suppliers. Its aim is to improve the business effectiveness of its member organizations and
cost effectiveness of capital projects through research, related initiatives and alliance among
organizations. The research by CII focus on 14 knowledge areas of engineering and
construction industry such as design optimization, project organization and planning and
project controls are to name a few. Their primary focus of research in CII is the current
practices employed by industries. For each knowledge area, CII identified best practices
(methods or processes, which lead to enhance project performance), other practices (methods
or processes that are not proven to enhance project performance) and research information
(which are neither method nor processes). (CII, 2012)
Consulting companies:
Professional consulting companies such as Independent Project Analysis (IPA), schlumberger
business consulting (SBC) have published their companys perspective and experience on
project management systems, project performance of engineering and construction projects.
Especially, IPA focuses primarily on project development and execution through its project
evaluation system (PES).

34

The professional consulting companies serve their customers globally thus representing their
findings on projects all over the world, along with publishing region based reports. These
consulting companies published mainly through their official publications.
Academic researchers:
Academic research group involves authors from academic institutions like technical and
business universities, research schools and sponsored academic research by organizations. The
focus of authors is to enhance theoretical and scientific knowledge base regarding overall
project management and specific domains of project management. Their research explores the
science and engineering to delineate unknown causes and potential solutions of practical
problems faced by industry and to find their theoretical solutions. The most applicable
findings are further explored and tested by industries before adopting them as practices. The
present research investigates (but do not limit itself) academic publications relevant to early
detection of problems in projects, problems in execution and their performance management.
Limitations of the review:
Terminology in project management is not uniform for early warning indicators of problems.
Some describes them as leading indicators, symptoms, early warnings, problem causes.
Moreover, many authors see actual problems as potential indicators of future problems.
The diverse approaches and many implicit mentioning of early warning indicators could make
the literature study a time consuming activity. Therefore, it is logical and necessary to adhere
to the discussion of more relevant literature, which explicitly deals with early warnings in
context of project execution. This approach will result in only a part of literature that could
possibly be relevant and might bring the risk of leaving block of literature which might
result in extra concepts.

4.3 CONCEPT OF PROJECT SUCCESS

4.3.1

Project management view:

Having a view on what O&C project are, what are their phases and their performance
management, a natural question arises what are successful projects, in other words How do
we perceive success of a project. Before answering this question, it is necessary to
understand the concept of project success. The Figure 9 shows the best-known and most used
representation of project success i.e. iron triangle with time, cost and scope (or
performance/quality) on its corners (See e.g. Freeman & Beale, 1992, Larsen & Gobeli, 1989,
Might & Fischer, 1985) and Oisen, 1971). From perspective of cost, time and scope, the green
colored triangle is seen as a successful project, whereas the dotted red triangle can be termed
as unsuccessful project, due to overrun in three dimensions of success. Although, this
approach has been seen as too narrow and often criticized. See: (Atkinson, 1999), (Raz &
Dvir, 2002) and (W.Hughes, Tippett, & Thomas, 2004)
To widen the concept of project success, Morris and Hough defined three dimensions of
project success (Moris & Hough, 1987):

35

I. Project functionality: to what extent does the project perform financially and or
technically in the way expected by the project sponsors?
II. Project management: how close is the implementation of the project to budget,
schedule and technical specification?
III. Contractors commercial performance: did the contractors have a commercial benefit
in either short or long term?

Figure 9: Iron triangle of projects

The project success dimensions comprehend project success from different perspectives of;
the customer, project execution contractors and sub contractor and other stakeholders.
However, in reality the perspectives differ more than they look. A project could be delayed,
but can be termed as a commercial success from client perspective given changes in his
strategic financial goals. This indicates that whether a project is success depends largely on
the perspective from which the project is viewed (Lientz & Rea, 1995).
(Lim & Mohamed, 1999) addressed the differences in perspective of stakeholders and defined
project success into two criteria: project completion criteria and satisfaction criteria. At macro
perspective the criteria involves both the project completion and satisfaction criteria. On the
other hand, the micro perspective only involves completion criteria. This is shown below in
Figure 10 below.

36

Figure 10: Project success criteria

(Source: Lim and Mohamed, 1999)

As mentioned in the scope of this research project, (see section 3.2) the present research
adopted the perspective of E&C contractors as they have the key responsibility of EPC phase
of project. In other words, the research is focused upon micro level criteria as defined by Lim
and Mohamed (1999).
However, it would be wrong to assume that the perspective of client and subcontractors are
ignored, because to achieve the sustainable success, an engineering and construction
contractor has to work collaboratively with its customer and suppliers by integrating their
perception of success into its own to the possible extent.
The success of a project is determined by evaluating its performance against success criteria
(Wit, 1988), which implies that performance needs to be measured to determine the
successfulness of a project.
Another noteworthy point regarding project performances is that intermediate project
performance varies with the time during project execution. A bad performing project could be
turned around by making necessary strategic changes or a good performing project could turn
into a poor performing project due to multiple reasons. However, final project performance is
static and determines the success or failures of a project.

4.3.2

Project performance measurement in O&C projects

Having adopted the micro level success, the next step is to develop performance measurement
criteria to measure the success. Menches and Hanna (2006) developed a performance
measurement index with the following six project outcomes:
I. Percentage budget overrun,
II. Percentage schedule overrun,
III. Actual percentage profit,
IV. Change in work hours

37

V. Number of change orders

VI. Communication between project team
The change in work hours is more of a factor rather than an outcome and contributes to the
final cost overrun of the project. Therefore, the use of numbers of change orders and change in
work hours as project performance criteria could be debated (Atkinson, 1999), (Shenhar &
Dvir, 1996) and (Hughes, Tippet, & Thomas, 2004).
The actual percentage profit also seems to be contradictory with definition of project success,
as it is highly dependent of the perspective of the stakeholder and type of contract. For
example in reimbursable contracts, the percentage profit for E&C contractor may increase
with scope and delay, whereas on contrary the project cost and schedule performance will
decrease.
With the adopted perspective of E&C contractors, it seems logical to limit and translate the
measures of project success to following project outcomes. Knowing that this is very limited
view on project success, yet they are the most commonly used across industry, therefore the
availability of actual data for these indicators are higher than other indicators.

I) Mechanical completion schedule of plant

Mechanical completion (MC) of plant is defined as The checking and testing of equipment
and construction to confirm that the installation is in accordance with drawings and
specifications and ready for commissioning in a safe manner and incompliance with project
requirements (Norwegian Technology Standards Institution, 2009). The scope of MC
includes construction validation, testing of equipments (dynamic and static) and handover for
start-up to owner. However, the testing phase could be excluded based on prior agreed upon
scope between E&C contractor and owner (Fluor Corporation, 2012). MC can be seen as an
important milestone from E&C contractor perspective as well as owner perspective.
For the E&C contractor, incentives or liabilities are attached with MC milestone. Moreover,
for an owner, completeness of MC marks as an indicator that plant is ready for startup. Delay
in MC could negatively affect its production plans and prior agreements with buyers, in other
words its revenue generation (Choi, Anderson, & Kim, 2006).
II) 95 % engineering complete
As explained in section 2.2.1, detailed engineering phase takes BOD as input (from FEED
phase) and transform conceptual engineering into detailed engineering documents. It provides
an input to procurement and construction. Although, the average engineering cost is only 20%
of the project cost (CII, 2012), but it has significant influence on the rest of 80 % cost. In
O&C project, the key deliverables of detailed engineering are as follows (Fluor Corporation,
2012):
Input to procurement:

Input to construction:

- Equipment data sheets

- Process and instrument diagrams

38

- Instrument data sheets

- Plant plot plan

- Bulk material take-offs

- Civil foundation drawings

- Technical bid reviews

- Structure fabrication drawings

- Pipe routing drawings (UG/AG)
- Piping isometrics
- Electrical single line diagrams
- Installation procedure and manuals

The milestone for 95 % engineering complete signifies the completion of all major
engineering activities including final issuance for key deliverables (Issue for construction). In
other words, marks the completion of E phase of EPC project. The rest 5 % of engineering
is designated to miscellaneous construction and start-up support, which could extend until
completion of construction or MC (Fluor Corporation, 2012). Therefore, in industry practice
95% engineering completion is seen as finish of engineering efforts. Figure 11 shows the 95 %
engineering milestone on EPC progress curves.

Figure 11: 95 % engineering completion milestone

Adapted from Fluor Corporation, 2012

III) Total installed cost of project

Total installed cost (TIC) by definition means that it is the total cost of installing a plant. TIC
includes the cost of engineering efforts, cost of all equipments, materials and construction and
other costs such as contingency, services fee, and escalation.
The most cost effective project execution is the one allowing lowest TIC consistent with as
sold estimates and owner requirements. TIC is an important project outcome for both E&C

39

contractor and the owner due to the simple fact that TIC is the important determinant factor in
net present value (NPV) of a plant.
In contractual terms, the dynamics of TIC on project economics of owner and E&C contractor
can be illustrated by following model (Berends, 2007):

Where:
P = Actual E&C profit
Pt = Target E&C profit
= E&C sharing cost related profit; 0 1 (Based on contract type)
Ct = Target/as sold TIC
C = Actual TIC cost
Cc = Owner contract cost
From Equation 3, it is evident on higher level that growth in TIC (C) will shrink the profit
margin for E&C contractor and at the same time will increase the cost for owner.
The cost performance in terms of TIC as project outcome can be assessed as follows:
IV) Engineering man-hours
The amount of engineering man-hours in a project can be seen as an indicator of engineering
efforts required in a project. Although from cost perspective, the cost of engineering efforts is
quite small as compared to the cost of equipments and construction (on average varies
between 10-15 % of TIC). In addition, the maximum engineering cost could be as high as 31%
of TIC and as low as 8 % (Bakker, 2012).
However, from project execution perspective engineering is the most important activity. As
the engineering set the basis for equipment, purchase documents and construction drawings
(see section 2.2.1). Any significant variation or a change in engineering man-hours has direct
effect of procurement and construction activities. Therefore, from project performance
perspective, a project has high chances of being a good performing project and successful
project, if it consume more or less the same hours as estimated.

4.4 CONCEPT OF EARLY WARNINGS

The secret of all victories lies in the organization of non-obvious
-Marcus Aurelius (Roman emperor, AD 161-180)

40

Marcus Aureliuss philosophy from days of Roman Empire suits well in the context of project
execution management. However, a balance between obvious and non-obvious needs to be
identified and monitored. In the present research, early warnings are the mixture of nonobvious and obvious, whos monitoring and subsequent appropriate actions based on them
could prevent manifestation of many problems.
The scientific relation of early warnings of problems goes back to 1970s theory of weak
signals proposed by Professor Igor Ansoff. His primary aim was to improve the strategic
planning methods in business environment. As per his proposition those methods did not work
satisfactorily when faced with sudden changes or unanticipated problems. Moreover, the
traditional strategic planning tools of trend monitoring and basing future planning on these
were not efficient neither successful in time of turbulence. Further, he emphasized that the
potential strategic surprises give advance information of themselves through weak signals
(Ansoff, 1984). He defined weak signals as follows:
An imprecise early indicator about impending impactful eventsall that is known is

that some threat or opportunity will undoubtedly arise, and their shape and nature and
source are unknown (Ansoff, 1984; p. 22).
The Ansoff theory of weak signals was advanced by many researches in a wide range of areas
such as communications research (Aberg, 1989), military science and research on
international security (Betts, 1982; Herman, 1996) and predicting bankruptcies (Morris R. ,
1997). However, the occurrence of weak signals in project environment was first addressed by
Lientz and Rea; they suggested that the potential problems have prior symptoms, which are
visible in the project environment (Lientz & Rea, 1995). They presented weak signals as
symptoms of problems. However, these symptoms are implicit in nature and seem to be
embedded in the project environment. Their identification relies heavily on the skills and proactiveness of project manager and team members.
There are other literature sources, which presented implicit discussions about early warnings
regardless of the term used in original text. Lewis presented a long list (10 pages) called
checklist for managing projects (Lewis, 1993, Ch. 24). The absence of elements mentioned
in this list could act is a cause of potential problems in projects, in other words those elements
could act as potential indicators. Kerzner, Cleland and Honko (Kerzner, 1995; Cleland 1995,
Honko, 1982) mention the similar lists or causes.
The first comprehensive and explicit study of early warning phenomenon in project
management was published by Nikander in the dissertation Early warnings: A phenomenon
in project management. His research contrasted on the deficiencies of traditional project
performance and control mechanisms for correct assessment of project performance status.
He went further on the forecasting techniques and questioned their efficacy by highlighting
their incapacity to accommodate possibly changing project circumstances. He proposed that
the potential problems during project execution could be detected beforehand via their early
warnings. The early detection of problems facilitates more informed decision-making and

41

provides an opportunity for the project manager to act proactively. He defined an early
warning as:
An early warning is an observation, a signal, a message or some other item that is can

be seen as an expression, an indication, a proof, or a sign of the existence of some future

or incipient positive or negative issue. It is a signal, omen, or indication of future
development (Nikander, 2002; p. 49).
The research conducted by Nikander marks a stepping-stone in the direction of early detection
of problems however, lacks the quantitative nature and ability to forecast project performance
in light of early warnings. In addition, the majority of early warnings identified within the
research composed of feeling and behavior of the project team and its stakeholders. Their
detection largely depends upon the experience and intuition of project manager.
Another relevant research titled Leading indicators to project outcomes was conducted by
CII to identify the leading indicators (beyond the conventional methods or standard practices
used to evaluate the status of projects) which may have a significant impact on project
outcomes. The research defines leading indicators as:
Leading indicators are fundamental project characteristics and/or events that reflect or

predict project health. Reveled in timely manner, these indicators allow for proactive
management to influence project outcomes (CII, 2006; p. 20).
The research had two objectives: identification of leading indicators and development of
tool to assess the health of a project based on those leading indicators (CII, 2006). Unlike
Lientz and Nikander, this study was more focused on industrial practices and relied heavily
on the inputs from the owner and E&C companies. The study emphasized on leading
indicators from industrial perspective and how they anticipate their impacts on project
outcomes. However, the research is purely qualitative in nature and the respective tool
utilizes user subjective information (based on perception) for reflecting on the status of a
project.
IPA also provided the O&C project industry with its services through its project evaluation
system (PES). The underlying premise behind the PES is that a relationship exists between
project drivers and the project's outcomes (IPA, 2012). The service by IPA is quantitative in
nature and utilizes the vast database of past projects.
The IPA performs the project audits primarily at FID and at project completion, which are the
battery limits for project execution phase; the dynamics with in project execution is ignored.
The IPA findings at FID could serve as early warnings for project execution i.e. if the project
is not well defined and basis of design is not of good quality. It could act as first warning for
the project manager and the deficient areas of project will most probably have problems
during execution, if no remedial action is taken. IPA being a service provider, their activities
is commercial in nature and only the outcomes and their illustration is known to the project
owners.

42

In the present research, the larger view focuses primarily on the management of project
performance with in project execution; therefore, audit key parameters from IPA could be
used as early warnings for continuous monitoring.

4.4.1

Potential value addition of early warnings:

By definition, early warnings are indicators of future development. Following the reasoning of
Nikander (2002), if early warnings of potential project problems are detected at early phases
of project, many options are still available at the discretion of project manager.
How early warnings can contribute to the cost effectiveness of project can be understood by
following simple mathematical expressions:

Where:
COP = Cost of problems in projects/unanticipated risks
C = Actual cost of project
Ct = Target cost
Cm = Cost due to unavoidable events
The cost of problems can be measured as the cost incurred to mitigate the negative impacts of
manifested problems. In the context of present research, the COP can be seen in two different
perspectives:
COPR = Cost associated with reactive approach (cost of corrective strategy after problem has
manifested)
COPA = Cost associated with active approach (Early warning detected and corrective action
taken)
Applying the reasoning of cost influence curve, explained via Figure 5, it can be concluded
that the cost of reactive approach is more significant as compared to the cost of proactive
approach.

43

With the identification of early warnings in projects, we can minimize the COP to COPA and
subsequently minimize the cost over-rum. In other words, EWI can help project in achieving
following state:

Figure 12 Potential benefits of EWIillustrates the relation between early warning and project
success more explicitly. As the project success is a function of project outcomes, undesired
project outcome can be interpreted as unsuccessful projects. Figure 13(i) show a situation in
which the potential problems was not anticipated well before and when it became visible. As a
reaction to the existence of problem, a corrective strategy was formulated to contain the
negative effects of problem, and subsequently implemented.
The implementation of corrective strategy, required additional cost and schedule requirements,
which were may not be anticipated while baseline finalization (COPR). Moreover, it might
cause non-satisfaction among owners, project team or suppliers. Figure 13 (ii) shows a similar
situation where the early warning indicated the presence of same potential problem in future
course of project execution. The project manager has additional time to make a proactive
decision, inform the stakeholders, and form preventive strategies and to implement the chosen
strategy. The preventive strategy utilizes much less resources, thus keeping the cost and
schedule close to desirable project outcomes (COPA). Furthermore, it vitalizes the sense of
confidence and predictability in project manager and project team.

44

Figure 12 Potential benefits of EWI

4.5 CONCLUSIONS AND DISCUSSIONS

The presented chapter aimed to explore the relevant literature and answer the first two
research questions i.e. What constitutes project success and what are performance assessment
criteria of O&C projects? and What do we understand by early warnings of project
problems?
it was observed that the project management literature sees project performance and success in
a broader sense. Lim and Mohamed comprehend the different views at two levels micro and
macro level. The present research adopted the project success criteria at micro level; the
decision was taken mainly because of two reasons
1) The research aims at EPC phase of the project and micro level criteria (time cost and scope)
is the most widely used criterion across E&C companies.
2) The present research relies heavily on the data from past project and availability of data
relevant to micro level criteria was higher than other mentioned criteria.
Based on the micro level criteria, following four project outcomes (PO) are defined to
determine the project performance
I. Mechanical completion of the project (Referred as MCI)
II. Total installed cost of the project (Referred as TIC)
III. 95% engineering completion schedule (Referred as ESI)
IV. Engineering man-hour of the project (Referred as MHI)

45

The first two PO (MCI and TIC) signifies the final cost and schedule of the project and the
rest two (ESI and MHI) can be seen as intermediate project outcomes, yet from an E&C
contractor perspective they do represent an important aspect of projects.
Review of the project management literature advocates the presence of similar notations to
early warnings in projects and addressed as symptoms of problems leading indicators and
even as checklist for managing projects. However, there is a limitation of explicit pragmatic
studies, which provides the evidence of early warnings engaged in management of projects.
The study by Nikander titled Early warning: phenomenon in project management lacks the
utilization capability in actual project work as the study focus on early warnings which are
either purely qualitative or composed of feelings and behaviors of project team. Another
research was conducted by CII titled Leading indicators to project outcomes identified the
leading indicators which might influence the project performance. However, this research is
also qualitative in nature and respective tool depends on user-subjective information. The
methodology adopted by IPA is of limited use, as it only asses static project health at FID and
project completion and does not provide active control mechanism during project execution.
Section 4.4.1 illustrated the potential benefits of early warnings. The detection of early
warnings and forecasting project performance based on them does provide valuable
information and additional time to project manager to correct the problem before its
manifestation. By the means of information and additional time, the reactive approach could
be converted into active approach, thus reducing the cost and schedule overrun of the projects.

46

EARLY WARNINGS IN PROJECTS

5.1 SUMMARY
The primary objective of this chapter is to find a set of early warnings in O&C projects, which
could be operationalized to predict the project performance. To achieve this objective, the
present chapter explores the early warnings from various sources (literature, experts) followed
by assessment of identified early warnings based on selection criteria (formulated from
present research perspective) to select fewer early warnings, which will to be carried forward
for further investigation.
The section 5.2 discuses the importance of measurability of early warnings and provides a
formal definition of early earnings used in present research. Section 5.3 presents the early
warning selection criteria with selection parameters and provides the argumentation for
selecting respective parameters. Section 5.4 discusses the further classification and potential
uses of early warnings after their evaluation on section parameters.
Further section 5.5 and Section 5.6 provides the overview of early warnings found in literature
sources and via expert interviews respectively. The comprehensive lists of early warnings are
attached as appendix A. Section 5.7 presents the final list of early warnings as EWI after
analyzing and rearranging the selected early warnings from respective sources.

5.2 IDENTIFICATION OF EARLY WARNINGS

Policy and decision makers are heard often saying, You can not achieve what you can not
measure and/or What get measured gets managed. This implies that if you are not able to
formulate a goal/objective in terms of a set of quantitative indicators, then it has not been
defined either in sufficient detail or still there is lack of census over its meaning. Bossel
(1999) described indicators as Indicators summarize complex information of value to the
observer. They are our link to the world. They condense its enormous complexity to a
manageable amount of meaningful information, to a small subset of observations informing
our decisions and directing our actions (Bossel, 1999)
This is also true in case of early warnings, as they need to present in measurable terms, so that
project manager or stakeholders can take decisions that are more informed. Therefore, the
early warnings as indicators altogether should provide the decision makers with a sufficiently
complete and accurate image of their status to enable them to decide whether proactive action
is required or not. For the identification of early warnings and practical issues, following
formal definition based on the aggregated academic and industrial research is determined.
Early warning indicators are inherited measurable project characteristics and/or
observations that indicate towards the development of potential problems in future course of
project execution.

47

With reference to the type of information presented, early warnings may be separated into two
categories: quantitative and qualitative. The former provides information in the form of hard
data extracted from system state (i.e. numbers) and latter are based on the subjective data
based on some ones perception (i.e. ordinal numbers or qualitative judgments such as good,
sufficient, bad).
The information, which can be treated as early warning in projects is available in both forms
i.e. qualitative (qualitative assessment) and quantitative (variations, percentage delays).
However, there is another set of early warnings, which is expressed as feeling, inter-personnel
behaviors and communications.

5.3 SELECTION CRITERIA OF EARLY WARNINGS

The selection criteria are formulated while keeping the main objective of the research in focus,
i.e. building of a prediction model. The collection and analysis process is explained and is
shown in Figure 13.

Figure 13: Early warning selection criteria

Measurable: The early warnings that are measured quantitatively or at least have the
potential to be measured
Early in project - The definition of early for identification of early warnings is somewhat
subjective. The screening of early warnings gives an indication that some of the early
warnings could exist all along the life cycle of projects which include early phases of projects.

48

There is no reference found in literature that could define the early phase of an O&C project.
Therefore, expert opinion was sought and based on the discussions; the period from start of
detailed engineering, up to 2/32 model review was defined as early. This choice seems quite
reasonable and can be argued based on design definition at 2/3 model review stage.
Data availability To establish the statistical relationship between early warnings and
project performance, it is of utmost importance that we can find the data in past projects
corresponding to selected early warnings.
However, there might be few early warnings, which might not have quantitative data attached
to them, yet the written sources of information in project reports could provide an opportunity
to make an inference regarding their existence. Such qualitative early warnings will be
accepted only for analysis during case studies.

5.4 SELECTION OF EARLY WARNINGS

Having defined the selection criteria for early warnings, each individual early warning was
assessed on the selection criteria. The primary objective was to select the early warnings,
which can be carried over the next phase for further analysis and for building of the prediction
model.
The early warnings, which are termed as important3 and cannot be selected for further
analysis, will be presented as recommendations. The rest of the early warnings will be
presented as for information. As a result, the selection process delivered three types of
early warnings
Early warning as indicators (EWI): To be a EWI, a particular early warning should
qualify on all three criteria. The EWI are further explored via case studies in chapter 6, and
ultimately acted as variables in the building of the performance prediction model. There are
few exceptions made for early warnings, which do not satisfy all three-selection criteria, but
are accepted as early warnings in case studies mainly due importance given to them by both
literature and interviewees.
Early warnings as recommendations: The early warnings which match any two criteria
and are either measurable or have potential to be measured. Apart from measurability, they
should occur at early stage in project execution and can provide useful information for project
management or future research.
Early warning for information: This category also includes early warnings, which are
either purely qualitative in nature and are not seen early enough in project. The main argument
for their selection as for information is that from the analysis perspective, its quite difficult
to quantify them and assign indicators, thought they can still provide useful information to
project manager to his capacity as the leader on the project. The PM could identify these early

Define 2/3 Model review

Important here refers to the importance given by the sources i.e. multiple literature or interviewee mention
the same early warnings (Frequency)

49

warning and can take necessary steps to make project environment more positive and
constructive.

5.5 EARLY WARNINGS FROM LITERATURE

As mentioned in section 4.4, project management literature does include few examples that
can be interpreted as early warnings. However, many of the examples are quite implicit and
difficult to operationalize for forecasting project performance. The deficiency emerged due to
the limited research and lack of consistency regarding what is an early warning and when it
is identified. Therefore, the effort was made to explore not only the academic database, but
also the publications by professional organizations.

5.5.1

Academic literature

The academic literature discusses the early warnings from different viewpoints such as
symptoms of problems, problems themselves, complexity factors or non-fulfillment of project
standard procedures (Lientz, 95; Honko, 82; Cleland, 84; Kerzner-95; Bosch-rekveldt, 2011).
From the analysis of early warnings found in this class, it could be readily observed that all of
them contain a clear indication of their warning like character. Explicit or implicit, all the
relevant quotes or elements are mentioned in appendix A.1 (L-1 to L-30).

5.5.2

Early warnings from CII

The CII study titled Leading indicators to project outcome investigated the leading
indicators (LI) of potential problems in projects. Given the accepted definition in chapter 5,
the mentioned LI probably can be seen as early warnings. The study concluded with 43 LI,
which were presented after three levels of evaluations.
All 43 LI were analyzed and after the preliminary screening, 15 top indicators were selected
based on the importance (in terms of score) given to them by E&C contractors. The early
warnings from CII are mentioned in appendix A.1 (L-31 to L-45)

5.5.3

Early warnings from IPA

As mentioned in section 4.4 IPA audit findings at completion of front-end design could serve
as early warnings for project execution. IPA project audits are purely quantitative and could
serve as excellent early warning indicators as per our adopted definition. IPA has defined its
own metrics and project database to validate those metrics. Apart from rating the project based
on its characteristics and metrics, they provide a benchmarked comparison of project with
other more or less similar projects. In addition, IPA findings on post project evaluation could
also provide key early variables, which makes difference between good performance and bad
performance of projects and can be seen as early warnings.
In the present research, the underlying principle of IPA PES is of interest, attempts could be
made to find the relevant data from past project and conclusions could be drawn based on the
findings. The early warnings from IPA PES are mentioned in appendix A.1 (L-45 to L-49)

50

5.5.4

Analysis of early warnings from literature

Based on the characterization and typology of early warnings found in literatures, it was
observed that the majority of the early warnings are in qualitative in nature (21 Nos, 45 % of
total). In addition, there were 17 nos of quantitative early warnings including 4 nos of
traditional performance indicators which could be seen as early warnings. The classification of
early warnings by literature source could be illustrated by following graph.
Early warnings from literature
30
Feeling, behavior
Quantitative
25

Qualitative

20
7
15

10
13
5

8
6

0
Acedemic Literature

CII

IPA

Figure 14 Classification of early warnings from literature by source

The early warnings can be further classified into sub categories. Most of the sub categories
consist of various early warnings primarily related to that sub categories. Figure 16 shows the
classification of early warnings into sub categories.

51

Early warnings by sub categories

30
Feeling, Behavior
Quantitative

25

Qualitative
9

20
3
15

10
15
4

1
2

4
Project Team
development &
integration

Change mgmt

Early Engineering

Procurement

3
Concurrency in
projects

Figure 15 Early warnings from literature by sub-category4

Note that the quantitative nature of early warning reflects the potential to measure and does
not necessary mean that it is measured. For example, early warnings number L-35, which
could be measured by monitoring the experience (Based on the grade), personnel need date of
actually deployed personnel against the planned requirements. However, not every project or
E&C Company measures the experience and skills of the project team quantitatively.
There are 27 early warnings that belong to quality of project team and its integration, which in
a sense reflects the importance of team in a project. The noteworthy point from above analysis
is that majority of early warnings in category team development and integration (24 out of
total 27) are either qualitative in nature or are feeling/behavior of team members.
Based on the selection criteria, following early warnings are selected for further analysis
Table 2: Early warnings from literature sources
Criteria

ID

Early warning

L-1

Monitoring of actual
allocation of resources
against the plan can
provide early warning of
lack of work

Measura
bility

Early in
project

Data
Availability

Remarks

The early warning is accepted

for early discipline. Project
level reporting may not reveal
the actual picture due to
aggregation factor

Here it should be noted that the quantitative nature of early warning reflects the potential to measure.

52

L-2

Noticeable change in
performance level for a
team or individual
members reflects
problems.

L-18

High rate of changes in

part prompted by errors
in assumptions and
mistakes

Adapted as cost of changes

and nos of changes in
project

Adapted as cost of changes

and nos of changes in
project

Accepted due to the

importance stated by experts
and literature
Data could be found as
write-up in internal reports

L-30

L-32

L-34

Owner and contractor is

requesting an excessive
number of contract
changes during project
execution
Project is experiencing a
high level of
engineering/design/
specifications changes
The project team is
lacking in the necessary
expertise, experience,
breath and depth to
successfully execute the
project

L-46

Poor front end definition

L-48

Poor team development

L-49

Major design changes at

later stage in design

The early warning is accepted

for early discipline. Project
level reporting may not reveal
the actual picture due to
aggregation factor
Segregation of change orders
from past projects at required
level of detail is difficult,
therefore the early warning is
mentioned as nos of changes

Accepted due to the

importance stated by experts
and literature
Data could be found as
write-up in internal reports
Accepted due to the
importance stated by experts
and literature
Data could be found as
write-up in internal reports
Changes after 1/3 model
review in project are
considered as late changes.

5.6 EARLY WARNINGS FROM EXPERTS

This part of research included 1) semi-structured interviews 2) preliminary analysis and 3)
selection of early warnings. The preliminary analysis included refining the interview findings
by understanding the early warning in context of O&C project execution. Furthermore,
interview transcripts were rewritten to understand how early warnings affect project
outcomes; the final selection is made on the criteria described in section 5.3.
The main objectives of interviews were defined as follows:
I. To find out whether information akin to early warnings can be detected in project
execution of O&C projects
II. If early warning can be detected, to form a description and understanding the
mechanism behind those early warnings

53

III. How EWI relate to project outcomes?

IV. To get practical example, (if exist) relating suggested early warnings with project
problems and project outcomes.
The group selected for interview was small, but at the same time, it was widely representative
of actual project experience with the professional field of capital O&C project execution.

5.6.1

Interview planning and execution:

Due to the unavailability of literature on early warnings in O&C project execution, a

development of redefined research questions was a challenging task. Therefore, semistructured interview approach was selected; all the interviewees were provided with a brief
introduction about the research with definition of early warnings and project outcomes. In
addition, the context, approach and possible research outcomes were explained to the
interviewees.
The interviews were conducted in October and November 2012, extending over period of 3
weeks. All the interviews were conducted face-to-face except one, which was conducted over
the phone. The interview lasted 1.5 hours on an average with maximum duration of 2 hours
and minimum of 1 hour. The interviewees base had been selected in consultation with
supervisors at Fluor and mainly comprised of project directors, senior management (with prior
PM experience) and project control managers (with prior PM experience). A complete list of
interviewees has been attached as appendix C.2.
Before the interview: Before the actual interview, the interviewee was emailed a
preparatory document explaining the background of the research project, key definitions,
proposed agenda of the research and the main practical matters. The document consists of
brief introduction, aim of the interview, relevant project outcomes and further steps in
research and attached as appendix C.1.
During the interview: The steps for the interview were as follows:
I. Personal introduction
II. Introduction to the research project
III. Setting the stage: types of EWI their basic characteristics, attributes, and potentially
affected project outcomes.
IV. Questions to understand the mechanism behind the suggested EWI to understand, how
and to which problem they reflect.
V. The measurability of suggested EWI and any practical project example
VI. Closing remarks

54

After a brief personal introduction, the interviewee was explained the conceptual design of
research, the concept of early warning indicators, EWIs intended use in present research and
the expected outcomes of the research project. The interviewee was asked to describe the key
early warning indicators in project execution, which they believe leads to the project problems
in project at later stage. Then further questions were asked to better understand of how the
particular EWI influences the project outcomes or key deliverables of the project and how it
enables the project manager to predict future problems.
After the discussing the particular early warning, the interviewee was asked to delineate the
time or instance on the project timeline at which this early warning can be identified along
with the possibility to measure this early warning quantitatively or qualitatively. In addition,
any example from interviewees practical experience was sought. The interview was
concluded with a discussion of a few practical matters on project execution and key
parameters to be kept in notice from a project manager point of view.
All interviews were taped after consent and notes of important points were taken during the
interviews. After the interviews, preliminary analysis was done. The final analysis was
conducted after all the interviews had been completed.

5.6.2

Early warnings, the results of the interview

The detailed transcripts provide the overview regarding each early warnings description,
which project outcome it affected, and how it could be measured. Based on the interviews the
list of early warnings from practical perspective is attached as appendix A.2:

5.6.3

Analysis of early warnings from experts

Based on the typology of early warnings found from expert interviews, it should be noted that
there is no early warning mentioned as feeling or behavior, which might be explained by the
fact that prior to interviews, every interviewee was made aware of the final objective of the
research i.e. building of quantitative model. Another explanation could be that the interviewee
base included people who had experience as PM or experience in project leadership, (PM,
PCM, Sr. mgmt) and being leaders, they might perceive feeling/behavior aspects as qualitative
to have meaningful interpretations from them rather than just behavior or feelings (Prati et al,
93; Pescolido, 02).

55

Early w arning from experts by sub-category

20
18

Quantitative

16
14

Qualitative
1

12
10

16

8
14
6

11
7

4
2
2

Early Engineering

Procurement

Concurrency in
projects

0
Project Team
development &
integration

Change mgmt

Figure 16: Early warning from experts by sub-category

On the contrary, to the early warnings from literature, experts see early engineering as most
prominent category of early warning (31%). In addition, the majority of early warnings in sub
category project team development and integration are qualitative in nature (14 out of 15),
similar to early warnings from literature.
Apart from the classification of early warnings, it should be noted that the early warnings
related to sub category change management and early engineering were mentioned by all
interviewees irrespective of the nature of early warning, implies towards the possible
importance of these two categories to determine project success. The following figure presents
the graphical representation of early warnings (category wise) as mentioned by interviewees.

56

Early w arnings mentioned by nos of experts (Sub category w ise)

14

12

13

13

11

10

9
8

0
Project Team
development &
integration

Change mgmt

Early Engineering

Procurement

Concurrency in
projects

Figure 17 Early warning mentioned by numbers of experts

Based on the selection criteria, following early warnings are selected for further analysis:
Table 3: Early warnings from Experts

ID

Early warning

I-3

Large
amount/Numbers of
scope changes
(additional items or
details) by clients

I-13
I-15
I-16
I-18
I-20

High Concurrency
level in projects
High Level of client
involvement
Late scope change
Overrun in process
engineering manhours
Delay in process
engineering

Criteria
Measurab
ility

Early in
project

Data
Availability

Remarks
Segregation of change
orders from past projects
at required level of detail is
difficult, therefore the early
warning is mentioned as
nos of changes
Data could be found as
write-up in internal reports

I-23

High Nos of
arguments between
E&C contractor and
owner

Accepted due to the

importance stated by
experts and literature
Data could be found as
write-up in internal reports

I-25

Low level of team

integration

Accepted due to the

importance stated by

57

experts and literature

Data could be found as
write-up in internal reports
I-26

I-27
I-31

I-35

I-40
I-43
I-52

Lack of
understanding
regarding project
objectives among
project team
Delay in purchase
order placing
Nos of changes
After P&ID's are
issued for design
due to rework
Changes in major
project objectives
Early engineering
discipline crossing
late curve
Higher numbers of
changes in project
at early stages
Incomplete Front
end design

Same as I-25

Accepted due to the

importance stated by
experts and literature
Data could be found as
write-up in internal reports
Same as I-20

Same as I-3
Data could be found as
write-up in internal reports

5.7 EARLY WARNING INDICATORS

After the evaluation, several early warnings were found to be similar in terms of their meaning
and were integrated as one early warning. The similarities are mentioned in the remark section
of selected early warnings. The exercise to integrate the early warning resulted into a smaller
numbers of early warnings.
Following is the list of early warnings, which are selected for analysis via case studies, each
early warning is defined in terms of its ID, name, measurability, source documents and the ID
of other early warnings, which has been integrated with it.
EWI related to Project team development and integration: From the analysis of early
warnings from literature (mentioned by academic, CII and IPA) and experts (mentioned by
eleven experts), it become quite evident that quality of the project team does play an important
role in making a difference between successful and unsuccessful project.
Yet the early warnings are qualitative in nature and no quantitative information could be found
in past projects for development of prediction model, it seems necessary and logical to include
them at-least in case study analysis. In addition, the past project literature does provide
information in form of write-ups regarding the quality of team and its integration. Therefore
following EWI were selected.

58

New
ID
LES

Early warning
indicator
Lack of
understanding of
project execution
strategy among
project team

PTE

Project team
lacks experience
required for the
project
Conflicts between
owner and E&C
contractor

COC

Measurability

Possible sources

Qualitative

Project review
meeting, Project
surveys

Qualitative

Deficiency in
required and
actual deployed
resources
Client survey,
review meetings,
contractual
disagreements

Qualitative

Integrated early
warnings
I-26, L-48

L-34

I-15, I-25

Note: In the perspective of present research, the term conflict has been used in negative sense
i.e. the conflicts which manifest the project problems. However, it should be noted that in
conflict is not always dysfunctional or destructive, though it also play constructive role by
generating new norms, and stimulating the engaged parties towards economic gains and
technological advancement (Coser, 1957)
EWI related to change management: The changes during project execution do affect the
project outcomes (often negatively) and this is confirmed by both literature and experts. One
of the basic principles of efficient work processes is that work is done once and done
correctly. However, there might be situations in the project where changes are not avoidable
due to safety, functionality or standards.
Ideally, in an EPC phase of an O&C project any change after completion of FEED stage
should be avoided. Any proposed change should pass the following criteria before
implementation (Bakker, 2012):
I. If the current design is not safe
II. If the current design does not work
III. If the current design is not as per standard
Changes are generally detrimental to the health of a project, as they often have major impact
on cost and schedule. It is also well documented that the cumulative impact of numbers of
small changes is much greater than sum of individual impacts. This is mainly due the fact that
the changes not only brings new addition of work, but also produces out of sequence activities
and disruption effect on ongoing activities.

59

Another noteworthy point is that impact of disruption and out of sequence activities is often
difficult to assess before implementing the change and often become visible at the later stage
of project.
The following EWIs were selected based on the selection criteria.
New
ID
NCO
CCO

Early warning
indicator
Numbers of change
orders
Cost impact of
changes

Measurability

Possible sources

Nos of changes in
project
Cost of
changes/estimated
cost of project

Change order log

Change order log

Integrated early
warnings
L-32, L-34, L-49, I-3, I43
L-30, L-32, L-34, L-49,
I-3, I-43

EWI related to early engineering: The early engineering in the EPC phase of an O&C
project can be seen in two perspectives:
1) Completion of FEED
2) Process engineering of EPC phase
The FEED phase of a project produces the basis of design (BOD) for detailed engineering.
Efforts at the FEED phase defines the quality of design basis (BOD), The BOD forms the
starting point for detailed engineering and the project baseline estimate. Therefore, an
incomplete FEED could not only disturb well-defined start of the detailed engineering design
but also affect the accuracy of estimate and resources for the project.
However, if the incompleteness of front-end design has been addressed by project team, it is
of utmost importance that incompleteness has been communicated well with in project team as
well as has been reflected in estimates, management reserves and contingency plans.
Process engineering works on BOD and set the design criteria for further discipline
deliverables. Process engineering delivers the critical documents, which act as the base
documents for subsequent engineering. In a case where process engineering is forecasting
significantly higher numbers of hours due to any reason (changes, missing items, inefficiency
etc,) it implies that the critical documents require additional efforts. The growth in process
hours is an indicator that the subsequent engineering discipline also might need additional
hours (to complete the added or modified work) and additional cost is required for material
and services.
The delay in process engineering implies that the critical documents will be issued with delay,
thus affecting the downstream disciplines in terms of work front and delay in their respective
deliverables. In addition, the delay in process engineering impacts the procurement cycle for
the project and this could result into late delivery of material and ultimately construction
delay.
Any early problems in these two areas should be seen as an early warning. Following EWI
were selected in this category

60

New
ID

Early warning indicator

FED

Percentage of missing
information in FEED
package

PH

Growth in process manhours

PS

Delay in process
engineering

Measurability
IPA FEL index
Non-conformities
and observations in
Gate-C Review
Forecasted or
actual hours /
Baseline hours
Actual %
completion
Planned %
completion

Possible
sources

Integrated early
warnings

IPA audit
report
Gate C review

I-52, L-46

Progress
reports

I-18, L-2

Progress
reports

I-20, I-40, L-1

EWI related to concurrent engineering: At the start of EPC phase, engineering planning
is carried out based on certain level of concurrency (baseline concurrency levels) via
information available from BOD, schedule and cost targets and PEP.
From the analysis of early warnings by literature and experts, it could be concluded that the
concurrent engineering execution is directly proportional to the amount of assumptions being
made in project. A high concurrency level means that an engineering discipline is either
working on assumptions or sufficient reliable information is available beforehand. In either
case, there exists a risk of assumed information proved wrong at later stage, which will
produce rework not only with in a specific discipline but also with the associated discipline
activities, bulk quantities and changes in vendors scope.
From an early warning perspective, it seems logical to identify the change in concurrency
levels early in the detailed engineering to have a check, if design is being carried on high level
of assumptions.
The engineering discipline process and piping has been identified as indicator of concurrency
in project. The choice has been made because of the following reasons
I. The engineering activities of the piping discipline constitute a significant part of early
engineering efforts are highly dependent on the inputs from process engineering.
II. Piping engineering constitutes of a substantial part of engineering and material cost.
III. The rework in piping could lead to the high impact of total project cost and schedule.
Following EWI were selected in this category:
New
ID
CE

Early warning indicator

Measurability

Change in concurrency
level between process
and piping engineering

Change in area
between process and
piping progress curves

Possible
sources
Discipline
progress
curves

Integrated early
warnings
I-13, L-32

61

EWI related to procurement: The procurement is of strategic importance as the equipment

and materials are not only required for construction, but they also form a critical part of
detailed engineering design. Many engineering deliverables are based on the information to be
provided by equipment vendors. Delay in issuing of purchase orders can have direct impact on
schedule of vendor information required for design. Based on the analysis of early warnings
related to procurement and selection criteria, the following EWI were selected for analysis.
New ID

Early warning

Measurability

DPO

Delay in
issuance of
purchase
orders

Actual procurement progress

- Planned procurement
progress

Possible
sources
Procurement
progress curves

Integrated early
warnings
I-27

5.8 DISCUSSION AND CONCLUSION

The primary aim of this chapter was to find a set of early warnings that can be operationalized
to build the performance prediction model, thereby answer the research question What early
warnings can be identified in project execution and Which early warnings can be
operationalized to build a performance prediction model. The aim was achieved sequentially
via investigation of academic literature and expert interviews followed by selection based on
defined criteria.
The chapter presented a selection criterion, which is composed of three parameters early in
project, data availability and measurability. For an early warning to be converted into
EWI, all of these criterions need to be satisfied. It was found that literature provides a balance
of quantitative and qualitative early warnings, whereas experts were found to be quantitative
focused. The difference among sources was mainly found due to the reason that the
interviewees were well informed regarding the aim of this research (I.e. the quantitative
nature).
Each early warning was evaluated on the selection criteria and few early warnings were
selected. Due to the similarity in nature of many early warnings, few of them were integrated
to form final ten early warnings. The ten early warnings are defined in detail, via how do they
affect the project performance and project outcomes. Moreover, they were assigned new ID
and defined in terms of their measurability, potential sources of information.

62

6 CASE STUDIES
6.1 SUMMARY
The primary aim of this chapter is to assess the presence and predictive capability of selected
early warnings (in section 5.7) in real life projects. To achieve this goal, a case study approach
is adopted. A multiple cases embedded design is used, in which each case represents a
completed project (Yin, 2002). The case study approach was chosen, mainly because the
present research is focused on contemporary real life situation. Therefore, it seemed necessary
to understand, how the suggested early warnings can be identified in projects, what problem
they can predict and how they influence the project outcomes.
In case studies, an effort has been made to recognize the link between early warnings, their
associated project problems and final project outcomes. In other words, the intention here is to
check and understand if an early warning provides a correct picture and illustration of their
intended purpose.
Section 6.2, discusses the basic design of case studies, including the possible data sources.
Section 6.3 describes the factors, which were considered while selecting the projects for case
studies followed by a brief overview of the selected cases. Section 6.4 to 6.7 discusses the
individual case studies by discussing the presence of early warnings, their associated project
problems and final project outcomes. Each case study is concluded by proving an overall case
summary.
The final section 6.8 provides a cross case analysis to argument the prediction capabilities of
early warnings in the contrast set by Successful and less than successful performing case
projects.

6.2 CASE STUDY DESIGN

The chosen unit of analysis for this part of research is a completed project in the O&C
industry. The case study design refers to the different sub units of analysis: early warnings,
major problems in projects and project performance.
A project is taken in following definition: it covers all activities after the start of EPC phase up
to the engineering completion for identifying the presence of early warnings; this choice was
made due to adopted definition of EWI (see section 4.4). However, the potential future
problems, which could be identified by EWI, were not limited to engineering phase only. The
case studies involved a study of written project archives, which involved closeout reports,
project monthly status reports and other relevant project archives.
The data collection method for case studies was uniform across all case studies. The following
sources of information were searched:
1) Project closeout reports 2) Monthly progress reports (MPR) narrative 3) Project deviation
log, 3) Project change log 4) FEED project closeout reports (if available) 5) Gate C reviews (if

63

available) 6) IPA audit reports (if available) 7) Discipline progress curves 8) Procurement
progress curves.
Wherever possible, an effort has been made to investigate more than one source for particular
evidence. On the top of above mentioned sources, informal talks with project team members
were conducted with an intention to understand the itinerary of project execution. The
information from project members also helped in identifying any special or abnormal events
that happened during the project execution, which might have influenced the project
outcomes.
The relationship between detected early warnings, manifested project problems and their
impact on project outcomes has been mapped by following framework.

Figure 18 Framework for mapping the relationship between early warnings, project problems, and project outcomes

In the above framework, the early warnings indicate the possible manifestation of specific
project problem, which cause an undesired growth in project outcome i.e. affects the project
performance. The relationship for each warning in all four case studies is mapped and has
been attached as appendix D.

6.3 CASE STUDY SELECTION

The cases selected for the study, covers both good performance projects and poor performance
projects defined in terms of meeting their schedule and as sold cost estimates (at micro
level, see section 4.3.1).
The case projects were selected with intent of having a balance of good performance and poor
performance projects. The good performance project can provide contrast in which the
prediction capability of early warnings is more visible. Under this assumption, the early
warnings in a less than successful project should be clearly visible or at-least have quantitative
values different from their values in successful project.

64

Having said that, three of the four selected case study projects were executed between 2008
and 2012. One case project was executed in year 2004 and 2005, which was selected due to its
extremely successful execution, and therefore could provide useful information while
comparing the cases.
In terms of project characteristics, the case projects are summarized in following table:
Case project
Case project 1

Cost of the project

(Estimated)
Confidential

Case project 2

Confidential

Case project 3

Confidential

Case project 4

Confidential

Project type

Contract Type

Chemical-green
field
Refinery upgradebrown field
Chemical-Green
field
Chemical-green
field

Cost-reimbursable
fixed Fee
Cost-plus fixed
Fee
Lump sum
Cost-reimbursable
fixed Fee

Year of EPC
execution
2008-2011
2004-2005
2011-2013
2010-2012

The case number one and four did experience growth all project outcomes. However, the
growth in final project outcomes i.e. TIC and MCI has been relatively low as compared to the
MHI and ESI. The good performance project did not show any significant growth in final
project outcomes i.e. TIC and MCI. The case project number two finished on schedule with
cost under. The case project three finished on schedule with less than 1% growth in TIC. The
case project three did experience growth in MHI, at the same time finished the detailed
engineering ahead of the schedule.

6.4 CASE 1 (LESS THAN SUCCESSFUL PROJECT)

6.4.1

Brief project description

The objective of discussed case project was to construct a new fully continuous integrated
chemical. The E&C contractor worked under fixed fee reimbursable contract.. The period for
EPCm execution of project was from 2008 to 2011. From project documentation, it was
concluded that during project execution, the project drivers were often changed. This is
evident by the multiple cost review and scope reduction exercises. The final project
experienced both cost and schedule overrun. Apart from cost and schedule, the performance
of the project was very good in terms of functionality and safety.

6.4.2

Project evaluation

An argumentative evaluation is presented in each section to understand how selected early

warnings could have predicted the project problems and their relation with the project
outcomes.
1) Team development and integration: The project execution team involved E&C
contractor team, client team, an external major sub-contractor and work share offices. The
summary of the findings on project team development and integration is given in following
table:

65

Early warning

Presence

Lack of understanding
regarding project execution
strategy among project team

Yes

Project team lacks experience

required for the project.

Yes

Conflicts between owner and

E&C contractor

Yes

Moment of
detection
4 months
after EPC
start
2 months
after EPC
start
2 months
after EPC
start

EWI mentioned as

Source

A Subcontractor did not

perform in line with the
requirements.
Few Inexperience issues
identified within project
team
Extreme involvement by
few stakeholders raised
conflicts

Project
status
report
Project
status
report
Project
status
report

Analysis of the project status reports indicated that the project team integration was a
challenge during project execution. The project team (E&C, Client) had some inexperience
issues. Extreme involvement by few stakeholders amplified the decision-making problems. A
sub-contractor did not seem to be aligned with the project execution strategy, which prompted
the E&C contractor to take corrective actions that resulted into some rework.
The analysis of project reports suggests that the early warning in team development and
integration domain has substantial potential to predict the problems at late engineering and
construction stage. The link between early warnings, their associated future project problems,
and their impact on final project outcomes is attached in appendix D.1.1:
2.) Change management: A strict change management process was placed since the start
of EPC phase, although project execution went through changes both in scope
addition/reduction and in detailing of design. The numbers of change orders were seen as
extremely high by the E&C contractor team and issues were raised.
In addition, the frequency of change orders was quite high, there were 70 change orders
between 15 to 30 % engineering. The significant numbers of change orders not only affected
the engineering efforts, but also affected proper assessment of their impacts.
The status of early warnings is shown in following table.
Early warning

Presence

Detection time in
project

Early warning
mentioned as

Source

Number of changes

Yes

7 Months after
EPC start

Nos of agreed
change Orders
132

Change order
Log

Cost impact of changes

Yes

7 Months after
EPC start

Cost of
Changes/TIC of
plant = 6 %

Change order
Log

The relationship between early warnings related to project changes, project problems and their
impact on final project outcomes is illustrated with figure in attachment D.1.2.

66

It was observed from the study of project archives that such high magnitude of changes
introduced many problems during project execution. The manifested problems include
engineering, cost and schedule related problems.
3.) Early engineering: The assessment of FEED package and process engineering is
explained further below.
I) Completion of FEED As explained in Figure 2 when a project moves from the FEED
phase to EPC, generally a review should be conducted (known as gate review).
Furthermore, the quality of a gate review should be thorough, rather than a tick the box
approach. In the present case project, the gate between FEED and EPC is known as Gate C.
The Gate C review of the FEED package determines the quality of BOD and ideally, all issues
at this stage should be resolved before moving into full scale EPC.
Gate C review of the project indicates that the E&C team identified non-conformities and
observation from organizations standards including few related to design required for
downstream disciplines. Furthermore, due to schedule pressure, the project team made a plan
to deal with non-conformities; however, it was mentioned in project documentation that
project moved into EPC too early. In summary, the real implications of missing information at
FEED stage only became visible at EPC phase and resulted late engineering problems and
enforced E&C contractor to work on concurrent execution.
Early warning

Presence

Detection time in
project

EWI mentioned
as

Source

Percentage of
missing
information in
FEED package

Yes

Start of EPC

Non-conformity
Observations

Gate C review

II) Process Engineering: As per interdisciplinary logics of engineering design, any design
problem in process engineering produces a negative effect on downstream engineering
disciplines.
The case project has few issues with deliverables in process engineering. The extra efforts
made, which had to be made by process engineering to meet with deficiencies from previous
phase enforced few scheduling problems within the discipline and for downstream disciplines
and affected some of the deliverables of the project. The total project team worked cohesively
to deal with problems, thus caused some overrun in manhours.
The status of early warnings at 2/3 model review stage is shown in following table:
Early warning

Presence

Growth in process manhours

Yes

Delay in process
engineering

Yes

Detection time in
project
7 Months after EPC
start
7 Months after EPC
start

Early warning
mentioned as
Growth in process
hours
Process
engineering behind
schedule

Source
Project status
report
Project status
report

67

The relationships between early warnings, project problems (discussed in above paragraphs)
and project outcomes are shown D.1.3 below.
4) Concurrency:
In the case of project 1, the concurrency level in process and piping discipline varied
throughout the engineering execution. It can be concluded that 15 % to 30 % engineering
duration, process forecasted growth in hours, and was behind schedule. There was
downstream effect in piping and to meet the schedule constrains piping engineering made
assumptions. Later, when the piping engineering was not able to sustain the assumption and
could not ignore the risk of high amount of rework, it re-optimized its schedule and forecasted
some delays thus concurrency level decreased, going beyond the planned concurrency.
However, the concurrent execution of piping engineering in the early stage of the project (up
to 45 % engineering completion) resulted into few problems related to cost and schedule. The
re-optimization and some rework affected bulk quantities and their subsequent purchase; in
addition to some delay in project milestones.
5. Procurement:
The case project did not experience much delay (4 %) in issuance of purchase orders for
equipment as due to favorable economic conditions (Buyer market) at early 2008, and major
equipment were already purchased (50 % PO issued at start of EPC). However, from the
project execution perspective the strategy met with some challenges, mainly due to the
changes in the project. The E&C contactor made corrective strategies to deal with the problem
and had some cost and schedule impact on project.
Early warning

Presence

Delay in PO

Yes

Detection time in
project 2/3 Model
review
7 Months after EPC
start

EWI mentioned as

Source

Delay in PO

Project status report

From the perspective of EWI, the delay in PO did not manifest any major problem during the
project. However when coupled with late changes in the project it did contribute to the cost
overrun.

6.5 CASE 2 (SUCCESSFUL PROJECT)

6.5.1

Brief project description

The case project 2 stared in January 2011 and the planned to be complete in august 2013. The
objective of case 2 project is to build a chemical plant. The project is a part of larger project,
aimed to construct a chemical complex owned by consortium petrochemical companies. The
EPC contract between E&C contractor and owner is on lump-sum basis. The EPC project
duration is 29 months with 14 months for detailed engineering. The peak engineering staffing

68

for project was 160 full time equivalents (FTE). Although the project is still in end of
construction phase, the project performance so far has been very good in terms of cost,
schedule, quality and HSE.

6.5.2

Project evaluation

As explained in section 6.1 (case study design) project evaluations are performed with an
intention to identify the presence and prediction potential of EWI. The present case project is
of lump-sum nature and still in the last phase of execution; therefore, project reporting was
selective and regarded as highly confidential.
1) Team development and integration: The project execution team consisted of owner
and E&C contractor and work-share office within E&C contractor. The project management
team consisted of an integrated management team (IMT) with owner and E&C contractor with
an intention to gather the experience of operations from owner. The IMT approach was
adopted since the FEED phase of the project. The IMT approach did work well and no
significant issues were identified in the team development and integration.
2) Change management: The changes implemented during the detailed engineering phase
are considered significantly low for project of such magnitude. The cost impact of
implemented changes was only 0.6% of estimated TIC, with the largest change being 0.24 %
of TIC in magnitude.
The early changes did produce some rework during engineering, but the project was capable
of absorbing the impact. However, the schedule impact of changes was contained within
schedule and the 95% engineering was achieved ahead of schedule. In addition, no period in
the project experienced relatively high numbers of changes.
The well executed no change policy helped the project in initial stages by not only
minimizing the impact on engineering efforts, but also to assess the impact of changes more
precisely. The figure 25 below underlines the above observations.
The status of EWI related to changes is as follows:
Early warning

Presence

Number of changes

Yes

Timing of changes

Yes

Cost impact of changes

No

Potential direct impact of

changes on engineering
Hours

No

Detection time in
project 2/3 Model
review
7 Months after EPC
start
7 Months after EPC
start
7 Months after EPC
start
7 Months after EPC
start

Early warning

Source

Nos of change Orders

20
Nos of changes after 1/3
model review 20
Cost of Changes/TIC of
plant = 0.4 %

Change
order Log
Change
order Log
Change
order Log

CO man-hours/original
budget = 0.3 %

Change
order Log

The mapping of EWI, associated project problems (discussed in above paragraphs) and their
impact on project outcomes has been illustrated in appendix D.2.1.

69

I) Completion of FEED: The Gate C review of the project was conducted 3 months prior to
the start of EPC phase with an objective to assess if the BOD and the associated work were
ready to proceed into EPC phase. The overall assessment suggested that the BOD was
sufficiently defined to move into EPC phase.
The results of Gate C review indicated few non-conformity and observations, however the
time available between the FEED and EPC phase allowed for the resolution of all the
observations. All non-conformities were rectified by E&C project team.
Early warning

Presence

Detection time in
project Gate C
Review

EWI mentioned as

Source

Percentage of
missing information
in FEED package

No

Start of EPC

Non-conformity 0
Observations 4

Gate C review

II) Process engineering: The process engineering execution for case project went
extremely well with ahead of schedule and with small person-hour growth. The process
deliverables were well on schedule with no or minimum scope changes at later stage, that
could be validated with only 0.82 % man-hour growth in process engineering caused by
changes. In addition, there were no significant project problems identified.
The status of EWI related to process engineering is shown below.
Early warning

Presence

Detection time in
project 2/3
Model review

Growth in process manhours

No

7 Months after EPC

start

Delay in process
engineering

No

7 Months after EPC

start

EWI mentioned as
Growth in process
hours 3%
(negative)
Process
engineering ahead
of schedule (3 %)

Source
Project progress
data
Project progress
data

4) Concurrency: The case project did experience increase in the concurrency between
process and piping engineering. However, the increase in concurrency did not manifest into
any major project problem. The status of CE has been shown in following table.
Early warning

Presence

Detection time in
project 2/3 Model
review

EWI mentioned as

Source

Change in concurrency
level between process and
piping engineering

Yes

7 Months after EPC

start

Increase in
concurrency by
20%

Discipline progress
curves.

5. Procurement: The case project did experience a delay in issuance of purchase orders for
equipment during early stages. However, the delay was recovered soon enough and purchase
orders were issued well in time to receive the required vendor information. From the

70

perspective of early warning, the delay in PO did not manifest any major problem during the
project.

Early warning

Presence

Detection time in
project 2/3 Model
review

EWI mentioned as

Source

Delay in PO

Yes

7 Months after
EPC start

Delay in PO 0.1
%

Project status report

6.6 CASE 3 (SUCCESSFUL PROJECT)

6.6.1

Brief project description

The case three project was aimed at up-gradation of a fuel refinery to produce cleaner fuels.
The project was brown field in nature and involved tie-in with existing operation plant. The
contract between E&C contractor and project owner was EPCm under cost reimbursable with
incentive arrangement. The period for project EPC execution was 2004 through 2005 with 13
months for detailed engineering and 19 months to MC. The project performed quite
extraordinary with under run in TIC and on time completion of plant. Furthermore, the project
performance in terms of safety and quality was in line with clients expectations.

6.6.2

Project evaluation

1) Team development and integration: The case project involved a Fluor project team,
client team, and Fluor work share offices. The study of internal status review documents
suggested that were no issues at all with client and internal interfaces. On few instances, the
project literature did mention conflicts with the client during initial stages, but they were
limited to negotiations. Later the contract negotiations concluded with satisfaction of all
parties concerned. The E&C team was open to client audits and there were no major
disagreement issues identified.
Early warning

Presence

Lack of understanding of project

execution strategy among project
team

Detection time
in project

EWI mentioned as

Source

No

No issues identified

Project status
report, IPA audit

Project team lacks experience

required for the project

No

The project team is well

integrated &
experienced

Project status
report, IPA audit

Conflicts between owner and

E&C contractor

No

No issues identified

Project status
report

2) Change management: The project change log reveals that there were no significant
changes in the project due to the no change culture from client and E&C contractor. The

71

successful implementation of a No change philosophy was possible due to the strong and
clear implementation of FEED package, management support and regular feedback to the
team regarding success of No change philosophy. The changes implemented in EPC phase
resulted in net cost saving. The engineering hours in change orders show 4 % growth with
respect to original budget, and the total TIC impact shows 1 % saving. The following
observation is shown by graph.
The EWI identification:
Detection time in
project 2/3
Model review
7 Months after
EPC start

Early warning

Presence

Number of changes

Yes

Timing of changes

Yes

7 Months after
EPC start

Cost impact of changes

No

7 Months after
EPC start

Hour impact of changes

Yes

7 Months after
EPC start

EWI mentioned as
Nos of change
Orders 14
Nos of changes
after 1/3 model
review 14
Cost of
Changes/TIC of
plant = -0.87%
CO manhours/original
budget = 4 %

Source
Change order Log
Change order Log
Change order Log
Change order Log

3) Early Engineering: The early engineering in EPC phase O&C project can be seen in two
perspectives:
I) Completion of FEED: The quality of FEED for case project was significantly high. The
audit by IPA suggested that the FEL index for FEED was 4.5, which is considered as best in
class. The project moved into EPC phase with just three non-conformities (As per Gate C
review). The gate-C audit report of the project concluded that the design is ready to move into
EPC phase. However, the report did recommend that project should wait for 2 weeks for the
start of few activities to include the information in BOD rather than working on assumptions.
No information could be found regarding project action on the recommendations of the audit
report.
Early warning
Percentage of
missing information
in FEED package

Presence

Yes

Detection time in
project Gate C
Review

EWI mentioned as

Source

Start of EPC

Numbers of Nonconformities
FEL Index 4.5
Best in class

Gate C audit
report
IPA Audit report

II) Process engineering: The case project had an exceptional well-executed process
design. As mentioned earlier, this is due to well defined FEED package and No change
philosophy employed in the project. The process engineering deliverables experience no major
changes and timely process deliverables enabled downstream disciplines to execute work as
planned. Furthermore, the process man-hours showed under run of 9 %. This statement could

72

be verified by the zero growth in process man-hours and on schedule execution of process
engineering.

Early warning

Presence

Detection time in
project 2/3 Model
review

EWI mentioned as

Source

Growth in
process manhours

No

7 Months after EPC

start

Growth in process hours

under-run by 16 %

Project status report

Delay in
process
engineering

No

7 Months after EPC

start

Process engineering
behind schedule 0 %

Project status report

4) Concurrency in engineering: The concurrency level between process and piping

engineering did not change much during project execution, only at one instance the
concurrency increased to 20 % level at 45 % engineering complete, which could be explained
be a slight delay in process engineering and its projection of optimistic schedule. Yet the
project was able to absorb the increase in slight concurrency in concurrency level without
compromising schedule or cost.
From the gathered information, no major problem could be identified which was evolved due
to concurrent engineering.
5) Procurement: The project did not experience any significant delay in issuance of purchase
orders, and no problems were mentioned in the project documentation that was a result of
delay in purchase orders.
Early warning

Presence

Delay in PO

Yes

Detection time in
project 2/3
Model review
7 Months after
EPC start

EWI mentioned
as

Source

Delay in PO 2
%

Project status report

6.7 CASE 4 (LESS THAN SUCCESSFUL PROJECT)

6.7.1

Project brief description

The case four project was aimed at construction of a chemical processing facility. The project
was green field in nature and was extension of an existing chemical complex. The contract
between E&C contractor and project owner was for EPCm under cost reimbursable with
liabilities. The schedule was the main project driver followed by cost and quality. The period
for project EPC execution was 2010 through 2012 with 11 months for detailed engineering
and 18 months to MC. The project did not perform as per expectations with over run in TIC

73

and schedule overrun. Furthermore, the project performance in terms of safety and quality was
in line with clients expectations.

6.7.2

Project evaluation

1) Team development and integration: The project execution team involved E&C
contractor, client team, and work share offices with in Fluor. The project adopted an
integrated project execution team led by clients project manager. The summary of the
findings on project team development and integration is given in following table:
Early warning

Presence

Lack of understanding
regarding project execution
strategy among project team5

Yes

Project team lacks experience

required for the project.

Yes

Conflicts between owner and

E&C contractor

Yes

Detection
time in
project
2 months
after EPC
start
2 months
after EPC
start
2 months
after EPC
start

EWI mentioned as
Different perception about project
execution strategy
The project required new software
system for piping design, on which
project team had limited experience.
Different underlying interests of
stakeholders.
Less synergy on impacts of proposed
changes on project

Source
Project
status
report
Project
status
report
Project
status
report

Analysis of the project status reports indicated that the project team integration had a good
start with establishment of integrated project management team, but soon challenges emerged
due to different underlying interest of stakeholders. At few instances, intervention level by
stakeholders was seen as significantly high and was a point discussion among project team.
Moreover, the schedule and cost impact of almost all changes were challenged by project team
with an intention to only accept necessary changes, which ultimately affected relations and
timing of change implementation.
The link between EWI, above discussed project problems and their impact on final project
outcomes is illustrated in attachment D.4.1:
2.) Change management: The basis for development of aggressive project schedule was
no change during engineering, however the project experienced introduction of heavy
changes since the start of EPC phase. Within 5 months of EPC, 51 change orders were issued.
These changes were in direct violation of assumptions made in schedule development. The
possible impacts of such high changes were highlighted by project E&C project management.
At intermediate model review, there were 125 approved changes in the project with a cost
impact of 4 %. The summary of the relevant EWI is presented in following table:

However, there is no specific reference to the difference in understanding of project execution strategy
among project team; the study of project document does suggest that there were differences in confidence
level of team members regarding highly concurrent execution model.

74

Detection time in
project 2/3 Model
review
7 Months after EPC
start

Early warning

Presence

Higher number of changes

Yes

Timing of changes

Yes

7 Months after EPC

start

Cost impact of changes

Yes

7 Months after EPC

start

Potential direct impact of

changes on engineering Hours

Yes

7 Months after EPC

start

EWI mentioned as

Source

Nos of change
Orders 125
Nos of changes
after 1/3 model
review 125
Cost of
Changes/TIC of
plant = 4 %
CO manhours/original
budget = 23 %

Change order
Log
Change order
Log
Change order
Log
Change order
Log

The analysis revealed that the project did acknowledge the impact of changes and their effect
of final engineering hours yet was a point of conflict.
In addition, the frequency of change orders was quite high as there were on average 26 change
orders between 0 to 60 % engineering. The significant numbers of change orders not only
affected the engineering efforts, but also brought inefficiency in assessing the impact of
changes. Moreover, relationship between EWI related to project changes, the project
problems, and their impact on final project outcomes is illustrated by attachment D.4.2
3.) Early engineering:
I) Completion of FEED The Gate C review conducted by project team indicates that the
FEED package had non-conformities and observations of deviations from the standards of
E&C organization. The literature shows there were deficiencies with the received BOD and
was not of good quality to proceed into EPC phase. However, due to schedule requirements
the project continued for a certain time with project team simultaneously improving BOD. In
summary, the real implications of missing information at FEED stage only became visible at
EPC phase and resulted late design changes and concurrency in design disciplines.
Early warning

Presence

Incomplete front
end engineering
design

Yes

Detection time in
project Gate C
Review
Start of EPC

EWI mentioned as

Source

Non-conformity
Observations

Gate C review

II) Process Engineering: The P&IDs during early design went through major changes as
soon as project moved into EPC phase. Because of the incompleteness in the BOD, the
activities had to be reworked. Moreover, the discovery of new work during process
engineering schedule and cost related problems in project. It was observed that the E&C
process engineering experienced & acknowledged the problems at very early stages, and
forecasted growth, which produced cascading effect into downstream discipline and ultimately
total engineering.

75

Early warning

Presence

Growth in process manhours

Yes

Delay in process
engineering

Yes

Detection time in
project 2/3 Model
review
7 Months after EPC
start

EWI mentioned as
Growth in process
hours

Project
report

status

7 Months after EPC

start

Process
engineering behind
schedule

Project
report

status

Source

The relationships between early warnings, project problems, and their impact on project
outcomes are shown in attachment D.4.3:
4) Concurrency:
In case of project 4, a concurrent project execution model with high concurrency level in
process and piping discipline was planned. An interesting observation was made that the
concurrency level did not change significantly throughout the project engineering duration. It
implies that the during the project execution the E&C team was proactive and acknowledge
the risk of concurrent execution and piping engineering forecasted the realistic progress plans
based on behavior of process engineering. However, at the later stage, the concurrency level
reduced further because frequent changes forced piping engineering re-optimize its planning.
From EWI perspective, the change in concurrency did not reveal any significant problem
mainly because the pre-set level of concurrency very initially high and as the detailed
engineering progressed in the project, the process and piping discipline addressed the early
problems and reforecast their realistic growth.
5. Procurement: The case project experienced delays in issuance of purchase order for
equipment. The delay in PO did affect the downstream disciplines regarding required vendor
information for their design.
Early warning

Presence

Delay in PO

Yes

Detection time in
project 2/3 Model
review
7 Months after EPC
start

EWI mentioned as

Source

Delay in PO

Project status
report

Delayed vendor design information produced a cascading effect and ultimately induced
schedule and cost related problems, which in turn affected the availability of work front for
mechanical and piping contracts at site. From the perspective of EWI, the delay in issuance of
PO resulted few project problems and ultimately contributed to the growth in project
outcomes. The relationship of early warning, project problems, and project outcomes is shown
in attachment in D.4.4.

76

6.8 CROSS CASE ANALYSIS

This section presents a cross case analysis to build a contrast by comparing the four case
projects. The cross case analysis is conducted by collective discussion of early warnings
across all four case studies.
Project team development and integration: All three early warnings (LES, PTE, COC) in
this category were identified in case of project 1 and case project 4, that were less than
successful projects.
Early warnings
LES
PTE
COC

Case 1

Case 2

Case 3
X
X
X

Case 4
X
X
X

Red Tick = Early warning detected and contributed to project problems, Green tick =
early warning detected but did not result into any problem, X = no early warnings was
detected
The effect of COC was observed to be important in all case projects. The conflicts in project
environment not only reduce the effectiveness of decision-making process, but also affect the
management of changes. From efficiency and communication perspective, the early warning
LES did play an important role during execution of case projects. In case of project two and
three, the positive synergy on project execution strategy among project team members did
improve the communication and enhanced the overall efficiency. In addition, the
understanding of project execution strategy among project team does result into a focused
approach towards projects main driver and help the project to adapt in case the project driver
changes (due to external or internal environment). In addition, it was found that incase of
projects with less than successful performance, lack of experience among few domains of
project team members did induced problems.
Change management: All four case projects did declare the use of No change policy at the
start of EPC phase. The no change prohibits the introduction of any change unless and until
the design is either unsafe, does not function or violate any procedure (Bakker, 2012).
However, almost all the projects did implement changes during their execution.
Early warnings
NOC
CCO

Case 1

Case 2

Case 3

Case 4

= Early warning detected and contributed to project problems, = early warning

detected but did not result into any problem, X = no early warnings was detected
In case of projects with less than successful performance, (case project one and four) the
changes were seen as extremely high and did contribute to the manifestation of project
problems. The major problems found to be associated with changes were rework (In E&C
domain and associated contracts and vendors), out of sequence activity execution.
Furthermore, the high nos and frequency of changes negatively affected the assessment and
management of changes.

77

The case project two experienced some rework due to early changes in the project that
resulted into growth in engineering man-hours and made small impact on the TIC. However,
the small nos of changes allowed proper assessment and management of those changes.
In case of project 3, the nos of changes were seen as small by project. In addition, the
majorities of the changes were small in their cost impact and intended towards cost savings
and no major problems were found to be associated with implemented changes during project
execution.
Early engineering: The early warnings FED, PH and PS did show a distinct behavior in
successful and less than successful projects. The results of early warnings in this category are
shown in following table.
Early warnings
FED
PH
PS

Case 1

Case 2

Case 3
X
X
X

Case 4

X
X

= Early warning detected and contributed to project problems, = early warning

detected but did not result into problems, X = no early warnings was detected
Significant non-conformities and observations were found in FEED design of case project one
and four that subsequently helped in manifestation of project problems. One of the major
problems found to be associated with FED was additional rework to improve the quality of
FEED, which was earlier not accounted in the EPC estimate. The second problem was
discovery of new work in form of changes that brought engineering problems and to a certain
extent affected the procurement efforts.
The both early warnings in process engineering domain i.e. PS and PH were detected in case
of project one and four, and contributed to project problems. From schedule perspective, the
delay in deliverables of process engineering did affect the activities of downstream disciplines
and key project milestones such as intermediate model review. In addition, the delay also
resulted into to concurrent engineering to maintain the project schedule.
The case projects two and three did not experience any significant growth in process hours or
delay in process engineering. Thus not only avoided the above-mentioned problems however
supported the downstream discipline with on schedule deliverables and concrete information.
Concurrency: The concurrent engineering design between process and piping engineering
was observed in case project one, two and three.
Early warnings
CE

Case 1

Case 2

Case 3

Case 4

= Early warning detected and contributed to project problems, = early warning

detected but did not result into problems, X = no early warnings was detected
However, the amount of change in concurrency differs with each project. The case project one
experienced a 20% increase in concurrency until 45% engineering completion followed by
reduction of 40% from baseline concurrency level. Such high change in concurrency level
represents the level of assumptions adopted by piping engineering. The subsequent reduction

78

in concurrency level was observed due to the high risk associated with assumptions thus
delaying the deliverables or actual information found to be different from assumptions, which
forced the piping discipline to forecast delay in their schedule.
In case of project two and three, the change in concurrency level was less significant as
compared (20%) to project one. The projects were able to absorb the changes and did not face
any major problems. The ability to absorb the increase in concurrency level could also be
explained by number of changes in these projects, as the project did not experience significant
amount of changes therefore it could be concluded that the assumptions made by piping
engineering did not change significantly.
From change in concurrency perspective, the case project four proved to be an exception. The
project was seen as extremely concurrent by project team members, therefore the piping
engineering discipline was proactive in forecasting its schedule with reference to schedule of
expected inputs from process engineering. The joint change in progress curves of these two
disciplines resulted into low change in concurrency level.
Procurement: The early warning, DPO was detected in all four case projects.
Early warnings
CE

Case 1

Case 2

Case 3

Case 4

= Early warning detected and contributed to project problems, = early warning

detected but did not result into problems, X = no early warnings was detected
In case of project one, majority of PO were issued before the start of EPC (due to favorable
market conditions) and project did not experience any significant delay in rest of the PO.
However, the adopted approach faced challenges due to the high number of changes. The
successful projects two and three did experience slight delay in issuance of PO. However,
their delay is limited to 30 % engineering completion. After which they delay was recovered
and did not contribute to any major problems.
The case project four did experience delay in issuance of PO. The delay was found to
associate with few major problems during project execution. The schedule of detailed
engineering was impacted due to delayed vendor design information. Consequently, the
estimation of bulk materials was delayed followed by delay of constriction drawings and work
at construction site.

6.9 DISCUSSION AND CONCLUSION

In this chapter the relationship between early warnings, project problems and project
outcomes were presented. It was investigated that whether the identified early warnings help
the manifestation of project problems or they themselves turn into problems and affect the
project performance. The four in-depth case studies showed the dynamic between early
warnings and project performance in form of project problems. From overall research
perspective, this chapter contributed to answering the research question What are the
dynamics between early warnings and project performance?
Understanding the dynamics between early warnings and project performance: Through
considerable analysis of project documentation, each early warning was mapped with the

79

specific project problems and project problems were analyzed to map their affect on project
performance. By looking at the time aspect of relationship, it was concluded that the early
warnings could predict the project problems and specific project problems manifest because of
problems in domain of these early warnings.
Observatory evidence to predictive capability of early warnings: The contrast set by
project with different performance project played a significant role in highlighting the
predictive capability of early warnings. The differences in detection of early warning and their
associated states (quantitative and qualitative) validate the importance of early warnings
during project execution. It can be concluded that if these early warnings are monitored and
interpreted as symptoms of potential problems, they could provide advance and crucial
information for proactive actions.

80

7 QUANTITATIVE ANALYSIS
SUMMARY
In the previous chapter, it was observed that in case of good performance projects, most of the
early warnings were absent or had quantitative values within acceptable limits (In the view of
project team). The projects with significant cost/schedule overrun exhibited early warnings
with significant high quantitative values. The difference in performance of projects in light of
early warnings became clearer during cross case analysis in section 6.8.
In the present chapter, an attempt has been made to explore the set of early warnings over a
larger number of projects than in chapter 6. As explained in section 5.7, it is not possible to
include qualitative early warnings (LES, COC and PTE) in development of the quantitative
prediction model without introducing user specific subjectivity.
In addition, the early warning completion of FEED could not be implemented, as it is static
in nature and provides information only at the start of EPC phase. Furthermore, if the
deficiencies in FEED are not corrected before diving into EPC phase, the other early warnings
such as process engineering should show warning like behavior.
Section 7.1 discuses the analysis approach at three levels, longitudinal behavior analysis of
early warnings, correlation analysis at 95% engineering completion and longitudinal
correlation analysis of the project. The description of data collected for past projects along
with an explicit description of early warnings as explanatory variables and project outcomes
as dependent variables has been attached as appendix G.
Later in the chapter, section 7.2.1 analyzes the longitudinal behavior of early warnings over
engineering duration. For this purpose, each EWI is plotted over the engineering duration of
the project. Subsequently to have the quantitative evidence of relationship between EWI and
project outcomes, the correlation analysis at 95 % engineering completion is performed in
section 7.3, followed by longitudinal correlation analysis over engineering duration. In the end
of this chapter, section 7.5 provides the discussion on three levels of analysis and conclusions
of this chapter.

7.1 ANALYSIS APPROACH

The quantitative analysis is performed at three levels. The first level of analysis investigates
the longitudinal behavior of early warnings over engineering duration. The second level of
analysis investigates the correlations of EWI with project outcomes at 95% engineering
completion. The third level of analysis combine the above two level and establish the
correlations between EWI and project outcomes longitudinally over engineering duration. The
third level of analysis will lay the foundation for development of dynamic prediction model.
The analysis approach is shown in Figure 19: Quantitative analysis approach and their
relation is explained in following paragraphs.

81

Figure 19: Quantitative analysis approach

Based on the finding of cross case analysis in section 6.8, a hypothesis is made that the EWI
do behave differently in case of successful and less than successful projects. The first level of
analysis tests this hypothesis by studying the longitudinal behavior of EWI over engineering
duration of past eight projects. If the results of the analysis are found to be affirmative towards
stated hypothesis, then the findings could be seen as first empirical evidence of predictive
capability of EWI. In addition, the results could validate the correlations found in subsequent
analysis and supports the causality between EWI and project performance.
The second level of analysis, i.e. correlations at 95 % engineering completion could be seen as
complementary to level one analysis and at the same time take the analysis further by mapping
the relationship of each EWI with parameters of project performance i.e. project outcomes.
The main question to be answered by this analysis is that which EWI relates to which project
outcome. As it could be observed in chapter 6 that not all the EWI contribute to all the
project problems and not all the project problems, affect all project outcomes. Therefore, the
second level analysis makes an effort to understand the relationship between EWI and project
outcomes.
The third level performs a longitudinal correlation analysis over the engineering duration.
This analysis could be seen as a statistical combination of first two analyses. In addition, it
provides the dynamic statistical relationship between EWI and project outcomes that will be
used for building of prediction model in subsequent chapter.

7.2 EXPLORATORY DATA ANALYSIS

To build the prediction model, data relevant to six early warnings and four project outcomes is
collected from past eight projects. The dataset constitutes a relatively balance type of project
in terms of size, cost and type. The overview of the past project has been provided in appendix
F. The final project outcomes of eight past projects are described in appendix E.

82

Note: Moreover, looking at the project outcomes in four projects (in appendix E) with
reference to the study presented by Merrow6 in chapter 1, it can be inferred that the project
execution by Fluor Corporation are well executed. (As no project experienced growth of 30%
in either TIC or MCI)
It could be observed from the figure that the data include a mixed performance projects. For
the classification purposes, the present research consider a project with less than successful
performance, if and only if the project has 15 % growth in either final cost (TIC) or schedule
delay of more than 10% in completion of project (MCI) and combination of both.
The scatter plot matrix was plotted and has been attached in appendix E. It is observed that the
there is a strong relationship between TIC and MHI; this implies that a significant growth in
engineering man-hours leads to the growth in TIC. From cost perspective, the observation
seems obvious as engineering manhours has cost associated with them. As explained in
section 4.3.2 the cost of engineering efforts is quite small as compared to the cost of
equipments and construction (on average varies between 10-15 % of TIC). Therefore, even a
100 % growth in manhours could only affect 10 to 15 % of TIC. However from logical
perspective the above observation implies that the impacts of problems in engineering design
is not limited to engineering itself, in addition they do affect the activities in procurement and
construction, therefore affect total project cost. The above statement could be validated by
found relationships between EWI, project problems and project outcomes in chapter 6.
The above observation is also true for MCI and ESI i.e. delay in completion of detailed
engineering leads to the delay in mechanical completion of the project. As discussed in section
4.3.2 engineering prepare and deliver key documents for procurement and construction
activities, therefore a relationship between MCI and ESI validate that the delay in issuance of
these key documents do impact the progress of procurement and construction activities.
Another important observation is that the growth in engineering manhours does not
necessarily result in improvement of schedule. This could be observed by absence of negative
relationship between MHI and MCI. Therefore, it can be concluded that in most of the past
projects the growth in man-hours was not caused by schedule acceleration, rather rework or
other problems.

7.2.1

Behavior of EWI over engineering duration

The next step in exploratory analysis was to study the behavior of each early warning to
check, if there exists a distinct behavior for projects that performed well and others whose
performance was not as per expectations. To achieve the above goal, each early warning was
plotted over the engineering duration of the project and results are attached as Appendix F and
discussed in following paragraphs.
Process schedule (PS): It is observed that PS showed a distinct behavior in good or
average performance projects and less than successful performance projects. An interesting

O&C industrys 50% project experience growth more than 30% in either cost or schedule.

83

observation is that in most of the projects, which did not perform well, process engineering
was not able to recover from delay until 60 % of the engineering duration. After which, almost
all the projects did show improvement. This could mean that the delay in early deliverables of
process engineering have more impact on the schedule and performance of downstream
disciplines and total project performance7.
Process hours (PH): The results are similar as PS. The PH had significantly low growth
values in good or average performance and high values in less than successful performance
projects. It could be observed from the figure that process engineering start projecting
significant growth after 30% engineering completion. When combined with delay in process
schedule, it can be concluded that process engineering forecast growth in manhours after
actual delay in early deliverables ( average 7.5 % delay in process at 30% engineering
completion) , which might be due to the changes or incomplete FEED.
Number of change orders (NOC): The analysis found that the good performance project did
experience less numbers of changes as compared to the less than successful performance
projects. The analysis indicated that the behavior of cumulative NOC in case of successful
projects (Green and blue) is significantly different as compared to less than successful projects
(Red). However at the early stages in the project, no clear distinction could be found between
projects (Except two), therefore the incremental number of changes were also explored and
difference became clearer as the less than successful performance projects did experience
relative higher number of changes within a short time span during their engineering execution.
Therefore, it can be concluded that changes combined with high incremental changes could
lead to bad performance of a project.
Cost of changes (COC): The costs of changes adopted during project execution do reflect
differences in projects, at-least after 45 % engineering completion. From zero to 45%
engineering duration, it is difficult to segregate between successful and less than successful
performance of the projects (same as NOC). However, the projects, which kept on introducing
changes even after 45 % engineering, ultimately resulted in undesired performance as
compared to the project that experienced little or no additional cost due to changes
Delay in purchase orders (DPO): The behavior of early warning DPO did not show a clear
distinction between the Successful and less than successful performance projects. Along with
successful projects, some less than successful performance project did not experience any
significant delay in issuance of purchase orders.
However, a deeper view on the projects provided more insights into the behavior of DPO.
Some less than successful performance projects issued their PO before the start of engineering
phase of the project, due to either favorable market conditions or the equipments had long lead
times. Despite the project strategic decisions, it can be concluded that the delayed issuance of

For more explanation, please see section 5.7

84

purchase order does affect the project performance as the related vendor information might be
delayed8.
In addition, the strategic advantage gained by early issuance of PO could be lost if changes
during detailed engineering impact the vendor equipments or services because that will cause
additional cost and schedule claims by the vendor and contribute to the cost and schedule
overrun of a project.
Concurrent engineering (CE): The early warning CE reflects the change in concurrency
level between process and piping engineering, i.e. what is the relative change in behavior of
piping engineering progress w.r.t process engineering. The analysis suggests that the in case
of less than successful performance projects (Shown in Red) the piping engineering progress
curve did came close to process engineering curve during early engineering, which implies
that they worked on concurrent engineering model9. However later, three out of five less than
successful performance projects were not able to sustain the concurrency level and piping
engineering subsequently projected schedule delay.

7.3 QUANTITATIVE ANALYSIS

The analysis and exploration of data in previous section did provide us with few encouraging
results. The analysis suggested that the six early warnings did behave differently in case of
successful and less than successful performance projects. In this section, the quantitative
analysis is performed to find the correlations between EWI and project outcomes.
The data from eight projects was collected at baseline, during detailed engineering (six data
points) and at end of the project10. The sample size contained eight projects, which is not
considered as significant for quantitative analysis; therefore, to obtain firm quantitative results
more data would be required. Since the data contained non-normally distributed variables,
spearmans rho correlation was used. The following classification for the correlation
strength was used (Cohan and Holiday, 1982) to classify correlation strength.
0.9 to 1: Very strong correlation
0.7 to 0.9: Strong, high correlation
0.4 to 0.7: Moderate correlation
0.2 to 0.4: weak, low correlation
0 to 0.2: weak to negligible correlation

The conclusion is drawn on the basis of information provided by interviewees and explained in section 5.7

High concurrency implies that the piping engineering has to work on assumptions due to non-availability
of information from process engineering. For detailed explanation please see section 5.7

10

For detailed explanation see section 0

85

Table 4 shows the correlations between project outcomes and PH at 95% engineering
completion. A strong and significant correlation was found between PH and ESI (rs = 0.74, p
= 0.0366), Very strong correlation between PH and MCI (rs = 0.97, p = 0.001), and between
PH and TIC (rs = 0.69, p = 0.058). A moderate correlation with 85 % significance was found
between PH and MHI (rs = 0.55, p = 0.15).
Table 4: Correlation between growth in process engineering and project outcomes
EWI

Project
outcomes

Correlation
strength (rs)

Significance (P)

ESI

0.74+++

0.03

Growth in progress hours results into

delay in 95 % engineering.

MCI

0.97++++

0.001

Growth in progress hours results into

delay in mechanical completion of the
project.

MHI

0.550*

0.15

Growth in progress hours results into

growth in total engineering manhours.

TIC

0.69+++

0.05

Growth in progress hours results into

growth in TIC of the project.

PH

Meaning of correlation

++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
The presence of these correlations with high strength and significance suggested that growth
in process engineering hours indeed affect the project outcomes and ultimately project
performance.
Table 5 below shows the correlations between delay in process engineering (PS) and project
outcomes. The negative sign in correlations means that delay in process results into growth in
project outcomes. A high and significant correlation was found between PS and ESI (rs = 0.68, p = 0.15) and between PS and TIC (rs = -0.75, p = 0.03). Very high and significant
correlation was found between PS and MHI (rs = -0.93, p = 0.007). However no significant
correlation was found between PS and MCI (rs = -0.52, p = 0.18)
Table 5: Correlation between delay in process engineering and project outcomes
EWI

Project
outcomes

Correlation
strength (rs)

Significance (P)

ESI

-0.68++

0.06

MCI

-0.520

0.18

MHI

-0.93++++

0.00

Delay in process engineering results

into growth in engineering man-hours.

TIC

-0.75+++

0.03

Delay in process engineering results

into growth in TIC of the project.

PS

Meaning of correlation
Delay in process engineering results
into delay in 95 % engineering
completion.
Delay in process engineering results
into delay in mechanical completion of
project.

86

++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
The above correlation suggests that delay in process engineering does affect the project
outcomes (3 out of 4) and ultimately project performance. The absence of significant
correlation between PS and MCI can be interpreted as that delay in process engineering might
be resulted into deployment of additional resources, which resulted into high manhours and
high TIC of the plant but did not impacted the MCI.
Table 6: Correlations between number of changes and project outcomes
EWI

Project
outcomes

Correlation
strength (rs)

Significance
(P)

ESI

0.79+++

0.02

MCI

0.83+++

0.01

MHI

0.6+

0.10

Higher number of change orders resulted in

growth in engineering manhours

TIC

0.520

0.18

Higher number of change orders resulted

into growth in TIC of the project.

NCO

Meaning of correlation
Higher number of change orders resulted
into delay in completion of 95 %
engineering
Higher number of change orders resulted
into delay in mechanical completion of
project.

++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
The above Table 6 provides the relationships between number of change orders and project
outcomes. The correlation analysis suggests that there is strong and significant correlations
between NCO and ESI (rs = 0.79, p = 0.02) and NCO and MCI (rs = 0.83, p = 0.01).
Moreover, moderate, low significant correlation between NCO and MHI (rs = 0.6, p = 0.10)
and NCO and TIC (rs = 0.52, p = 0.18).
The correlations strength and values suggests that the high number of change orders does
affect the schedule outcomes i.e. 95 % engineering completion and mechanical completion of
the plant. However, does not have significant relationship with growth in manhours and
growth in TIC of the plant.
The Table 7 below shows the correlations between cost of changes and project outcomes. In
case of COC strong and significant correlation was found between CCO and TIC of the plant
(rs = 0.69, p = 0.05) and moderate and low significant correlation between CCO and MHI (rs =
0.52, p = 0.15). The correlation implies that the cost of changes does affect the project
outcomes associated with cost i.e. MHI and TIC of the project. In addition, it implies that cost
of changes does include additional engineering man-hour cost and cost does not only include
material cost.
Table 7: Correlations between cost of changes and project outcomes
EWI

Project
outcomes

Correlation
strength (rs)

Significance (P)

Meaning of correlation

87

ESI

0.240

0.57

Changes with high cost results into delay in

completion of 95 % engineering

MCI

0.350

0.39

Changes with high cost results into delay in

mechanical completion of the project

MHI

0.550*

0.15

Changes with high cost results into growth in

total engineering manhours

TIC

0.69++

0.05

Changes with high cost results into growth in

TIC of the project

CCO

++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
The Table 8 below shows the correlations between delay in purchase orders and project
outcomes. The negative sign in correlations implies that the delay in purchase orders results
into growth in project outcomes. The analysis found a very strong and highly significant
correlation between delay in DPO and MHI (rs = -0.99, p = 0.001). A strong and significant
correlation was found between DPO and TIC (rs = -0.73, p = 0.03). Moderate the low
significant correlations exits between DPO and ESI (rs = -0.50, p = 0.20) and between DPO
and MCI (rs = -0.49, p = 0.21).
The correlations strength and significance suggests that delay in issuance of purchase orders
resulted into growth in cost aspects of the project i.e. MHI and TIC, which could imply that
the projects intend to stay on schedule by either working on assumption, due to delayed
vendor information. Subsequently, then spend additional manhours in re-verifying the
assumption or re-working in case the assumptions were wrong and accepted high growth in
manhours and final TIC of the plant.
Table 8: correlations between delay in purchase orders and project outcomes
EWI

Project outcomes

Correlation
strength (rs)

Significance (P)

ESI

-0.50

0.2039

MCI

-0.490

0.2199

MHI

-0.99++++

0.001

TIC

-0.73+++

0.0396

DPO

Meaning of correlation
Delay in purchase orders resulted
into delay in completion of 895 %
engineering.
Delay in purchase orders resulted
into delay in mechanical
completion of the project.
Delay in purchase orders resulted
into growth in total engineering
manhours of the project
Delay in purchase orders resulted
into growth in TIC of the project

++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8
The Table 9 below shows the correlations between the final changes (at
completion) in concurrency between process and piping engineering and
The positive sign in the correlations means that positive change (low
planned concurrency level) results into growth in project outcomes.

95 % engineering
project outcomes.
concurrency w.r.t
A moderate and

88

significant correlation was found between CE and MHI (rs = 0.64, p = 0.08). This means that
the delay in progress of piping engineering due to delay in inputs from process engineering
ultimately results into growth in engineering man-hours.
Table 9: Correlation between Change in concurrency and project outcomes
EWI

Project
outcomes

Correlation
strength (rs)

Significance (P)

ESI

0.140

0.73

MCI

0.20

0.62

MHI

0.64++

0.08

TIC

0.380

0.35

CE

Meaning of correlation
Significant change in concurrency between
process and piping engineering results into
delay in completion of 95% engineering.
Significant change in concurrency between
process and piping engineering results into
delay in mechanical completion the project
Significant change in concurrency between
process and piping engineering results into
growth in total engineering man-hours.
Significant change in concurrency between
process and piping engineering results into
growth in final TIC of the project.

++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level,
* significant at 0.15 level, N = 8

7.4 CORRELATIONS OVER ENGINEERING DURATION

The analysis of correlations over engineering duration indicates that not all the EWI have
correlations with all the project outcomes at all prediction moments. The summary of
significant correlations w.r.t each prediction moment is provided in Figure 20. Furthermore,
the correlations per project outcome over engineering completion are provided in Figure 21 .
The following paragraphs discuss the analysis of correlation in Figure 20 and Figure 21.
The overview of the significant correlations in figure 44 indicates that the number of
correlations increased as the engineering progress i.e. minimum correlations at 15 %
engineering and maximum correlations at 75% engineering completion. This observation
could be explained, based on the behavior of EWI over engineering duration.
As explained in section 7.2, the difference in behavior of EWI becomes more significant in
successful and less than successful performance projects as the engineering progress. The
correlations did seem to be in accordance with the behavior of EWI over engineering duration,
for example PS & PH found to be highly correlated start from 15% engineering and NCO
become significant, after 45% engineering completion.
Another interesting observation is that throughout the engineering duration CCO, CE and
DPO were not found to be significant for schedule outcomes i.e. ESI and MCI. This could
imply the schedule impact of changes was considered less than actual impact (conservative
estimate) and additional efforts were made to absorb the schedule impact of higher
concurrency and delay in purchase orders. The effects of conservative schedule estimate of
changes and optimistic planning reflected in increase in MHI and TIC (Which found to be
correlated with CCO, CE and DPO after 45% engineering).

89

Looking at the incremental changes in EWI, the time duration before and after 45%
engineering completion seemed to be critical. Significant correlations were found between
project outcomes, PH, and CCO. This could imply that most of the project with less than
successful performance did has significant incremental growth in PH and CCO around 45%
engineering duration.
The figure 45 provides the overview of significant correlations per project outcome over
engineering duration. It was observed that the cost related project outcomes i.e. TIC and MHI
have more correlations in general as compared to schedule project outcomes i.e. ESI and MCI.
The phenomena could imply that either the schedule impact of changes, delay in purchase
orders and high concurrent engineering was seen optimistically or ignored until 60 %
engineering completion or schedule was given priority over cost.
The ESI found to be consistently correlated with PH. This could lead to the inference that the
problems in process engineering do produce downstream cascading effect and delay the
subsequent engineering disciplines.
The change in concurrency only become significant after 60 % engineering, And found to be
positively highly correlated with MHI. This could mean that until 60% engineering, the piping
engineering tried to work on assumptions and keep its schedule as planned. However, when it
could not sustain the concurrency due to changes in assumptions (and need rework) and/or
need concrete information to issue documents it project the growth in man-hours.

90

Significant correlations among Project outcomes and EWI

EWI

ESI

15% Engineering completion

MCI
MHI
TIC

30% Engineering completion

ESI
MCI
MHI
TIC

0.79+++

0.64++

CCO

ESI

45% Engineering completion

MCI
MHI
TIC
0.64++

ESI

60%Engineering completion
MCI
MHI
TIC

0.83+++

0.67++

ESI

75% Engineering completion

MCI
MHI
TIC

0.74+++

0.67++

0.59+

CE
-0.76+++

DPO

-0.69++

PH

-0.56+

0.6+

PS

-0.74+++ -0.81+++

0.62+

-0.9++++

-0.89++++

0.83+++
-0.75+++

0.66++

0.79+++

-0.66++

-0.86++++ -0.9++++

0.64++

-0.6+

-0.67++

0.65++

0.74+++

0.62+

0.62+

0.83+++

0.88++++ 0.71++

0.83+++

0.62+

0.74+++

0.87++++ 0.57+

NCO

0.74+++

-0.64++

-0.69++

0.62+

0.74+++

-0.62+

0.62+
-0.57+

Significant correlations among Project outcomes and incremental change in EWI

EWI

ESI

15% Engineering completion

MCI
MHI
TIC

ESI

30% Engineering completion

MCI
MHI
TIC

ESI

45% Engineering completion

MCI
MHI
TIC
0.64++

CCO

ESI

60% Engineering completion

MCI
MHI
TIC

0.88++++ 0.74+++

ESI

75% Engineering completion

MCI
MHI
TIC

0.8+++
0.76+++

CE
-0.6+

DPO

0.69++

0.77+++

0.55+
0.8+++

NCO
0.57+

PH
-0.86++++

PS
++++
+++
++
+

0.001<P value<0.01
0.01<P value<0.05
0.05< P value<0.1
0.1< P value<0.15

-0.79+++

0.76+++

0.83+++

0.67++

1.0
0.9
0.7
o.4

0.66++

0.8+++

0.63++
0.78+++

-0.550
0.9 to
0.7 to
0.4.to
0.2 to

0.66++

Very strong correlation

Strong correlation
Moderate
Weak correlation

Figure 20 Significant correlations between EWI and project outcomes at each prediction moment.

Significant correlations among Project outcomes and EWI

TIC
Project duration

CCO

15
0.79+++

30
0.64++

45
0.83+++

MHI
60
0.74+++

75

15

30

0.74+++

45

60

75

0.64++

0.67++

0.67++

0.59+

0.64++

CE
-0.76+++

DPO

-0.69++

NCO

PS

-0.67++

-0.69++

-0.6+

-0.64++

0.62+

0.74+++

0.74+++

0.62+

0.62+

PH
-0.9++++

-0.75+++

ESI

-0.9++++

0.62+
-0.57+

0.62+
-0.81+++

0.79+++

0.83+++

-0.86++++

15

30

MCI

45

60

75

15

30

45

0.62+

0.57+

0.6+

0.83+++

0.88++++

0.74+++

-0.56+
-0.74+++

-0.62+

0.66++
-0.89++++

60

75

0.65++

0.83+++

0.71++

0.87++++

-0.66++

Significant correlations among Project outcomes and incremental change in EWI

TIC
Project duration

15

30

45

MHI
60

75

15

30

0.88++++

CCO

45

ESI
60

75

15

30

MCI

45

0.64++

60

15

30

45

0.74+++

60

-0.6+

DPO

0.69++
0.23+++

NCO

0.8+++
0.57+

-0.79+++
++++
+++
++
+

0.67++

-0.86++++
0.001<P value<0.01
0.01<P value<0.05
0.05< P value<0.1
0.1< P value<0.15

0.66++
0.77+++

0.66++

PH

0.9 to 1.0
0.7 to 0.9
0.4.to 0.7
0.2 to o.4

0.8+++
0.76+++

0.83+++
-0.550

Very strong correlation

Strong correlation
Moderate
Weak correlation

Figure 21 Significant correlations of project outcomes with EWI over engineering duration.

92

75

0.8+++
0.76+++

CE

PS

75

0.63++
0.78+++

7.5 DISCUSSION AND CONCLUSIONS

7.5.1

Discussion on behavioral analysis

By analyzing the behavior of behavior of EWI over engineering duration, it was found that the
early warnings do behave differently; however, their visibility is not uniform over all EWI.
The cumulative values of PH along the engineering duration do provide visible difference in
successful and less than successful performance projects. The same is true for PS it was found
that process engineering in projects that experienced significant cost or schedule overrun, fall
behind its schedule consistently until 45 % engineering completion.
In case of number of change orders (NCO), it as observed that in case of less than successful
performance project, cumulative numbers of change orders keep growing at an consistent rate
(consistent slope of the curve in Error! Reference source not found.) However at the early
stages of the project, it is difficult to see the difference between projects. Therefore,
incremental number of change orders was also explored. Via analysis of incremental changes,
it was observed that less than successful performance projects did experience a large number
of incremental changes, on the contrary the successful projects did not experience any higher
incremental changes. The cost of change orders (COC), reflected a similar behavior as NCO,
yet the distinction between projects was found to be clearer than NOC.
The analysis of DPO proved to be complicated EWI, as few of the project that did not
performed well enough, showed a similar behavior as good performance projects. A more
detailed analysis revealed that some of the project do issue purchase orders well before the
start of EPC phase due to reason such as lead-time and market conditions. However, it was
observed good performance projects did not experience any significant delay in issuance of
purchase orders (Not more than 5%).
The EWI CE, which represents the change in concurrency between process and piping
engineering, also showed a distinct behavior in successful and less than successful
performance projects. The analysis revealed that the concurrency level does increase until 45
% engineering and thereafter the concurrency level decreases beyond the baseline concurrency
levels. This implies that in less than successful performance projects, piping engineering do
work on assumptions and try to stay on schedule, yet at the later stage due to changes in
assumptions and subsequent rework the progress of piping engineering get delayed.
From the discussion above, an overall conclusion can be made that the EWI do behave
differently either in absolute numbers or their incremental changes. Therefore, it seemed wise
to look not only at the absolute values of EWI, but at their incremental changes also.

7.5.2

Discussion on correlation analysis at 95 % engineering completion

The analysis was performed to find the correlations between each EWI and four project
outcomes at 95% engineering completion. The found correlations between EWI and project
outcomes are discussed in Table 4 to Table 9.

Growth in process engineering (PH) was found to be highly correlated with three project
outcomes (ESI, MCI and TIC) and moderately correlated with MHI. The delay in the process
engineering was found to be highly correlated with ESI, MHI and TIC. The correlation of PS
with project outcomes is negative in nature, implying if process engineering is delayed, the
project outcomes will see growth.
The numbers of change orders was found to be strongly correlated with schedule outcomes i.e.
ESI and MCI and less correlated with cost outcomes. Where as the other aspect of change
orders COC found to be highly correlated with project outcomes related to cost i.e. TIC. This
implies that higher number of changes do affect the schedule of project and when combined
with high cost changes, it will also affect the cost of the project.
The EWI, Delay in purchase orders was found to be highly correlated with cost project
outcomes (TIC and MHI). The possible conclusion can be drawn here that if issuance of
purchase orders is delayed, then extra engineering manhours are required to process the
delayed vendor information, thus results in growth of engineering manhours. Another possible
conclusion could be that due to delay in issuance of purchase orders, projects do place costly
orders, thus results into cost overrun, However there is no sound evidence was found, which
support this conclusion.
The last EWI, change in concurrency level between process and piping engineering CE, was
found to be correlated with growth in engineering manhours. The relationship implies that in
case the piping engineering work on assumptions (negative change in concurrency) during
early engineering, (due to delayed inputs from process engineering) then there are chances that
the assumptions might turned to be wrong or modified, which results in rework and delayed
piping progress (positive change in concurrency).
The table below explores the correlations among the early warnings. A very strong and highly
significant correlation was found between the PS and DPO. The correlation implies that the
projects that experienced delay in process engineering deliverables also issued purchase orders
late, thus validating the importance of process information for PO.
Another strong correlation was found between PH and NCO. The correlation suggests that if
process engineering is projecting significant growth in their manhours, probably the project
will experience high number of changes. In addition, a moderate negative correlation was
found between PS and NCO. Therefore, an inference could be made that if process
information could is frozen on schedule and without rework then there are chances that the
project will experience less number of changes, thus there are higher chances of success.
The correlation between DPO and CE (negative correlation) suggests that the project that
experienced delay in purchase orders also ended up with less concurrency than the baseline,
implying that the planned concurrency level might not be sustainable if the purchase orders
are delayed. The above observation point towards the dependence of piping deliverables on
information form equipment vendors.
Early
warning

Correlations
parameters

CCO

CCO

rs

CE

Early warning
DPO
NCO

PH

PS

94

Sig.
rs
Sig.

0.55
0.16

CE

rs
Sig.

-0.49
0.2166

-0.68+++
0.0621

DPO

rs
Sig.

0.55
0.16

0.5
0.207

-0.57
0.1361

NCO

rs
Sig.

0.38
0.3518

0.21
0.6103

-0.53
0.1796

0.86++++
0.0065

PH

rs

-0.56

-0.48

0.91++++

-0.63++

-0.57*

Sig.

0.1463

0.229

0.0017

0.0909

0.1361

PS

++++ significant at 0.01 level, +++ significant at 0.05 level, ++ significant at 0.1 level, * significant at 0.15
level, N = 8

7.5.3

Discussion on correlations between EWI and project outcomes over

engineering duration

Looking at the analysis from previous subsections, it was important and interesting to
investigate the dynamic relationship between EWI and project outcomes over engineering
duration of the project. The relationship was investigated at two levels, between absolute
values of EWI and project outcomes and between incremental values and project outcomes.
All significant correlations are shown in Figure 20 and Figure 21.
The dynamic nature of found correlations during engineering exhibit the similar nature as
behavior of early warnings explained in section 7.2.1. As the engineering progresses the
number of correlations between EWI and project outcomes increases i.e. minimum
correlations at 15 % engineering completion and maximum at 75% engineering completion.
Looking at the individual EWI, the correlations of PH were found to be similar to its behavior
in Error! Reference source not found., and it is true for other EWI. For example, cumulative
value of NCO did not found to be correlation with project outcomes until 60 % engineering,
after which it was found to be strongly correlated with schedule outcomes. In a sense, the
result of the dynamic correlations validates the distinct behavior of EWI in case of less than
successful and good performance projects.

7.5.4

General conclusions

Looking back at the results of quantitative analysis and case studies, there is strong evidence
that the EWI are indeed related with project performance and has the predictive capability to
predict final performance of projects. It is also showing that not all the EWI are correlated
with all the project outcomes at all moment during engineering execution. Therefore a
dynamic prediction model would be required, which can utilize the related EWI at different
moments and predict the future performance based on them.

95

96

8 DEVELOPMENT OF PREDICTION MODEL

From the analysis in chapter 6 and chapter 7, enough evidence of relationship between EWI
and project outcomes was found. In chapter 7, it was also argued that the dynamic prediction
capability has more value addition and could be developed via dynamic behavior of EWI.
Therefore, in this chapter an attempt has been made to develop a conceptual prediction model.
Section 8.1 presents a list of requirement and guiding principles for development of
conceptual prediction model. Section 8.2 presents the various statistical methods that are in
general implemented to predict the future sates of systems followed by selection of prediction
methodology for present research. Section 8.3 comprehends the information from previous
chapter and discusses the implemented approach for development of prediction model.
Stepwise approach has been introduced, which include correlation analyses between EWI and
project outcomes followed by investigation of inter correlations between EWI. The aim here is
to include EWI that are highly correlated with project outcomes and at the same time
independent from other EWI to avoid multicollinearity.
Section 8.4 presents the actual development of prediction model via multi regression and
provides a list of regression models at each prediction moment. Section 8.6; analyze the errors
found in prediction of project outcomes using the same data.
In the end of the chapter, section discusses the preciseness of predictions and errors following
by conclusions.

8.1 REQUIREMENTS AND GUIDELINES FOR PERFORMANCE

PREDICTION MODEL
This section defines the requirements that an operational performance prediction model should
fulfill in order to achieve the value addition described in section 4.4.1 i.e. to act proactively
to prevent future potential problems and their impact on project performance. The set of
requirements is derived from 1) goals and deliverables of the research 2) problems that exist in
traditional control methods and 3) from discussions with the stakeholders of this research
project.
Requirements:
Performance prediction Model should predict the final performance of the
project that
is accurate, timely, and comprehensive information to assist
management in making decisions to prevent future problems
is quantitative and continuous(not discrete) in nature so that the
severity of potential cost and schedule overrun is known
is free from optimism bias
The prediction model should be able to explain its reasoning of prediction
Guiding principles
When developing a prediction model, the following should strive for
Easy to use and understandable

Derived from

Reference

Goals

R1

Goals

R2

Knowledge
gaps
stakeholders

R3
R4

stakeholders

G1

97

Should not provide negative incentives to hide or manipulate information

Capability to upgrade the model by adding new projects

stakeholders
stakeholders

G2
G3

The performance prediction model should be able to make accurate prediction of future
performance during early phases. The prediction information should be comprehensive in
nature i.e. the potential cost and schedule overrun (if there might be any) along with the
potential future problems that might cause that overrun. The prediction should be continuous
in nature rather than categorical, so that the severity of potential cost and schedule overrun
could be known beforehand. Furthermore, categorical prediction i.e. successful or less than
successful could be made from continuous prediction by setting the threshold for project
outcomes.
The prediction should be objective and completely based on the facts from the previous
projects. The current project controls techniques utilize the forecast made by project team
based on their assessment of project performance thus are susceptible to optimism bias. An
additional unbiased performance prediction could provide an opportunity for project team to
reflect upon their forecast in combination with status of EWI. If case of any warning from
these EWI and undesired performance prediction an investigation and corrective strategy
could be launched to avoid any potential problem.
Furthermore, due to the limited past project data and conceptual nature of the first prediction
model, it seemed necessary to adopt a methodology, which can explain its reasoning for
prediction of future performance. The open knowledge from prediction model could be
applied at a later stage to larger set of past project. In addition, if the quantitative dynamics is
understood at conceptual stage, a more sophisticated prediction methodology could with larger
dataset of past projects.
In addition to the above requirement, few general guidelines are developed which should be
strived for when developing an operational model. The first guideline is that the model should
be easy to use and results should be understandable. The second guiding principle is that the
prediction model should not provide incentives for strategic behavior by project team
members. The strategic behavior into context of prediction model can be described as hiding
or manipulation of information to get desired state of project performance.

8.2 SELECTION OF PREDICTION METHODOLOGY

The field of predictive analytics offers a verity of techniques from statistics, data modeling,
and machine learning that can analyze historical facts to make predictions about the future
performance of projects. The final product of predictive analytics is generally a predictive
model that uses one or combination of above-mentioned techniques. A predictive model is
defined as A predictive model is simply a mathematical function that is able to learn the
mapping between a set of input data variables, usually bundled into a record, and a response or
target variable (IBM, 2012).
Predictive models are often used in insurance and financial industry to assess the risk in
investments, FMCG (Fast Moving Consumer Good) industry to direct their marketing efforts
and other industries such as retail, travel to name a few. The most common prediction

98

techniques are linear regression, logistic regression, and probit regression, neural networks,
support vector machines, nave Bayesian classifier.
Furthermore, the predictive techniques can be classified into two classes based on how the
model learns the prediction i.e. supervised learning and unsupervised learning. Supervised
learning refers to a function that maps the relationship between predictors and outcomes (i.e.
their place in function are known), whereas unsupervised learning refers to the techniques
which infers from set of observations without labeling them as predictors or outcomes (e.g.
clustering) (Hastie et al. 2008).
The selection of predictive modeling technique is primarily guided by type of variable to be
predicted and its desired state of information i.e. numeric or categorical. In the present
research, the selection of prediction methodology is guided by the requirements stated in
section 8.1. The requirements R3 state that the prediction model should predict project
outcomes that are numeric and continuous, so that the severity of potential cost or schedule
overrun is known. The requirement R3 rule out the classification prediction techniques that
could classify the project as successful or less than successful such as logistic regression,
nearest neighbors classification, nave Bayes classifiers and support vector machine
classification.
Furthermore, R4 requires that the model should be able to explain its prediction. The
requirement ruled out the so-called black-box prediction techniques such as Kernel
regression, and support vector machine regression. These techniques are known as
nonparametric regression and they fit the model by penalizing the errors in prediction by using
weights proportional to the inverse of variance (Mayers, 1990). In addition, the nonparametric
regression model requires larger sample sizes than parametric models because the data must
supply the model structure as well as the model estimates. In addition, with small data size
there are high chances of model over fitting (Welling, 2010; Hastie et al. 2008; Mayers, 1990)
Looking the associated merits and demerits of above mentioned prediction models and
available data, the linear multi regression models seemed to be reasonable choice. They are
simple and often provide an adequate and interpretable description of relationship between
inputs and outputs. They are based on least squares method, in which the model selects the
variable coefficients to minimize the residual sum of squares (Hastie et al. 2008; pg. 44).

8.3 PREDICTION MODEL DEVELOPMENT APPROACH

Before the actual development of prediction model, its seemed necessary to provide explicit
answers to few questions such as what exactly need to be predicted, when to predict and
how to predict. The answers to the above questions are summarized in Table 10.
The aim is to predict the project outcomes of the project i.e. at completion of EPC phase. The
final project outcome altogether determines the final performance of the project. The answer
to second question is that we want to predict the final performance at early stages in the
project, for the reason that in case of undesired performance, the project team should have the
time and resources to act proactively and take corrective action. Another important aspect is
that based on applied corrective action, what change has occurred in performance should be

99

visible i.e. the prediction model should be time-dependent. Therefore, the prediction moments
are selected from 15% engineering completion to 75% engineering completion.
Table 10
What to be predicted

The final performance of the

project i.e. performance at the end
of EPC phase

When to be predicted

How to predict

- At early stages of the project i.e.

Using the EWI at prediction

moment

- At 15 % engineering completion

EWI at 15
completion

- At 30 % engineering completion

EWI at 30 %, 15% engineering

completion

- At 45 % engineering completion

EWI at 45 % , 30%, engineering

completion

- At 60 % engineering completion

EWI at 60 %, 45%, engineering

completion

- At 75 % engineering completion

EWI at 75% ,60 %, engineering

completion

engineering

The answer to third question How to predict is drawn from section 7.2.1 and 7.3. It was
observed that the EWI do behave differently in successful and less than successful
performance projects and in the subsequently section, 7.3 significant statistical relationship
were found between EWI and project outcomes. Therefore, EWI were used as predictors to
predict final project outcomes.
The stepwise approach to develop prediction model is provided in Figure 22 : below. At each
prediction moment, the first step is to establish the correlations between EWI and project
outcomes. Based on the analysis of the correlations, the EWI with at least moderate
correlations (rs = 0.5) and 85 % probability (p=0.15) with project outcomes will be selected.
In the second step, the selected EWI were checked for multicollinearity11 i.e. the correlations
among selected EWI. The aim was to select EWI, which were more independent and highly
correlated with a project outcome. The criteria for multicollinearity is set at minimum rs = 0.7
and p = 0.05. This criterion is selected after a round of investigation into inter-correlations
between EWI.
After the check for multicollinearity, independent and significant EWI were used for multi
regression to predict the final project outcomes. The minimum requirement for EWI was set at
two and due to the less numbers of past projects; the maximum limit for EWI in a regression
model was limited to four. Therefore, in case of more than four EWI, the EWI were selected

11

Multicollinearity refers to a statistical phenomenon in which two or more predictor variables in a multiple
regression model are highly correlated. Due to the inter-correlations, the coefficient estimates may change
erratically in response to small changes in the model or the data. Multicollinearity does not reduce the
predictive power or reliability of the model as a whole, at least within the sample data themselves; it only
affects calculations regarding individual predictors. (Aiken & West, 1991)

100

based on their regression coefficients i.e. the top four EWI with highest correlations with a
project outcome were used.
Establish correlations
between EWI and project
outcomes

Check the correlations between

EWI and project outcomes

Analysis of
correlations

Analysis the correlations via

correlation coefficients rs and pvalues

Select EWI
with high and
significant
correlations

Select the EWI with correlations

minimum and above
rs = 0.5 and p = 0.15

Establish correlations
between EWI to check
Multicollinearity

Select
independent EWI
with high and
more significant
correlations

Predict the project

outcomes with selected
EWI

Activities

Check the correlations among

selected EWI with coefficients rs
and p-values

Rank the EWI based on coefficients rs

and select the EWI
(Maximum 4 nos, minimum -2
nos)

Build multi regression model with

EWI as explanatory variables and
Project outcomes as dependent
variables

Decisions

Figure 22 : Step approach for development of prediction model

101

8.4 DEVELOPMENT OF PILOT PREDICTION MODEL

The approach defined in Figure 22 : Step approach for development of prediction modelis
adopted at each prediction moment. The correlations between EWI and project outcomes and
correlations among EWI were investigated at each prediction moment. For explanation
purpose, the development of prediction models at 15% engineering completion is explained
below.
At 15% engineering completion
The growth in TIC was found to be correlated with only CCO (rs = 0.79, p = 0.02) and PS (rs =
0.99, p = 0.001) and there was no significant correlation found between these two EWI (rs = 0.5, p = 0.20). Therefore, CCO and PS were used to predict the growth in TIC. The multi
regression model was termed at R.1.
The delay in engineering manhours (MHI) found to be correlated with PS (rs = -0.81, p = 0.01)
only. As explained in section 8.3, the aim is to have minimum of two EWI. Therefore, DPO
was used as another EWI.
The delay in mechanical completion (MCI) was found be correlated with PS (rs = -0.89, p =
0.002) only. After stepwise investigation, DPO and NCO were also added in the model
because no significant correlation was found among them. For ESI, there were no significant
correlations were found. Therefore, no prediction can be made at 15% engineering completion
for ESI.
The summary of regression models is provided in following table.
Table 11 : Prediction models at 15 % engineering completion
Model
No

Time

R.1

15%

R.2

15%

R.3

15%

R.4

Variables
CCO-15
PS-15
DPO-15
PS-15
DPO-15
NCO-15
PS-15
-

Project
outcome

Model equation

Rsquare
value

Model P
value

TIC

0.01581+18.38801*(CCO-15)0.01946*(PS-15)

0.9567

0.0003906

MHI

0.26452-4.07028*(DPO-15)0.04660*(PS-15)

0.7917

0.01981

MCI

0.148173+2.054449*(DPO15)+0.009458*(NCO-15)0.024970*(PS-15)

0.9449

0.0216

The approach was applied to each prediction moment and the summary of regression models
is attached below.
Table 12: Prediction models at 30 % engineering completion
Model
No

Time

Variables

Project
outcome

Model equation

R.5

30%

DPO-30

TIC

0.001121+13.097283*(CCO-

Rsquare
value
0.9728

Model P
value
0.01104

102

R.6

30%

R.7

30%

R.8

30%

PS-30
CCO-15
PS-15
PH-15
PS-15
PS-30
PH-30
PS-30
PH-30
PS-15

15)-0.243072*(DPO-30)0.016940*(PS-15)0.001786*(PS-30)
MCI

0.03280+0.09575*(PH-15)0.00618*(PS-15)-0.01853*(PS30)

0.6322

0.22

ESI

0.02292+0.41004*(PH-30)0.02487*(PS-30)

0.5772

0.1163

MHI

0.32199+1.16856*(PH-30)0.05965*(PS-15)

0.6966

0.0507

Table 13: Prediction Models at 45% engineering completion

Model
No

Time

R.9

45%

R.10

45%

R.11

45%

R.12

45%

Variables
CCO-45
DPO-30
DPO-45
PS-45
CCO-45
PH-45
PS-45
DPO-45-1st
PH-45-1st
PS-30
PS-45
DPO-45-1st
PH-45
PH-45-1st

Project
outcome

Model equation

Rsquare
value

Model
P value

TIC

-0.014780+1.948622*(CCO45)-1.696703*(DPO30)+1.126065*(DPO-45)0.002113*(PS-45)

0.9303

0.0440
8

MHI

0.04258+7.37035*(CCO45)+1.13930*(PH-45)0.01938*(PS-45)

0.836

0.0475
6

MCI

0.033925+0.860814*(DPO-451st)+0.407402*(PH-45-1st)0.023325*(PS30)+0.012497*(PS-45)

0.9771

0.0085
42

ESI

-0.004889-0.097507*(DPO1st)+0.671001*(PH45)+0.673656*(PH-45-1st)

0.8699

0.0303
1

Table 14 Prediction Models at 60% engineering completion

Model
No

Time

R.13

60%

R.14

60%

R.15

60%

R.16

60%

Variables
CCO-60
NCO-60-1st
PS-45
CCO-60
CE-60
NCO-60
PH-60

Project
outcome

Model equation

Rsquare
value

Model P
value

TIC

-0.011655+0.723199*(CCO60)+0.003017(NCO-60-1st)0.014405*(PS-45)

0.6549

0.1957

MHI

0.353659+3.366194*(CCO60)+0.489701*(CE-60)0.004420*(NCO60)+1.711740*(PH-60)

0.9255

0.04854

MCI

-0.034538+9.445831*(CCO60-1st)+0.006913*(NCO-601st)+0.035976*(PH-60)

0.5813

0.2785

0.8307

0.05063

CCO-60-1st
NCO-60-1st
PH-60
CCO-60-1st)
PH-45-1st

ESI

-0.00519+7.91209*(CCO-601st)+0.56122*(PH-45-

103

1st)+0.52772*(PH-60)

PH-60

Table 15: Regression models at 75 % engineering completion

Model
No

R.17

Time

75%

R.18

75%

R.19

75%

R.20

75%

Variables
CCO-75
DPO-75
NCO-75
PH-75
CCO-75
CE-75
PH-60
PS-75
NCO-75
PH-75-1st
PH-75
CCO-60-1st
CE-75-1st
PH-60
PH-75

Project
outcome

Model equation

Rsquare
value

Model P
value

TIC

-0.098785-5.979603*(CCO-75)2.748513*(DPO75)+0.002186*(NCO75)+0.245588*(PH-75)

0.8528

0.1288

MHI

0.29162+1.90469*(CCO75)+0.50289*(CE75)+0.45872*(PH-60)0.03505(PS-75)

0.9537

0.02422

MCI

0.0550667-0.0013335*(NCO75)+0.5382666*(PH-751st)+0.4170979(PH-75)

0.8867

0.02315

ESI

-0.04344+22.50128*(CCO-601st)+0.56418*(CE-751st)+0.61536*(PH-60)0.11908*(PH-75)

0.8117

0.1512

8.5 MODEL EVALUATION METHODS

As stated in section 3.3 and shown in Figure 8, the evaluation of prediction model is
performed in two ways (depending on the data used for evaluation).
I. Analysis of predictions within the data set that has been used to build the model
II. External evaluation by using independent data from an external project
The first evaluation is discussed in section 8.6 and further divided into two levels. The first
level of analysis compares the forecast made by prediction models and by traditional methods
(section 8.6.1). This evaluation method uses the error obtained by evaluating the different
forecasts with the known project outcomes. The choice has been made to check the prediction
accuracy of model as compared to traditional methods. The second analysis focuses on the
forecasts made by prediction model and their comparison with the actual project outcomes.
This analysis will provide us useful insights about the accuracy level of prediction model. The
analysis and associated results has been presented in section 8.6.2.
External validation is performed by using data from an external project and is presented in
section 8.7. The external validation of pilot prediction models tests their prediction accuracy in
order to determine its use in current form to predict the future performance of O&C projects.

104

8.6 ANALYSIS OF PREDICTIONS

8.6.1

Comparison of prediction model with traditional methods

For the first level analysis, predicted values of project outcomes via prediction model are
compared with values predicted by traditional forecasting methods. For the purpose of
explanation, Figure 23 provides an example in which the delay in mechanical completion of
project is predicted by two methods longitudinally over engineering duration of the project.
The yellow curve represents the actual final delay in mechanical completion of the project.
The blue curve represents the prediction by traditional method. The pink curve represents the
prediction made by prediction model. It can be observed that the pink curve is quite close to
actual curve. Whereas, the forecast made by traditional methods is changing from zero to 40
% and then coming close to actual value.
Prediction of MCI (Project 1)
0.25

1.2
Error by Traditional method
P1 Traditional Method
P1 Actual

Error by prediction Model

P1 Prediction Model

0.2

0.8

Absolute % error in prediction

Percentage growth

0.15
0.6
0.1
0.4

0.05

0.2

0
15

30

45

60

75

Engineering percentage complete

Figure 23 Predictive capability comparison of traditional method and developed prediction tool.

The bars in above figure represent the absolute error made in prediction by two methods. The
errors are measured as predicted value minus actual value of project outcome divided over
actual value. The pink bar represents the error made by traditional methods and violet bar
represents the absolute error made by prediction model. The error in prediction model is far
less than traditional method.
The above example is just one case; therefore basing our conclusion on this case is not
practical. The concept of absolute error is applied to all past projects (same as used in building
the model) and following paragraphs discuss the results.

105

The Table 16 below shows the comparison of forecast made by traditional methods and
forecast by prediction model. The signs in table are defined as follows:
+ sign means that the absolute error made by traditional methods is larger than the absolute
error by prediction model, implying forecast made by prediction model is more accurate.
- sign means that the absolute error made by traditional methods is smaller than the
absolute error by prediction model, implying forecast made by prediction model is less
accurate.
Table 16 : Predictive capability comparison of traditional methods and prediction model
Green = Good performance, Blue = Average performance
Red = less than successful performance

Prediction of TIC
Engineering
completion
15%
30%
45%
60%
75%

Engineering
completion

15%
30%
45%
60%
75%

Engineering
completion

15%
30%
45%
60%
75%

Engineering

1
+
+
+
+
+

1
+
+
+
+
+

+
+
+
+
+

+
+
+
-

+
+
+
+

Project Number
4
5

+
+
+
+
+

+
+
+
-

2
+
+
+
+
-

Prediction of ESI
Project Number
3
4
5
+
+
+
+
+
+

6
+
-

7
+
+
+
+
+

8
+
+
+
+
+

2
+
+
+
-

Prediction of MCI
Project Number
3
4
5
+
+
+
+
+
+
+
+

6
-

7
+
+
+
+
+

8
+
+
+
+
+

+
+
+
+
+

+
+
+
+
+

Prediction of MHI
Project Number

106

completion

15%
30%
45%
60%
75%

1
+
+
+
+
-

2
+
+
+
+
+

3
-

4
+
+
+
+
+

5
+
+
+
+

6
+
+
-

7
+
+
+
+
+

8
+
+
+
+
+

It could be observed that at majority of the prediction moments, the forecast made by
prediction model is more accurate than traditional methods.
Another important observation can be made from the analysis that in case of good
performance projects, the traditional methods were found to be more accurate than prediction
model and reverse is true in less than successful performance projects. Two possible
explanations could be provided for this observation 1) presence of optimism bias in traditional
methods. As discussed by Flyvbjerg and Bruzelius, many forecasts believe that their project is
less prone to problems therefore keep their forecast close to the desired state, until the problem
has already manifested (Flyvbjerg & Bruzelius, 2003). 2) Presence of possible bias towards
less than successful projects in dataset due to relatively high number of less than successful
projects.

8.6.2

Analysis of errors

The regression models developed in above section showed sufficient R-square value. Yet the
R-square was not seen as sufficient explanation of error, therefore the prediction models were
used to predict the project outcomes of same projects used in building of the model.
The errors between predicted and actual values of project outcomes were plotted over
engineering duration of projects. The errors were measures in absolute terms irrespective of
the sign. The errors are measures as difference in predicted value minus actual value of project
outcome divided over actual value. For example, absolute percentage error is TIC is measured
as difference between predicted values minus the actual cost of the project divided over actual
cost of the project.
On similar terms, the errors in project outcomes were plotted and analyzed. The following
paragraphs discuss the errors per project outcomes.
Errors in TIC: The error found in prediction of project TIC is shown in Figure 24, each
curve represents an individual project. The curves reflects that for most of the projects, the
error stays with in 6 % range till 45% engineering completion, Thereafter, absolute error
become as high as 12 % for few projects. This is in direct conflict with correlations found in
chapter 7.

107

Error in prediction of TIC

0.2
0.18
0.16

Absolute % error

0.14
0.12
0.1
0.08
0.06

0.04
0.02
0
15

30

45

60

75

Engineering % Complete

Figure 24 : Errors in prediction of final TIC

The Figure 25 shows the errors in prediction of ESI. No prediction could be made at 15 %
engineering completion due to insufficient numbers of EWI. The figure shows that the errors
decrease from 30 to 45 % engineering and then remain more of less stable (within range of 8
to 9%). This could be explained by low number of correlated EWI with ESI. Figure 21,
validate the above statement and shows that only PH showed a consistent correlation with ESI
other than few incremental changes in EWI.
Error in prediction of ESI

Error in % of baseline duration

0.24

0.2

0.16

0.12

0.08

0.04

0
15

30

45

60

75

Engineering % complete

Figure 25 Errors in prediction of ESI

The following Figure 26 shows the errors in prediction of MHI, the errors found to be in
accordance with the established correlations in chapter 7. As, until 45 % engineering

108

completion, the numbers of correlated EWI is significantly less, thereafter 45 % engineering

the errors decrease as number of predictor EWI increase.
Error in prediction of MHI
0.48

Error in % of Baseline Man-hours

0.44
0.4
0.36
0.32
0.28
0.24
0.2
0.16
0.12
0.08
0.04
0
15

30

45

60

75

Engineering % complete

Figure 26 Errors in prediction of MHI

The same is true for MCI, The Figure 27 shows the errors in prediction of MCI, and it can be
observed that on overall level, the errors seem to be decreasing from 30 % onwards. In
addition, the error stays within 4% ranges (except one project) in prediction after the
completion of 45 % engineering.

109

Error in MCI

Error in % of baseline duration

0.2

0.16

0.12

0.08

0.04

0
15

30

45

60

75

Engineering % complete

Figure 27 Errors in prediction of MCI

8.7 EXTERNAL VALIDATION

The validation of pilot model is done by using an external completed project. The objective of
the completed project was to construct the utilities to support a petrochemical plant. The
contract between E&C contractor and client was cost reimbursable with EPCm scope. The
project cost was estimated to be $350 million and completed in 2005 to 2008. The final project
cost was 22% over the estimated cost with 12% schedule overrun.
The prediction of project outcomes is done at three levels, point estimate, upper and lower
range predictions. The prediction interval has been calculated by using the standard error of
the models in combination with critical t value for required confidence interval (Anderson et.
al, 2007). Initially, prediction interval with various confidence intervals was explored, being
95% as the highest confidence and 30% as the lowest confidence. The ranges were found to be
very wide for 95% confidence intervals. In case of 30 % confidence interval, the prediction
ranges were found to be close to actual values however, 30% confidence level seemed too
low. Finally, two ranges were selected with 50% confidence (addressed as P50) and with 80%
confidence (addressed at P80).
The prediction of four project outcomes is presented and discussed in following paragraphs
The growth in TIC is predicted longitudinally over engineering duration of the project. The
results are shown in Figure 28. The point prediction for growth was found to be in acceptable
limit from 15% to 60% engineering completion. However, at 75% prediction the error was
found to be quite high (16%). The difference between upper and lower prediction at P50 and
P80 is presented by following values:

Predication confidence

Average

Min.

Max.

110

P50
P80

tolerance to point
estimate
(growth in TIC)
7.5%
15.5%

tolerance to point
estimate
(growth in TIC)
3%
6.5%

tolerance to point
estimate
(growth in TIC)
14%
30%

Figure 28: Model validation: prediction of TIC

The actual value is within or close to P50 range. The maximum difference between lower and
upper range at 60% engineering completion was found to be too wide, however the point
estimate is closest to actual value. From 15 % to 60% engineering completion, the estimated
ranges are quite reasonable as compared to actual value. Longitudinally, the estimates
improve. However, at later stages, the lower bound values show a possibility of under run. In
case of P80, the average difference between the ranges is too wide ( 15.5 %), therefore
considered as less reliable.
The validation results for MCI are shown in Figure 29. The point estimate did improve
longitudinally, but showed a less growth at 75% engineering completion. The actual value is
more or less within P50 range. The statistics of prediction ranges are shown in following table.
Predication confidence

P50
P80

Average
tolerance to point
estimate
(growth in MCI)
6.5%
13.5%

Min.
tolerance to point
estimate
(growth in MCI)
3.5%
7%

Max.
tolerance to point
estimate
(growth in MCI)
11%
22%

111

Looking at the point estimate, the prediction does seem to improve longitudinally. However
goes below the actual value after 45% engineering completion and predict a lower growth at
60% and 75% engineering completion.

Figure 29 Model validation: prediction of MCI

The validation results from ESI are shown in Figure 30. No prediction model was found at
15% engineering completion. After the 30% engineering completion, the point estimate for
growth in ESI proved to be on lower side of actual growth. The P50 range estimate covers the
actual growth and upper estimates are close to the actual value, however overall range is
found to be too wide to make any policy decision.
The prediction ranges for P50 and P80 are shown in following table:
Predication confidence

P50
P80

Average
tolerance to point
estimate
(growth in ESI)
11.5%
24.5%

Min.
tolerance to point
estimate
(growth in ESI)
10%
21%

Max.
tolerance to point
estimate
(growth in ESI)
13%
28%

112

Figure 30: Model validation- prediction of ESI

For prediction of MHI, the project proved to be exceptionally different from dataset. In the
data set utilized to build the model, for average 20% growth in TIC, 77% average growth was
observed in MHI, whereas in case of this project the final growth in MHI was found to be
21%. The errors in the prediction of MHI are found to be high. The actual value is outside the
prediction ranges of P50.

113

8.8 FINAL EVALUATION

The final evaluation of pilot prediction model is performed by evaluating the results of this
chapter against the requirements set in section 8.1. The model can be evaluated based on the
reasoning to what extent the requirements stated in section 8.1 have been satisfied. The
guidelines formed in section 8.1 are used to recommend the implementation strategy in
chapter 9.
Reference
R1
R2
R3
R4

Requirements:
is accurate, timely, and comprehensive information to assist
management in making decisions to prevent future problems
quantitative and continuous prediction so that the severity of
potential cost and schedule overrun is known
free from optimism bias
The prediction model should be able to explain its reasoning of
prediction

Evaluation
(Meet the
requirement?)
Partially
yes
yes
yes

The most important requirement for prediction model is to predict the future performance that
is accurate, timely and provide comprehensive information to guide the interventions during
project execution. The current form of prediction model partially meet this requirement, as the
model predict the future performance at early stages, thus allowing the time for proactive
decision-making. However, the predictions made by current form of model have limited
accuracy (errors) and reliability (low confidence level of predictions). In spite of limited

114

accuracy and reliability, the stepwise development model does provide a useful framework to
enhance the accuracy and reliability of model by adding more projects.
In addition, the level of comprehensive information could be further improved by adding the
potential problems associated with EWI. Few of the potential information have been identified
in chapter 6 and an effort has been made for their integration in prediction model during
formation of implementation strategy in section 8.9.
The second and third requirements are fully meet by pilot prediction model. The predictions
made by model are continuous and quantitative, thus do have the capability to reflect upon the
severity of cost of schedule overruns. The data used for training the model is purely
quantitative and objective. In addition, the data set is based on the previous completed
projects; therefore predict the future performance from actual situation in similar projects and
could be considered as free from optimism bias.
The fourth requirement is met by using the multi-regression as prediction technique. The
mathematical models provide the explicit relations between EWI and project outcomes, thus
the effect of each EWI in a model could be understood by magnitude of its coefficient and its
sign in model.

8.9 INTEGRATION OF PROJECT PROBLEMS WITH

PREDICTION MODEL
The case studies mapped the relationship of EWI with project problems and project outcomes.
Insights from the relationships could be used in the model to provide casual explanation of
performance prediction using EWI. The project problems mentioned below are drawn from
cases in chapter 6 in addition to the associated problems (with selected EWI) mentioned by the
expert interviews. The major problems found to associate with these early warnings are
described below:
Problems associated with changes:
EWI

Potential problems

NCO

- Out of sequence activity execution bring inefficiency

- Rework due to changes (direct impact on targeted activities and indirect impact on
associated activities)
- Mismanagement of changes
- Subcontractor and vendor claims of schedule and cost
- Violation of baseline estimate (cost and schedule)

CCO

- Violation of baseline estimate (cost and schedule)

- subcontractor and vendor claims of schedule and cost
- Additional cost associated with discovered work and disruption due to out of
sequence execution

Problems associated with early engineering:

115

EWI

Potential problems

PH & PS

- Delay in critical information to downstream disciplines, procurement

activities
- inefficiency due to out of sequence execution
- concurrent engineering execution

Problems associated with concurrent engineering execution of process and piping

engineering:
EWI

Potential problems

CE

- Possibility of assumed data changing at later stage, thus resulting into

rework
- Delay in critical project milestones such as intermediate model reviews,
final bulk quantity estimations
- delay in Issuance of mechanical contracts for fabrication and
construction
- Inefficiency and frustration among project team members due to change
of inputs or their assumptions being proved wrong.

Problems associated with delay in issuance of purchase orders;

EWI

Potential problems

DPO

- Delay in vendor design information,

- Delay in issuance of final deliverables to fabrication and construction.
- Lack of work front for construction sub contractors resulting into idle
cost

The integration of above mentioned project problems could be illustrated by following

example:
Prediction of growth in TIC at 45% engineering completion: The illustration used an
actual past project. The prediction model at 45% engineering completion uses three EWI i.e.
CCO, DPO and PS. In case the selected project, the model predicted a cost overrun of 15%.
The associated values of EWI are shown in following table
Project
outcome
TIC
Predicted
growth = 15%

Prediction Moment

45% engineering completion

EWI And values

PS = 6 % behind schedule
CCO = 5% of estimated TIC
DPO = 6% at 30% engineering completion
=4% at 45 engineering completion

116

Based on the EWI values and associated potential problems, following problems could
manifest and drive the TIC to predicted growth:
Problem
predictor
PS

Potential problems
- Delay in critical information to downstream disciplines, procurement activities
- Concurrent engineering execution

CCO

- Subcontractor and vendor claims of schedule and cost

- Violation of baseline estimate (cost and schedule)
- Additional cost associated with discovered work and disruption due to out of
sequence execution

DPO

- Delay in issuance of final deliverables to fabrication and construction.

- Lack of work front for construction sub contractors resulting into idle cost

In the above illustration, the undesired growth in TIC is predicted, which can triggered the
investigation into potential problems associated with EWI and could enable the informed
decision making to avoid above mentioned problems.

8.10 DISCUSSION AND CONCLUSION

This chapter presented a first pilot prediction model to predict the performance of O&C
projects, thereby answering research question how prediction model predicts the performance
of the O&C projects research question The model is based on purely quantitative data from
past projects and used stepwise regression as prediction technique.
The amount of data on which the prediction model is build is limited therefore it is difficult to
draw a conclusion with high confidence. However, it seemed worthwhile to check the
preciseness of prediction with traditional forecasting methods. At each prediction moment, the
forecasts made by traditional methods were recorded and error was found between the actual
final value and forecasted value. The found errors were compared with the errors in forecasts
made by prediction model. On the overall level, it was found that forecasts by prediction
model are more accurate. (It is important to remember that the comparisons were done using
the same data as used for development of model.)
However, the analysis showed another limitation of dataset, bias towards less than successful
performance projects. Therefore, to have a more accurate prediction in good projects, a dataset
is required which contains a balance of Successful and less than successful performance
projects.
Looking back at the errors found in the predictions from prediction model in section 8.6.2, it
was found that in case TIC, the errors did not decrease as we add more information to our
predictions. At 60 % engineering completion, The errors were found to be as high as 12%,
before and after 60% engineering the errors stays with in range of 4% to 6%. At case of other
project outcomes, the results are more encouraging, yet there is always one or two projects in
data set that show higher errors than others.

117

The results of external validation reflected the limited reliability of pilot prediction model. On
average, the point predictions of project outcomes except MHI were found to be close to
actual values, however lacks consistent behavior. The prediction ranges with 50% confidence
(P50) embrace the actual value and could be seen as reasonable prediction ranges. The
prediction ranges with 80% confidence interval were found to be too wise and in current state
could not be used for guiding the decision-making.
Based upon the findings in this chapter, it should be concluded that the accuracy level of
prediction model developed by this research need more preciseness and reliability. To have
more preciseness, a larger dataset with balance of Successful and less than successful
performance project is required. Nonetheless, it can be concluded that if the approach
developed by this research is applied to a larger and rich dataset, a more precise and better
prediction model could be developed.

118

9 INSIGHTS AND RECOMMENDATIONS FOR

IMPLEMENTATION
In the previous chapter, a dynamic pilot model for performance prediction was developed and
analysis of its prediction accuracy and reliability was determined. This chapter will focus on
comprehension of knowledge created in previous chapters and strategic deployment of pilot
prediction model for Fluor Corporation.
In addition to the prediction model, useful insights gained through the extensive analysis in
chapter 6 to 8 are comprehended in section Error! Reference source not found.. Based on
the insights, few general recommendations are presented. The insights and recommendations
are confidential and not be made public.
Section Error! Reference source not found. shows the usability of prediction model in very
simplistic form and formulate the recommendations for its implementation on short term and
long-term perspective.

9.1 INSIGHTS AND RECOMMENDATIONS

Insights-1: The first set of insights can be drawn from identified early warnings in chapter 5.
A comprehensive list of early warnings is identified from literature sources and expert
interviews. In addition to EWI that are used for prediction modeling, another category of early
warnings identified as recommendations is mentioned in section 5.6.2 and 5.6.3. The list
contains early warnings that has been identified as early in the project execution and has the
ability to be measured objectively (quantitative). The early warnings could be seen as the
focus points during the early phase of the project.
The recommended early warnings should be presented among the project team and a survey
could be conducted to rank the recommended early warnings. The top ranked early warnings
then should be measured (on project level and discipline level) and used in project reporting
along with key performance indicators (KPI). There is a possibility of overlapping between
early warnings and KPI. Nonetheless, the exercise will provide a new perspective towards the
KPIs and their use as early warnings of project problems rather than just as performance
indicators.
In addition to the prediction model, a new section as Potential problems should be added in
the project monthly reporting with status of EWI as justification for their anticipation.
Insights-2: The third sets of insights are drawn from quantitative analysis performed in
chapter 7. It should be noted that the insights in this section are solely based on the limited
data set used for quantitative analysis. Therefore, the generalizations are subjected to
validation from experts. Based on their quantitative analysis following insights can be drawn:
1. The early (until 45-60% engineering completion) and continuous delay in process
engineering was observed in less than successful projects. Whereas in successful projects the

119

continuity of delay was not observed neither any significant early delays. The above insight
imply towards the criticality of early process deliverables.
2. In projects with of less than successful performance, process engineering forecasted
significant growth in their hours (on average 25%) from 15% to 45% engineering completion,
this phenomenon was not observed in successful projects.
3. A high numbers of incremental changes were observed in projects with less than successful
performance. In addition, the time of high numbers of incremental changes does affect the
severity of cost and schedule over run. It was observed that the projects in which high
incremental changes were implemented late (after 45% engineering completion) they
experienced more severe overruns.
4. Almost all the projects did experience increase in concurrency until 45% engineering
completion, however less than successful projects did experience positive change (piping
projecting schedule delay) of more than 30% in concurrency between 45% and 75%
engineering completion.
Based on the above insights, following general recommendations could be made:
1. Focus on status of early process deliverables for example critical unit P&IDs and reasons
for growth in process engineering hours.
2. The growth in proposed changes should be monitored both commutative and incrementally
and in case of high number of changes within one or two reporting cycles, efforts should be
made to first avoid the changes that could be avoided, secondly proper assessment of accepted
changes should be performed (may be add additional temporary resources). The assessment of
changes should not include only the cost and schedule impact, but also the cost and hours of
rework, impact on associated vendors and contracts.
3. The concurrency between process and piping engineering disciplines should be monitored
by tracking the relative change in their progress curve. Any significant change in concurrency
should be scrutinized by checking the level and types of assumptions made by piping
engineering disciplines.

9.2 RECOMMENDATIONS FOR IMPLEMENTATION

The proposed pilot prediction model met three of the four requirement stated in section 8.1.
However, meet partially with the most important requirement (R1) i.e. accurate, timely
prediction of project performance. Thus, accuracy and reliability are few key aspects of the
model that require additional efforts and time for improvement. This section first discusses the
general usability of prediction model for Fluor. In subsequent sections, recommendations are
formed for implementation with in short term and long term. The short-term refers to the
aspects of model that can be used within 0-9 months and long term refers to the time from
beyond 9 months.

120

9.2.1

General usability of the model

The total results of the study, prediction model, EWI, potential problems can apparently be
considered as new, additional project control and management method created for the field of
project control. The usability areas of the research results can be examined both as separate
results that obtained from research phase one to five and as total, integrated prediction model.
In most simplistic form, the use of prediction model is shown by following figure:

Prediction model user

Decision maker

Figure 31: Usability of prediction Model

For any current project, and at any prediction moment t1, the prediction model user collects the
data regarding status of EWI. The quantitative values for EWI are then used to predict the
future performance of project outcomes. In case any undesired performance, the relevant
explicit relation between EWI and potential problem and between potential problem and
project outcomes is mapped. Subsequently, decision-maker himself or with support from key
stakeholder plan and implement proactive response to mitigate the impact of potential project
problems.

9.2.2

Short-term implementation

The goal of short-term implementation is two fold. 1) The aspects of the research that is to be
implemented within the short term 2) The recommendations for improvement and further
development of prediction model and its use as a new method for management of project
execution.

121

The aspects of the research that could be implemented in 0-9 month duration are the use of
EWI, their relation with potential problems (identified in chapter 6 and again comprehended in
section 8.9). This can be achieved by mapping the behavior of EWI in current projects with
the behavior of EWI from past projects. Incase the behavior and status of EWI was found to
be similar to one of the lass than successful projects, an outside reflection could be provided
on potential overrun/under runs in project outcomes.
Before the prediction model is implemented as actual practice, additional efforts are required.
These efforts are formed as recommendations and categorized into two domains 1) Technical
2) Institutionalization.
Technical design: The recommendations in the technical domain are focused on design of
prediction model as a system. The recommendations for technical design of prediction system
are focused on associated aspects (can be termed as subsystems) and their interactions that are
necessary to act as whole.
1) Automated data collection process: The data collection process from project should be
automated and standardized. The standardization should progress in two directions i.e.
collection from current projects 2) collection from past projects. An automated excel sheet
could be developed which can be send to different Fluor offices (that execute similar types of
projects) across the projects to collect more data for increasing the accuracy and reliability of
prediction model.
2) Platform and technical architecture to make prediction model more automated: In the
current research, R project has been used as main statistical tool for predication modeling in
combination with Microsoft excel. From usability perspective, R project does not seem to be a
most preferred approach, however Microsoft excel is used extensively and considered as a
preferred option.
The prediction algorithms and development approach could be transferred to Microsoft excel
as Visual basic (VB) codes. The technical architecture for prediction model includes series of
processes and decisions moments. On higher level, the main processes that should to be
modeled are:
I. Prediction algorithms that can guide the selection of specific model and required EWIs
based on prediction moment.
II. Automated upgrading process that could provide the capability to enter new data from
past project and can upgrade the prediction model based on the new information
III. Database of relationships between EWI, project problems and project outcomes. The
system should be able to extract the relationships based on the status of EWI and
predictions of project outcomes.
2) Institutionalization: For successful implementation of prediction system, it is necessary to
institutionalize and foster the right incentives to its users and project managers. Following
recommendations could be made to institutionalize the prediction in short term.

122

I. Create and sustain the top management commitment: Commitment from top
management is a necessity for successful implementation of prediction system. In
present case, the dynamic prediction system is relatively new, therefore commitments
from top management need to be created and sustained. A way to do that is to make
the role of dynamic prediction of project performance more clear and highlight the
added value.
II. The second recommendation is to gather the support of users and project team
members. The prediction model and associated knowledge base should be distributed
among it potential users and project managers. The feedback from users and project
mangers could be accommodated into design of prediction system.
III. To provide the right incentives to its users and affected stakeholders, it is necessary to
build an open and constructive environment for participation. The message that should
be received by users, project managers and project team as whole is that they are not
just at the receiving end of prediction system. However, their contribution and
participation is critical for successful management of project execution. In addition,
the positive antecedent for their desired state of behaviors should be mapped and
implemented.

9.2.3

Long-term implementation (beyond 9 months)

Once the prediction system has been developed, test and being institutionalized within
organization system and is supported by its sponsors, top management; prediction system
could be embedded within projects of Fluor. By this time, the following products are to be
available:
I. Prediction model equipped with sufficient dataset of part project
II. Second automated pilot of prediction system Excel based
III. Organizational requirements and draft procedure for implementation of prediction
system
The long-term implementation contains the further improvement in prediction system and its
introduction as independent system for management for project execution. The long-term
implementation will possibly induce additional improvement cycles. The Possible
recommendations for long-term implementation can be summarized as follows.
I. A transition from excel based prediction system to more sophisticated platform such as
web-based system with more robust database of past projects
II. A self-learning design to add current project data along with its predictions
III. Customized reporting capability to assess and quantify the impacts of interventions in
project execution, Furthermore automatic storing of interventions along with its
impacts, applicability, reference projects; to offer the decision-makers, a
comprehensive list of corrective actions.

123

IV. Capability to extract quantitative information for benchmarking and planning purposes
V. Integration with lesson learned framework to receive quantitative information along
with lessons.
VI. Sustain the top management commitment
In addition to further institutionalization of prediction system, a training module could be
developed to make the staff more familiar with prediction system, tools, its principles and
changes to the current procedures.
To widen the acceptability of prediction system beyond Fluor Corporation and gain the
acceptance of Fluors main clients, a joint research project could be setup. This will provide a
dual benefit for Fluor; 1) more projects could be added to the prediction system, therefore
increasing the accuracy and reliability of its predictions, 2) the acceptability of prediction
model will increase by incorporating the client needs, thus improve the project management
capabilities and competitive advantage of Fluor Corporation in E&C industry.

124

10 CONCLUSIONS AND REFLECTIONS

In this chapter, first the research questions are revisited again and answer to these questions
are summarized, referring back to the chapter in this thesis in which the foundations for these
answers has been laid (section 10.1). After summarizing the answers to research question, the
attention is focused on main research question. A reality check has been provided by
discussion on how well the research answered the main question.
Later, the chapter (section 10.3) discusses how well the research achieved the stated goal.
Subsequently, the contribution of research in academic field and O&C project industry is
discussed. In the last section, reflections are mentioned along with the future research
opportunities.

10.1 REVISITING RESEARCH QUESTIONS

RQ.1 What constitutes project success and what are performance assessment criteria of O&C
projects?
The section 4.3.1, discuss the project management view on project success and performance
measurement. It was found that the project management literature addresses the project
success and performance both at macro and at micro level, yet the famous project iron triangle
was found to be most widely practiced. After an argumentative analysis of literature, the
industry view (perspective of an E&C contractor) was accepted. Therefore, the following
criteria were mentioned as measurement of project performance.
I. Total project cost (TIC)
II. Mechanical completion achievement (MCI)
III. 95% engineering schedule achievement (ESI)
IV. Engineering manhours (MHI)
RQ.2 what do we understand by early warnings of project problems?
The answer to second research question could be found in section 4.4 and the observatory
evidence has been provided in section 6.4 to section 6.7.
At first, the available literature on early warnings in project was kept in focus and literature
review was performed in section 4.4. It was found that there had been number of research and
studies concerning early detection of problems. However, the literature review revealed that
most of the studies were either qualitative in nature and lack the practical usability.
Therefore, the professional researches were also explored, which partially filled the gap within
academic researches. The study by CII was focused on usability but lacked the objectivity
because its use of user subjective information, moreover it lacked the utilization of knowledge
from past projects. The work by IPA was found to be purely quantitative in nature, however

125

their use during EPC was found to be limited. IPA conducted its audits primarily at FID
(before the start of EPC phase) and at project completion and therefore are static in nature,
whereas the presented research is focused on dynamic performance management over EPC
phase i.e. project execution.
From literature review, we adopted our own definition of early warnings, which defined early
warnings as follows and therefore answer the second research question.
Early warnings are inherited measurable project characteristics and/or observations that
indicate towards the development of potential problems in future course of project
execution.
Subsequently, in chapter 4, the potential benefits of early warnings were presented. If well
detected in time, the early warning could provide an opportunity for the project manager to act
proactively and mitigate the problems, before they actually affect the project cost and
schedule.
RQ.3 What early warnings could be identified in project execution?
Having defined the early warnings, the next task was to identify the early warnings during
project execution. For that purpose, two main type of sources were selected, the literature
(academic as well as professional) and experts from industry. A selection criterion with three
parameters was designed.
Thereafter, the early warnings from literature and experts were selected irrespective of their
nature i.e. qualitative or quantitative. The analysis of early warnings (section 5.5.4 and section
5.6.3) from both literature and experts revealed that there are early warnings related to the
quality of project team and its integration that are critical for project success. All the early
warnings were evaluated on three selection parameters and finally ten early warnings were
selected for further analysis.
Therefore, answer to research question three is two fold, the comprehensive list in appendix A
provides the full list of all the early earnings that could be identified in project execution and
the list of selected early warnings from perspective of present research.
RQ.4 what early warnings can be operationalized to build a performance prediction model?
In an attempt to find answer to above research question early warning selection criteria was
developed. As the main objective was to have a quantitative model that could use the data
from past project, therefore measurability and data availability was two of three selection
parameters.
After the careful evaluation of each identified early warning on selection criteria, section 5.7
provides a list of early warnings (with their detailed explanation) that were selected for further
analysis. The following table provides the list of selected early warnings.

New ID

Early warning indicator

126

LES
PTE
COC
NCO
CCO
FED
PH
PS
CE
DPO

Lack of understanding of project execution strategy among project team

Project team lacks experience required for the project
Conflicts between owner and E&C contractor
Numbers of change orders
Cost impact of changes
Percentage of missing information in FEED package
Growth in process man-hours
Delay in process engineering
Change in concurrency level between process and piping engineering
Delay in issuance of purchase orders

However, from operationalization perspective, few exceptions were made. The early warnings
related to project team development and integration (LES, PTE and COC) were included
because of their importance stated by both literature and experts. Therefore, a decision was
taken to explore them further at least during case studies. Another early warning FED
(percentage of missing information in FEED package) was selected for the same reasons.
RQ.5 What are the dynamics between early warnings and project performance?
Having selected the early warnings of future potential problems, a logical question arises that
how early warnings that belongs to the early phase of engineering can predict the problems in
late engineering, procurement or even construction. The above question has three parts, early
earnings, project problems and project performance (assessed via project outcomes). The first
part i.e. the research questions has been provided in the previous section of this search
(Section 5.7)
To connect the future project problems and project performance, a case study approach was
adopted. Chapter 6 explored the four past projects in detail (section 6.4 to section 6.7) and
provided the answer to research question five. The case study investigation found that there is
a relationship between early warning indicators, project problems and project performance.
For example, the problems associated with early warning indicator, number of changes (NCO)
are mentioned in Figure 32. The past project literature mentions those identified problems as
main reason for their less than successful performance.

127

Figure 32 future project problems and project outcomes associated with NCO

RQ.6 What early warnings are indicative of deviation in project performance?

RQ.7 How performance prediction model could predict the future performance of O&C
projects?
To find an answer to above-mentioned research questions, a detailed quantitative investigation
was performed in chapter 7 and chapter 8.
In chapter 7, the relationship between EWI and project problems was investigated at three
levels, behavioral analysis, static correlation analysis at 95% engineering completion and
dynamic correlations over engineering duration. The quantitative analysis did present
interesting and encouraging results. The main results are mentioned as follows:
I. Early warnings do behave differently in case of less than successful and good
performance projects, few in terms of their absolute value and few in their incremental
changes.
II. Correlations do exist between EWI and project outcomes, however not all the EWI
found to be correlated with all project outcomes.
III. The EWI indicators do shows a dynamic quantitative relationship with project
outcomes over engineering duration of the project.
The above three results does validate the predictive capability of EWI. However, it is very
important to remember that the quantitative analysis was performed on very limited data set

128

i.e. eight projects. Therefore, to have more concrete conclusion and evidence, more data
would be required.
Based on the results in chapter 7, chapter 8 explored the conceptual development of prediction
model with a well structured development approach. On the overall level, it was found that
predictions by prediction model are more accurate than traditional forecasting methods.
However, it is important to remember that the results are very much limited to small dataset
used for development of prediction model.
Furthermore, the analysis showed another limitation of dataset, the bias towards less than
successful performance projects. Therefore, to have a more accurate prediction in good
projects, a dataset is required which contains a balance of Successful and less than successful
performance projects.
Based upon the findings in this chapter, it should be concluded that the accuracy level of
prediction model developed by this research need more preciseness and validation.
Nonetheless, it can be concluded that if the approach developed by this research is applied to a
larger and rich dataset, a more precise and better prediction model could be developed.
Therefore, the chapter provided a limited answer to our last research question.

10.2 ANSWER TO THE MAIN RESEARCH QUESTION

The main research question of this research was presented as follows
How can future problems and performance of a current O&C project be predicted at early
stages using knowledge and experience from past projects in an EPC environment?
The Figure 33, below show the synthesis of answer to main research question from subquestions answered in previous section.

Figure 33: synthesis of answer to main research question

The research question one to four deals with the project performance and early warnings that
are used as main instrument for predicting future problems and final project performance via

129

literature and experience of experts. Answer to research question five provided the
observatory evidence of relationship between early warnings, future problems and project
performance using past projects. The answer to final sub-question six utilized the past projects
quantitative knowledge and developed an approach with first conceptual prediction model.
Based on so far research it can be concluded that early warnings can be used for predicting the
future performance of the project. In addition, the potential future problems associated with
early warnings could also be known before they affect the project performance.
However, the above answer to main research question is partial in nature due to very limited
dataset used in this research; therefore limit the generalization of findings. To have more
concrete conclusions and better prediction model, a larger dataset consisting of more projects
with similar nature should be used. Even though, the findings and prediction model
development approach presented in this research can be used to build a useful model and
applied in O&C project industry.

10.3 DISCUSSION ON RESEARCH GOALS AND DELIVERABLES

Looking back at the research goals stated in section 2.5, it can be concluded that the research
did achieve its stated goals; however, the achievement is subjected to limited dataset of past
projects.
The first set goal was to provide a new scientific base for understanding and analyzing early
detection of project problems. The chapter five and more importantly chapter six provides the
mechanism (as base) to predict the future project problems through early warnings (as
instrument).
The second set goal i.e. an approach that can facilitate the constructive utilization of past
project and explore the power of prediction modeling for successful management of O&C
project has been achieved in chapter 6 to chapter 8, which used the quantitative information
and knowledge from past project to develop a conceptual performance prediction model.
However, it is recommended to use sophisticated and advance prediction methodologies using
large and balanced dataset of past projects.
In terms of its intended deliverables, as mentioned in chapter 8, the deliverable a pilot
performance prediction model is subjected to limited availability of data from past projects.

10.4 CONTRIBUTION TO SCIENTIFIC COMMUNITY

From scientific perspective, the research possibly has two main contributions to project
management science.

10.4.1 A shift from reactive project management to proactive project

management
The present research focuses on the management of problems before they affect the project
execution. In other words, marking a shift in the attention of project management from
manage what has happened to manage what will happen. Chapter 5 discusses the early

130

warnings of project problems and chapter 6, provided the observatory evidence of relationship
between early warnings and project problems. Further, the chapter 8 used the indirect
relationships (proved in chapter 6) between early warnings and project performance to build a
conceptual prediction model. The presented approach could provide the opportunity for
project managers to act well in advance incase the early warnings shows undesired
performance.

10.4.2 A new and constructive role of past projects

The project management literature and professional researchers sees the past project as
sources of qualitative information (lesson learns) and benchmarking data to improve the
project management practices. Another use of past projects is to perform a post
implementation analysis to find why a project performed good or bad.
In the perspective of present research the above uses of past projects is termed as
underutilization of the information they generate. Every year, there are hundreds of project
being executed around the world. Yet so limited information is extracted from them and used
for limited purposes. The present research advocates for a larger role of past projects and their
project execution information to improve upon the performance of current projects. Chapter 6
and 8 of the research demonstrates exactly how it should be done.
Another noteworthy point is that if the execution information from past project can be
gathered, it will add value to the already existed use of their information. In other words, the
lesson learned from the previous projects could use the quantitative information from past
project as evidence in their support.

10.5 CONTRIBUTION TO O&C PROJECT INDUSTRY

In this thesis, two main concepts are identified 1) early detection of future potential problems
2) future performance prediction using past projects. The first concept does exist, but
implicitly and no proper mechanism was found to be in performance management of projects.
The second concept was found to be missing in O&C project industry. Therefore, the above
two can be identified as main contribution to O&C project industry.

10.5.1 Early detection of future potential problems

Although the O&C industry is aware of the importance of domains, which the EWI represents
for example, change management, project team development and integration. However, the
relationship between their behavior at early stages in project and future project problems is
somewhat unclear. The chapter 6 of the present research explicitly highlights this relationship.
It was found in the case study analysis that the project problems do have their origination in
the early stage of project i.e. detailed engineering. Therefore, the present research do
contribute in development of a new project performance management practice, which can
transform the existed implicit knowledge in O&C project industry into a structured approach
which can predict the problem before their manifestation.

131

10.5.2 An approach to capitalize on past projects to improve project

performance management
As stated previously, in the O&C project industry the past projects are seen in two
perspectives 1) learning lessons 2) improving benchmarks.
The two perspectives has been researched and promoted by both academicians and
professional research organizations (CII, IPA). Whereas, the value of quantitative data
generated during project execution is somewhat lost after the project completion. The present
research stressed on capitalizing on that lost value to improve the performance management
of current projects. Chapter 7 of this research demonstrated that information generated during
project execution do hold value and chapter 8 capitalized on that information to predict the
future performance of projects.
In addition, the chapter 8 compared the forecasts made by prediction model and forecasts by
traditional forecasting. Most of the results were found to be in favor of prediction model (build
via information from past projects). However, the results are subjective to limited data set. The
validations of results require further research and defiantly a larger dataset. In spite of
deficiencies the present research holds, it does demonstrate the potential benefits of past
project information and advocate for a stronger quantitative perspective on project
performance management.

10.6 FINAL REFLECTIONS

In the previous section conclusion were drawn. This section focus on reflecting on research
project itself. In the first section, research approach and process is reflected upon, including a
reflection on the theories used. The second section contains the reflection on the research
results i.e. the degree to which the research met intended objectives, including a reflection on
the validity of the perspective results and recommendations for future research. The final
section presents a personal reflection on the research project.

10.6.1 Reflection on research approach and methods

When reflecting on the scope and complexity of the research, it can be said that the research
problem at hand was quite complex and very much new to the project management
community. The knowledge base was lacking and no comprehensive theory on prediction of
project performance was found. Furthermore, the absence of clear and objective early
warnings in project execution did pose a substantial challenge. The choice to make a dynamic
prediction model further enhanced the complexity of the problem. The scope of the research
was very much ambitious and there could be a possibility to narrow down the scope further.
One possible option could have been to limit the scope to just establishing the quantitative and
qualitative relationship between EWI and project problems. Nonetheless, the insights gained
during this research do compensate for selecting such as large scope.
When reflecting on the fundamental research approach, it can be said that the research design
seems to be logical and coherent, as each building block for prediction model was collected in
phases; each phase receiving the required inputs form previous phase and delivering its added
knowledge to next phase. The research approach has been very useful in bringing structure to

132

the research. When reflecting on the desk research (in phase one and two) it would have been
better to limit the literature to select early warnings. From the preliminary analysis of
literature, it became evident that the there is a lack in the scientific literature on this topic,
therefore expert interviews could have used earlier. The expert interview proved to effective
and efficient method to get the early warnings in project execution. The notion of
measurability in early warnings could have been defined more preciously; the interviews
resulted into many early warnings that could be measured, however not necessarily being
measured in actual practices.
The case study analysis as a research method and its execution did prove to be highly
beneficial to overall research. The in-depth case study analysis revealed the dynamics between
EWI and project performance i.e. the potential project problems. The cross-case analysis set
the background in which the evidence for prediction capability of EWI was obtained.
The data collection from past project proved to be most problematic and cumbersome task.
The corresponding data for each EWI has to be collected at 8 moments in project execution
and there were no standard reports were found from which the data could be readily obtained.
Moreover, data for few of the project had to be retrieved from archives and was available in
size of several GB. Nonetheless, to keep the quality of data high, the efforts were made to
collect the data from more than one source.
The statistical research methods i.e. correlation analysis and multi regression could be termed
as reasonable choice, however subjected to limited data availability. Furthermore, the data
from past projects could not be normalized (except the duration), because the variable that
cannot be predicted, it is not possible to normalize them. However, the similarity in nature of
projects and their sizes could be compensating for the absence of normalization.

10.6.2 Reflection on results of the research

The reflection on the results of the research is two fold: a reflection on the gathered practical
and scientific knowledge and a reflection of the accuracy and reliability of the prediction
model.
A dynamic prediction model is developed in chapter 8. Looking at the errors found in the
predictions from prediction model, it can be said that the prediction model certainly have an
advantage over traditional methods. However, the results of external validation reflected the
limited accuracy and reliability of pilot prediction model. The prediction ranges with 50%
confidence (P50) embrace the actual value and could be seen as reasonable prediction ranges.
The prediction ranges with 80% confidence interval were found to be too wise and in current
state could not be used for guiding the decision-making.
Based upon results of prediction model, it should be concluded that the accuracy level of
prediction model developed by this research certainly need more preciseness and reliability.
To have more preciseness, a larger dataset with balance of successful and less than successful
performance project is required. Nonetheless, it can be concluded that if the approach
developed by this research is applied to a larger and rich dataset, a more precise and better
prediction model could be developed.

133

Reflection on the gathered knowledge could be made by looking at the contributions made by
present research to practical and scientific domains. From scientific perspective on project
management, it can be concluded that the research does add to the dynamic control of project
execution. By highlighting the early warnings of potential problems and establishing its
relationship with overall project performance, the research does mark a shift prom reactive
project management practices to proactive project management. In addition, the research
provides a constructive use of past project, by learning from factual dynamic situation to
reduce the future uncertainties in current projects. From practical perspective, it can be
concluded that the prediction model is still far from being off the shelf method, thus have
limited usability in current form. However, as stated earlier the current research project sets
fertile ground for improvement in accuracy and reliability of prediction model.

10.7 FUTURE RESEARCH OPPORTUNITIES

The natural continuation of this research project would be collection of more data from past
project and improve upon the reliability and accuracy of the prediction model. This requires:
I. A collection of past project data with more of less similar nature and size
II. Normalization of data to address the difference in projects (Size, technology type,
location)
III. More rigorous statistical prediction (neural networks, technique with large sample
size
In this instance, the following research questions are proposed as the base for future research
opportunities in this topic:
Q1. How can the historical data collection and retrieval systems be improved so that the
current project can utilize their information to reduce the uncertainty in their execution?
Q2. What simulation and modeling techniques could be applied to prediction models in
project management to improve their accuracy and reliability?
Q3. What prediction system can be designed for management of O&C project execution in
EPC environment?

134

11 REFERENCES
1. A.J.Shenhar, & Dvir, D. (1996). Towards a typology theory of project management.
Research policy, 25, 607-622.
2. Aberg, L. (1989). The facet theory of communication. University of Helsinki.
3. Anderson, D. R., Sweeney, D. J., & Williams, T. A. (2007). Essentials of Modern
Business Statistics with Microsoft Excel. Cengage learning.
4. Ansoff, I. H. (1984). Implanting Strategic Management. Prentice/Hall International.
5. Artto, K. A., Lehtonen, J.-M., & Saranen, J. (2001). Managing projects front-end:
incorporating a strategic early view to project management with simulation.
International Journal of Project Management, 19(5), Pg. 255264.
6. Atkinson, R. (1999). Project management: cost time and quality - two best guesses
and a phenomenon, Its time to accept other success criteria. International journel of
project management, 17(6), 337-342.
7. Barber, E. (2004). Benchmarking the management of projects: a review of current
thinking. International Journal of Project Management, 22(4).
8. Berends, T. (2007). Contracting economics of large engineering and construction
projects. TU Delft.
9. Betts, R. K. (1982). Surprise Attack, Lessons fro defence planning. Washington
D.C: The Brookings institution.
10. Booz Allen & hamilton. (2008). Capital project execution in the oil and gas industry.
11. Choi, J., Anderson, S., & Kim, S. (2006). Forecasting potential risks through leading
indicators to project outcome. Construction industry institute.
12. CII. (2004). Model Planning and controlling system for EPC of industrial projects.
Construction industry institute.
13. CII. (2012). Retrieved from https://www.constructioninstitute.org/scriptcontent/basics.cfm?section=aboutcii
14. CII. (2012). Manageing EPC projects in oil and gas.
15. Cleland, D. I. (1994). Project Management, Strategic Design and Implementation,
(2nd ed.). McGraw-Hill Inc. USA.

135

16. Coser, L. A. (1957). Social Conflict and the Theory of Social Change. The British
Journal of Sociology, 8(3), pp. 197-207.
17. Crawford, L., Pollack, J., & England, D. (2006). Uncovering the trends in project
management: Journal emphases over the last 10 years. International Journal of
Project Management, 24(2), 175-184.
18. Davies, H., & Ellis, P. (2000). Porters Competitive Advantage Of Nations: Time
For The Final Judgement? Journal of Management Studies, 1189-1214.
19. Fluor Corporation. (2012). Standard engineering work process.
20. Fluor, C. (2010). Guidlines for Detailed engineering. Guideline 000 200 1505.
21. Flyvbjerg, B., & Bruzelius, N. (2003). Mega project and risk - An Anatomy of
ambition. 207.
22. Freeman, M., & Beale, P. (1992). Measuring project success. Project management
journal, 1, 8-17.
23. Harpum, P. (2004). Project control in The Wiley Guide to Managing Projects (eds P.
W. G. Morris and J. K. Pinto. John Wiley & Sons.
24. Herman, M. (1996). Intelligence power in peace and war. Cambridge university
press.
25. Hughes, S., Tippet, D., & Thomas, W. (2004). Measuring project success in
construction industry. Engineering management journal, 16(3), 31-37.
26. IPA. (2012). Retrieved from http://www.ipaglobal.com/
27. Kerzner, H. (1994). Project Management, A Systems Approach to Planning, (Fifth
ed., Vol. Fifth addition). Van Nostrand Reinhold Co, USA.
28. Koontz, H. (1958). A PRELIMINARY STATEMENT OF PRINCIPLES OF
PLANNING AND CONTROL. 1, 45-61.
29. L.Lientz, B., & P.Rea, K. (1995). Project management for the 21st. Academic Press,
USA.
30. Larsen, E., & Gobeli, D. (1989). Singificance of project management structure on
project success. IEEE Transactions on engineering management, 36(2), 119-125.
31. Lientz, B., & Rea, K. (1995). Project management for 21st century. Academic press.
32. Lim, C., & Mohamed, M. (1999). Criteria of project success: an exploratory reexamination. International Journal of Project Management, 17(4), Pg. 243-248.

136

33. Merrow, E. w. (2012). Oil and gas industry mega projects:Our recent track record.
Society of petroleum engineers.
34. Might, R., & Fischer, W. (1985). THe role of structural factors in determining
project management success. IEEE Transactions on engineering management, 32,
71-77.
35. Morris, P., & Hough, G. (1987). The Anatomy of Major Projects. Wiley guide of
project management.
36. Morris, R. (1997). Early warning indicators of corporate failure, A critical review of
previous research and further empirical evidence. Ashgate publishing Limited.
37. Newcombea, R. (2003). From client to project stakeholders: a stakeholder mapping
approach. Construction Management and Economics, 21(8), Pg. 841-848.
38. Nikander, I. O. (2002). Early warning: A phenomenon in project magement.
Helsinki university of technology.
39. Nikander, I. O., & Eloranta, E. (2001). Project management by early warning.
International journal of project management, 19, 385-399.
40. Norwegian Technology Standards Institution. (1999). MECHANICAL
COMPLETION AND COMMISSIONING. Retrieved from
http://www.standard.no/pagefiles/965/z-007.pdf
41. OCED. (2011). Environmental Outlook to 2050: Draft Health and Environment
Chapter. OECD.
42. Oisen, R. (1971). Can project management be defined. Project management
Quarterly, 2(1), 12-14.
43. P.Lewis, J. (1993). The Project Managers Desk Reference. Probus Publishing Com.
44. PMI. (2012). Project management body of knowledge. Project management institute.
45. R.H.Mayers. (1990). Classical and modern regression analysis with applications.
PWS-Kent.
46. Raz, T. S., & Dvir, D. (2002). Risk management, project success and technological
uncertanity. R&D Management, 32(2), Pg-101-109.
47. Schlumberger business consulting. (2012). E&P Mega projects. Schlumberger
business consulting.
48. Thamhain, H., & Wilemon, D. (1986). Criteria for controlling projects according to
plan. Project management Journal, 17(2), 75-81.

137

49. Turner, R. (1999). Handbook of project based management: Leading strategic

change in organizations. Mcgraw-hill.
50. Vanhoucke, M. (2011). Measuring efficiency of project controls using fictitious and
empirical project data. International journal of project management, 30, 252-263.
51. W.G.Morris, & J.Pinto. (2004). The Wiley guide to managing projects. John Wiley
& Sons.
52. W.Hughes, S., Tippett, D. D., & Thomas, W. K. (2004). Measuring Project Success
in the construction industry. Engineering Management Journal, 16(3), Pg-31- 37.
53. Wallace, W. L. (1971). The Logic of Science in Sociology. Aldine De Gruyter.
54. Williams, T. (2004). Identifying the hard lessons from projects easily. International
Journal of Project Management, 22(4), Pg. 273279.
55. Williams, T. (2004). Identifying the hard lessons from projects easily. International
Journal of Project Management, 22(4), 273-279.
56. Wit, A. d. (1988). Measurement of project success. International Journal of Project
Management, 6(3), Pg - 164170.
57. Wit, A. d. (1988). Measurement of project success. Project management journal, Pg
164-170.
58. Yin, R. (2003). Applications of case study research. Sage publications.

138

APPENDIXA.1
EWI from literature

ID
L-1

L-2

L-3

L-4

L-6
L-7

L-8
L-9
L10

early warnings
Monitoring of
actual allocation
against the plan
can provide early
warning of lack of
manpower or lack
of work
Noticeable change
in performance
level for a team or
individual
members reflects
problems.
Changes in cost
estimates of
management
reserve, cost
variance, budget
slips
Everyone is
writing protection
memos, before
everything was
verbal
Hidden feelings,
conflicts avoidance
at all costs
Undefined clear
objectives and
fluid commitment
levels from key
participants
Duplication of
efforts
Undefined
deliverables
Argumentive
sessions at
meetings

Source
Bent - 82

Feature
Qualitati
ve

Quantitativ
e

Feeling,
Behavior

Typology
Descripti
ve

Performan
ce
indicator12

Cleland 84

Fleming83

Kerzner95

Kerzner95

Kerzner95

Kleim and
Ludin-94
Kleim and
Ludin-94
Kleim and
Ludin-94

12

Performance indicators are indicators with pre-established thresholds and deviation from thresholds is
noticed and reported on project level (for more information, see section 5.4.2).

139

L11
L12
L13
L14
L15
L16
L17
L18

L19
L20

L21
L22

L23
L24

L25

Project objectives
are cut back.
Project team
members are
reassigned.
Contingency plans
to project surface
No discussion on
task status
Difficulty in
getting resources
for project
High turnover rate
of personnel
High rate of
rework
High rate of
changes in part
prompted by
mistakes and
errors
Confusion and loss
of enthusiasm and
motivation
People withhold
information on a
problem in a hope
that the problem
will go away
An excess of no
cost no time
effects
High Level of
subcontractors
claims and
extension of time
claims
When people work
too much or too
little
Letters exchanged
between main
parties showing
there was not a
common definition
of what one partys
responsibility
is
Continually
unfulfilled
promises,
frequently
changing
decisions, or lack
of commitment to
make decisions

Lientz et al
- 95
Lientz et al
- 95
Lientz et al
- 95
Lientz et al
- 95
Lientz et al
- 95
Lientz et al
- 95
Obradovit
ch - 90
Obradovit
ch - 90

Obradovit
ch - 90

Smith-83

Williams
et al. 2012

Williams
et al. 2012

Williams
et al. 2012

Williams
et al. 2012

Williams
et al. 2012

140

L26
L27
L28
L29
L30

L31

L32

L33

L34

L35

L36

Unaligned goals
Conflicting norms
and standards
Lack of trust in
project team
Lack of companys
internal support
Lack of experience
in project team
Owner and
contractor is
requesting an
excessive number
of contract
changes during
project execution
Significant project
scope items are
omitted from bid
packages
Project is
experiencing a
high level of
engineering/design
/ specifications
errors and changes
The project team is
frequently asking
vendors, suppliers
and service
providers to
perform functions
outside their area
of expertise and
experience
The project team is
lacking in the
necessary
expertise,
experience, breath
and depth to
successfully
execute the project
Design reviews fail
to include qualified
personnel who can
analyze safety and
loss prevention
features of plans
and specifications

Boschrekveldt,2
011
Boschrekveldt,2
011
Boschrekveldt,2
011
Boschrekveldt,2
011
Boschrekveldt,2
011

CII-2006

CII-2006

CII-2006

CII-2006

CII-2006

CII-2006

141

L37

L38

L39

L40

L41

L42

L43

The project lacks

sufficient skilled
craft and is
experiencing high
craft turnover due
to competition
from other
projects, low
wages and shorter
work schedules
The project team is
failing to identify
or address missing
requirements
during design
reviews
The project
manager is lacking
required expertise
or experience and
skills
Project quality
control results are
reflecting high
rejection rates for
equipments and
materials under
fabrication in the
factory and/or
materials in place
through testing in
the field.
Project team
response to RFI,
questions and
changing events
that can
significantly
impact the project
results is slow,
inadequate or
incomplete
Forecast to
complete based on
actual project
experience
combined with
actual
commitments and
expenditures are
consistently
projecting
overruns in E, P
and C budgets
The project team is
experiencing a

CII-2006

CII-2006

CII-2006

CII-2006

CII-2006

CII-2006

CII-2006

142

L44

L45
L46
L47
L48
L49
L49

high turnover rates

and instability in
team membership
Business goals,
project objectives,
priorities, and
critical success
factors are not
being consistently
used by project
team and key
stakeholders to
guide decisionmaking.
Commitments are
not made with the
intention of being
made and are
usually not met.
Poor front end
loading
Limited use of
value improving
practices (VIP)
Poor team
development index
Poor project
control index
Major design
changes at later
stage in project

CII-2006

CII-2006

IPA, 2012

IPA, 2012

IPA, 2012

IPA, 2012

IPA, 2012

143

APPENDIX A.2
Early warnings from Experts

Feature
ID
I-1
I-2
I-3
I-4

I-5

I-6

I-7
I-8
I-9
I-10
I-11
I-12
I-13
I-14
I-15
I-16
I-17
I-18

Early warning
Delay in Critical unit final
P&ID,s,
Lack of understanding of
execution strategy by project
team
Large amount/Numbers of
scope changes (additional
items) by clients
Large amount/Numbers of
scope changes (In details)
by clients
Lack of communication
between engineering
disciplines regarding
critical interfaces
Lack of communication
between engineering
disciplines regarding
assumptions and their
verifications
Low productivity of project
team members
Delay in 1/3 Model review
Delay in 2/3 Model review
Delay in Vendor design
information for critical
equipments
higher nos of 1/3 Model
review tags
Higher numbers of 2/3
Model review tags
High Concurrency level in
projects
High ratio of open changes
vs. approved changes
High Level of client
involvement
Late scope change
Variation in contingency
rundown rate
Overrun in process
engineering man-hours

Qualitative

Typology
Quantitative

Feeling,
Behavior

Descriptive

Performance
indicator

144

I-19
I-20
I-21
I-22
I-23
I-24
I-25
I-26
I-27
I-28
I-29
I-30
I-31
I-32
I-33
I-34
I-35
I-36
I-37
I-38
I-39
I-40
I-41
I-42

Overrun in piping
engineering man-hours
Delay in process
engineering
Ration of changes induced
by client vs. casual changes
Forecasted growth in
engineering man-hours
High Nos of arguments
between E&C contractor
and owner
Limited Level of risk
identification
Low level of team
integration
Lack of understanding
regarding project objectives
among project team
Delay in purchase order
placing
Growth in man-hours per
equipment
Growth in man-hours/
P&ID
The distance between peaks
staffing of engineering
disciplines is diminishing
Nos of changes After
P&ID's are issued for
design
The percentage of total RFQ
delayed over planned RFQs.
Growth in man-hours per
key quantities
Large increase in numbers
of engineering drawings
Changes in major project
objectives
Delay in critical equipment
specifications and data
sheets
Low progress earned for
critical Activities
Unrealistic monthly
incremental schedules
Delay in interface milestone
of engineering and
procurement
Early engineering discipline
crossing late schedule curve
Undue importance or focus
of parts of facility, while
ignoring the other parts
Presence of complex
interfaces with in project

145

I-43
I-44
I-45
I-46

I-47
I-48
I-49
I-50
I-51
I-52
I-53

I-54

I-55

I-56

I-57

execution methodology.
Higher numbers of changes
in project at early stages
Low motivation of project
team especially when its
induced by numerous
changes
High numbers of project
deviation notices
Delay in vendor design
information for detailed
engineering
Quality of project team in
terms of technical
experience and experience
with adopted procedures
Opportunistic behavior by
vendors
over optimistic monthly
incremental schedules
early spending of
Contingency (faster
rundown rate)
Changing project drivers
during execution
Incomplete Front end design
Failure to incorporate
technical complexity while
estimating the budget hours
for engineering
Undue delay in approval or
rejection of proposed
changes
Failure to build and
harmonized project team,
addressing cultural
differences
Low confidence of
engineering leads towards
schedule and budgeted
hours.
High variation in
construction quantities w.r.t
original estimate

146

APPENDIX B: RECOMMENDED EARLY

WARNINGS

ID

Early warning

I-1

Delay in Critical unit final P&ID,s,

I-2

Lack of understanding of execution

strategy by project team

I-8

Delay in 1/3 Model review

I-9

Delay in 2/3 Model review

I-10

Delay in Vendor design information

for critical equipments

I-11

higher nos of 1/3 Model review tags

I-12

Higher numbers of 2/3 Model review

tags

I-21

Ration of changes induced by client

vs casual changes

I-24
I-28
I-29
I-30
I-34
I-36
I-37
I-42
I-54
I-56

Limited Level of risk identification

Growth in manhours per equipment
Growth in man-hours/ P&ID
The distance between peaks staffing
of engineering disciplines is
diminishing
the increase in numbers of
engineering drawings due to
revisions
Delay in critical equipment
specifications and data sheets
Low progress earned for critical
Activities
Presence of complex interfaces with
in project execution methodology.
Undue delay in approval or rejection
of proposed changes
Low confidence of engineering leads
towards schedule and budgeted
hours.

Measurab
ility

Criteria
Early in
project

Recommendation

Recommendation

Recommendation

Recommendation

Recommendation

Recommendation

Recommendation

Recommendation
Recommendation
Recommendation
Recommendation

Data
Availability

Status

Recommendation

Recommendation

Recommendation

Recommendation

Recommendation

Recommendation

Recommendation

147

APPENDIX C.1: INTRODUCTION TO

INTERVIEWEES
Brief introduction
1. Background: The primary purpose of this research is to develop a prediction model for predicting the
future project performance during early phases of project execution. The main problem with traditional
project performance systems is that they are based on the information from the project history, which
implies that the forecast made from them is extrapolative and always somewhat late. Moreover, these
measure or indices cannot anticipate the potential problems in projects. When problems became visible,
they have significant negative impacts on project performance and project managers have to adopt reactive
approach. If early warnings of potential project problems can be detected at early phases of project, many
options are still available at discretion of project manager. The project manager has additional time to make
a proactive decision, inform the key stakeholders, and form preventive strategies and subsequently
implement them. The preventive strategy utilizes much less resources, thus keeping the cost and schedule
close to desirable project outcomes as compared to reactive approach. Therefore, if a tool is developed,
which can provide a vision for project managers to predict the future project outcomes in light of these key
early warning indicators (EWI), the probability of project success can be enhanced significantly.
2. Aim of the interview: The interviews aimed to find key EWI during following phases of a project:
Detailed engineering/design, procurement. A EWI should have negative effect on following four project
outcomes.
1.)
2.)
3.)
4.)
5.)

Cost
Schedule
Quality
Functionality
Safety

3. Key definitions:
3.1) Early warning indicators (EWI)
Early warning indicators are inherited project characteristics and/or observations that indicate towards the
development of potential problems in future course of project execution. Reveled in timely manner, these
indicators allow proactive management to influence project outcomes.

3.2) Project outcomes:

3.2.1) Cost - Cost performance is viewed as overall final cost versus the established baseline budget.
Secondary cost outcomes can include cash flow deviations, and the rate of contingency consumption.
3.2.2) Schedule - Schedule performance is viewed as the overall actual duration versus baseline project
duration. Secondary schedule performance can include overall engineering discipline deviation and
deviation in construction schedule.

148

3.3.3) Quality Quality performance is viewed as the amount of rework being done on project. Secondary
quality performance can include more document review cycles than target.
3.4.4) Functionality: Functionality performance can be seen as the actual numbers or cost of change orders
processed during project versus target.
3.5.5) Safety: Safety performance is viewed as OSHA incident rate (Actual Vs Zero)

4. Further steps: The selected EWI from literature study and expert interviews will be evaluated in the
context of project reality via quantitative analysis. The data related to selected EWI and project outcomes
will be collected from past-completed projects to establish the correlation between these EWI and project
outcomes. Furthermore, the correlations will be used in modeling of performance prediction tool.
The complete approach for this research is attached below.

Attachment 1: Research approach

149

APPENDIX C.2 LIST OF INTERVIEWEES

(In alphabetical order)

150

APPENDIX D CONFIDENTIAL
APPENDIX E : CONFIDENTIAL

151

APPENDIXF: DESCRIPTION OF VARIABLES

AND OUTCOMES
EW as
Variable
Growth in process
hours

Variable
ID
PH

Delay in Process
hours

PS

Cost of change
orders

CCO

Numbers of
change orders
Delay in purchase
order

NCO

Change in
concurrency
between process
and piping
engineering

CE

Project outcomes
as dependent
variables

ID

Description

Measurement units

Total installed
cost

TIC

The final cost of the plant w.r.t

estimated baseline cost

Percentage growth
+ve = Cost over-run
-ve = Cost under run

Engineering Man
hours

MHI

The final engineering manhours w.r.t estimated baseline

man-hours

Percentage growth w.r.t. baseline

+ve = Man-hour over-run
-ve = Man-hour under run

95 % engineering
schedule

ESI

Mechanical
completion

MCI

The 95 % engineering actual

completion w.r.t baseline
schedule
The mechanical completion of
plant w.r.t baseline schedule

Percentage delay
+ve = Delay
-ve = ahead
Percentage delay
+ve = Delay
-ve = ahead

DPO

Description

Measurement units

Process engineering is
consuming or forecasting
more/less hours than baseline
hours.
Process engineering is
behind/ahead of baseline
schedule
The cost impact of implemented
or to be implemented changes

Percentage change w.r.t baseline

+ve = Growth
-ve = reduction

The numbers of implemented to

be implemented changes
The delay in issuance of
purchase orders for main
equipments
Change in the distance between
process and piping engg. curves
w.r.t baseline distance

Numeric
+ve = ahead
-ve = delay
Percentage of estimated TIC
+ve = Growth
-ve = reduction
Numeric
Percentage delay
+ve = ahead
-ve = delay
Percentage change
+ve = Low concurrency w.r.t. baseline
-ve = High concurrency w.r.t. baseline

152

Data collection points

The data collected at T1 corresponds to the baseline values. Thereafter, the data was collected
at every 15 % progress in engineering until 95 % engineering completion. The values at T-8
i.e. completion of the project represents the final project outcomes and determines the final
performance of the project. The data collection process developed above enabled the
normalization of data collection moments so that projects with different project durations can
be compared with each other.

Figure 34: Data collection moments

153

APPENDIXG:
G.1 Correlations among EWI and project outcomes at 15 % engineering completion

Early warning
indicator

Correlations
parameters

rs
CCO

Sig.

CE

Sig.

DPO

Sig.

rs
rs
rs
NCO

Sig.

PH

Sig.

rs
rs
PS

Sig.

Project outcome
ESI

MCI

MHI

TIC

-0.10
0.8225

0.130
0.7558

0.330
0.4198

0.79+++
0.0208

-0.050
0.9087

-0.10
0.8171

0.340
0.4076

0.120
0.7735

0.190
0.6514

0.040
0.9327

-0.50
0.207

0.430
0.207

0.140
0.7358

0.40
0.3325

0.310
0.4556

0.520
0.1827

-0.150
0.719

-0.56+
0.1476

-0.340
0.4063

-0.320
0.4441

-0.450
0.2604

-0.74+++
0.0349

-0.81+++
0.0149

-0.9++++
0.002

154

G.2 Correlations among EWI and project outcomes at 30 % engineering completion

Early warning
indicator

Project outcome
MCI
MHI

Correlations
parameters

ESI

rs

0.120

0.360

0.260

0.64++

CCO-30

Sig.

0.7789

0.3821

0.5309

0.0856

rs
CE-30

Sig.

0.170
0.6915

-0.230
0.5754

0.250
0.5479

-0.310
0.4528

DPO-30

Sig.

-0.290
0.4927

-0.240
0.5678

-0.50
0.207

0.64+++
0.028

NCO-30

Sig.

-0.220
0.6081

-0.050
0.8985

0.160
0.7128

0.260
0.5284

0.6+
0.1195

0.410
0.3167

0.62+
0.1017

0.380
0.3518

-0.540
0.1681

-0.89++++
0.0029

-0.540
0.1681

-0.75+++
0.0305

rs
rs
rs
PH-30

Sig.

rs
PS-30

Sig.

TIC

G.2.1 Correlations among incremental EWI and project outcomes at 30 % engineering

completion
Early warning
indicator

Correlations
parameters

rs

CCO-1st-30

Sig.

CE-1st-30

Sig.

DPO-1st-30

Sig.

st

rs
rs
rs

NCO-1 -30

Sig.

PH-1st-30

Sig.

PS-1st-30

Sig.

rs
rs

ESI

Project outcome
MCI
MHI

TIC

0.210
0.6103

0.190
0.6494

0.450
0.2604

0.290
0.4927

0.140
0.7358

-0.240
0.5678

0.140
0.7358

-0.480
0.2329

-0.520
0.1827

-0.350
0.3993

-0.330
0.4198

-0.6+
0.1195

0.20
0.6265

0.120
0.786

0.230
0.5855

0.270
0.5258

0.40
0.3199

0.420
0.3013

0.57+
0.139

0.290
0.4927

-0.050
0.9098

-0.020
0.9659

0.510
0.2007

0.230
0.5855

155

G.3 Correlations among EWI and project outcomes at 45 % engineering completion

Early
warning
indicator
CCO-45

Project outcome

Correlations
parameters

ESI

MCI

MHI

TIC

rs

0.260

0.440

0.64++

0.83+++

Sig.

0.5309

0.2715

0.0856

0.0102

rs

0.40

0.230

0.260

-0.260

CE-45

Sig.

0.3199

0.5878

0.5309

0.5309

rs
DPO-45

Sig.

0.050
0.9108

0.020
0.9551

-0.380
0.3518

-0.69++
0.058

NCO-45

Sig.

-0.050
0.9087

00
1

0.340
0.4076

0.220
0.6013

0.83+++
0.0102

0.66++
0.0757

0.79+++
0.0208

0.520
0.1827

-0.550
0.16

-0.66++
0.0757

-0.86++++
0.0065

-0.9++++
0.002

rs
rs
PH-45

Sig.

rs
PS-45

Sig.

G.3.1 Correlations among incremental EWI and project outcomes at 45 % engineering

completion
Early
warning
indicator

Correlations
parameters

CCO-1st-45

Sig.

CE-1st-45

Sig.

rs
rs
rs

DPO-1st-45

Sig.

NCO-1st-45

Sig.

PH-1st-45

Sig.

PS-1st-45

Sig.

rs
rs
rs

Project outcome
ESI

MCI

MHI

TIC

0.140
0.7358

0.460
0.2572

0.64++
0.0856

0.88++++
0.0039

0.120
0.7789

0.070
0.8657

-0.070
0.8665

-0.050
0.9108

0.69++
0.058

0.77+++
0.0265

0.550
0.16

0.40
0.3199

0.070
0.8657

0.130
0.7544

0.470
0.2433

0.23+++
0.028

0.76+++
0.028

0.83+++
0.0114

0.67++
0.071

0.570
0.5878

0.020
0.9554

0.320
0.4346

-0.330
0.4198

-0.14+
0.139

156

G.4 Correlations among EWI and project outcomes at 60 % engineering completion

Early warning
indicator

Correlations
parameters

rs
CCO-60

Sig.

CE-60

Sig.

DPO-60

Sig.

NCO-60

Sig.

PH-60

Sig.

rs
rs
rs
rs
rs
PS-60

Sig.

ESI
0.310

Project outcome
MCI
MHI
0.40
0.67++

TIC
0.74+++

0.4556
0.070
0.8657
-0.380
0.3518

0.3325
0.150
0.7219
-0.20
0.6287

0.071
0.59+
0.1262
-0.6+
0.1195

0.0366
0.360
0.3821
-0.67++
0.071

0.520
0.1827
0.88++++
0.0039
-0.490
0.2166

0.65++
0.0831
0.71++
0.0501
-0.190
0.6474

0.74+++
0.0366
0.83+++
0.0102
-0.420
0.3013

0.62+
0.1017
0.62+
0.1017
-0.50
0.2039

G.4.1 Correlations among incremental EWI and project outcomes at 60 % engineering

completion
Early warning
indicator

Correlations
parameters

rs

CCO-1st-60

Sig.

CE-1st-60

Sig.

DPO-1st-60

Sig.

rs
rs
rs

st

NCO-1 -60

Sig.

PH-1st-60

Sig.

PS-1st-60

Sig.

rs
rs

ESI

Project outcome
MCI
MHI

TIC

0.74+++
0.0366

0.8+++
0.0165

0.290
0.4927

0.310
0.4556

-0.190
0.6514

0.020
0.9551

0.020
0.9554

0.450
0.2604

-0.070
0.8665

0.010
0.9775

-0.10
0.8225

0.020
0.9554

0.440
0.2715

0.8+++
0.0183

0.66++
0.0757

0.8+++
0.0165

0.10
0.8225

0.140
0.7342

0.070
0.8665

0.450
0.2604

-0.550
0.16

-0.30
0.4713

0.10
0.8225

-0.020
0.9554

157

G.5 Correlations among EWI and project outcomes at 75 % engineering completion

Early warning
indicator

Correlations
parameters

CCO-75

Sig.

CE-75

Sig.

DPO-75

Sig.

NCO-75

Sig.

PH-75

Sig.

rs
rs
rs
rs
rs
rs
PS-75

Sig.

ESI

Project outcome
MCI
MHI

TIC

0.310
0.4556

0.40
0.3325

0.67++
0.071

0.74+++
0.0366

0.140
0.7358

0.20
0.6287

0.64++
0.0856

0.380
0.3518

-0.360
0.3851

-0.20
0.6287

-0.64++
0.0856

0.50
0.058

0.62+
0.1017

0.83+++
0.0114

0.62+
0.1017

0.74+++
0.0366

0.74+++
0.0366

0.87++++
0.0045

0.57+
0.139

0.62+
0.1017

-0.480
0.2329

-0.140
0.7342

-0.62+
0.1017

-0.57+
0.139

G.5.1 Correlations among incremental EWI and project outcomes at 75 % engineering

completion
Early warning
indicator

Correlations
parameters

CCO-1st-75

Sig.

CE-1st-75

Sig.

DPO-1st-75

Sig.

rs
rs
rs
rs

NCO-1st-75

Sig.

PH-1st-75

Sig.

PS-1st-75

Sig.

rs
rs

ESI

Project outcome
MCI
MHI

TIC

0.480
0.2329

0.240
0.5678

-0.070
0.8665

0.020
0.9554

0.76+++
0.0274

0.66++
0.0758

0.460
0.2471

0.380
0.3505

0.020
0.9551

-0.160
0.7111

-0.410
0.3167

-0.280
0.5091

0.380
0.3518

0.63++
0.0909

00
1

0.450
0.2604

0.360
0.3851

0.78+++
0.0229

0.330
0.4198

0.430
0.2894

0.210
0.6103

0.410
0.3167

0.050
0.9108

0.330
0.4198

158