Austroad Review of Potential Benefit of PBC

Austroads Review of Performance Contracts
Paul Hardy, Opus International Consultants

Page 1
AUSTROADS REVIEW OF PERFORMANCE CONTRACTS:
THE POTENTIAL BENEFITS OF PERFORMANCE
CONTRACTS
Paul Hardy
Opus International Consultants

ABSTRACT

The author is a highway maintenance manager with a particular interest in
procurement methods. Prior to coming to New Zealand in 1995 he was
involved in maintenance of trunk roads and motorways in the UK. In New
Zealand he has been involved in the management of the Nelson State
Highway network and has also assisted to prepare contractor lead tenders for
highway maintenance management contracts in Australia and Scotland. He
has recently been involved in a review of performance contracts as a project
for Austroads. The paper draws on the authors experience gained via this
project and his involvement in tendering 2 PSMC contracts in Western
Australia. The paper examines the potential benefits of adopting a
performance contract approach. It offers the authors opinion on the feasibility
of achieving these benefits and the practicality of measuring them.

Paul Hardy
Opus International Consultants Ltd
Nelson Office, 4
th
Floor Civic House,
106 Trafalgar Street
Private Bag 36, Nelson
Tel (03) 548 1099
Fax (03 548 9528
Email paul.hardy@opus.co.nz
Page 2

1 INTRODUCTION
1.1 THE CHANGING ROLES OF ROAD AUTHORITIES

Over the past 10 years or so Road Authorities (RAs) around the world have
undergone significant changes. In many instances this has meant a
redefinition of their role. Whereas previously they were providers of services,
including design, construction and maintenance, with expertise in carrying out
these functions, they are increasingly becoming asset managers and the
purchasers of services. The levels of traditional technical skills required in the
design, construction and maintenance of road networks has been replaced in
many RAs with expertise in asset management and procurement of the
outcomes desired by their customers. For these RAs their role is evolving
from that of expert supplier to informed purchaser.
1.2 EVOLUTION OF CONTRACTS
Evolution of contracts used to procure road construction and maintenance has
occurred as a result of various changes in Government policies, in particular
the outsourcing of technical and construction activities. Many RAs have
adopted forms of performance contracts and specifications to the extent that
some form of performance contracts or specifications have been used by RAs
in most parts of the developed world. This has been particularly the case in
New Zealand and Australia were 11 performance specified maintenance
contracts have been let over the last 6 years. Despite this there is no common
understanding of what is meant by, and what can realistically be achieved by,
a performance contract.

1.3 AUSTROADS PROJECT
In response to this Austroads commissioned a project to facilitate a common
understanding between Austroads member authorities and industry of what is
meant by, and what can realistically be achieved through, performance
contracts. Specifically the project looked at:

The potential benefits of performance contracts
The applicability of performance indicators and models
Experience to date with performance contracts

This paper comments upon the first element, the potential benefits. It draws
heavily upon the research undertaken for the study but reflects the authors
opinion on the findings.
Page 3

2 CONTEXT
To define the context, within which performance contracts are used, a model
of the hierarchical needs of a typical road authority is shown below, figure 1.
This model will be used to explain the drivers behind the adoption of
performance contracts.

2.1 ROAD AUTHORITIES HIERARCHY OF NEEDS
Figure 1, illustrates the linkage between individual tasks (inputs) and their
overall objective to provide a road network that meets the needs of their
customers. Increasingly, RAs are endeavouring to align their contracts with
their higher-level goals, i.e. the outcomes they wish to provide to their
customers. Performance contracts are in part driven by a desire to focus on
the higher level needs rather than the means of achieving them and to allow
suppliers increased freedom to determine how to achieve the desired
outcomes. To explain this we need to consider how various current road
maintenance contracts align with the model above.

Customer
Satisfaction

High Level Outcomes

Network Level Outcomes

.

Outputs

Inputs
Figure 1: Road Authorities Hierarchy of Needs
Customer Satisfaction: is the ultimate objective of any
RA.
High Level Outcomes: look to identify the broad social,
economic and environmental outcomes required and are
generally reflected in the RAs mission statements in
such term as the provision of a safe and efficient
network in an environmentally acceptable manner
Network Level Outcomes: are a means of defining how
the high level outcomes are translated into actions on
the road network and reflect customer expectations of
the road network. These are the outcomes currently
measured by most RAs e.g. Austroads National
Performance Indicators measure outcomes at this level.
Outputs: are the elements of work that contribute
towards achieving an outcome, for example areas of
rehabilitation, reseal etc. Outputs deal with when thing
get done.
Inputs: are the means by which the outputs are
delivered. They are governed by technical specifications
and may dictate work method, material specifications
etc. Inputs deal with what gets done and how.
Page 4

2.2 THE CONTRACT CONTINUUM
A range of contract types is currently used for the procurement of road
maintenance. This model can be used to allow the incremental changes of
moving from one contract type to another to be considered in context. Moving
left to right across the continuum, the contract type focuses increasingly upon
the higher levels of the hierarchy of needs i.e. there is an increasing focus on
the outcome rather than the means of achieving it. Implicit in this is the notion
that greater focus on the customers expectations is achieved. The benefits of
performance contracts rely upon the premise that as you moving to the right
along the contract continuum there is a progressive increase in the value
achieved.

Figure 2: The Contract Continuum

Maintenance contracts align with the three drivers shown as follows:
Input driven contracts; focus on what needs to be done and how.
They typically provide detailed methodologies for each task by the use
of method specifications and payment is made on the individual inputs
provided i.e. labour and plant by the hour and materials by quantity.
They rely the contract managers to decide the treatments and scope of
works required. Examples of input driven contract include in house-
service agreements with direct labour organisations
Output driven contracts; focus on when thing need to be done. They
generally comprise a schedule of items for each repair type. There is
some flexibility in repair method afforded to the contractor. Payment is
per item completed and may include some lump sum items that
combine cyclic and routine works items. A greater level of risk and
responsibility is passed to the contractor than in input driven contracts.
Contractors can usually carry out output driven contracts without the
need to hire in new technical skills. Conventional routine maintenance
contracts fall into this category.

Inputs

Network
Level
Outcomes

High
Level
Outcomes

Customer

Satisfaction

Outputs
What & How When Customer
Expectations
Outcome Driven
Output Driven
Input Driven
Page 5
End product specifications, fall somewhere between output and
outcome driven. These specifications are performance-based with the
time frame being end of construction with a 12-month liability period.
E.g. Transits P17 Resurfacing contracts
Outcome driven contracts; focus on the delivery of the outcome
desired by the customers and endeavour to align the contract
objectives with the higher level needs of the RA. They specify only the
desired outcomes and allow flexibility in methodology. Payment is by a
single lump sum, with an element of the payment at risk against
measured performance. Performance contracts are outcome driven
contracts. Examples include:
Transits PSMCs 01,02 for Auckland Harbour Bridge and
409km of predominantly rural state highway respectively
Transits hybrid contracts fall somewhere between an
outcome and an output-based contract.
Outcome driven contracts and performance contracts are synonymous. They
have developed generally as an evolution of existing contract arrangement
with the intent of providing better value than the current contracts and a
service more closely aligned with the outcomes desired.
Page 6

3 TERMINOLOGY
The definition of what constitutes a performance contract or specification
varies significantly between RAs and also from country to country. Defining a
common terminology for use by the industry is a key output from the study.
3.1 DEFINITIONS
The following definitions were proposed as means of fostering a common
understanding of what a performance contract is and were adopted for the
study:

Performance Specification: A specification that describes how the end
product should perform over time.

Example: The average roughness of the road shall always be below 75
NAASRA counts

A Performance Contract: defines obligations and outcomes in terms of
performance indicators for the product or service and is of sufficient duration
to enable measurement of the contractors performance over time.
Examples of current performance contracts are:
Design, Construct and Maintain Contracts (DCM)
Performance Specified Maintenance Contracts (PSMC)
Build, Own, Operate and Transfer (BOOT)
Design, Build, Finance and Operate (DBFO)
3.2 KEY ELEMENTS OF A PERFORMANCE CONTRACT

The key elements of a performance contract are:
A definition of the performance required (as opposed to the definition of
the method to be used or simply the end product at the completion of
construction)
A requirement that a defined performance shall prevail over a suitable
period of time
Page 7
The definitions are required as much to exclude what isnt a performance
contract as to identify what is. Terms such as performance based and
performance related have been specifically excluded.

4 THE TIME ELEMENT
4.1 PERFORMANCE OVER TIME
The definition of a performance contract requires the inclusion of a time
element. Many performance contracts endeavour to measure performance
using performance indicators that predict the expected performance, i.e.
measurements taken at the end of a maintenance period are used to infer the
expected life of the product. Design and construct contracts are examples of
this approach as are the performance based resurfacing contracts being used
by Transit New Zealand (P17) which use measures such as texture depth at
the end of a maintenance period to predict the life of the new surface.
4.2 CONTRACT DURATION
The definition of a performance contract includes the time element, however it
is deliberately silent on what the duration of the contract should be. Ideally, a
performance contract would have a duration that covers a significant
proportion of the expected life of the product. In practice this is difficult to
define, e.g. what is the expected life of a road network? Elements of it may
have a life cycle of 10 years or less; but the network as whole needs to
endure well beyond the 10 years and needs to be managed accordingly.
4.3 CHOICE OF DURATION
The choice of duration for performance contract is a key consideration. A 10-
year duration has been adopted for most current PSMCs. 10 years has been
chosen to generate the economic incentives that are the trade-off for the
obligation to meet performance objectives. It is long enough to allow
amortisation of risks and investments by the contractor allows:
Investment, in physical resources & an 'asset ownership' culture
The flexibility to design appropriate treatments,
Investment up-front to reduce ongoing costs,
10 years recognises the planning horizon of most RAs. It also limits the
potential changes in traffic volume, axle loading and weather events and the
consequent changes in asset condition that could be expected during the
contract period as a result of such changes
Page 8
The definitions given above are academic. The driver behind the adoption of
this form of contract is a desire to create efficiency gains and therefore obtain
better value for money. A performance contract is merely means to an end.
Debate over whether it is possible to measure performance is largely an
esoteric argument.
Page 9
5 POTENTIAL BENEFITS
5.1 WHAT IS A BENEFIT?
For any contracting strategy to be considered to have provided a benefit it
must result in the customer receiving:
The same level of service at a reduced cost or
A better level of service at the same cost or
A better level of service at a reduced cost

If one of these overall outcomes is not achieved there is no benefit.
By definition, the customer must receive the benefit, i.e. an improvement that
provides a benefit for one party in the supply chain but does not provide a
benefit to the customer is not a benefit. This is key point, as a number of the
claimed benefits of performance contracts are not directly beneficial to the
customer.
5.2 ARE THE BENEFITS UNIQUE TO PERFORMANCE CONTRACTS?
I ncrement al Benef i t s?
I nput
Dri ven
Out put
Dri ven
Out come
Dr i ven
B
e
n
e
f
i
t Own
Forces
Per f or mance
Cont ract
Convent i onal
cont ract s
?

The scale of benefits attributed to a performance contract has to be
established with reference to the contract arrangements (if any) prior to the
establishment of the contract. A benefit of a performance contract is the
incremental benefit that has accrued as the result of moving to a
performance model over and above benefits that could have been achieved
through other contracting methods. An array of benefits has been attributed
to the exposure of services to competition. These have most often been
achieved by the use of competitive tendering. These benefits have to be
discounted before an assessment of the benefit of the performance contract
can be attempted.
Page 10

5.3 EXTENT OF EXPERIENCE
The majority of performance contracts have durations of 10 years. The two
longest running contracts have been in existence for 5 years, the remaining
contracts are all less than 3 years old. In the context of the term (duration) of
the contracts, experience is limited and consequently so is the evidence to
support the benefits claimed.

Contract 1995 1996 1997 1998 1999 2000
NSW RTA North Sydney

Virginia USA (5 + 5.5 yrs)

Tasmania Southern Road Network

PSMC01 (Auckland Harbour Bri dge)

PSCM02 (SH3 etc)

PSMC03 (Northland)

Main Roads Western Australia (6
contracts)

Table 1: Extent of Experience: Current Performance Specified Maintenance Contracts
5.4 INDUSTRY OPINION
As part of the study 3 workshops were run and were attended by 54 people
representing a range of interests within the industry including contractors,
consultants, road authorities and funding agencies. In addition a
questionnaire was sent out to a wide industry group. 19 replies were
received. Whilst this is statistically a small sample the respondents almost all
had significant experience with performance contracts. Their views are
consequently worthy of comment.

In the questionnaire we posed the question in relation to the potential benefits,
Are the claimed benefits achievable? The responses are summarised in
Figure 3.

Page 11

Figure 3: Industry Questionnaire Response: Are Potential Benefits Achievable?

There was strong agreement amongst the respondents that the benefits
claimed could be achieved. The questionnaire went on to ask are these
benefits quantifiable? The responses received are summarised in figure 4.
Figure 4: Potential Benefits: Can They Be Quantified?
Potential Benefits: Can they be achieved?
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
B
e
tte
r V
a
lu
e
fo
r M
o
n
e
y
R
e
d
u
c
e
d
A
d
m
in
is
tra
tio
n
R
e
d
u
c
e
d
A
g
e
n
c
y
R
is
k
B
u
d
g
e
ta
ry
C
e
rta
in
ty
In
c
re
a
s
e
d
In
n
o
v
a
tio
n
Im
p
ro
v
e
d
U
tilis
a
tio
n
o
f In
d
u
s
try
S
k
ills
B
e
tte
r
C
u
s
to
m
e
r
F
o
c
u
s
B
e
tte
r D
e
fin
itio
n
o
f B
e
tte
r S
e
rv
ic
e
R
e
q
u
ire
d
B
e
tte
r U
n
d
e
s
ta
n
d
in
g
o
f P
e
rfo
rm
a
n
c
e
b
y
R
o
a
d
M
a
n
a
g
e
rs
C
le
a
re
r A
c
c
o
u
n
ta
b
ility
b
e
tw
e
e
n
S
ta
k
e
h
o
ld
e
rs
In
c
re
a
s
e
d

O
w
n
e
rs
h
ip
P
e
r
c
e
n
t
a
g
e

A
g
r
e
e
Industry
Road Authority
Both
Potential Benefits: Are they quantifiable?
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
B
e
tte
r V
a
lu
e
fo
r M
o
n
e
y
R
e
d
u
c
e
d

A
d
m
in
is
tra
tio
n
R
e
d
u
c
e
d
A
g
e
n
c
y
R
is
k
B
u
d
g
e
ta
ry

C
e
rta
in
ty
In
c
re
a
s
e
d
In
n
o
v
a
tio
n
Im
p
ro
v
e
d
U
tilis
a
tio
n
o
f In
d
u
s
try
S
k
ills
B
e
tte
r C
u
s
to
m
e
r F
o
c
u
s
B
e
tte
r D
e
fin
itio
n
o
f B
e
tte
r S
e
rv
ic
e
R
e
q
u
ire
d
B
e
tte
r U
n
d
e
s
ta
n
d
in
g
o
f P
e
rfo
rm
a
n
c
e
b
y
R
o
a
d
M
a
n
a
g
e
rs
C
le
a
re
r A
c
c
o
u
n
ta
b
ility
b
e
tw
e
e
n
S
ta
k
e
h
o
ld
e
rs
In
c
re
a
s
e
d
O
w
n
e
rs
h
ip
P
e
r
c
e
n
t
a
g
e

A
g
r
e
e
Industry
Road Authority
Both
Page 12

Clearly there is less confidence that the benefits can be quantified than there
is that they can be achieved. However it is worth noting that the greatest
agreement on this question is with the benefits of better value for money and
better definition of the level of service required. Most road authorities would
be happy to achieve these two benefits alone and would consider other
potential benefits to be subsidiary to them.

There was discussion within the workshops are to whether these benefits
could only be achieved by the use of performance contacts. This resulted in
the production of the following table. A indicates general agreement. A ?
indicates a mixture of opinion between the workshop attendees.

Potential Benefit
Achieved only
through a
Performance
Contract?
Easier through
a Performance
Contract?
Aligned with Road User Requirements
Ownership by Contractor (?)
Better Value ?
Better Knowledge of Asset ?
Risk transfer to those who can manage it best

Encouraging move to Pavement deterioration
modelling

Intervene at optimal time

Longer term planning

Encourages innovation

Price certainty for level of service

Up-skilling staff

Greater uniformity of measures

Better programming / flexibility
Table 2: Workshop Output: Comparative Ease of Achieving Benefits

Interestingly the workshop attendees were divided on whether the two key
benefits (better value and better knowledge of the asset) could be more easily
achieved using a performance contract compared to a traditional contract.
This I belief reflects a generally mixed opinion amongst the industry as to the
credibility of the benefits claimed for performance contracts.

The following section comments upon the specific benefits that have been
claimed by various parties as attributable to the adoption of performance
contracts and in particular relates to the use of performance specified
maintenance contracts (PSMC).
Page 13

6 SPECIFIC POTENTIAL BENEFITS
6.1 VALUE FOR MONEY
Performance contracts are purported to provide better value for money by
providing the same, or better, level of service at reduced price. Better value
for money is ostensibly easy to quantify. Every contract let to date in Australia
and New Zealand has resulted in a tendered price that has been assessed as
a significant saving to the RA letting the contract. Figures for savings
compared with traditional contracts range from approx. 38% quoted for the
latest TRANSIT PSMC to 15% in Virginia in the USA as shown in Table 2
below.
The context of the contract arrangements preceding a change to a
performance contract is important. The process of contracting out and the
exposure of services to competition have led in most cases to significant cost
reductions. When the contracting out process is combined with a move to the
use of a performance contract model it can be reasonably assumed that some
of the economic benefit has been achieved through the process of
competition.
The maturity of the contracting environment is another important
consideration. Where contracting out has been repeated over a sustained
period there has generally been successive cost reductions as contractors
discover more cost effective methods of delivering the services required.
Where performance contracts have been preceded by a short, 3 years or less
pilot contracting initiative, it could be argued that part of the saving could have
been achieved by simply re-tendering on the basis of a conventional
contracting arrangement. This is not the case with the New Zealand contracts
listed in Table 1, where the preceding contract arrangement was a succession
of conventional contracts over a period of over 10 years.

In terms of initial reduction of cost the evidence at present would appear to be
compelling.

Page 14

Contract:
(Award Date)
Quoted Saving
(Source of information)
Preceding Contract Arrangement
NSW RTA
Northern Sydney
(1996)
35%
(Improving Quality and
Cutting Cost through
performance contracts
M Frost 1996)
2-year pilot trial contracts split
between private contractor and
in house resources.
Virginia USA
(1997)
15%
(Review of VDOTs
Administration of the
Interstate Asset
Management Contract
Dec. 2000)
NB this reports
questions the validity of
these savings
In house resources.
NB this contract resulted from
an unsolicited offer by the
contractor.
Tasmania
(July 1998)
20% ($20m)
(MRWA Position Paper
Ten Year Contracting
Strategy Road
Maintenance - Aug
1998)
1 no. 3 year conventional
maintenance contract
Western Australia
(1999-2000)
Contracts range of
savings 15 to 35 %
(Outsourcing Road
Design, Construction
and Maintenance in WA,
- G Martin, 2000)
Mixture of 3 year conventional
contracts and in house
resources
New Zealand: PSMC01, 02 &
03

(1999 2001)
15% to 38%
(TRANSIT press release
Transit to let 4 new
10 year maintenance
contracts following
success of the first
May 2001)

PSMC01, 02 Conventional
maintenance contracts for the
last 10 years re-tendered every
3 years.
PSMC03 Traditional
maintenance contracts, followed
by 18month hybrid contract
leading into the PSMC
Table 2: Quoted Savings of PSMCs

It is too early to judge whether these contracts are providing better value for
money, it is clear that they almost always lead to tendered prices, which are
less than the estimated costs of using other procurement methods. The acid
test will be to monitor long-term performance and determine whether:
The level of service provided meet long term expectations
The tendered price remains intact over the contract period
Page 15
Most of the current contracts are for 10 years and only 2 of them are older
than 3 years. It is therefore early days for these contracts. These apparent
savings will only be able to be confirmed once there has been time to ratify
that the specified performance measures deliver the level of service expected
over the contract period. In general terms, experience is mixed although
clearly positive from a Transit point of view in New Zealand.
There is concern held by some that over the period of the contract the asset
will be consumed, i.e. the condition of the asset at the end of the contract will
not be of equal or better value than it was at the start of the contract. The fact
that it may take 10 years or more to draw definitive conclusions regarding the
cost savings that might be derived from performance contracts is significant.
The contracts deal with assets that may have life spans of 20 to 40 years on
which changes in condition happen slowly. Currently, there is consequently
little objective data from which to assess the whole of life cost benefits of
these contracts. As a result, there is considerable debate within the industry
over how real these cost savings are. In particular there is the question of the
duration of the contract in comparison to the deterioration life cycle of the
pavement, i.e. will the deterioration of the network within the contract period
be masked by the short contract period. In other words will the contracts
effectively mine the condition of the asset over the contract duration creating a
backlog to be addressed by the following contract?
The key question is whether the same or better value is being delivered, not
simply is the cost reduced. The definition of a level of service for maintenance
of a road network is complex. How sure can an authority be that the key
performance measures given in the contract adequately represent the service
required by the road users? Even if the measures are correct can they be
accurately measured?
The successful contractors have long-term revenue streams. This provides
them with the potential to make long term investment decisions for both
research and development (R&D) and training. This is a clear benefit to them
in comparison to shorter contracts, but it prompts the question of whether the
potential economic benefit is a result of the longer duration, or the form of
contract.

There is to date insufficient substantive evidence to state categorically
that the initial savings are a true reflection of increased value for
money. It is, however, similarly impossible to state that this is not the
case.
Page 16

6.2 BETTER CUSTOMER FOCUS
Supporters of performance contracts assert that the process of defining
performance measures has led to a greater understanding of customer needs.
This is feasible and could be quantified by simply asking the customers. A
customer satisfaction measure is, however, conspicuous by its absence from
current performance contracts and it is not clear whether RAs in establishing
their performance measures have involved their customers in determining the
target levels.
It is true that customer satisfaction is dependent upon an array of issues;
many of which are external to the contract scope e.g. progress with desirable
improvements. As the customer is the ultimate arbiter of the quality of service
provided, this is an important consideration. The technically oriented will look
to correlate the cost with the technical interpretation of the customers needs
but this may miss the point in terms of customer satisfaction.
RAs are increasingly focusing on the needs of their Customers. The
translation of these needs into robust performance measures (KPIs), which
match the performance of the asset being created or maintained, is a
considerable challenge. Historically, many of these needs have been implicit
with little documentary evidence of their existence and, more importantly there
has been no common understanding of customer needs amongst the parties
delivering them. Who determines the level of service/performance required is
a crucial consideration.
The adoption of performance contracts has led to attempts to define perceived
customer needs in term of measures that can be used in a contract. It does
not necessarily follow, however, that these measures accurately reflect
customers needs. This can only be determined by reference to the customers
and current contracts do not explicitly address this.

Performance contracts by their nature impose a discipline on all
involved, which facilitates greater communication and discussion of
customer needs. A performance contract is not required for this to
happen but has in many instances provided the catalyst for a greater
consideration than ever before.

Page 17

6.3 IMPROVED RISK RECOGNITION, ALLOCATION AND MANAGEMENT
Performance contracts transfer significant risks to the contractor that have
been traditionally been held by the RA. In this regard the most significant
additional risk transferred is that of the network condition. At the same time,
the RA takes on the risk that their performance measures may not reflect their
requirements.
Risk transfer does not necessarily mean better risk management. The
transfer of risk from the RA to the contractor only provides a benefit to the
customer if the contractor is better able to manage the risk
Current performance contracts tabulate the risks associated with the contract
and either allocates them to the parties or provides a framework for risk
sharing on a negotiated basis. There is thus a much more explicit recognition
of risk than in traditional contract arrangements. The benefit of this is that risk
may be managed better as a result of this greater focus. Assessing whether a
risk was better managed is, however, difficult and, debatably, could only be
quantified by a subjective assessment carried out if the risk eventuated.
Performance contracts offer the RA the opportunity of shedding or sharing a
variety of risks previously borne solely by the RA. Most current contract
offered options to the contractor in terms of how risks are allocated and
managed. For example the MRWA PSMCs invited contractors to propose the
costs associated with transfer of a table of identified risks from the RA to the
contractor. These figures then formed part of the contract negotiation with the
successful contractor. This method has the advantage of allowing the RA to
evaluate the cost effectiveness of reducing their risk exposure. Other
contracts have placed a cap on certain risks. For example` Transit New
Zealands PSMC01 transferred some weather risks to the contractor but,
placed a limit upon the extent of these i.e. land slips up to a specified size
were allocated to the contractor and had to be allowed for in the lump sum
price, but the cost of remedying larger slips remained with Transit.
There is an inherent difference in the manner in which contractors and RAs
manage risks. RAs are required to be careful if not necessarily conservative in
how they handle risk. In contrast, contractors are generally amenable to taking
on additional risks, provided they are accompanied by a commensurate
opportunity.

The notion that performance contracts in themselves force a better
management of risk is in my opinion at best unproven.
Page 18

6.4 REDUCED ADMINISTRATION
Customers are not interested in administration. A reduction in administration is
only a benefit if it assists in providing a better level of service or a reduction in
cost.
Traditional measure and value contracts can require double handling of
information between the parties involved (RA, consultant and contractor).
Performance contracts are generally single supplier arrangements that negate
the need for a portion of this work. In addition, performance contracts
advocate a hands off approach to contract administration. The input required
from supervisory staff and contract administration staff is reduced due to the
change of role from day to day surveillance to the measurement of the
specified performance outcomes.
Little direct evidence is available to quantify this except for the staff reduction
in some RAs following the change in contract arrangements. Some of this
administration is, however, passed onto the contractor.
In instances where the previous contracting arrangement included 3 parties,
(the RA, consultant and contractor), the consultants role gets absorbed into
the contractors. There is at least one less contract to administer and it is
logical to assume that whilst the contractor will take on some of the functions
previously undertaken by the consultant, there will be an efficiency gain and
thus some reduction in the administration.
Anecdotal evidence suggests that although performance contracts lead to a
reduction in the number of supervisory staff required to administer the
contract, there is a need for a higher level of skill and experience in the staff
involved.
For some RA staff the change in role is challenging as it creates a feeling of
loss of control. For example the decision making on treatment selection
transfers to the contractor and the RA no longer controls directly what
happens on the road.

Reduced administration is an internal benefit. It only provides a
benefit to the customer if it is significant enough to provide better value
for money. The scale of savings quoted for these contracts are
significantly greater than could be achieved simply by reducing
administration. Reduced administration is not an outcome and should
not be a major consideration in evaluation of the benefits.
Page 19

6.5 CERTAINTY OF EXPENDITURE
Most performance contracts are tendered on a single lump sum basis. The
lump sum is then subject to price adjustments for performance achieved as
measured by KPIs, changes to the asset etc. The potential benefit is that this
approach reduces the risk of cost over-runs and budget blow- outs.
Budgetary certainty is an issue that is largely internal to the supply chain. It
may not in fact be desirable to the RA as it could lead to a global inflexibility of
expenditure. The customer is interested in value for money, rather than
certainty of expenditure. It is therefore debatable whether this is a benefit at
all.
Greater certainty of expenditure is feasible and is relatively easy to measure
through a simple comparison of tendered cost against out-turn cost. In many
respects a measurement of this may be useful to demonstrate whether the
contract has delivered the savings indicated at the time of tender. If the price
increases during the contract through variations careful consideration will
need to be given as to whether these have eaten into the savings declared
at the commencement of the contract
The downside of being committed to lump sum payment for 10 years is
covered in the limitation section that follows. Many authorities, particularly
local authorities that may be managing other assets as well as roads, prefer
the ability to manage their budget globally. They may, for example, wish to
reduce road expenditure in one year to allow funds to be allocated to utility
upgrades and then catch up the deferred road maintenance in the subsequent
years. Fixed lump sum performance contracts reduce this flexibility.

Certainty of expenditure is only a benefit if it is something that the road
authority desires. Even then it only benefits the road authority rather
than necessarily the customer. International experience includes
examples in South America where the prime motivation for moving to a
long term contract was to secure future funding by making it
contractually difficult for funds to be diverted to other causes.
Page 20

6.6 PROMOTE INNOVATION
There are two key considerations in determining if the customer benefits from
increased innovation through the adoption of a performance contract
Does a performance contract provide greater incentive to a contractor
to innovate?
If so, does the benefit get to the customer?
As previously described, the guaranteed long-term revenue streams provided
by PSMCs might assist contractors fund the development of innovations that
improve effectiveness and hence reduce the cost of their operations. It is
highly likely that in tendering contractors will anticipate some efficiency gains
during the contract and reduce their price accordingly. The current speed of
technological advancement means that it is likely that during the life of the
contract improved techniques will be available to the contractor. The longer
term means that the contractor has a much greater incentive to invest in the
development of new products and techniques.
The negative aspect of this is that innovations may be carried out on a
commercial-i n-confidence basis that may lead to duplication of effort and loss
of industry wide benefits. The cost savings from innovation may not be
passed on to the road user. Some contracts have endeavoured to address the
potential loss of innovation, due to contractors retaining the benefits for their
own commercial gain, by requiring compliance with their existing technical
specifications as a starting point for the contract. The contract then allows
negotiation of changes to these if they are seen as an improvement. This
allows the RA the opportunity to share in the benefit of innovations developed
and implemented under the contract.

Quantifying innovation is difficult in any context performance contracts
are no exception. We should no confuse true innovation from efficiency
gains. True innovation is generated by the forward thinkers in the
industry and often need the support of RAs to come to fruition. I do not
believe that performance contracts significantly increase the ability of
the industry to innovate.
Page 21

6.7 IMPROVED INDUSTRY SKILL BASE
If the result of the implementation of performance contracts is an overall
increase in the skill level in the industry, there will ultimately be a benefit to the
customer through more effective and efficient services. It is debatable,
however, whether there is an overall increase is skills, or simply redistribution
within the industry? I.e. have technical staff who previously worked for the RA
or consultants, moved to a contracting organisation?
The shift of some RAs to more asset management focus has resulted in
increased specialist technical knowledge in the contracting industry. There is
clear evidence of this where some large contracting firms have recruited
technical expertise and are actively targeting a greater scope of works i.e.
many are looking to offer a comprehensive service that incorporates both
design and construction. Whilst this trend is easy to observe and could be
quantified, it would not give any true indication of whether there had been an
overall increase in the skill levels in the industry.

Whilst it is clear that some skills have simply transferred from one party
to another, performance contracts have created an increased focus on
performance measurement, risk management and deterioration
modelling/prediction. Enhanced skills in these areas have almost
certainly been created as a result of being involved in the tendering and
running of PSMCs.
Page 22
7 THE DOWNSIDE: RISKS AND CONSTRAINTS
To obtain a balanced view of the potential benefits the potential risks and
constraint associated with the adoption of a performance contract must also
be evaluated. There are some significant constraints limiting the wide-scale
adoption of performance contracts
7.1 INAPPROPRIATE PERFORMANCE MEASURES
The measurement of performance is a complex and challenging area and
whilst the benefits noted above are possible without robust performance
measures which truly reflect the performance required these benefits may
either not be realised or be overstated i.e. the desired outcome may not be
achieved even if the KPIs are met.
If the base asset data is inaccurate, or not repeatable, or there is a lack of
historic data this will have a big impact on the ability of the tenderers to
produce maintenance programs that will be correct in the future years. Hence
the risks of asset consumption or future contractual dispute over the
performance measures are increased.
The development of key performance indicators is a complex and challenging
task. In many respects this concept is in its infancy and it is likely that
considerable knowledge will be gained over the next 5 to 10 years. This
knowledge may allow this risk to be reduced significantly.
Even if appropriate measures can be determined is it possible to measure
these to sufficient accuracy and repeatability? This is particularly pertinent to
the expected life of a pavement and hence the inability to measure asset
value and remaining life.

7.2 REDUCED COMPETITION
The majority of performance contracts let to date have been large contracts in
excess of $100m. This is to ensure that there is sufficient economy of scale, to
provide financial incentive to contractors to offset the high initial tendering
costs and to counteract the potential downside of taking on additional risk (in
particular the network condition risk). The downside to this is that the scale of
the contracts and the high tendering cost is such that it effectively precludes
the smaller contractors from tendering and it will become increasingly difficult
for them to stay in the market.
In addition, the length of the contracts means the size of the market is
reduced for the duration of the contract, thus placing greater pressure on the
small and medium sized contractors.
Competition is a key driver of efficiency and needs to be retained in any long-
term procurement strategy. The size of the market and the ability to retain a
Page 23
competitive market place are key considerations in deciding whether to take
the performance contract path. There will undoubtedly be instances whereby
the adoption of a single large contract would significantly reduce future
competition by locking up the majority of locally available work with one
contractor. This is particularly the case in networks linking small rural
communities.
7.3 THE COST OF TENDERING
Tendering is an involved and costly process requiring legal and financial
advice to support the business decision in addition to the detailed technical
analysis required to enable a price to be determined. This limits the parties
able to tender for the contracts and raises the question of who should bear
these costs. The preparation and tendering of the contract is also a complex
and costly process for the RA. It generally involves not only producing a set
of a typical contract documents but also compiling and verifying asset
inventory and condition data and calculating programmes and estimates to be
used in the tender evaluation.
7.4 INCREASED SIGNIFICANCE OF POOR CONTRACTORS PERFORMANCE
The possibility of poor contractor performance exists in all contracts. As with
other types of contract remedies exist in performance contracts to address
poor performance. The consequential effects of prematurely terminating a
performance contract are, however, potentially more difficult to handle than for
a traditional contract. E.g. the lead in time and cost of re-tendering a
performance contact are particularly significant.
A similar situation would exist if a contractor were to go into liquidation. It
could be argued that the transfer of significantly greater risk onto the
contractor increases the possibility of this occurring, especially if the
contractor is not able to quantify the potential costs of these risks and include
adequate provision for them in their bid price.
All contracts contain the risk that a contractor may not perform to the standard
required. With PSMCs the net effect of poor contractor performance is
potentially greater, especially if the extent of the inadequacy is sufficient for
the RA to contemplate terminating the contract. Termination of such a large
contract would always be a last resort, but it is consequently particularly
important that the contracts contain appropriate provisions to deal with poor
performance.
7.5 REDUCED ABILITY TO DEAL WITH CHANGE
During the life of a performance contract many factors affecting the contract
could change. These could be physical, political or environmental. The lump
sum nature of performance contracts means that the ability to deal with such
changes is potentially reduced by being locked in to outdated contract
Page 24
provisions. It is always possible deal with variations but they invariably result
in increased costs to the RA and the lack of contract rates may make
negotiation of the variation more difficult.
7.6 REDUCED TECHNICAL CAPABILITY IN RAS
The transfer of responsibility for design and technical specification means that
RAs will require a reduced level of technical expertise. There is a risk that if
this downsizing in key technology areas is taken to an extreme level, then that
RA could loose the ability to be an informed purchaser. In addition, if RAs do
not retain sufficient technical skills to keep pace with developing technologies
they risk being unable to take advantage of emerging technologies in future
contracts. This is more of an item to be aware of than a significant risk. The
developing role of RAs as procurement specialists still requires technical skills
and in many respects requires a higher level of technical skills focused on
asset management rather than day-to-day design and supervision.
7.7 LOSS OF CONTROL
A common concern amongst RA staff about performance contracts is the loss
of control of the network. Ownership passes from the RA to the contractor
who has the greatest say in what gets done on the network. The perceived
risk is that this loss of control will lead to a reduction of the standard of service
provided. The risk that is reflected by these concerns is really the risk that
performance measures are inadequately defined. It is immaterial who
controls or owns the network as long as an adequate level of service is
provided.
7.8 LOSS OF INNOVATION TO THE PUBLIC DOMAIN
Whilst one of the purported potential benefit is to encourage innovation, the
availability of innovation to the wider market place wi ll be restricted if they are
locked in to one supplier. This is highly dependent upon how this aspect is
dealt with in each contract. The risk clearly exists, but the real questions are
how significant a risk is it and is it any different to what happens under normal
contract arrangements?
The risk is predicated on the assumption that contractor developed
innovation in some way replaces RA funded research and development. If
RAs retain R&D then any innovations coming out of performance contracts
can be considered a bonus. Comparison to RA instigated R&D is not valid if
RA R&D continues.
It is likely that groundbreaking innovations will continue to come from a variety
of sources. Whilst the performance contracts may provide additional incentive
to investigate new methods and products the risk of these being lost to the
public domain is no greater under a performance model arrangement that
under a conventional contract arrangement.
Page 25
7.9 LOSS OF FINANCIAL FLEXIBILITY (RA).
As described above lump sum contracts can provide greater certainty of
expenditure, but this assumes that this is desirable. Not all authorities wish to
be locked in to a level of expenditure for a period as long as 10 years. For
example, many local authorities manage their funds more actively than this
and wish to retain the ability to defer expenditure from year to year in order to
balance the books and seek community input into levels of expenditure and
service. Whilst performance contracts do not preclude varying the specified
level of service and re-negotiation of the total expenditure it is more difficult
than under a traditional contract model.
The ability to vary budgets on a year-by-year basis is restricted by PSMCs.
Whether this perceived increase in price certainty is desirable or not will
depend upon the wishes of the particular RA and may be a significant
consideration is deciding whether or not to pursue the performance contract
route.
7.10 THE ABILITY TO JUSTIFY THE BENEFITS
The potential benefits but also comments upon the difficulty of quantifying
these. Currently there appears to be minimal objective data available for use
in documenting the perceived benefits. Whilst this doubt exists it will always
present an obstacle to the wide scale adoption of performance contracts.
7.11 LACK OF EXPERIENCE OR PREPAREDNESS (IN BOTH THE INDUSTRY
AND RAS).
Performance contracts demand different skills and a different approach to that
taken traditionally by contractors. If there were to be wide-scale adoption of
performance contracts contractors would require sufficient warning to allow
them to up-skill in some key areas particularly asset management. The RAs
require a similar shift in skills with the supervision role undertaken by the RAs
representative changing to one of observation/auditing requiring the
acquisition of new skills. The limitation of the industry in terms of size and
ability can thus be an obstacle to the wide-scale adoption of performance
contracts.
The contracts themselves are large, generally over $10m for PSMCs and over
$100m for DCMs. The scale and complexity of the contracts means that in
order to provide a suitable team to attack the contracts contractors may wish
to form joint ventures or associations with other contractors or consultants.
Experience to date shows a mixture of large companies and joint ventures has
been successful to date in acquiring these contracts.
7.12 LACK OF ROBUST ASSET DATA
Performance contracts rely heavily upon the use of complete, accurate and up
to date asset data. Whilst most RAs will have asset inventory and condition
Page 26
databases, the robustness of this data may never have been tested in
earnest. If asset data is incomplete, or insufficient to allow the contractor to
make reasonable pricing assumptions then significant risk to the contract is
introduced.
7.13 ABILITY TO FORECAST FUTURE CONDITION
PSMC contracts use forecasts of future condition to determine the works
required to maintain asset condition. For the contractor this means using a
forecast as a means of determining a tender price. For the RA the forecast
will be used to determine a cost estimate and to assist with the evaluation of
tenders. The ability to forecast future condition is a crucial item in the success
of the contract. A range of views exists within the industry over the accuracy
and reliability of the tools used currently to predict future conditions. This is
particularly the case with the residual life of the pavements. Without a good
estimate of the effect on residual life the long-term (whole of life) benefits are
difficult to assess. Confidence levels in deterioration models therefore
represent a constraint to the wide scale adoption of performance contracts.
7.14 REGIONAL DEVELOPMENT AND EMPLOYMENT
In many rural areas road maintenance activities provide vital employment that
assists in sustaining the viability of small communities. The efficiency gains
that large region wide performance contracts create place pressures on this
employment. That is contractors may not retain staff in all the communities
that currently employ a small road maintenance work force. There is
consequently a potential social cost to rural communities of implementing a
performance contract, which may override the economic arguments.
Organisational politics within the RA may provide another constraint. The
threat of job losses in the RA and the erosion of traditionally held roles and
responsibilities may produce a resistance to change that makes it difficult to
make the changes required to implement a performance contract.
Page 27
8 SUMMARY & CONCLUSIONS
8.1 POTENTIAL BENEFITS
Significant benefits have been claimed to result from the implementation of
performance contracts. The most significant of which is a substantial cost
savings/increase in value for money. Other associated benefits claimed
include a better identification and management of risk, greater focus on the
needs of the customer and greater incentives for innovation from suppliers
(contractors). Whilst significant benefits have been claimed for all the
performance contracts let to date the majority of current performance
contracts are less than 3 years old. It is accepted by most of the parties
involved that it is too early to determine the true nature and extent of the
benefits derived from these contracts.
It is also difficult to accurately quantify many of the claimed. Even the benefit
that is ostensibly the easiest to measure i.e. cost savings has significant
difficulties associated with it e.g. against what benchmark should savings be
assessed and how sure can we be that the asset is being maintained. (Asset
value and remaining life are difficult to measure). So, how real are these
benefits? It is certainly true that costs are reduced but is sufficient effort being
put into the development, testing and publishing of performance measures
that can be benchmarked between networks and between procurement
models?
8.2 DOWNSIDE: RISKS AND CONSTRAINTS

Is it possible to accurately define the level of performance required and
if so is it possible to measure it? Current measures are based upon that
which we know we can measure. Whether or not these measures truly reflect
the desired outcome has never been tested in earnest. Similarly the
sensitivity of these measures to relative inactivity is untried i.e. would some of
the targets be met even if no work was carried out due to the slow rate of
deterioration thus creating a backlog of deferred maintenance for some future
contract to deal with?
Are Road Authorities able to relate the level of service they provide to
the requirements of their customers? There appears to be a reluctance to
involve the customers view at the level at which these contracts operate. It
will be interesting to see how any local authorities that take up the model deal
with this, as they by necessity keep closer contact with the road users than
other road authorities.
Could the claimed benefits of performance contracts be achieved by
using another model? If the objectives were as clearly stated, as
performance contracts require them to be? . Massive cost reductions are not
Page 28
generally achieved simply through efficiency gains. The reduction achieved
by not doing a treatment is much greater than that which can be achieved by
doing it more efficiently. What would be the effect on the network condition if
the road authority were to gratuitously cut the funding to the level of expected
saving that would be achieved by a PSMC and simply instruct their suppliers
to prioritise?

Would the further expansion of the use of performance contracts
ultimately lead to a loss of competitive market due to the dominance of a
few large players? In New Zealand the roading a few large players already
dominate industry. This applies equally to consulting and contracting. Real
competition cannot in my opinion be maintained if new entrants are locked out
by the entry costs of large and technically challenging contracts.

8.3 CONCLUSION
Performance contracts have prompted thought and consideration of the
outcome of maintenance that had not occurred prior to their implementation.
This may ultimately be the most significant benefit they provide i.e. they
provide a catalyst for a much greater focus on delivering good outcomes and
hence ultimately better outcomes. The method of achieving these will always
be secondary. The present challenge is to provide robust methods of
measuring performance that can be used to benchmark performance between
networks and between procurement methods. Only when such
measurements have been tested will there be a significant increase in
confidence that the level of claimed benefits are real.
PSMC are not appropriate in all cases. They do however provide a viable
alternative to traditional procurement methods that should be considered
when determining the optimal procurement model for any network.

Austroad Review of Potential Benefit of PBC

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Austroad Review of Potential Benefit of PBC

Uploaded by

Copyright:

Available Formats

Austroads Review of Performance Contracts

Paul Hardy, Opus International Consultants

You might also like