W H I T E P A P E R

EXECUTIVE BRIEFING SERIES

Vol. 1, No. 3 February 2006

Identifying Software
Development Project Risks:
What You Don’t Know
Will Hurt You

1441 Branding Lane

Suite 240
Downers Grove, IL 60515
V: 630.434.8200
F: 630.434.8391
geneca.com

©2006 Geneca LLC

Identifying Software Development
Project Risks:
What You Don’t Know Will Hurt You

Table of Contents:

Introduction........................................................................1

Typical Software Risks ....................................................2

Technical Risks ................................................................3
Process Risks ..................................................................4
Engagement Risks ..........................................................5
Organizational Risks........................................................6
Identifying Software Development Project Risks ....8
Conclusion ........................................................................9
About the Author..............................................................9
Introduction

ccording to the Airlie Software Council — a group composed of the nation's

A most distinguished Software Engineering experts — risk management is the

most important best practice for software developers to master.

While most software practitioners have a healthy paranoia about the importance of
early identification and management of project risks, identifying risks in the develop-
ment process might not be as clear cut as you think.

Although the majority of technology people believe that most projects fail because
of either technology or process, we have found just the opposite to be true:
Technology and process risks are, in fact, easy to mitigate. The risks capable of
killing your project are the organizational and engagement risks. Why? Because
these risks have more to do with ingrained behavioral and culture issues rather than
mechanical ones.

Identifying project risks is the first step in the development of a healthy risk manage-
ment program – or the systematic identification and elimination of risks that could
cause a project's ROI to be compromised. This paper represents battle-tested think-
ing on risk identification and includes a comprehensive framework to check against.

1
“If You’re Not Managing Risk,
You’re Not Managing”
oftware engineers are known to be eternal optimists: We either assume

S everything will go according to plan or we maintain that the creative nature

of software development makes predictability too difficult to achieve. Both of
these attitudes lead to unexpected events that have the potential to throw your
project off track.

Risk management is becoming recognized as a best practice in the software indus-

try for reducing the surprise factor. While we can never predict the future with
certainty, we can apply structured processes to contain risk by managing concerns
before they become crises.

Symptoms that an organization is not successfully practicing risk management

include a persistent state of project instability, frequent fire-fighting, multiple
schedule slippages because of recurring surprise factors, and continuously
operating in crisis management mode. Formal risk management greatly improves the
likelihood of successful project completion while reducing the negative conse-
quences of the risks that cannot be avoided.

Typical Software
Risks
Typical Software Risks
The list of things that can befall a software project is miserably long. The enlight-
ened project manager will begin to build extensive lists of potential risks from the
get go to help the team deal with as many concerns as early as possible in the
planning process.

Typically, we look for four categories of risks. Most projects include risks from
each of these categories:
1. Technical risks involve the technology being used on the project. These risks
include working on unproven software, difficult technical objectives (performance
goals), and solutions to technical design problems that are not known.
2. Process risks involve the processes used to accomplish project tasks. These
include processes not being followed, inconsistent processes, lack of process for
a particular task, and processes that may have been omitted (example: testing,
project management, risk analysis)
3. Engagement risks cover general engagement issues such as client and vendor
behavior, aggressive time frames, and insufficient funding.
4. Organizational risks cover the structural organization of the development team.
Risks include unproductive resources, staff turnover, ineffective coordination
between multiple teams and companies, project personalities, and running remote
projects.

The key to managing risks is to understand where to look for potential risks, develop
contingency plans for these risks, and build enough time into your project schedule
to mitigate risks that you do not know about.
2
Technical Risks

Type of Risk Difficulty of Identification Difficulty of Mitigation

Technical Easy Easy

Process Easy Moderate
Engagement Moderate Moderate
Organizational Difficult Difficult

Technical Risks
Our checklists are Technical risks involve the technology being used on the project.This is the category
living things and not that usually first comes to mind when people think of risks.
meant to be the
Technical risks include working on unproven software, difficult technical objectives
definitive list of all (performance goals), and solutions to technical design problems that are not known.
risks. We revise it These risks also involve the processes used to accomplish these tasks and include
constantly, adding processes not being followed, inconsistent processes, lack of process for a particu-
new risks as they lar task, and processes that may have been omitted (Examples: Testing, project
management, risk analysis).
crop up on our
projects. Maintain Like process risks, technology risks are usually not fatal to a project for a couple of
your own inventory reasons. First, technical risks are for the most easy to identify. You pretty much know
of risks and expand when you are working with hardware, software, or networking components that are
risky. Second, they are usually easy to mitigate. You can either build prototypes to
it over time. Revisit
resolve the risks or there are usually substitute products or solutions that you can
often. One of the use.
biggest traps you can
fall into is believing
Technical Risk Checklist:
in static lists. Technical risks cover hardware, software, and networking.
Unproven components
Complicated logic
Complex architecture
Features not feasible
Third party software
Re-usable components available
Technology match
Hardware Constraints
Performance factors
Unproven hardware
Access (for remote teams)
Networking Constraints
Capacity

3
Process Risks
Scope and Feature Creep
(So what else is new?)
Your client agrees to a
requirement for a Logon
page: The client enters their
user id/password; the pass-
word is validated and
allows entry upon success-
ful validation. Simple
enough. However, during a
meeting immediately prior
to coding, the client tells
your PM that they now
want a daily report that
shows how many people
log in each day. The client
figures that since we
already have the login
information, it will only take
a few minutes to automate
a report. Although this
sounds simple to the client,
it requires many different
things to happen. First, the
PM needs to amend the
requirement document. The
programmer needs to
understand the new
requirement. The testing
team must design new test
scenarios. The documenta-
tion team must now include
this report in the documen-
tation. And, finally, the user
acceptance team must plan
to test this.
Process Risks
Process risks are associated with the methods and procedures that we use when
we conduct a project. Process risks are slightly more deadly than technical risks so
we are moving up the deadliness scale.
The most famous risk here is scope creep and has to do with requirements. Many
projects face uncertainty and turmoil around the product’s requirements. While
some of this uncertainty is tolerable in the early stages, the threat to success
increases if such issues are not resolved as the project progresses. If we don’t con-
trol requirements-related risk factors, we might either build the wrong product, or
build the right product badly. Either situation results in unpleasant surprises and
unhappy customers.
Almost as common though, especially among small to medium size businesses, is
the lack of established and well defined QA procedures. Identifying risks here is
pretty straightforward. Just think of the major processes that need to be present for
the project to be a success. Then ask yourself if the processes are present and how
well defined they are.
Become familiar with established requirements gathering and project management
practices, and watch out for these risk factors:
Process Risk Checklist:
• Requirements • Project management approach
Lack of clear product vision Micro management
Lack of agreement on product Too hands-off
requirements Lack of experience
Unprioritized requirements Lack of organization skills
New market with uncertain needs Lack of leadership
New applications with uncertain Lack of authority
requirements
Rapidly changing requirements • Design
Ineffective requirements change Functionality
management process Difficulty
Inadequate impact analysis of Interfaces
requirements changes
Unclear project ownership and • Performance
decision making • Change Requests
Inadequate planning and task • QA
identification • Deployment
Inadequate visibility into actual • No-risk management process
project status
• Development
Feasibility
Suitability
Reliability
Deliverability
4
Engagement Risks

Engagement Risks
The risks that can really kill you are the organizational and engagement risks. Why?
Because these have to do more with behavioral and cultural issues rather than
mechanical issues that are straightforward to deal with. Unfortunately, although most
technology folks would rather not have to deal with organizational and engagement
issues, these are the ones with the highest impact and the most difficult to mitigate.
This makes for a deadly combination.

Engagement risks are associated with how the project is conducted. Examples of
an engagement risk include projects that are done under an extremely short time-
frame, projects under budget constraints, and the risk that probably gets the most
people riled up – expectations.

Engagement risks can also arise due to dependencies from outside firms or factors
– such as client-supplied items or subcontractor relationships. Because we cannot
usually control these external dependencies, mitigation strategies may involve con-
tingency plans to acquire a necessary component from an alternative source or
working closely with the source of the dependency to manage looming problems.

Engagement risks typically cause the most tension on a project because often times
they are the most visible of risks, especially to executives. As with the other risks, go
through the checklist and identify those most likely to occur:

Engagement Risk Checklist:

• Timelines
Aggressive timelines – "Death marches"
Relaxed timelines – "No sense of urgency"
Unrealistic commitments made – often for the wrong reasons
Unrealistic expectations from managers and clients

• Cost
Total project budget
Contractual arrangements
Mid-project course changes

• Scope
Are we solving the right problem?
What specifically will we build?
What are the overall project goals? Is everyone behind them?
Project necessity and importance
Clarity of objectives
Clarity of requirements
Conflicting goals

• Expectations
Expectations set inappropriately
Failing to manage expectations

5
Organizational Risks

Organizational Risks
So what are organizational risks? Think people. Anything that has to do with people:
Skill sets, personalities, location, and environment go into the organizational risks
section.

In thinking about organizational risks, we are reminded of an old Tootise Pop com-
mercial where a boy is trying to find out how many licks it takes to get to the center
of a tootsie pop. He goes to the wise old owl who licks the tootsie pop three times
and tells the boy its takes three licks. In our version, we ask how many high mainte-
nance people can a project stand? The answer, from the wise old owl, is three (or
less if you have a small project).

We recommend that you spend a lot of time thinking about organizational risks
since they can be deadly. They tend to be deadly because these risks tend to go
unseen until the last minute. There are two reasons for this:
• First, it can be hard to read interactions between people or the impact an
environment has on a team.
• Second, most technologists do not have expertise in organizational development
and tend to underestimate the impact these sort of issues have on a project.

There is one risk here that we want to call special attention to because we see it so
often. We call this a "Fault Line" risk and it particularly occurs when you have multi-
vendor teams. Faultline is a term from geology that refers to a crack in the earth's
crust. At a faultline, you usually have pressure and friction caused by the two sides
rubbing against each other. Teams are the same way. Sometimes teams fracture into
cliques that often rub against each other and build up pressure.

Another organizational risk is lack of knowledge. The rapid rate of change of soft-
ware technologies, and the increasing shortage of skilled staff, mean that our
project teams may not have the skills we need to be successful. The key is to recog-
nize the risk areas early enough so that appropriate preventive actions — such as
obtaining training, hiring consultants, and bringing the right people together on the
project team – can be taken in a timely matter.

Although organizational shortcomings impede the success of many projects, don’t

be surprised if your risk management plan doesn’t list very many of these. After all,
the project manager is usually the person who is writing the risk management plan,
and most people don’t wish to air their own weaknesses (assuming they even
recognize them) in public.

6
Organizational Risks -con’t.
The organizational risks listed below are, in our experience, among the
deadliest of all risks.
Organizational Risk Checklist:
• Risks emanating from individuals
Technical skills and talents
Inadequate training
Poor understanding of
methods, tools, and techniques
Inadequate application domain
experience
Unfamiliar technologies or
development methods
Staff personality conflicts
Experience
Lack of team spirit
Personnel issues: Personality
conflicts; cooperation and
communications; skill mix and
experience; resource availability
Focus issues: Alignment; is
everyone pulling on the oars at
the same time?
Morale: Bataan Death March;
rats leaving the ship; getting on
the bandwagon
• Leadership abilities
Poor communication
Inability to compromise
• Productivity
Lack of/inconsistent
productivity
People on the critical path are
not the most productive
members
• Application area
Risks arising from team
dynamics
Fault lines and factions
• Cultural
Work ethic
Attitudes
• Contractual and compensation
Fixed price
Time and materials
None
• Departmental
Networking
Database
Application development
• Ideology
Split along ideological lines.
Examples include: Microsoft vs.
Open Source;
Mainframes vs. PCs; Java vs.
Visual Basic
• Environmental risks
Work Environment: Distributed
locations; office space; support
infrastructure
Business Environment:
Organizational stability; is the
organization stable or constantly
undergoing reorganizations?
• Executive involvement
What sort of commitment and
involvement come from the execu-
tives? Too little (this is the classic
lack of executive support) or too
much (this can lead to scope creep
or analysis paralysis)?
Nature: Supportive, antagonistic, or
neutral
Triggers: From project inception or
when things get closer to crunch
time
• Organizational politics
Vendors
Management
End Users
• Overall support 7
Identifying Software Development
Project Risks
Risks

Risk Description Impact Mitigation Plan

Technical
Multi-branded Application must support the ability to Low – • Expertise with CSS 2.0
multi-lingual website be multi-branded. being mitigated • Define extent of multi-branding
capabilities

UI-middleware Testing between the web application Medium – • Finish current phase and use as model of
interface and the middleware layers has taken being mitigated future development along with prototype
place with the existing UI. • Mitigate by extensive testing

Feasibility/complexity There may be issues with expanding High • Test transactions in prototype
of backend some of the backend transactions
transactions through the software interface or directly.

Environmental issues Issues with the IT infrastructure may
continue to arise from time to time.
These issues arose with the Citrix
development environment and the test
environment.
Hot • Monitor and resolve issues as they arise
during phase I
• Development
— Move development off of Citrix
— Move development offsite
• Testing
— Set up separate test environment
— Setup remote access to web server

Communication with Several questions remain regarding Low • Test communication paths using prototype
external services communication to imaging, bill payment, developed in phase I
could be an issue e-statements, and credit card systems. • Set up a contact with each one of the
vendors

Process

Regression testing Currently no automated testing Low for • Evaluate, select, and install automated
infrastructure is in place. Automated short term test tools
testing tools like Quick Test Pro should • Create automated test scripts to
be investigated and brought in. perform regression testing

Scope creep Expanding scope on a project of this Hot • Freeze requirements quickly
complexity coupled with an aggressive • Establish strict change control
timeframe must be limited at all costs. procedures to eliminate scope creep

Project coordination Issues are increasing at a rapid rate. Hot • Create, maintain, and distribute central
Not all team members are aware of the issues list
issues or their ongoing resolution and • Publish weekly status reports
status. Project communication and
coordination must be improved
immediately.

As illustrated in this example above, the key to successful risk management is to

identify all risks you know about and build time in for risks you do not know about.
The mechanics of doing this is to simply list the risks, and figure out the loss (in hours)
you expect would happen if the risk occurs, and then list the probability of the risk
happening. Then you can factor that risk into your project schedule by multiplying the
loss by the probability.

8
Conclusion

Conclusion
It is not a law of Risk management is becoming recognized as a best practice in the software
nature that software industry for reducing the surprise factor. While we can never predict the future with
certainty, we can apply structured processes to contain risk by managing concerns
projects will run late before they become crises.
or over budget. A
Risks must not be passively monitored. Rather, a given risk and its associated
careful program of exposure must be identified for the sole purpose of increasing the likelihood of
risk analysis and delivering a successful project by decreasing the probability and impact of the risk.
abatement can reduce
the probability of About the Author:
major problems. Kevin Mason, Geneca Client Partner
Kevin is a seasoned technology professional with over two decades of experience in
software development and architecture, risk management and mitigation, business
analysis, IT strategic planning, systems thinking, and project management.

Kevin created the role of Client Partner at Geneca to insure that all Geneca clients
experience the support, certainty and peace of mind they are promised throughout
the software development process. Prior to Geneca, Kevin worked for groundbreak-
ing firms such as SHL Systemhouse and AGENCY.COM. His former clients include
ABN Amro, First Chicago (now Bank One), State Farm, U.S Bank, Motorola,
McDonalds, Purina, and AT&T. Universal Card, Purina, 3M, Medtronic, and Compaq.

Kevin is an industry speaker and has presented at Internet Commerce expos and
management seminars on topics that include IT strategy, project solutions, technolo-
gy perspectives and Rational Unified Process. Additionally, he has written numerous
articles on risk management and web technologies.