Professional Documents
Culture Documents
Table of Contents
Table of Contents.......................................................................................................2
Introduction...............................................................................................................3
Executive Summary....................................................................................................3
Recommendation Summary.........................................................................................3
Evaluation of Alternatives.........................................................................................4
Evaluation Methodology..............................................................................................4
PPM Alternatives Descriptions......................................................................................4
Conclusion.................................................................................................................6
Appendix A................................................................................................................7
TOTAL COST OF OWNERSHIP....................................................................................7
Appendix B..............................................................................................................12
SURVEY OF FUNCTIONAL EVALUATION CRITERIA......................................................12
Appendix C..............................................................................................................16
FUNCTIONAL EVALUATION MATRIX..........................................................................16
Appendix D..............................................................................................................20
City Services Auditor Project Management Software Solution Evaluation Spring 2008 ...20
Appendix E..............................................................................................................27
VENDOR QUOTE INFORMATION ..............................................................................27
Page 2
PPM Alternatives Results & Recommendation Report
May 26, 2009
Introduction
This is an evaluation with recommendations report for the Office of the CIO, which had
indicated the need for a review of cost alternatives for a Project Portfolio Management (PPM)
software application. A PPM application assists DT Management to view, analyze and
collectively manage all current or proposed projects, or groups of projects, across the
organization. Our objective is to identify suitable applications with the optimal Total Cost of
Ownership (TCO), as well as the best functional fit with our business and technical
requirements.
Executive Summary
DT’s Project Management Office (PMO) currently is using CA-Clarity for DT’s PPM
application. It is the central repository of all projects for which DT is responsible, including
COIT projects. This is what we call our “Book of Business”. However, the high-cost of the
Clarity solution prompted DT to evaluate possible alternatives. To this end, Blair Adams,
DT’s Chief Technology Officer, and Chair of the COIT Performance Subcommittee, undertook
a project to accomplish this.
1. Project.Net
2. NetSuite - OpenAir
3. MS - Project Server
4. CA - Clarity
Note that this report does not preclude a more formal RFI/RFP process in the future.
Recommendation Summary
Based on the weighted scores from our evaluation matrix (Appendix C), our
recommendation is to test Project.Net, on a 90-day trial basis.
Page 3
PPM Alternatives Results & Recommendation Report
May 26, 2009
Evaluation of Alternatives
MS-Project NetSuite
Server CA-Clarity OpenAir Project. Net
501.47 567.13 697.73 736.20
Evaluation Methodology
We began by gathering the business needs of those in DT who are currently using Clarity
(PMO, COIT Staff, and PMs), as well as those in DT who are seeking similar functionality
(Contracts Section). From this a list of functional requirements was developed, and a
voluntary web survey was sent out to IT Managers across the City (see Appendix B), which
asked respondents to assign relative weights to each requirement. From the survey
responses a weighted functional evaluation matrix score sheet was derived (see Appendix
C).
Since it was important to consider all aspects of cost, a method to calculate Total Cost of
Ownership (TCO) was identified and adopted as well (see Appendix A). We made every
effort to obtain cost information that was as accurate as possible from vendors, as well as
from DT’s relevant known internal costs. When direct vendor cost quotations could not be
obtained in a timely manner, cost estimates were extrapolated from similar vendor sources.
Costs were examined for two modes of service: 1) On-Demand/Software-As-A-Service
(SaaS), which is remote hosting by the vendor; and, 2) On Premises/In-House hosting by
DT.
After preliminary discussions with the alternate vendors, each one was then asked to demo
their solution to PMO Director Joe Armenta and R&D Manager Jay Nath, and each
performance was rated against our functional evaluation matrix score sheet. Since we are
currently using Clarity, we were able to evaluate this ourselves. Finally, the Criteria Weights
derived from the survey were applied, and the final weighted scores were calculated
(Appendix C).
Page 4
PPM Alternatives Results & Recommendation Report
May 26, 2009
The project manager can communicate project plans and task assignments to team
members via home pages in Web Access. Team members can communicate status and
changes to keep the project manager up to date.
Resource workloads can be analyzed by project and by resource with the Resource Center,
allowing organizations to forecast future resource requirements and make more efficient use
of resources.
However, at this time Microsoft seems to be relying upon third-party vendors to provide
true project portfolio capability (dashboards, aggregate views, etc.) Also, since Windows
SharePoint seems to be a technical prerequisite for implementing Microsoft’s vision in this
arena, the true cost of implementation has the potential to be much higher.
CA Clarity PPM supports core project and portfolio management functionality. This
functionality provides the ability to unify PPM information and processes into a single system
of record. For individual projects, and groups of projects (portfolios), Clarity can track
resources, organizational structures, budget and financial information, issues, and individual
resource-task timekeeping. A Project Administrator can control security defining users and
access rights. Access to Clarity is via web browser.
Because Clarity has a limited innate project scheduling capability (CA offers its own open-
source product Open Workbench for this), it supports the importing of project schedules
from either MS-Project (with limitations), or CA Open Workbench (preferred).
Clarity also supports the attachment of external documents at the project and portfolio
levels, through its collaboration feature.
In our experience with using Clarity, there have been some difficulties with configurability
and ease-of-use.
NetSuite OpenAir is currently being piloted by the Controller’s Audit Division. The Controller
initially looked at CA-Clarity, and some other candidate software offerings (see APPENDIX
D), before selecting OpenAir for pilot. OpenAir is aimed at project-based businesses,
Page 5
PPM Alternatives Results & Recommendation Report
May 26, 2009
providing automated time and expense tracking, billing, reporting, and basic project
management and work breakdown structures.
While OpenAir provides most of the desired project portfolio functionality, some of the
desired graphics capability; such as, Gantt charting at the portfolio level, etc. are still “in the
works”, and are not yet available. Nevertheless, OpenAir provides strong resources tracking
and analysis tools; is easily customizable; and has a robust reporting suite.
Project.net
The application can prioritize and measure projects using scorecards, and can create
business graphs of portfolio information such as status, budget variance, and completion
time line. The graphics support project status color codes and arrows to indicate
improvement.
More attractively from the individual Project Manager’s standpoint, the application also
provides robust collaboration tools; such as, centralized discussion boards for all project
participants, the ability to attach documents and links to other project objects, and the
tracking of who has read each message. Project.Net is the only one of the four candidate
products which could demonstrate this capability. The application can also import project
plans from Microsoft Project.
Conclusion
Besides the all-important cost considerations, we strongly believe that migrating to
Project.Net or OpenAir for DT’s PPM solution will also provide the added benefits of
enhanced
• Project collaboration tools for the PMs, and project team members
Page 6
PPM Alternatives Results & Recommendation Report
May 26, 2009
Appendix A
TOTAL COST OF OWNERSHIP
The following worksheets summarize the Total Cost of Ownership (TCO) for each of the PPM
solutions which were evaluated. Each vendor submitted a TCO worksheet, except for
current vendor CA/EnPointe. (See APPENDIX E, for additional information.) We made every
effort to obtain cost information that was as accurate as possible from vendors, as well as
from DT’s relevant known internal costs. When direct vendor cost quotations could not be
obtained in a timely manner, cost estimates were extrapolated from similar vendor sources.
Below is the summary table of TCO for the PPM applications evaluated, followed by the
detailed worksheets for each one.
Cost Summary
PPM TCO Per User Annual Cost Per TCO Per User Annual Cost Per
Application (Three Years) User (Three Years) User
Page 7
PPM Alternatives Results & Recommendation Report
May 26, 2009
MS-Project Server
There were two additional factors here which proved difficult to estimate. The first was the
fact that this vendor has third-party business partners providing the possible solutions, and
the second was that most of them require MS Sharepoint to be in place. Both these factors
have the potential to boost these estimates considerably. However this might be
mitigated, by Enterprise Agreement pricing.
5/13/2009
Year 1 = 2009
On-Demand
On-Premises
MS Project Serve
Number of users (ma
S ubscription fee per
Page 8
PPM Alternatives Results & Recommendation Report
May 26, 2009
CA-Clarity
DT is currently using Clarity, and this pricing reflects the actual cost amounts based on 150
users, which is what CA is willing to quote over three years. However, only about 50 DT
users actually use the application, hence the difference between the number of
subscriptions per user per month, and the TCO per User.
5/27/2009
Year 1 = 2009
O n-Demand
O n-Premises
C A-C larity
N umber of users (m
S ubscription fee pe
Page 9
PPM Alternatives Results & Recommendation Report
May 26, 2009
OpenAir
This vendor does not offer their software to be hosted in-house by the customer.
5/13/2009
Year 1 = 2009
O n-Demand
O n-Premises
OpenAir
N umber of users (m
S ubscription fee pe
Page 10
PPM Alternatives Results & Recommendation Report
May 26, 2009
Project.Net
Although this solution’s software source code is Commercial Open Source, there are still
costs involved.
5/13/2009
Year 1 = 2009
O n-Demand
O n-Premises
Project.net
N umber of users (m
S ubscription fee pe
Page 11
PPM Alternatives Results & Recommendation Report
May 26, 2009
Appendix B
SURVEY OF FUNCTIONAL EVALUATION CRITERIA
Other CCSF Departments were solicited for their input via Web Survey. Their responses
and/or participation, was strictly voluntary. The survey asked respondents to weigh the
relative importance of functional requirements for a PPM software application. The final
weight for each criterion is the average of the responses received.
Response 2 4 5 6 7 8 9 11 15 16 17 18 22 23 24 AVG
Page 12
PPM Alternatives Results & Recommendation Report
May 26, 2009
Page 13
PPM Alternatives Results & Recommendation Report
May 26, 2009
Page 14
PPM Alternatives Results & Recommendation Report
May 26, 2009
years
Please add any comments, suggestions, or requirements we may have missed. Thanks!
The product(S) selected must add value for the PM... i.e., the PM has to want to use the tools in his/her daily
job. It can't be simply another vehicle to report information about projects that use other products to actually
manage and control.
Page 15
PPM Alternatives Results & Recommendation Report
May 26, 2009
Appendix C
FUNCTIONAL EVALUATION MATRIX
The following matrix computes the weighted score for each of the PPM solutions which were
evaluated. Raw scores were determined by Joe Armenta and Jay Nath.
Criteria
Weight
CRITERIA (1-9) Raw Score 1 (poor) - 5 (excellent) Weighted Score
Total Cost of Ownership
Relatively Inexpensive 3.00 2 1 3 4 6.00 3.00 9.00 12.00
TCO
Management
Requirements
Ability to view a list of
projects for an
organizational unit with basic
information (start/end,
budget, etc) and status
indicator(s) 4.40 4 5 5 5 17.60 22.00 22.00 22.00
Page 16
PPM Alternatives Results & Recommendation Report
May 26, 2009
Ability to establish
milestones and phases 4.53 5 4 5 5 22.67 18.13 22.67 22.67
Page 17
PPM Alternatives Results & Recommendation Report
May 26, 2009
Page 18
PPM Alternatives Results & Recommendation Report
May 26, 2009
Permission/Security allows
for isolation between
organizational units,
projects, and project artifacts 4.27 2 5 5 5 8.53 21.33 21.33 21.33
Page 19
PPM Alternatives Results & Recommendation Report
May 26, 2009
Appendix D
City Services Auditor Project Management Software Solution
Evaluation Spring 2008
• Reporting
Page 20
PPM Alternatives Results & Recommendation Report
May 26, 2009
• Nice ability to ‘group’ projects by priority • Cannot enter payroll categories for
for viewing. projects (dealbreaker??)--May require
integration with a separate timesheet
• Good gantt views of project progress,
program.
phase status.
• Timesheets are generated for any period
• Good high level view of projects and
designated by user-may lack sufficient
alignment with business need. controls for our purposes.
Individual Project Management -
Reporting-
• Clunky interface, via web applet. It is • Customized reports are not intuitive –
functional however-similar to MS
difficult to use and lacks documentation.
Project, though more difficult to use and
not laid out as well. • Out of box reports generally do not meet
our needs.
Page 21
PPM Alternatives Results & Recommendation Report
May 26, 2009
• Can select payroll codes and activity • Clunky gantt interface for project
codes for hours. task/subtask entry-may need to interface
with MS Project.
• Can select specific project tasks in
Reporting-
timesheet.
Page 22
PPM Alternatives Results & Recommendation Report
May 26, 2009
• Good user support with this software • Cannot select payroll codes and activity
(included in licensing). codes for hours-cannot add these fields.
Page 23
PPM Alternatives Results & Recommendation Report
May 26, 2009
• Web Resource • Web Timesheet has robust time entry • Web Resource does not allow for portfolio
and reporting capabilities—it is the planning and has very elementary project
• Web ONLY timesheet software that will allow scheduling/assigning features- rule out
Timesheet for allocating of project hours to multiple for CSA
funding sources without
customization/work arounds. • This is primarily a payroll/time capture
oriented program and will require
• Reasonable cost ($6,000 for licenses integration with other software solution to
plus $2,000/year for support) meet CSA’s needs.
Page 24
PPM Alternatives Results & Recommendation Report
May 26, 2009
• TeamSchedul • Continued use would avoid resource • TeamSchedule does not allow for projects
e costs associated with implementation of to be scheduled in more than one ‘plan’-
a new system (i.e., need for must either create a ‘new’ project or roll
• TeamTec development, training, etc.). over the plan.
Page 25
PPM Alternatives Results & Recommendation Report
May 26, 2009
• Reasonably priced ($30/user, plus • Payroll report may not be adequate for
$18,000 for first year ‘quickstart’ CSA.
installation package) – includes hosting.
• Custom report creation through the use of
• Full-featured work-flow system – may pivot tables may be inadequate for CSA
allow for integration of CSA needs.
policies/procedures (e.g. training
• Tracking of staff availability for
approvals, leave requests…) into
assignments is confusing/cumbersome
software system.
(grid method).
• Notification emails work with Lotus
• Heavy reliance on dashboard views.
Notes.
Reports are not as useful.
• Can track biweekly schedules (per sales
– need to verify) • Rude/aggressive salesmen
Time Capture-
• Allows for “split-billings”-not thoroughly
investigated
• Cannot select ‘category’ attributes for
Portfolio Planning- hours – overtime must be automatically
calculated by system in dashboard view –
• Can prioritize, sort, group and view may not meet CSA’s requirements.
portfolio—meets CSA needs.
• May not be able to automatically calculate
Resource Assignment- overtime since two-week schedules MAY
not be supported in current version—may
• Can do ‘soft’ assignments to staff and have work around according to salesman.
view availability--interface (grids) a little
clumsy (not as nice as Daptiv). • Cannot select ‘activity’—however, if this is
explicity in the task title, it should not be a
Individual Project Management – concern.
Page 26
PPM Alternatives Results & Recommendation Report
May 26, 2009
Appendix E
VENDOR QUOTE INFORMATION
The following is additional correspondence from vendors, which was submitted in addition to
the TCO Worksheets.
*******************************************
Jay,
Pricing =
Project Pro $ 650.00 x 50 comes w CAL
"For Project Server so no additional cost for current 50 Project managers. They may already own
these since they are using it now."
Project Server $4400.00 x 2
Portfolio Server $4400.00 x 2
Portfolio CAL $220.00 x 10
Thanks for the extra time. Let me know if you have questions...
Jeff
Page 27
PPM Alternatives Results & Recommendation Report
May 26, 2009
Hello Ollie,
Here is the quote for the first year of the three-year CA Clarity SAAS subscription agreement which has
been negotiated with Michele Oushakoff. This quote replaces the existing licensing quote for the previous
agreement's second-year (also attached for your reference), and represents a much lower cost and
extended period of license coverage.
Additionally, I am working with Michelle and the CA services group to create two phased SOWs and
quotes for additional assistance in upgrading the CCSF instance configuration and functionality, as well
as mentoring/training for DT IT staff. CA's scope of work for these services won't be finished until mid-
June, and I will be sending those documents at that time.
Please let me know if you have any questions about the attached quote.
Thank you.
Regards,
En Pointe Customer Service (800) 800-4214 x 5777 Hours: 5:00 a.m.-5:00 p.m. (PST)
Page 28
PPM Alternatives Results & Recommendation Report
May 26, 2009
Ordered By:
Dept. Contact Phone #:
Department Approval:
(Signature & Date )
Other Approval:
(Signature & Date )
DT Approval: (Signature
& Date ) Page 29
PPM Alternatives Results & Recommendation Report
May 26, 2009
Joe, Jay – Thanks for your patience. I’ve confirmed that we will not be offering the on-
premise as an option moving forward. Growing as a SaaS company, well over 99% of our
clients are leveraging that technology at this point. While we are committed to supporting
those clients who purchased on-premise models years ago, our firm feels that it is in the
best interest of our clients and shareholders to wholeheartedly commit to Software-as-a-
Service.
Jay – I’d also be happy to align another demonstration, analogous to the one I gave Joe last
week, at your convenience.
Kindest regards,
Grant
www.twitter.com/openairPSA
Hello Jay,
Here is the TCO document. We can make alterations after you have had a chance to review
the numbers.
Best Regards,
Tim
Tim Hok
VP, Business Development
http://www.project.net
617 621-0060 x123
Page 30