Performance Improvement Complements IT Best Practices Frameworks

By Neil Barton
(http://www2.cio.com/analyst/report2669.html) An executive seeking to advance the efficiency of an IT organization has no shortage of options. Not-for-profit organizations have developed a wide range of objective, independent improvement methodologies, including the Capability Maturity Model, the IT Infrastructure Library, and Control Objectives for IT. While these and other approaches vary in scope and emphasis, they all offer detailed guides to implementing processes and practices designed to yield optimal performance in both operations and development environments. Which model is best? Can they be effectively employed together? And should they be used to complement commercial consulting services? The Capability Maturity Model The Capability Maturity Model (CMM www.sei.cmu.edu/cmm/cmm.html) describes practices and processes around application development. CMM for software development was established in 1986 at Carnegie Mellon University by the Software Engineering Institute (SEI), with assistance from the MITRE Corporation. Originally developed for the U.S. Defense Department to manage large and complex development projects, the CMM has gradually gained recognition in both the public and private sectors as a useful framework for improving software development processes and application quality. Five Stages of Maturity The CMM defines five stages of organizational maturity with respect to software development. The characteristics of the five stages are defined by CMM as follows: Initial. The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics. Repeatable. Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications. Defined. The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization's standard software process for developing and maintaining software. Managed. Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled. Optimizing. Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies.

Predictability, effectiveness, and control of an organization's software processes are believed to improve as the organization moves up these five levels.

Integrated Model A recent enhancement of the CMM model - Capability Maturity Model Integration, or CMMI - is designed to align development efforts more closely with business requirements. Specifically, CMMI best practices aim to achieve the following: more explicitly link their management and engineering activities to their business objectives expand their visibility into their product life cycle and engineering activities to ensure that their products and services meet customer expectations incorporate lessons learned from additional areas of best practice (for example, measurement, risk management, and supplier management) implement more robust high-maturity practices address additional organizational functions critical to their products and services more fully comply with relevant ISO standards

CMM and Performance Improvement A performance improvement analysis can quantify in business terms the benefits delivered by adhering to CMM guidelines. If, for example, a development organization were to report to its business units that it had advanced from maturity Level Two to Level Three, the business executives would likely be unimpressed. If, on the other hand, the development organization can cite improved development processes, resulting in quicker responses to requests, a reduction in errors and bugs, increased productivity, and reduced costs, then the business executives might take notice. Again, a performance improvement analysis enables these benefits to be quantified. The Information Technology Infrastructure Library ITIL (www.itil.co.uk) is a repository of information cataloguing leading practices in IT Service Management, with a focus on operational service delivery. Available to the public, ITIL is championed by a not-for-profit organization called ITSMF (www.itsmf.com). ITIL has a strong following in Europe, especially in the government sector, for which it was initially written in the late 1980s. It has a growing following in North America, and has been used as the basis for a number of vendor offerings, including Microsoft's MOF, and HP's ITSM. ITIL describes 11 major entities, processes, and disciplines within the two broad categories of Service Support and Service Delivery. These are as follows: Service Support Service Desk Incident Management Problem Management

Configuration Management Change Management Release Management Service Delivery Service Level Management IT Financial Management Capacity Management IT Service Continuity Availability Management

ITIL also touches on several associated processes of key importance such as IT Security Management, Supplier Management etc. ITIL Compliance To help IT organizations assess their compliance with ITIL guidelines, a number of ITIL "checklists" are available These checklists describe each ITIL practice in detail and are used by the auditor to assess how well the practice is being followed, typically on a scale from 1 to 5. IT organizations can "self-audit" themselves, or hire external consultants to do so. Audits provide a qualitative and subjective rating of ITIL adoption in an organization, but do not provide hard quantitative indicators of the cost, productivity, or quality of a process. ITIL and Performance Improvement A quantitative functional model - applied as a complement to ITIL - can show where implementing ITIL will yield a return, and whether and to what extent ITIL processes have resulted in performance gains. Such a model can also be used to develop an ITIL "scorecard" to track performance on an ongoing basis. Consider the ITIL category of Incident Management. ITIL examines the processes involved in identifying, managing, escalating, resolving, and reporting an incident that affects a user, as well as outlines relevant performance measures, such as the average time to resolve an incident, and number of incidents handled per agent. A quantitative performance improvement analysis - applied to the ITIL framework - can provide an effective roadmap to positive change through a fact-based view of what is actually going on in an organization, where problems might be arising, and what actions can be taken to address them. For example, while ITIL defines guidelines for identifying and escalating an incident from first to second level, a quantitative performance improvement initiative can analyze an organization's actual performance with respect to incident management. In other words, an effective IT organization must know not just how to identify and route incidents, but how many incidents are in fact coming through the organization, how many incidents are being handled by one agent, and through what channels. Understanding which technologies are generating incidents, moreover, can aid in identifying root cause analysis. And, by comparing performance against a reference group of top-tier

organizations, such an analysis can gauge whether performance is above or below average. In summary, a performance improvement initiative can demonstrate the need for an ITIL implementation, manage ITIL processes, and show the business benefits delivered by ITIL practices. To maximize the benefits, the performance improvement methodology should be ITIL-compliant, and the consultants involved trained and qualified in ITIL processes. COBIT Control Objectives for IT (COBIT www.isaca.org) is in some respects a hybrid of other management methodologies. Broader in scope than ITIL and CMM, COBIT covers the breadth of IT management - both infrastructure operations and application development. COBIT, moreover, has adopted CMM's maturity rating for organizational compliance to performance standards. COBIT was initially developed by the IS auditor community, and is championed ITGI (IT Governance Institute), the commercial arm of ISCA, the Information Systems Audit and Control Association. COBIT focuses on issues of corporate risk management, and defines a checklist of IT best practices and a practical guide to due diligence. The COBIT methodology divides IT into four domains, which are subsequently broken down into 34 processes, and then further divided into 304 control objectives. For each of Control Objective, COBIT defines: A list of critical success factors that provides succinct, non-technical best practices for each IT process Key Goal Indicators and Key Performance Indicators (outcome measures and performance drivers for all IT processes) Maturity models to assist in benchmarking and decision-making for capability improvements

To avoid confusion, it should be mentioned that COBIT borrows the technique of five numbered maturity levels from CMM. In contrast to CMM, however, COBIT defines maturity levels in terms of its specific COBIT Control Objectives. COBIT and Compass COBIT defines good governance processes, and indicates the maturity of practices. It also provides access to benchmarking data (online, for registered members) and allows COBIT users to share the maturity levels they achieve. While COBIT defines some key performance indicators, they tend not to be detailed enough to be used for operational management. As such, Compass performance analysis can quantify cost and service levels delivered through the successful application of COBIT standards. For example, successfully implementing COBIT's change management procedures might result in a lower cost per MAC, a shorter target time for MAC completion, fewer problems introduced as a result of changes, and so on. Which Approach(es) Work(s) Best?

Process improvement methodologies are designed to address a wide range of areas and concerns. No one approach is superior to the others, and each has specific attributes tailored to particular management challenges. The key point is that, whatever the approach is taken to process enhancement (in terms of implementing leading practices), a fact-based performance improvement initiative can play a critical complementary role by identifying and quantifying the benefits delivered by process improvement. Neil Barton is director of consulting services for Compass. He is based in Guildford, UK.