NELSONHALL PERSPECTIVE

NelsonHall

NelsonHall Perspective

IBM Testing Services

IBM Global Services had in 2009 revenues of $55bn and a pre-tax margin of 14.3%.

1. Background In Testing
IBM Testing Services (IBM TS) is a service line that exists in all IBM Global Services countries. It falls under IBM Global Business Services, the consulting and application services arm of IBM. NelsonHall estimates that IBM:

Derived approximately $550m in standalone software testing revenues for enterprise clients (i.e. excluding internal work for IBM Software) Employs ~8,000 career testers Has in total 20,000 personnel with testing skills, whether career tester or occasional testers.

2. Recent Offerings
IBM testing Services has developed several offerings over the last two years. Total Test Quality Together with IBM Research and IBM Software, IBM Testing Services has developed its Total Test Quality (TTQ) framework. TTQ focuses on the overall software development cycle. It has several services including:

Source code analysis: uses several tools including IBM Rational Appscan and Purify Plus, to scan the code the code and look for vulnerabilities in terms of performance, security and overall structural issues in the code. IBM argues the source code analysis allows to detect defects e.g. memory leakage that are very hard to find during traditional testing. Also, this allows defects to be found earlier in the lifecycle reducing the cost of defect rectification. Source code analysis is run as a service and does not require investment in software licenses The offering can focus on specific areas such as security and performance vulnerabilities.

Defect analysis: the offering analyzes historical defect data in order to prevent new defects being introduced and to find defects early in the software development lifecycle. Service is delivered both in India for collection of client application and classification of data and onshore, in the U.S., for the final analysis. It is based on IBMs Defect Reduction Method IBM differentiates between what it calls valid defects e.g. code defects and invalid defects e.g. defects external to the code such as test data, test scripts and test environments. The companys experience is that invalid defects can be the most expensive cost to solve by the client

2010 by NelsonHall.

March 2010

NELSONHALL PERSPECTIVE

NelsonHall

Test Practitioner Workbench: developed with IBM Research, it provides test planning and estimation functionality. It helps in particular estimating the effort, resources and costs for a test campaign. It provides predictive test modeling and estimates the impact of putting more/fewer resources on the defects level at the end of the testing campaign. The workbench functions on risk and requirement basis

Commercial offerings Fixed Price Contracts IBM Testing Services has also worked on expanding the way it contracts with its clients. The unit has developed a systematic manner to offer fixed prices to its clients. To do so, it provides an assessment of the application to test in terms of interfaces/integration tools, changes in the scope of the application e.g. build vs. enhancements, etc. Once the assessment is completed, IBM TS is able to estimate the testing effort on a man-day basis. The fixed price approach is valid for testing services such as functional testing and performance testing. Outcome/SLA-based Contracts IBM Testing Services has several contracts which are based on the number of defects and overall on software quality found in the application at the end of a testing campaign

IBM is more likely to commit on defect levels as part of long-term contracts, during which it has gained knowledge of the clients applications The contract includes defined entry criteria e.g. level of defects found in the application before IBM proceeds to its testing effort

Warranty Period IBM has extended the notion of its SLA-based contract to a warranty offering. Under this offering, IBM commits to being paid at the end of a warranty period of several years, based on the number of defects found in production during the warranty period.

The warranty offering is available for both contract types where IBM is in charge of the build and run or is only involved in the independent testing service It covers functional testing services as well as non-functional IBM has primary responsibility for deciding how to conduct the test effort. It will conduct testing based on a risk analysis and for critical requirements This offering is currently available for applications of a moderate complexity and is better fitted for application management contracts rather than for complex systems integration contracts IBM Testing Services has conducted this warranty effort for a limited number of clients. It is expecting to gain further experience and maturity for its next round of clients in understanding when and how exactly to offer the warranty offering.

Cloud Computing IBM TS provides three main forms of cloud computing services:

It is deploying virtualization technology over the client-owned servers aiming to maximize the level of usage of the servers in the test environment. A virtualized infrastructure applies mainly to functional testing such as systems integration and user acceptance testing. Other non-functional testing forms like performance and security testing still require a physical environment IBM is working with ISV iTKO for its Lisa product. The Lisa technology virtualizes applications and allows to commission test environments. Examples of usage of this technology include simulating a mainframe without actually having to invest into such a system. It therefore allows to simulate a mainframe backend without having to invest into such a system. IBM Testing Services is using this technology for system testing

2010 by NelsonHall.

March 2010

NELSONHALL PERSPECTIVE

NelsonHall

Finally, IBM Testing Services provides mainframe testing environment capacity and related services under a pay-per-use offering. This offering extends a traditional way of procuring mainframes to the testing environment level.

3. Reusable Assets
IBM Testing Services is aiming to increase the level of artifacts across clients in selected areas:

For test plans and to a lesser extent for test cases and scripts For non-regression testing and system testing For several packaged applications such as iflex (Oracle) and SAP For several sector i.e banking.

However, IBM highlights that two constraints exist around the reuse of artifacts:

IP rights: artifact rights that belong to the client Level of customization of the application to specific needs make it sometimes hard to reuse to other clients.

4. Delivery
NelsonHall estimates that IBM has ~8,000 career testers, of whom two thirds are located in India or other low-cost countries. The company has standardized its delivery units, whether onshore or offshore, on standard processes and tools (IBM Rational). The company accommodates testing tools from other vendors, when delivery is taking place onsite, in the client premises.

5. NH Perspective
NelsonHall estimates that IBM has one the two largest testing practice in the world. The IBM testing offering has a significantly different value proposition from most competitors

Schematically, Indian vendors are, in addition to their cost advantage, taking a domain expertise route that builds on the large number of testers they have. They are developing tools and accelerators to make testing execution more efficient and are expanding into test design and optimizing the reuse of artifacts and assets. The capacity of Indian vendors to ramp up resources remains unmatched Vendors like Capgemini and Logica have a strong methodology background, still have a strong onshore presence and have very significantly expanded into offshore (in particular Capgemini). They provide a balanced alternative to an all-offshoring strategy. Among the other European vendors active in testing, Atos Origin stands out by its approach to infrastructure-led testing There is a third angle that is best characterized by IBM and HP. In particular their approach is based A technology and automation approach that expands from pure execution into the whole software development life cycle. This technology approach translates into code analysis and predictive modeling techniques Synergies with IBM Software allow IBM Global Services to have a SaaS offerings of IBM Rational testing tools IBM and, to a lesser extent, HP have expanded into offshoring and are bringing over time the accelerators and domain expertise that Indian vendors have developed.

2010 by NelsonHall.

March 2010