Tutorial:

Building Automation Frameworks for

Web Services Testing

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

Table of Contents
Important Terms
Business Case
Framework Architecture
Benefits
Customization
What's next?

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

Important terms
SOA Service Oriented Architecture: is the structure that defines how a
collection of software components communicate with each other to provide
application functionality as services.

Regression Testing: is the testing conducted after changes in the software /

application to ensure that new defects have not been introduced as a result of
the change.
Data Driven Framework: is an automation framework that comprises of reusable test logic and multiple data sets that can be used repeatedly to ensure
a wide coverage of variable inputs.
Batch File: is a script file (text file) containing a series of commands and is
executed through the command interpreter.
SOAP UI: is an open source web service testing application for SOA.

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

Key Features

Data Driven
Framework

Batch
Process
Execution

Reporting
Structure

Solution
SOAPUI tool using groovy scripting
and build through ANT
3

Data Driven Framework:

reduces maintenance and
improves test coverage
Batch Process Execution:
facilitates easy execution of
multiple test cases
Reporting Structure:
enables easy comparison of
test results against base
responses. Concise HTML
reports are generated
showing pass/fail status for
each scenario

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

Evolution of the Framework

Automation of manual testing
Base Response Comparison
Version 1.0 Test results Summary Report
Record Not Found Scenario
Graphical User Interface
Version 2.0

Data Mismatch Scenario

Database response comparison
Version 3.0

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

Framework Architecture v1.0

1.3

Soap Request

1.1

1.2 Response

1.4

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

Framework Architecture v2.0

2.1 Update Test case with Status Flag if Record Not Found Scenario occurs

2.3 Update
Test Data
sheet with new
Data

2.2 Execute Record Not Found

Queries

2.5 Update the Expected Base Response

2.4

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

Framework Architecture v3.0

Query results

3.3 Compare
DB results
with Request
parameters

3.4 Update Request

Parameters

3.2

Data
Mismatch
Matched

Not Matched

3.5 Update Expected Base Response

3.1 Execute Queries for Data Mismatch Scenario on DB
7

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

Benefits Quantitative
Hrs.

1600

R1 (19)

R2 (19+42)

R3
(19+42+23)

1400
1200

Manual Effort
(Estimated)
Manual Effort
(Actual)
Automation Effort
(Actuals)

1000
800

600
400
200
0
1 2 3 4 5 6 7 8 1 2 3 1 2 3

#Cycles

Comparison of Automation vs. Manual per Regression cycle

8

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

Benefits Qualitative

Reusability
Reliability
Repeatable

Increased turn
out time

This framework can be

used across multiple
projects and domains.

This suite correctly

diagnosed 70% of all
interfaces and
Provided consistent results
across 11 execution cycles
and releases.
Saved ~ 6000 hours in
manual testing effort.

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

Customization
Framework (v3.0)

Customizable

10

Re- usable

SOAP UI Requests

Test data sheet

format/framework

Expected base
response

Outcome comparison
tool

Database
environment

SOAP UI Database
Interaction

Reports

Reporting Structure

Data Mismatch
queries

Data Mismatch Logic

Record Not Found

queries

Record Not Found

logic

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

What's Next?
Proposed Version 4.0 features:
a. Test data creator - to facilitate the creation of appropriate test data for
the test scenarios to be executed; as the automation framework is
dependent on test data.
b. Better Graphical User Interface Enhance the GUI to enable the user to
create predefined test suites which are capable of scheduling regression
tests.

11

Copyright 2013 Deloitte Consulting LLC. All rights reserved.

About Deloitte
Deloitte refers to one or more of Deloitte Touche Tohmatsu, a Swiss Verein, and its network of member firms, each
of which is a legally separate and independent entity. Please see www.deloitte.com/about for a detailed description
of the legal structure of Deloitte Touche Tohmatsu and its member firms. Please see www.deloitte.com/us/about for
a detailed description of the legal structure of Deloitte LLP and its subsidiaries.
Copyright 2013 Deloitte Development LLC. All rights reserved.
Member of Deloitte Touche Tohmatsu