Test Levels for
Web-based applications
By G. Bazzana, F. Basili, E. Fagnoni
The last years have seen an explosive
growth in the WWW. Currently the Web
is the most popular and fastest growing
information system deployed on the
Internet, representing more than 80% of
its traffic.
Additional trends are:
‹Interaction of Web-based solutions with
large DBMS;
‹Web-portals;
‹ Usage of Web-based interfaces for
‹ Intranet/ Extranet applications that
directly interface the company legacy
system;
‹Usage of Web-based approaches for
critical applications (e.g.: on-line trading)
‹Access to the Web by different media
(e.g.: mobile phones, TV)
‹Need to allow equal opportunities to
Web access also for impaired or disabled
people, in order not to exclude them from
the new “Information Society”.
This has increased the complexity and
criticality of applications, requiring the
adoption of systematic testing activities
also in the Web-based realm that is far too
often wrongly considered an application
domain populated mostly by hackers.
The increasing importance and reliance
of Web applications ask therefore for
more and more testing levels to be
applied, as highlighted in the lower part
of the picture.
To date, we can say that Web-based appli-
cations deserve a high level of all soft-
ware quality characteristics defined in the
ISO 9126 standard, namely:
‹Functionality: Verified content of Web
must be ensured as well as fitness for
intended purpose
‹Reliability: Security and availability are
of utmost importance especially for
applications that required trusted transac-
tions or that must exclude the possibility
that information is tampered
‹Efficiency: Response times are one of
the success criteria for on-line services
‹Usability: High user satisfaction is the
basis for success
‹ Portability: Platform independence
must be ensured at client level
‹Maintainability: High evolution speed
of services (a “Web Year” normally lasts
a couple of months) requires that applica-
tions can be evolved very quickly.
In the experiences of the authors, Web-
based applications are characterised by
the following project management pecu-
liarities:
‹Development is managed in accordance
with Rapid Application Development
(RAD) approach;
‹This implies that analysis and design
are scarce when compared to “standard”
applications: the goal is to sketch-out an
innovative idea into a service, rather than
to build a product starting from very pre-
cise specifications and architectural
design
‹Proof of concept presentations are the
normal way to set-up “live” specifica-
tions
‹Limited formalisation of analysis and
design automatically implies that usage
of defect prevention techniques can only
be marginal: most of things to be checked
are thus let to dynamic testing
‹Round-trip engineering is followed, by
which we do not have a waterfall model
but rather we: “design a little, implement
a little, test a little” several times on
incremental versions;
Such characteristics have marked the suc-
cess of the Web; hence we do not think
that Web development has to be adjusted
in order to fulfil traditional software engi-
neering practices, but rather testing tech-
niques and tools have to be capable of
operating within such innovative
approach.
The following aspects further complicate
the picture:
‹Designers are often not professional
software developers or at least are not
aligned with conventional software engi-
neering practices;
‹The turnaround of people involved in
Web projects is extremely high
‹Compressed deadlines for services are
normal considering the pace of innova-
tion in the field;
‹Evolutionary maintenance is a must; we
can say that Web-based products seldom
reach a mature status, since they are
replaced by newer version when they still
are beta-tested;
‹Underpinning technologies are chang-
site are in relative forms (e.g.
../images/image.gif),...).
3. Usability tests are concerning with the
following aspects with respect to normal
behavior, destructive behavior, and inex-
perienced users:
‹Coherence of look and feel
‹navigational aids;
‹user interactions;
‹help messages;
4. Fast loading/link tests are concerning
with the following problems:
‹home page weight should be less
than 45k;
‹every page weight should be less
than 50k;
‹every web-site should have a fast
loading abstract/index;
‹every IMG tags must have WIDTH
and HEIGHT attribute.
Fast loading test is very important if we
consider that 85% of Web Users indicate
slow loading times as the reason for
avoiding further visits to Web Sites.
Moreover, a survey made by Zona
Research Inc. in April 1999 highlighted
very high bail-out rates for pages with a
weight resulting in more than 8 seconds
to wait;1 second load-time improvement
brought to a reduction in bail-out rates
from 30% to 7% !
ONION static testing checklist
In order to have a pragmatic approach to
Web syntactic testing, a standard check-
list, containing more than 100 checks,
was devised by ONION to be applied
both for acceptance purposes and for
regression testing activities. This check-
list covers the following aspects (for each
class the number of tests is given, togeth-
er with some of the aspects checked):
‹stylistic problems (9 tests, including:
spelling errors, particular tags, use of
obsolete mark-up, particular content-free
expression, empty container elements,
etc.),
‹lexical problems (5 tests, including: use
of character sets, formatting-related prob-
lems, using white spaces around element
tags, etc.),
‹syntax problems (12 tests, including:
illegal elements, illegal attributes,
unclosed container elements, malformed
URLs and attribute values, etc.),
‹fast loading related problems (26 tests,
including: bandwidth consumption,
images syntax, etc.),
‹document structure problems (4 tests
applicable both to tables and forms),
‹portability problems (17 tests includ-
ing: accessibility by various browsers and
platforms, mark-up inside comments, use
of single quotation marks for attribute
value, use of specific mark-up not sup-
ported by all browsers, liberal usage of
file naming, etc.),
‹structural integrity problems (4 tests
including: no index file for a directory,
dead links, limbo pages, etc.),
‹security problems (7 tests including: no
confidential data passed through form
without SSL, no user form field exposed
to shell in CGI programs, etc.)
The adoption of supporting tools can
allow for setting up a test factory and thus
run almost automatically a large propor-
tion of the defined tests.
Tools for Testing Static Web
Applications
In this section, the most known tools for
Testing Web Static Applications are
quickly examined.
This does not imply any endorsement by
the authors of any of the listed tools.
Information on listed tools might not be
always fully aligned with their latest ver-
sion, owing to the fast evolution pace. In
general we can say that no one tool is
fully comprehensive in its coverage, and
is best used in combination with addi-
tional tools. For most tools, coverage
improves with every release
The World-Wide Web Consortium
(W3C), hosted by MIT, INRIA and
University of Keyo, has been committed
from its beginning, under the leadership
of Tim Berners-Lee, the inventor of the
WWW, to developing a neutral, open
forum for the evolution of Web
Technology.
Like its partner standard body, the
Internet Engineering Task Force (IETF)
W3C is committed to developing open,
technically sound specifications, backed
by running sample code. As a conse-
quence W3C has developed various tools
for Web Testing including:
‹ An HTML Validator, which allows
HTML documents to be validated against
the DTDs for HTML, including HTML
4.0
‹A CSS Validator, which allows the user
to validate the CSS style sheets used by
HTML and XML pages
‹HTML Tidy, a free utility for correcting
HTML syntax automatically and produc-
ing clean mark-up. Tidy can be used to
convert existing HTML content into
compliant XML
All W3C software is Open Source and
can be retrieved from http://www.w3.org/
Doctor HTML
Doctor HTML is a Web site analysis
product, whose main features are:
‹Check the document for spelling errors:
This test looks for spelling errors in the
document. This is very helpful because it
removes HTML directives and accented
text before running the document through
a spelling checker, eliminating most of
the false alarms;
‹Perform an analysis of the images: This
section loads all the images in a docu-
ment and determines a few important
properties of each image. The most
important information is the bandwidth
consumed by each image, and roughly
how long it will take to download over a
14.4kbps modem (now the most common
speed for dial-up access users). Excessive
load times for individual images are high-
lighted. The program also reports the size
and number of colors in the image, which
has a direct bearing on how much band-
width the image consumes;
‹Test the document structure: This fea-
ture tests the main document structure
(excluding tables, which are dealt with
separately). The test looks for unclosed
HTML codes that may cause problems on
some browsers. When used in conjunc-
tion with other features, this can be help-
ful in hunting down extra HTML tags;
‹Look at image syntax: This test deals
with one of the most common mistakes in
HTML coding: overlooked image com-
mand tags. Specifically, it checks each
image command for HEIGHT, WIDTH
and ALT tags, and reports if they are
absent. These tags are important for quick
image loading and page formatting, as
well as providing information for
browsers lacking images;
‹Examine table structure: This feature
tests the table structure on the page. It
specifically looks for unclosed TR, TH
and TD tags inside a properly defined
table (e.g.: one which has both an open
and close table tag). It also reports on TR,
TH and TD tags that appear outside of
any properly defined table, since these
may cause formatting errors on some
browsers;
‹Verify that all hyper-links are valid:
long time on the market for C language.
Among the various development/ test
environments offering such features it is
worth remembering at least: SUN’s Java
Test Tools, TCAT for Java, White Box
Deep Cover for Java and RST Test tools
(inclusive of: Deep Cover for coverage
analysis; Assert Mate for pre-conditions,
post-conditions and data assertions test-
ing; Total Metric for static analysis).
Testing of dynamic Web-
based applications
Module testing
Testing of Dynamic Web-based applica-
tions deserves much of the challenges of
client-server applications, with additional
constraints posed by the underlying
architecture, which can be summarised as
represented by Figure 2.
Dynamic WWW development can be
done, at the current level of technology,
with two main approaches: CGI
Programming (Perl, C, TCL..) or Server
Extension Programming (ASP, PHP,
Apache Server API, Netscape Server
API, ..) It has to be noted that the risk
level associated to the two techniques is
utterly different, as clearly highlighted by
Figures 3 and 4.
In fact, whereas when a CGI calls fail just
a program fails, when a server extension
fails, the whole server might crash!
For Testing Server Side, an approach
similar to client-server testing has to be
taken, covering both GUI and HTTP test-
ing, as shown in Figures 5 and 6.
Besides the well known techniques for
client-server testing, you should beware
of complexity from included software
layers; it has to be remembered that often
Figure 3: CGI Programming
HTTP SERVER
CGI interface
CGI PROGRAM
DB LEGACY
APPLICATION
Browser
WEB SERVER
Server
Extension MIS
Transaction
DBM CGI Mng
Figure 2: Architecture & Challenges of Dynamic Web Applications
more than 90% of the software is out of be read back from the browser. This is
the developers’ control, being re-used useful for having the browser remember
from other sources. some specific information. The problems
related with testing are that cookies
Special care has to be devoted to the fol- expire and that users can disable them in
lowing aspects: browser.
‹Focus on usability test.
‹Focus on performance test. Besides the basic Security checking per-
‹Focus on load-stress. formed during the previous test level,
‹Focus on installation test if special specific security testing has to be per-
plug-ins are present formed when Web applications make
‹Perform well-managed beta test. usage of sensitive data.
First of all it shall be clear that on the
Integration/ Security
WWW there is no silver bullet for
Testing absolute security, likewise in real life,
and that security techniques and checks
Integration/Security tests are concerned shall be tailored depending on the value
with to be protected.
‹Included components
‹Cookies Moreover, it is important to underline
‹Proxies/ caching that security enforcing involves both
‹Frames organisational and technical issues;
namely organisational issues are often
A peculiar aspect of the testing of dynam- much more important than technical
ic WWW applications is testing of cook- ones, at least in Intranet and Extranet
ies. The web is a memory-less system, applications.
with no concept of session. To overcome
these issues you can use cookies: a small In such cases, the approach is to define a
piece of information sent by a web server “Security Policy” at company level and
to store on a web browser so it can later
Figure 4: Server Extension Programming
virtual users; HTTP requests can be edit-
ed, changed or parameterised
‹Web Scenario Wizard: orchestrates vir-
tual users into a multi-user scenario; up to
4.3 million hits per day like real users
surfing
‹Visual Load Testing Controller: drives,
monitors and synchronises interactions
Data analysis statistics include number of
virtual users, transaction performance,
completed transactions per sec., connec-
tions per sec., and throughput.
Web-specific supporting features include:
cookies, proxy servers, user authentica-
tion, session Ids, CGI scripts, API calls,
and HTML forms.
The tool can be integrated with Astra Site
Manager and SSL Plug-in Pack.
Testing ERP and WWW
Tool providers are performing certified
integration with ERP systems.
Future challenges
Future challenges are related to the evo-
lution of the WWW, namely:
‹ New Generation HTTP Protocol
(HTTP-NG)
‹Integration between TV and the Web
‹Emergence of XML
‹New references for User Interfaces:
evolution of HTML (for publishing docu-
ments), on MathML (for publishing
Math), on SMIL (for multimedia presen-
tation), on SVG (for publishing diagrams
and vector-based graphics)
‹Mobile access to the WWW as well as
techniques for using voice interaction for
accessing the Web
‹Privacy issues
‹Digital signatures
‹Micro-payments

integration
These emerging technologies and servic-
Future Intranet/ Extranet applications es will require Internet testing approach-
will require more significant work and es to be continually fine-tuned, to guaran-
more sophisticated skill set. In fact, tee the reliability and quality of service
Intranets will evolve into a component of required by the global Information
the IT infrastructure making distributed Society.
computing more open, simpler and more
manageable. This will make possible the ONION S.p.A.
delivery of more flexible, manageable Via L. Gussalli, 9 - 25131 BRESCIA
distributed business processes. (Italy)
E-mail: info@onion.it - Web:
From a technical point of view, Web- http://net.onion.it/
enabled business applications will be
based on transaction-oriented business
processes; hence Intranet based applica-
tions will merge with Extranet-based
business-to-business transactions, EDI
and electronic commerce transactions.

Already today, to multiply benefits, com-

panies need to integrate Web technology
with transaction-oriented business appli-
cations, group-ware and infrastructure
services, integrating Web-based applica-
tion and MIS and setting-up simple,
cross-platform applications on top of a
simple-to-manage and more centralised
IT infrastructure.

As far as testing is concerned, challenges

are on:
‹security,
‹load testing
‹User authentication
‹Server authentication
‹Connection privacy
‹Message Integrity
‹Payment security.