White Paper

WHAT CAN YOU DO WITH GOMEZ?

The Case for a Web Experience Management Platform for Web 2.0 and Beyond

WINTER 2008

Executive Summary
Customer demand for the superior web experiences that Web 2.0 applications can provide
-- including increased user interactivity, relevant real-time content and rich media, content cre-
ated by social networks -- is driving leading web companies to design complex new end-user
facing web applications. But building these applications often requires moving functionality
from the well-understood server environment to inside the user’s browser itself. For the major-
ity of companies accustomed to developing applications in the traditional way, the Web 2.0
paradigm represents an unprecedented loss of control. Many continue to develop applications
behind the firewall, test with conventional tools, and launch into production with minimal
regard for the actual customer experience on the desktop.
Along with these new end-user requirements comes the need for an integrated platform of
tools to painlessly support those tasked with designing, developing and deploying those new
applications. The ExperienceFirst platform from Gomez described in this white paper provides
the industry’s first end-to-end solution for web application experience management through-
out the application lifecycle. This technology promises to help business and IT managers to:
Ensure end-users have quality Web experiences;
Set performance and web experience goals early in the design phase and test against those
goals throughout development and quality assurance (QA) phases;
Monitor and manage application performance levels and the impact of third-party content
across multiple browser types and operating systems;
Measure the relationship of performance and customer satisfaction with analysis of yield and
conversion rates of marketing funnels; and
Understand the impact of changes – in everything from partner relations, vendor sources,
capacity, addition of new applications and site modifications – on end user experience.

It’s a composite world today, so much of the web experience a company provides hinges on content, applications and
infrastructure delivered by third parties. As we move toward rich Internet technologies such as Ajax, even less of the
customer experience is under our direct control. Yet the customer still holds the host brand accountable for
the web experience and expects much richer services delivered instantaneously.”

Steve Trimbo
Director of Online Operations Best Buy.com
 W hite Pa p e r Managing User Experience and the Organization

Contents

Introduction.......................................................................................................................................................................................3

A Closer Look at Web 2.0 Composite Applications....................................................................................................................3

Inside the Gomez ExperienceFirst Platform................................................................................................................................5

Putting It All Together......................................................................................................................................................................8

What can you do with Gomez?.......................................................................................................................................................9

Appendix A: Web Experience Management in Practice –

A Customer Example..................................................................................................................................................................... 10

The information contained in this document represents the current view of Gomez, Inc. on the issues discussed as of the
date of publication. Because Gomez must respond to changing market conditions, it should not be interpreted to be a
commitment on the part of Gomez, and Gomez or its respective suppliers cannot guarantee the accuracy of any informa-
tion presented after the date of publication.

This white paper is for informational purposes only. GOMEZ AND ITS RESPECTIVE SUPPLIERS MAKE NO WARRANTIES,
EXPRESS OR IMPLIED, AS TO THE INFORMATION IN THIS DOCUMENT.

Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright,
no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form
or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express
written permission of Gomez, Inc.

Gomez may have patents, patent applications, trademarks, copyrights or other intellectual property rights covering subject
matter in this document. Except as expressly provided in any written license agreement from Gomez, the furnishing of this
document does not give you any license to these patents, trademarks, copyrights or other intellectual property.

©2008 by Gomez, Inc. All rights reserved. Gomez® is a registered service mark, and ExperienceFirst™, ExperienceFirst
SCoE™, ExperienceFirst Network™, Active Network™ XF, Private Network™ XF, Active Last Mile™ XF, Private Locations™
XF, Actual Experience™ XF, Reality Check™ XF, Reality Load™ XF, Reality View™ XF, the taglines “Ensuring Quality Web
Experiences™”, “Quality of Web Experience™”, and QoWE™ are service marks of Gomez, Inc. All other trademarks and
service marks are the property of their respective owners.

Gomez, Inc. 10 Maguire Road, Suite 330 Lexington, MA 02421 USA

This document contains information of a proprietary nature. All information contained herein shall be kept in confidence
and shall be for the original recipient’s use only. Any unauthorized reproduction by any other party shall constitute an
infringement of copyright.

2
Gomez, Inc. 10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com
T: 781.778.2700 F: 781.778.2799
 W hite Pa p e r Managing User Experience and the Organization

Introduction
Website development has completely transformed in a very short amount of time. First-
generation sites circa 1997 – 2003 were architected in a fairly straightforward manner, Site
control resided neatly on the server side. Tens of thousands of sites were built following a basic
three-tiered layered approach where each layer was neatly segmented by function and all the
underlying site components resided in the data center. The back-end mix of databases, applica-
tion servers and web servers were all under the control of developers, and by extension, the
business and brand behind the site.

Year 2000: IT Controls Web Experience Year 2007: Web Application Lifecycle Distributed

thin/
dumb client smart clients

www.mybrand.com www.mybrand.com Customer experience

is browser dependent
with AJAX, FLEX,
Flash Technologies

Web Server Web Server

App Acceleration, Content
Web Container Web Container
Quotes, Bill Payment,
Application Server Application Server Store Maps, Package
Tracking, etc.
EAI EAI
Zone of Zone of
Direct Direct
Control Control
Application components
outside of companies
control

Fast-forward to today’s Web 2.0 world and the difference is astonishing. Top web applications
today have leaped from yesterday’s simple setup to a more complex architecture focused on
providing superior online customer experiences. More user interactivity, relevant content and
rich media is what leading web companies want to deliver in order to achieve a superior user
experience. Outsourcing this new content and functionality saves time and money and makes
good business sense. Implementing these technologies almost always requires moving more
functionality from the well-understood server environment to inside the browser itself. A big
challenge today is that only a few companies know how to do this well.
Unlike ten years ago when companies clearly maintained ownership of the entire web experi-
ence by controlling it from the data center, today there is no single owner of the user experi-
ence. Modern web applications are typically comprised of code and content coming from
numerous parties, only one of which is the company whose logo and brand are displayed
prominently on the page. This mosaic of content and services that comes together in the
browser relies far more on local processing than ever before. With today’s multitude of browser
types, computer operating systems, and versions, there is a real need to ensure that web appli-
cations work well for all end-users.
Outside of a select few developers and business managers in the savviest web companies, few
people realize how much of a site’s content and functionality depends on outside sources. Yet
building sites that use an increasing number of outside services is increasingly the norm.

A Closer Look at Web 2.0 Composite Applications

Leading organizations are striving to deliver increasingly interesting, useful, or exhilarating
web experiences. With Google Maps, for example, users can “speed drag” satellite images
of their neighborhood, city, state or country at will. That’s because the underlying Ajax code
(Asynchronous JavaScript and XML) anticipates their movements and makes server calls behind
the scenes. Mash up Google Maps with a real-estate brokerage home finder, and a Web 2.0
developer can suddenly have a lot going on at the browser level that not long ago would have
happened on the server, where they could control it.

3
 W hite Pa p e r Managing User Experience and the Organization

Unbeknownst to most users, adding layer upon layer of third party content to web applications
is becoming increasingly common. For example, users may think they’re just paying bills on
their bank’s web site. In reality, they are interacting with a third-party bill-paying application
that is enlisting another provider’s hidden web analytics service. Similarly, comparison shopping
services are often outsourced to providers who add features such as local shopping to web-
only national businesses. In the new paradigm, it’s not unusual for sites to have 10 or more
service providers contributing the website content and functionality that end-users rely on.
Meanwhile, businesses increasingly rely on
service-oriented architectures (SOA) to expand
What Rich Internet Applications (RIAs) have you deployed or are planning to deploy in the next year?
(Percentages are based on the total number of responses) the functionality of their web applications. These
applications leverage blocks of external applica-
tion code or web services to add a wide range
of features and functions to a site. In many cases,
these applets are supplied not only by the com-
pany itself but by partners invisible to end-users.
With all this added complexity, few companies
today really understand how their applications
are performing from the end-user perspective.
The fact is, in today’s composite world, it’s dif-
ficult to tell when some combination of external
content and the user’s browser or operating sys-
tem breaks down and spoils the user experience,
particularly if the organization is using conven-
tional testing tools.
Ajax Flash/Flex Streaming Silverlight Adobe Air Other
63.1% 57.1% 28.9% 4.7% 2.7% 14.1%
The user experience trumps all. When done
properly, pushing more Ajax and Flash code to
the browser enhances the user experience and
makes the application more rich and respon-
sive. However, as more content and code goes
Why did you choose to use these Rich Internet Applications (RIA’s)?
(Percentages are based on the total number of responses) directly to the browser, it’s harder for developers
to predict performance. Often, design prob-
lems don’t emerge until after deployment in a
live customer-facing environment. When web
applications fail outright or perform poorly, the
danger for the business is lost revenue to com-
petitors and long-term damage to the brand. For
developers, it means spending time on rework
rather than focusing on future application devel-
opment.
Worryingly, many organizations still aren’t think-
ing about the increasingly important role of
extended web architectures. Following traditional
User Demand Inadequacy of existing Keeping up with the Desire of internal Executive Mandate Other development procedures, they continue to test
29.5% technologies to meet market/competitive developers to use new 7.5% 11.6%
design goals
39.7%
pressure
56.2%
technologies
36.3%
what’s inside their zone of control and to see the
effects of external content and code only after
their application is launched into production.
With increasingly sophisticated user expectations,
those tasked with developing, deploying, and managing applications can no longer afford to
wait to test, tweak, and improve web applications and practices. The stakes for their companies
are too high. Rather, they must start early on and set their targets to define success criteria for
the user experience in the Web 2.0 world. What’s needed is a new, more fully integrated devel-
opment and management platform that starts with the end-user experience in mind, builds on
and combines the time-tested processes that have traditionally been favored.

4
Gomez, Inc. 10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com
T: 781.778.2700 F: 781.778.2799
 W hite Pa p e r Managing User Experience and the Organization

Inside the Gomez ExperienceFirst Platform

To meet customer demand for superior online experiences, a new generation of web perfor-
mance management tools has emerged that gives control back to the company. The Gomez
ExperienceFirst™ Platform is the first on-demand platform to focus on ensuring a superior web
experience throughout the entire web application lifecycle. This platform uniquely supports all
aspects of the web development effort, including the design, development, validation, deploy-
ment, operation, analysis, management and continuous refinement of web applications. While
other solutions test or monitor the shrinking behind-the-firewall portion of a web application, the
ExperienceFirst Platform takes a holistic view by focusing on the entire web application as it is
delivered in today’s fragmented Web 2.0 world, to the actual end-user.
A Great User Experience Starts Early. In order to minimize costs and maxi-
mize the chances of delivering satisfactory experiences to users, planning for and
integrating the user experience into today’s Web applications should occur at
the earliest stages of development, not later or as an afterthought. At the early
design and development stages, two services in the Gomez ExperienceFirst plat-
form work side by side to ensure that web applications both appear and work as
intended, for all users on all browser and OS combinations. Reality View XF and
Reality Check XF employ easy to use, Ajax-based web interfaces to help develop-
ers understand the impact of their programming on end users.
Gomez Reality View XF is a cross-browser compatibility and performance test-
ing service that profiles the behavior of web pages across multiple browsers
such as IE6, IE7, Safari, Opera and Firefox. It provides a one-click screen capture
view of how page prototypes on a site appear and perform when viewed across
multiple operating systems, including Windows XP, Windows Vista, Linux and the
Macintosh. The tool shows how well applications perform under various browser
screen sizes, and reports back details on how each browser type renders on a
page. This helps designers and developers to optimize ck pages as needed, long
before they are delivered to an end-user’s browser.
Reality View XF Gomez Reality Check XF is a full-featured functional testing tool for every type
of web application, including complex Web 2.0 applications dependent on Ajax
and other Rich Internet Application frameworks that rely on browser-side pro-
cessing. It makes certain that each function and third party component going
into an application works as intended and displays properly in the user’s browser.
Reality Check XF can also be used to record and automatically replay specific
user transactions, such as searching for a book or placing an item in a shopping
cart. Developers can easily unit test across different types of browsers or operat-
ing systems to make sure everything on each page is working as expected. In
addition, a movie capture mode in Reality Check XF helps developers to see
dynamic page interactions as they occur, and verify how those interactions will
be perceived by the end-user, across multiple browsers and operating systems,
before the application goes live.
More Meaningful Load Testing. Once an application has been developed
and tested from the customer perspective under normal conditions, it’s impor-
tant to understand how the application will perform when it’s forced to bend.
By understanding what type of load it takes to degrade application performance,
or what it takes to bring an application or a vendor to its knees, companies can
be better prepared for those situations and better understand what it takes to
consistently deliver a quality web experience to their end-users.
Reality Load XF
Gomez’ Reality Load XF is different because it is an external load testing service that is designed
to mirror the characteristics, demographics, actions, and volumes of your company’s actual user
population. Utilizing the Gomez Last Mile panel of 25,000+ global testing agents – the largest
and only of its kind -- Reality Load XF can test an application under development or in production
by tapping a population that represents a companies’ real-life customer base, rather than from a
series of vendor designated “load centers”. Additionally, companies can determine which specific
geographical areas should receive the most load (e.g. Tokyo, California, Europe), giving developers
greater confidence that the code they are writing will work as intended for real-world customers
when the application goes live in production.

5
 W hite Pa p e r Managing User Experience and the Organization

Preview the Customer’s Web Experience. Once development teams are sure their applica-
tion will look and function properly for end-users across different browsers and OS’, it’s time to
transition the application to the Operations team and take testing one step closer to the cus-
tomer. Using Gomez’ active web application testing and monitoring products, companies can
test their applications in a pre-production environment to understand how they will perform for
customers around the world, without actually having to expose the applications to customers.

Private Locations XF
Validate the speed and reliability of your
partners and corporate end users

WEB APP
SERVER SERVER
LOCAL ISP

MAJOR
DATABASE PROVIDER CONSUMER

DESKTOPS
3RD PARTY
CONTENT
Active Last Mile XF
Extend performance insights down
to the desktop
Private Network XF Active Network XF
Behind-the-firewall network monitoring to Real-time monitoring from
quickly determine performance problems the Internet Backbone

MAJOR
PROVIDER

LOCAL ISP

ACTUAL
CUSTOMER

Actual Experience XF
Measure your online customer’s
experience from their browser

Gomez Monitoring Products

Active Network XF - Measure web application performance from the Internet backbone in
over 80 major global cities so you can baseline performance, troubleshoot potential issues,
and accurately measure improvements.
Active Last Mile XF - Measure web application performance from a population representa-
tive of your customer base by testing web application performance from over 25,000 broad
band and dial-up machines across the globe.
Private Locations XF – Measure web application performance from the locations most
important to you. Use Gomez software to create your own testing network of customers,
partners, branch offices, kiosks, or remote employees.
Private Network XF – Measure web application performance from inside your firewall to
determine whether a performance problem is hardware or network related.

6
Gomez, Inc. 10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com
T: 781.778.2700 F: 781.778.2799
W hite Pa p e r Managing User Experience and the Organization

Ensure Quality Web Experiences in Production. Because revenue is not generated in your
test environments or inside your firewall, Gomez provides the same active monitoring services
that companies can use for pre-launch testing to monitor and troubleshoot the performance
of those applications once they are released and customer facing. Additional monitoring tools
that leverage these agents, including Gomez ExperienceFirst Alerts, ExperienceFirst Dashboards,
and ExperienceFirst Screen Capture on Error (SCoE), and Performance Analyzer also serve to help
companies identify and resolve problems with pre- and post-production web applications more
quickly.

ExperienceFirst Dashboard
Measuring Experience from the Customer’s Browser. Using Actual Experience XF, compa-
nies gain an authentic picture of customer web experiences, individually and collectively, by mea-
suring performance directly from the end-users’ browsers as they visit web pages. As such, Actual
Experience XF is a real-user monitoring service that expands the ExperienceFirst platform to
address the new challenges facing both business line owners and Web 2.0 developers concerned
with the performance impact on customer web experience.
Using a few lines of proprietary code embedded on each page (similar to web analytics technol-
ogy), Actual Experience XF reports back on the end-user experience with the portion of the appli-
cation delivered by the servers and on the portion running locally in end-users’ browser (e.g. Web
2.0 technologies like Ajax and Flash).
What it measures. From its vantage point in the browser, Actual Experience XF measures all
aspects of the end user experience, including the perceived performance of a web application,
download times, abandonment rates, service consistency, geographic disparities, browser win-
dow size, browser type, cache utilization and first time/repeat visitor status. Armed with this infor-
mation, a company can:
Analyze the actual experience of specific customers and segments, and understand which
buyer segments have poor performance and why;
Know which pieces of a company’s website contribute to (or detract from) a favorable cus-
tomer experience, including third party content providers, use of multiple browsers, and rich
media types; and
Make better decisions as to what Web components to monitor and improve by analyzing
performance and how they impact business performance.
Actual Experience XF dives deeply into many factors influencing customer satisfaction and far
beyond what’s found in standard web analytics. For example, the tool can uniquely show the
impact of application performance on conversion by tracking both page level abandonment rates
(e.g., when a user clicks on the stop button of a browser to stop the page from downloading) and
business process level abandonment (e.g., when a user abandons a multi-step process before
reaching the last step, such as putting an item in the shopping cart but never checking out).

7
 W hite Pa p e r Managing User Experience and the Organization

Putting It All Together

The demand for superior web experiences has necessitated and accelerated changes in web-
site development considerably over the last few years. Sites have evolved from simple process
applications to more complex architectures. The emphasis is on providing a superior customer
experience with more interactivity and use of rich media. Much of the code that adds to the
experience resides in the browser, not on the company’s server. Yet, few developers today
know how well their local code is performing – a dangerous situation considering that some
combinations of the browser or operating system may not work as expected.
The Gomez ExperienceFirst Platform reaches beyond production and monitoring and back to
the early stages of development and QA. It emphasizes the user all the way from early develop-
ment to post-production and business analytics.
Using a single web-based user interface and delivered in on-demand model, business and
technical users alike can use the platform to understand how web performance impacts busi-
ness results and track performance of business processes at every stage of an application’s life-
cycle. With ExperienceFirst, designers, developers, QA teams, operations and business manag-
ers can benefit from a unified platform where they can employ the best tools to optimize their
sites for the best possible user experience.
With the ExperienceFirst platform, companies can:
Test the customer experience much earlier in Application Lifecycle;
Exploit a Web 2.0 ready pre/post deployment testing platform;
Continuously monitor actual & synthetic customer experience for all their web applications;
Leverage integrated scripting, reporting, alerting, and analytics on data from the World
Largest (Over +25,000 POPs) Experience Testing Network;
Improve collaboration on Web experience among design, operations and business manage-
ment; and
Better ensure quality Web experiences.

TM

Develop
& Test Monitor
Design

The first on-demand web experience

management platform focused on
Ensuring Quality Web Experiences
from the customer perspective.

8
Gomez, Inc. 10 Maguire Rd, Lexington, M A 02421-3119 www.gomez.com
T: 781.778.2700 F: 781.778.2799
 W hite Pa p e r Managing User Experience and the Organization

What can you do with Gomez?

If the goal is to improve your functional testing processes, get better insight in to customer
experience, improve the efficiency of Web application teams, or resolve problems more quickly,
the suite of on-demand products in the Gomez ExperienceFirst Platform can help.

Table 1. What Can You Do With Gomez?

Find web application problems at the user,

browser, application, network or server level.
Identify visual issues end users will experience
using different browsers and mobile devices.
Check functionality in QA & Development across
multiple browsers and operating systems to
minimize issues in production.
Test your website under load generated from
where it counts most -- at the user’s computer.
Validate the performance and readiness of ap-
plications from the end–user perspective prior
to launch.
Establish historical baselines of Web application
performance.
Monitor the performance of production ap-
plication continuously to find problems before
users are impacted.
Develop alert mechanisms that automatically
detect and create alarms for application and user
experience issues.
See competitive baselines that compare your
organization’s performance to others in your
industry.
Observe and measure the performance of your
third-party partners.
Hold partners and internal groups to standards
set in Service Level Agreements (SLAs).
Segment users by type, geography, and value to
your organization to measure experience.
Discover how web experience is impacting your
businesses bottom line.
Build common goals for response times,
availability, consistency and other experience
metrics.
Apply tools & Best Practices that can help your
team measure, monitor and improve end use
experience.
© Gomez 2008

9
W hite Pa p e r Managing User Experience and the Organization

Appendix A: Web Experience Management in Practice –

A Customer Example
Gomez agent-based active testing can be combined with real-user actual testing to reveal exactly how well
an application is performing under a variety of conditions for user populations around the world. The follow-
ing scenario illustrates how the Gomez tools work together to help several departments in a company, share
metrics, align on the same goals, and provide the best possible user experience.

A Retailer doing significant business in the United States is releasing a new Web 2.0 shopping
cart application, but wants to make sure that the release will not negatively impact the cus-
tomer web experience – and therefore the businesses revenue. Based on information captured
with Actual Experience XF, business managers have benchmarked the current end-user satisfac-
tion levels and also determined that the large majority of top customers in the US are accessing
the company’s Website using a variety of browser/OS combinations, but are primarily on one
ISP using low-broadband and dial-up connections around the Western US.
Armed with this information, the company’s developers went to work with the goals of meet-
ing or exceeding the current end-user performance and experience levels. Using Reality View
XF and Reality Check XF, the developers are able to test the code as they build it. To make sure
the new application both functions and appears as intended from all the different browser and
operating system combinations, including those of the high value customers.
Once the application had been developed, the quality assurance team decides that it’s ready to
put the new application to the test. Using information gathered with Actual Experience XF, the
company uses Gomez RealityLoad XF to create a load testing population scenarios that are rep-
resentative of their customer base. They then test the pre-production application with the real
traffic levels that it expects to see in the upcoming busy season.
Assured that scalability is not a concern, the operations team is now able to use Active Network
XF and Active Last Mile XF to test the beta application and measure how it responds from the
perspective of the low-broadband and dial-up customers. Fortunately, this testing exposes a
period issue with their third party content acceleration vendor that was causing a 50% increase
in response time. After sharing the data with the vendor, the company then validates that the
discussed fix has been implemented and has resolved the problem.
Satisfied with the results of the load test and the operations team’s initial testing, they share the
results and expected user experience levels internally to review that the initial goals have been
met. After getting approval from all the involved units, the operations team sets up continuous
monitoring to catch any new customer or application problems that might arise and launches
the new application into production with confidence that the new application will exceed the
quality of web experiences delivered by the old application.
The business managers, eager to measure the impact of the new release, quickly correlates the
new Actual Experience XF measurements with the old measurements to determine that they
have indeed successfully launched the new application, and improved performance for that key
customer segment more than anticipated, and achieved the web application experience goals
they set at the beginning of the process.

Gomez, Incorporated ©2008 by Gomez, Inc. All rights reserved. Gomez® is a registered service mark, and
ExperienceFirst™, ExperienceFirst SCoE™, ExperienceFirst Network™, Active Network™ XF,
10 Maguire Rd, Bldg 3, Suite 330 Lexington, MA 02421 Private Network™ XF, Active Last Mile™ XF, Private Locations™ XF, Actual Experience™ XF,
General: +1 781-778-2700 Reality Check™ XF, Reality Load™ XF, Reality View™ XF, the taglines “Ensuring Quality Web
Sales: USA +1 781-778-2760; UK +44 (0)207-554-5154; Experiences™”, “Quality of Web Experience™”, and QoWE™ are service marks of Gomez, Inc.
All other trademarks are the property of their respective owners.
Germany +49 (0)40-53299-167.
www.gomez.com Specifications may change. Consult the technical documentation for the most current information.
Some features described are extra cost options. Ask your sales representative for details.

wp2008003