Solution Brief

HTTP Optimization
Riverbed RiOS 6.1

HTTP Optimization

HTTP OPTIMIZATION
Overview
A typical Web page is not a single file that is downloaded all at once. Instead, Web pages are composed of
dozens of separate objectsincluding .jpg and .gif images, JavaScript code, cascading style sheets, and
moreeach of which must be requested and retrieved separately, one after the other. Given the presence of
latency, this behavior is highly detrimental to the performance of Web-based applications over the WAN.
The higher the latency, the longer it takes to fetch each individual object and, ultimately, to display the entire
page.
RiOS enables several mechanisms to further optimize HTTP(S) traffic, in addition to standard data streamlining and transport
streamlining functionality.
For static web content, a learning mechanism, that allows a client-side Steelhead to track the objects that are requested for a
particular web page, accelerates future requests by using the learned information and pre-fetching associated content. In
addition, HTTP(S) leverages the learned information to send normally sequential data requests in parallel creating additional
optimization benefits.
For dynamic web content, RiOS performs a parse-and-pre-fetch of embedded objects on dynamic web pages. When requests for
dynamic content occur, RiOS parses the retrieved dynamic HTML page and immediately pre-fetches embedded objects to
accelerate webpage load times. The net result is a significant reduction in roundtrips across the WAN for dynamic content that is
often leveraged by web-based enterprise applications.
Another performance enhancing feature for HTTP(S) is the object prefetch table. This enables the Steelhead appliance and
Virtual Steelhead datastore to cache complete web page objects, allowing these to be served up immediately as a whole locally
rather than reassembled from data references or transferred across the WAN. Unlike other cache approaches, consistency and
freshness is maintained as RiOS will always still deliver the latest version of the object being requested.
RiOS accelerates HTTP(S) traffic further by also performing authentication tuning. This eliminates roundtrips across the WAN and
minimizes delay for end-users.
This combined multilayer approach to HTTP(S) optimization delivers acceleration benefits to a range of web content and
application scenarios.
The RiOS HTTP optimization works for most web-based (HTTP and HTTPS) applications, including:

SAP
Microsoft SharePoint
Customer Relationship Management
Enterprise Resource Planning
Financials, Document Management
Intranet portals
Internet traffic (from branch, routed through data-center)

All HTTP optimization features are driven by the client-side Steelhead appliance. The following features can be used individually
or in combination with each other:

HTTP Optimization Features

Basic Tuning
Strip Compression In order to conserve bandwidth, nearly all browsers in use today support compression. In order to
maximize the benefit of SDR, data coming from the server should be uncompressed in order to allow SDR to de-duplicate this
data. When Strip Compression is enabled, the HTTP module will remove the Accept-Encoding line from the request header
before sending the request to the server. Since the modified request does not contain any supported compression scheme, the

2010 Riverbed Technology. All rights reserved.

HTTP Optimization

server will therefore respond to the request without any compression. This will then allow SDR to de-duplicate the data. Note that
with this option enabled, the amount of LAN-side traffic will increase as the server will no longer send the traffic in a compressed
format. Strip compression is enabled by default.
Insert Cookie The HTTP module relies on cookies to distinguish between different users. If the server does not support
cookies, the HTTP module inserts its own cookie so that it can distinguish between different users. This option should only be
enabled if the server does not issue its own cookies.
Insert Keep-Alive Keep-alive, or persistent connection, is required in order for the HTTP module to perform pre-fetches. The
Steelhead appliance uses an existing TCP connection between a client and a server to prefetch objects from the Web server that
it determines are about to be requested by the client. Many Web browsers open multiple TCP connections to the Web server
when requesting embedded objects. Enabling this feature allows the HTTP module to keep the connection open between the
server-side Steelhead appliance and the server and perform the necessary pre-fetches. However, the connection between the
client and the client-side Steelhead will be closed per the request of the client. Note that this option does not apply to situations
whereby the client supports keep-alive but the server does not.
Prefetch Schemes
URL Learning - The Steelhead appliance learns associations between a base request and a follow-on
request. This feature is most effective for Web applications with large amounts of static content, for
example, images, style sheets, and so on. Instead of saving each object transaction, the Steelhead
appliance saves only the request URL of object transactions in a Knowledge Base and then generates
related transactions from the list. This feature uses the Referer header field to generate relationships
between object requests and the base HTML page that referenced them and to group embedded
objects. This information is stored in an internal HTTP database. The following objects are retrieved by
default: .gif, .jpg, .css, .js, .png. You can add more object types to be retrieved.
Parse and Pre-fetch - The Steelhead appliance includes a specialized algorithm that determines which
objects are going to be requested for a given Web page and pre-fetches them so that they are readily
available when the client makes its requests. This feature complements the URL Learning feature by
handling dynamically generated pages and URLs that include state information.
Parse and Pre-fetch reads a page, finds HTML tags that it recognizes as containing a pre-fetchable object,
and sends out pre-fetch requests for those objects. Typically, a client would need to request the base
page, parse it, and then send out requests for each of these objects. This still occurs, but with Parse and
Pre-fetch the Steelhead appliance has quietly perused the page before the client receives it and has
already sent out the requests. This allows it to serve the objects as soon as the client requests them,
rather than forcing the client to wait on a slow WAN link.
For example, when an HTML page contains the tag <img src=my_picture.gif>, the Steelhead
appliance pre-fetches the image my_picture.gif because it parses an img tag with an attribute of src by
default. The HTML tags that are pre-fetched by default are base/href, body/background, img/src,
link/href, and script/src. You can add additional object types to be pre-fetched.
Object Prefetch Table - The Steelhead appliance stores object prefetches from HTTP GET requests for
cascading style sheets, static images, and JavaScript files. This helps the client-side Steelhead appliance
respond to If-Modified-Since (IMS) requests and regular requests from the client, thus cutting back on
round trips across the WAN. This feature is useful for applications that use a lot of cacheable content.
Authentication Tuning
Background
The HTTP module supports authentication optimization, for handling the various inefficient browser authentication behaviors.
Furthermore, the HTTP authentication optimization attempts to modify the client-to-server behavior in such a way that it would
maximize the benefit of the HTTP module.
The default authentication behavior on Microsofts IIS server is per-request authentication for Kerberos and per-connection
authentication for NTLM.

2010 Riverbed Technology. All rights reserved.

HTTP Optimization

NTLM is a Microsoft authentication protocol which employs a challenge-response mechanism for authentication,
in which clients are required to prove their identities without sending a password to a server. NTLM
requires the transmission of three messages between the client (wanting to authenticate) and the server
(requesting authentication).
Kerberos replaces the challenge/response-based NTLM protocol used in Windows NT. It provides a number of significant
advantages:
Eliminates need for server to contact domain controller to verify presented credentials.
Allows clients to verify identity of servers (mutual authentication)
Supports delegated authentication (impersonation) with forward-able and proxy-able tickets
Simplifies management of trust relationships. Tickets issued by a domain are valid anywhere within the domain's forest.
Kerberos utilizes symmetric encryption (and optionally asymmetric PKI encryption) in combination with a set of trust relationships
to allow clients and servers to establish secure connections based on knowledge of shared keys. Clients and servers trust Key
Distribution Centers (KDCs) to maintain the keys and construct properly encrypted tickets granting clients access to services.
Steelhead Authentication Tuning Features
Re-use NTLM Auth With URL Learning and Parse-and-Prefetch, a particular request from the browser can trigger the HTTP
module to pre-fetch the objects of a webpage. The browser would typically open parallel TCP connections to the server to
download the objects. If the Re-use NTLM Auth feature is not enabled, the HTTP module cannot serve out the objects to the
client even though it may already have the objects in its database. With Re-use NTLM Auth enabled, the HTTP module assumes
that the session has already been authenticated and hence it would be safe (i.e. without violating any permissions) to serve the
pre-fetched objects to client regardless of whether the connection is authenticated or not. This feature has the same effect as if
the client only uses a single connection to download all the objects in serial. This feature assumes that the server is configured
with per-connection NTLM authentication.
Force NTLM When the server is configured for per-request authentication, two features from the HTTP module can no longer
function: URL Learning and Parse-and-Prefetch. The default authentication behavior on Microsofts IIS server is per-request
authentication for Kerberos and per-connection authentication for NTLM. For this reason, the HTTP module can be configured to
change the client-to-server negotiation so that the client will choose an authentication that will maximize the benefit of the HTTP
module. Although Kerberos provides stronger security, it is less efficient over the WAN because of the per-request authentication
and secondly because the client must contact the Doamin Controller to answer the server authentication challenge.
Strip Auth Header With the Strip Auth Header feature enabled, when the HTTP module detects an authentication request on
an already authenticated connection, it will remove the authentication header before sending the request to the server. Since the
connection is already authenticated, the server should deliver the object to the client without having to go through the entire
authentication process again. This feature assumes the server is configured for per-connection authentication. If this is not the
case (i.e. per-request authentication is in-use), then do not enable this feature.
Gratuitous 401 this feature can be used with both per-request and per-connection authentication but its most effective when
used with per-request authentication. With per-request authentication, every request must be authenticated against the server
before the server would serve the object to the client. However, most browsers do not cache the servers response requiring
authentication and hence it will waste one round-trip for every GET request. With Gratuitous 401, the client-side Steelhead
appliance will cache the server response and when the client sends the GET request without any authentication headers, it will
locally respond with a 401 Unauthorized message and therefore saving a round trip. Note that the HTTP module does not
participate in the actual authentication itself. What the HTTP module does is to inform the client that the server requires
authentication without requiring it to waste one round trip.

2010 Riverbed Technology. All rights reserved.

HTTP Optimization

HTTP Optimization Implementation

All HTTP optimization features are driven by the client-side Steelhead appliance. The client-side Steelhead
appliance sends the pre-fetched information to the server-side Steelhead appliance. Pre-fetched data and
object pre-fetches are served from the client-side Steelhead appliance upon request from the browser.
Optimization schemes that applies to all HTTP traffic can be set-up, or else individual schemes for each
server subnet can be created. Therefore, you can configure an optimization scheme that includes your choice of pre-fetch
optimizations for one range of server addresses, with that range encompassing as large a network as you
need, from a single address to all possible addresses.

The following situations might affect HTTP optimization:

Fat Client - Not all applications accessed through a Web browser use the HTTP protocol. This is
especially true for fat clients that run inside a Web browser which might use proprietary protocols to
communicate with a server. HTTP optimization does not improve performance in such cases.
Digest for Authentication - Some Web servers might require users to authenticate themselves before
allowing them access to certain Web content. Digest Authentication is one of the less popular
Authentication schemes, although it is still supported by most Web servers and browsers. Digest
Authentication requires the browser to include a secret value which only the browser and server know
how to generate and decode. Because the Steelhead appliance cannot generate these secret values, it
cannot pre-fetch objects protected by Digest Authentication.

2010 Riverbed Technology. All rights reserved.

HTTP Optimization

Authentication Parameters
The following table lists the recommended authentication features to enable on the HTTP module, based on the application IIS
Authentication method.

IIS Authentication
per-connection NTLM
per-request Kerberos

per-request NTLM
per-connection Kerberos

dont know

Recommended Configuration
Reuse Auth. + Strip Auth. Header + Grat. 401
Reuse Auth. + Strip Auth. Header
Reuse Auth. + Force NTLM + Strip Auth. Header +
Grat. 401
Reuse Auth. + Force NTLM + Strip Auth. Header
Grat. 401
Grat. 401
Reuse Auth. + Force NTLM + Strip Auth. Header
Reuse Auth.
Change to per-request Kerberos and turn on Grat.
401.
Reuse Auth. + Strip Auth. Header + Grat. 401
Re-use Auth. + Strip Auth. Header

Notes
N/A if NTLM is not an option
N/A if NTLM is not an option
N/A if NTLM is not an option

Recommended HTTP Parse & Prefetch features per application

The HTTP latency optimizations features target different types of Web applications.
The following table provides the general guidelines on HTTP optimization features to enable, based on application. Note that
applications may be tuned to suit the particular customer deployment, thus it is advisable to test the specific application with the
HTTP module features, to ensure optimal results are achieved.

URL Learning

Parse and
Prefetch

OPT

SAP/Netweaver

No

No

Yes

Microsoft CRM

Yes

No

Yes

Agile

No

No

Yes

Pivotal CRM

No

Yes

Yes

Sharepoint

No

Yes

Yes

2010 Riverbed Technology. All rights reserved.

HTTP Optimization

Optimizing Internet Traffic

For many customers, access to the Internet from the branch offices may be routed to the data center first, before going out to the
Internet see diagram 1 below.

Internet

Corporate WAN

Diagram 1: Accessing the internet via the data-center

Internet-bound traffic for remote sites is often routed to the data-center and then uses the Internet connection at the data-center to
get to the outside world. The question thus arises should this traffic bound for the Internet be routed through the Steelheads and
HTTP optimization enabled?
The recommendation in general is yes enabling HTTP optimization can reduce the amount of traffic on the local connection,
freeing bandwidth for other applications that may need it and also improve the overall user experience. Things to consider
however include the latency between the server-side Steelhead and the server (i.e. in this case the internet). If this latency is
much greater than the latency between the client-side Steelhead and server-side Steelhead, it is recommended to disable Parse &
Prefetch and URL Learning.

Sharepoint 2010 Performance Improvement with HTTP module

Sharepoint 2010 performance improvements of the order of 28x can be achieved with Riverbed Steelhead.
The HTTP module in RiOS also includes the ability to optimize authentication and authorization traffic. Sharepoint can benefit from
the HTTP authentication features by mitigating the chatty behavior of authorization requests required for each object.

The following settings were enabled on the Steelhead RiOS HTTP application streamlining module for Sharepoint 2010 testing:
RiOS HTTP blade
Strip Compression
Parse & Pre-fetch
Object Prefetch Table
Gratuitous 401

Table 1: HTTP features enabled for optimizing Sharepoint traffic

2010 Riverbed Technology. All rights reserved.

HTTP Optimization

Testing showed Sharepoint 2010 performance improvements of the order of 28x for download operations..
Sharepoint 2010 Test
Description

Time without Steelhead (seconds)

Time with Steelhead (Cold)

Time with Steelhead

(Warm)

Performance Improvement

HTTP 6.3MB .ppt File

Download

37

16

1.3

28x

HTTP 6.5MB .doc File

Upload

42

1.5

28x

HTTPS 6.3MB .xls

File Download

28

28x

Table 2: Performance Improvement results for Sharepoint 2010 with Riverbed Steelhead

Bandwidth usage was reduced by up to 99% with the Steelhead appliance

Sharepoint 2010 Test
Description

Bytes transferred without

Steelhead

Bytes transferred with

Steelhead (cold)

Bytes transferred with

Steelhead (warm)

Bandwidth Reductrion

HTTP 6.3MB ppt Upload

14665490

1430005

252814

98%

HTTPS Word .doc Edit

7622124

509276

342444

96%

Table 3: Bandwidth Reduction results for Sharepoint 2010 file with Riverbed Steelhead

Conclusion
By optimizing the performance of HTTP traffic over the network, users can defer upgrading bandwidth and may also be able to
leverage a smaller and less expensive network connection. Depending on the distances involved, this could result in significant
savings. Riverbed also can help maintain a centralized deployment model for HTTP applications, which can effectively service
distributed workers. This can result in reduced IT maintenance costs.
Riverbed WAN optimization solutions enable enterprises to maximize the benefits of HTTP applications across a distributed
environment by providing a very positive and productive user experience.

About Riverbed
Riverbed Technology is the IT infrastructure performance company. The Riverbed family of wide area network (WAN) optimization
solutions liberates businesses from common IT constraints by increasing application performance, enabling consolidation, and
providing enterprise-wide network and application visibility all while eliminating the need to increase bandwidth, storage or
servers. Thousands of companies with distributed operations use Riverbed to make their IT infrastructure faster, less expensive
and more responsive. Additional information about Riverbed (NASDAQ: RVBD) is available at www.riverbed.com

Riverbed Technology, Inc.

199 Fremont Street
San Francisco, CA 94105
Tel: (415) 247-8800
www.riverbed.com

2010 Riverbed Technology. All rights reserved.

Riverbed Technology Ltd.

Farley Hall, London Road, Level 2
Binfield, Bracknell
Berks
RG42 4EU
Tel: +44 1344 401900

Riverbed Technology Pte. Ltd.

391A Orchard Road #22-06/10
Ngee Ann City Tower A
Singapore 238873
Tel: +65 6508-7400

Riverbed Technology K.K.

Shiba-Koen Plaza Building 9F
3-6-9, Shiba, Minato-ku
Tokyo, Japan 105-0014
Tel: +81 3 5419 1990