Cisco ACE Training

VDC Design ACE Training
November 2008
Andrew Holding
andrew.holding@bt.com
British Telecommunications plc
Agenda
Scope ACE Overview What is the ACE Module? Software, Licences and Virtualisation Deployment options Resource Management Redundancy Key ACE terminology Layer 3-7 Server Load Balancing TCP Review Layer 3-4 Server Load Balancing Layer 7 Server Load Balancing Configuration Examples & Important ACE features Layer 3, Layer 4, Layer 7 load-balancing Route Health Injection Persistent Rebalance ACE Connection handling Stickiness Source IP, Cookie SNAT SSL ACE Configuration Spreadsheet ACE Documentation Q & A
Agenda
Scope
The scope of this training is to ensure that network designers understand the ACE topology and the configuration options used within the VDC design This is a high-level training to explain basic features and ACE behaviour It is assumed that attendees have basic load-balancing knowledge
Agenda
What is the ACE Module?
Application Control Engine
Layer 3-7 content-aware, virtualised, application loadbalancer with SSL termination & initiation and security
What is the ACE Module?
CSM
ACE Module
Appliances
CSS 11506
ACE Appliance
Cat6K Modules
CSS 11503
CSS 11501
The Evolution of L4 to L7 Services

Previous
Now with Application Control Engine
Integrated Layer 4 and Layer 7 Rules
Infrastructure simplification with L47 Services integration Converged policy creation, management, and troubleshooting Reduced latency (single TCP termination for all functions)
ACE Hardware Architecture
Control Plane SAN OS

100M 2G
NP1
10G
NP2
10G 8G
Sup Connect
CDE Switch 60Gbps

16G 10G
Daughter Card 1
8G
Switch Fabric Interface

Daughter Card 2
SSL Crypto
ACE Performance/Features
Max of 4 ACEs per chassis (64Gbps) 4Gbps, 8Gbps, 16Gbps single link to Backplane 4Million Concurrent connections ~350K L4 connections per second
Onboard SSL Offload (1K to 15K tps throughput)

Virtualisation (250 Contexts) TCP Reuse
DDoS protection
etc
Agenda
ACE software versions

Version number for ACE: Based on SanOS release BU identifier is ACE software version 3.0(0)A1(6.3b) 3.0(0) A 1.6(3b)
SanOS info has now (A2.x) been dropped for simplification;

show ver :Software loader: Version 12.2[118] system: Version A2(1.1) [build 3.0(0)A2(1.1) adbuild_00:25:02-2008/06/05_/a uto/adbu-rel3/rel_a2_1_1_throttle/REL_3_0_0_A2_1_1] system image file: [LCP] disk0:c6ace-t1k9-mz.A2_1_1.bin installed license: ACE-08G-LIC ACE-VIRT-050 ACE-SEC-LIC-K9 ACE-SSL-05K-K9
Note: ACE Module and ACE Appliance use different software images
ACE software versions (contd)

BNCMNSSW01>show mod Mod Ports Card Type --- ----- -------------------------------------1 1 Application Control Engine Module 2 6 Firewall Module 3 48 CEF720 48 port 10/100/1000mb Ethernet 4 1 SSL Module 5 2 Supervisor Engine 720 (Active) 6 4 SLB Application Processor Complex 7 4 SLB Application Processor Complex 8 4 CEF720 4 port 10-Gigabit Ethernet 9 4 CEF720 4 port 10-Gigabit Ethernet
Model -----------------ACE10-6500-K9 WS-SVC-FWM-1 WS-X6748-GE-TX WS-SVC-SSL-1 WS-SUP720-3B WS-X6066-SLB-APC WS-X6066-SLB-APC WS-X6704-10GE WS-X6704-10GE
Serial No. ----------SAD1021076N SAD100202V9 SAL1005CBZP SAD094307LT SAL1004BPJU SAD1006061M SAD100301YX SAL1005C12A SAD100204FK Status ------Ok Ok Ok Ok Ok Ok Ok Ok Ok
Mod MAC addresses Hw Fw Sw --- ---------------------------------- ------ ------------ -----------1 0030.f275.b454 to 0030.f275.b45b 1.1 8.7(0.22)ACE A2(1.1) 2 0013.c39f.63f8 to 0013.c39f.63ff 4.0 7.2(1) 3.2(4) 3 0016.c810.3284 to 0016.c810.32b3 2.3 12.2(14r)S5 12.2(18)SXF1 4 0030.f274.f702 to 0030.f274.f709 4.0 7.2(1) 2.1(9) 5 0013.c43a.8cb0 to 0013.c43a.8cb3 4.5 8.1(3) 12.2(18)SXF1 6 0013.c39f.cce0 to 0013.c39f.cce7 1.9 4.2(3a) 7 0013.c39f.8530 to 0013.c39f.8537 1.9 4.2(3a) 8 0016.c75a.a700 to 0016.c75a.a703 2.2 12.2(14r)S5 12.2(18)SXF1 9 0015.62e1.aee8 to 0015.62e1.aeeb 2.2 12.2(14r)S5 12.2(18)SXF1
ACE licensing
Base = 5 contexts (plus Admin), 1000 SSL tps, 4Gbps Contexts = 50, 100 or 250 Throughput = 8 or 16Gbps SSL = 5,000, 10,000, 15,000 tps
bncmnace02/Admin# show license status Licensed Feature -----------------------------SSL transactions per second Virtualized contexts Module bandwidth in Gbps Count ----5000 50 8
bncmnace02/Admin# show ver . Software loader: Version 12.2[118] system: Version A2(1.1) [build 3.0(0)A2(1.1) adbuild_00:25:02-2008/06/05_/a uto/adbu-rel3/rel_a2_1_1_throttle/REL_3_0_0_A2_1_1] system image file: [LCP] disk0:c6ace-t1k9-mz.A2_1_1.bin British Telecommunications plc installed license: ACE-08G-LIC ACE-VIRT-050 ACE-SEC-LIC-K9 ACE-SSL-05K-K9
ACE Virtualisation
One Physical Device
100%
Multiple Virtual Systems (Dedicated Control and Data Path)

25% 25% 15% 15% 20%
Traditional Device: Cisco Application Services Virtualisation:

Single configuration file Single routing table Limited RBAC (Role Based Access Control) Limited resource allocation Distinct configuration files Separate routing tables RBAC with contexts, roles, domains Management and data resource control Independent application rule sets Global administration and monitoring
ACE Multiple Contexts

Physical Device
Admin Context
Context Definition, Resource Allocation, FT Config
Context 1
Context 2
Context 3
Management station AAA
Admin Context + 250 Contexts (Licensed: five contexts in base code)

Agenda
ACE Deployment
Physical View
Web Client
6500 with ACE
Web Server
ACE VLANs
Logical View
Catalyst 6500
Web Client
Client-side VLAN
ACE
Server-side VLAN
Web Server
Bridged (Layer 2) Mode
Server Default Gateway: Upstream Router
ACE Bridging
VLAN 10
VLAN 20 Subnet A
Routed (Layer 3) Mode
Server Default Gateway: ACE IP
ACE Routing
Subnet A VLAN 10
Subnet B VLAN 20
One-Armed Mode
Server Default Gateway: Upstream Router Subnet C VLAN 30
Subnet A VLAN 10
Subnet B VLAN 20
ACE not in path PBR or SNAT required for return traffic

Routed, Bridged or One-Armed Mode?

All of these modes can be mixed within, and between, contexts - the same context can have bridged interfaces, routed interfaces and one-armed interfaces
Advantages of bridged vs routed are; + Routing protocols can be exchanged through the ACE + Multicast packets can be passed through the ACE Disadvantage of bridged vs routed;
Potential for bridge-loop if both ACEs go active-active (RPVST+ used to minimise impact. Note: MST not supported) If SNAT required, then traffic must be load-balanced
One-armed (ACE is not inline for load-balanced traffic)

+ Removes potential bottleneck - PBR or SNAT required
VDC ACE Topology

Firewall Block
%cust1-fw1-vrf-name%
VLAN 501
EIGRP
VLAN%cust1-ace1-ns-vlan% %ace-blade1-hostname%-001/002
Subnet A
ACE Block
VLAN 601
VLAN7 VLAN%cust1-ace1-ss-vlan%
%cust1-ace1-vrf-name%
ACE has a static default route with a next-hop of the FW1 VRF, and server-subnet routes with a next-hop of the Cust VRF
ACE Interface Configuration

Routed interfaces:
interface vlan 231 description Client vlan ip address 172.16.31.5 255.255.255.0 no shutdown
Bridged interfaces:
interface vlan bridge-group no shutdown interface vlan bridge-group no shutdown 231 3 232 3
interface bvi 3 description Server Access vlan ip address 172.16.31.5 255.255.255.0 no shutdown
Which slot is the ACE in?

Cat6k>show mod Mod Ports Card Type --- ----- -------------------------------------1 1 Application Control Engine Module 2 6 Firewall Module 3 48 CEF720 48 port 10/100/1000mb Ethernet 4 1 SSL Module 5 2 Supervisor Engine 720 (Active) 6 4 SLB Application Processor Complex 7 4 SLB Application Processor Complex 8 4 CEF720 4 port 10-Gigabit Ethernet 9 4 CEF720 4 port 10-Gigabit Ethernet Model -----------------ACE10-6500-K9 WS-SVC-FWM-1 WS-X6748-GE-TX WS-SVC-SSL-1 WS-SUP720-3B WS-X6066-SLB-APC WS-X6066-SLB-APC WS-X6704-10GE WS-X6704-10GE Serial No. ----------SAD1021076N SAD100202V9 SAL1005CBZP SAD094307LT SAL1004BPJU SAD1006061M SAD100301YX SAL1005C12A SAD100204FK Status ------Ok Ok Ok Ok Ok Ok Ok Ok Ok
Mod MAC addresses Hw Fw Sw --- ---------------------------------- ------ ------------ -----------1 0030.f275.b454 to 0030.f275.b45b 1.1 8.7(0.22)ACE A2(1.1) 2 0013.c39f.63f8 to 0013.c39f.63ff 4.0 7.2(1) 3.2(4) 3 0016.c810.3284 to 0016.c810.32b3 2.3 12.2(14r)S5 12.2(18)SXF1 4 0030.f274.f702 to 0030.f274.f709 4.0 7.2(1) 2.1(9) 5 0013.c43a.8cb0 to 0013.c43a.8cb3 4.5 8.1(3) 12.2(18)SXF1 6 0013.c39f.cce0 to 0013.c39f.cce7 1.9 4.2(3a) 7 0013.c39f.8530 to 0013.c39f.8537 1.9 4.2(3a) 8 0016.c75a.a700 to 0016.c75a.a703 2.2 12.2(14r)S5 12.2(18)SXF1 9 0015.62e1.aee8 to 0015.62e1.aeeb 2.2 12.2(14r)S5 12.2(18)SXF1
Configuring ACE VLANs
Create the necessary VLANs on the Cat6k. Group the VLANs into service line card VLAN groups. Assign the VLAN groups to individual ACE modules.
vlan 7,2001-2003, 3502,3504 svclc multiple-vlan-interfaces svclc module 1 vlan-group 1 svclc vlan-group 1 7,2001,2002
Configuring ACE VLANs (contd)
Configuring ACE VLANs (contd)
Verify Cat6k Setup

Cat6k>show svclc vlan-group Display vlan-groups created by both ACE module and FWSM commands
Group ----1 2 3
Created by ---------ACE FWSM ACE
vlans ----7, 2001-2002 201-206,401-406,999-1000 2003
Cat6k>show svclc module Module Vlan-groups ------ ----------01 1,3
Cat6k>show firewall module Module Vlan-groups ------ ----------02 2,3

v401 Group 2
v2003 Group 3
v2001 Group 1
Accessing the ACE
Connect to the ACE from IOS:

Cat6k#session slot 1 processor 0
Processor 0 = Control Plane CPU for configuration Processor 1 = NP1 Processor 2 = NP2
Creating ACE Contexts

1. Create Context from within Admin context 2. Allocate Interfaces
bncmnace02/Admin# show vlan Vlans configured on SUP for this module vlan7 vlan2001-2003 bncmnace02/Admin#config Enter configuration commands, one bncmnace02/Admin(config)# context bncmnace02/Admin(config-context)# bncmnace02/Admin(config-context)# bncmnace02/Admin(config-context)# bncmnace02/Admin(config)# exit
per line. End with CNTL/Z. development allocate-interface vlan 7 allocate-interface vlan 2001-2003 exit
Verifying ACE Setup
ACE-Module/Admin# show context development
Name: development , Id: 117 Description: Resource-class: default Vlans: Vlan7, Vlan2001-2003
ACE-Module/Admin# show run Generating configuration.... context development allocate-interface vlan 7 allocate-interface vlan 2001-2003
Accessing ACE Contexts

From Admin Context
bncmnace02/Admin# changeto development bncmnace02/development#
Access new context
or can Telnet/SSH direct to management interface of the relevant context (once it has been created)
[Prompt shows ACE hostname and current context]

Agenda
ACE Resource Management

Per Context Control:
Resource levels for each context Support for oversubscription
Rates
Memory
Bandwidth Data connections per sec. Management connections per sec. SSL bandwidth Syslogs per sec.
Access lists Regular expressions Data connections Management connections SSL connections Xlates Sticky entries
Maximum Unlimited
Maximum Equal To Minimum
Minimum Guarantee
Minimum Guarantee
Total ACE resources
Oversubscribed Global Pool (unreserved resources)
Context 4 Minimum Context 3 Minimum Context 2 Minimum Context 1 Minimum

ACE-Module/Admin(config)# resource-class gold ACE-Module/Admin(config-resource)# limit-resource all minimum 10% maximum unlimited
ACE-Module/Admin(config)# context development ACE-Module/Admin(config-context)# member gold

ACE-Module/Admin# show resource allocation ----------------------------------------------------------Parameter Min Max Class ----------------------------------------------------------acl-memory 0.00% 100.00% default 20.00% 200.00% gold syslog buffer ... 0.00% 20.00% 100.00% 200.00% default gold
default resource class = 0% minimum, unlimited maximum gold resource class = 10% minimum, unlimited maximum Looking at the above figures, the gold class is applied to 2 contexts, meaning there is a 200% oversubscription By default a context is a member of the default resource group
ACE Resource Management gotchas

Only allocate the minimum resources required/estimated initially (its hard to recoup resources later), and ensure you have a reserve Unlike other resources, sticky resources are not allocated by using the all keyword. Sticky resources must be allocated individually if required
resource-class gold limit-resource all minimum 20.00 maximum equal-to-min limit-resource sticky minimum 20.00 maximum equal-to-min
Bandwidth value is shown in Bytes (not Bits)

bncmnace02/Admin# show resource usage Allocation Resource Current Peak Min Max Denied -------------------------------------------------------------------------Context: development <snip> throughput 316 6125 0 500000000 0 <snip>
500,000,000Bps = 4Gbps
ACE Resources and Licence Upgrades

ACE licence can be upgraded from 4-8-16Gbps, and SSL 1K, 5K and 15K SSL tps These ACE resources can be limited however a percentage figure is used, not an absolute amount This means the amount of resources allocated will vary depending upon the current licence
20% of the 4Gbps licence is 800Mbps, whereas 20% of the 8Gbps = 1.6Gbps 10% of 1000 SSL tps = 100tps, whereas 10% of 5000 SSL tps = 500 tps
When upgrading an ACE licence, the percentage figure in the resource-class does not change, therefore you must change the percentage allocated if you want the same amount of resources to be allocated to members of that resource-class after the upgrade
Agenda
ACE Redundancy
Two ACEs form a Redundancy pair Single FT VLAN required between ACEs (not one per context) Redundant ACEs can be in the same, or different, Catalyst 6500 Chassis
Each pair of contexts (on two distinct ACE modules) form a redundancy group, one being active and the other standby Both ACE modules can be active at the same time, processing traffic for different contexts, and backing-up each other (stateful redundancy)
ACE-1
Example: 2 ACE modules 4 FT groups 4 Virtual Contexts (A,B,C,D)

A
Active FT VLAN
B
Active
C C
Active
FT group 3
D D
Active
FT group 4
Standby Standby
A
ACE-2
FT group 1
B
FT group 2
Standby Standby
ACE Redundancy
Fault-Tolerant (FT) VLAN (/30) carries FT packets, heart beats, config-sync packets, state replication packets Configuration synchronisation (bulk and incremental) & state replication is enabled by default SSL files (keys and certs) are not replicated Much like HSRP, each Context is assigned a priority, and the highest priority will become master (if pre-emption enabled) Normally recommend pre-emption is only used for operations (failing back to a recovered ACE) Possible to oversubscribe resources on both ACEs (active/active), however, a failure of one of the ACEs (or path to the ACE) will reduce capacity by half
Agenda
Key ACE terms

North (Client) side & South (Server) side VLANs Real Server load-balanced servers Serverfarm a group of Real servers Probe keepalive to Real Servers Predictor load-balancing algorithm (e.g. roundrobin, least-connections etc) VIP Virtual IP Address. Typically NATed to the address of the real-servers. Has no dependence on connected subnets. Route-Health Injection (RHI). ACE Module* can advertise the reachability of the VIP to the MSFC
* RHI not supported on ACE Appliance
ACE Key Terms
Server Farm RHI if VIP is Active
Client/North side VLAN
Server/South side VLAN Real Server
Think of the ACE as a Firewall By default, traffic is not allowed through or to the ACE Access-list type management is required for traffic to the ACE IP access-list is required for traffic through the ACE
N.B. Access-list type ethertype required in order to allow STP BPDUs (when ACE is in Bridged mode)

access-list nonip ethertype permit bpdu access-list permit-all line 10 extended permit ip any any interface vlan 2001
description Client_VLAN
bridge-group 1 access-group input nonip access-group input permit-all no shutdown interface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip
access-group input permit-all

no shutdown interface bvi 1
ip address 10.1.1.4 255.255.255.0 no shutdown
ACE routes
Routes are not shared between contexts Each load-balancing context requires route(s) to servers AND a route back to the client, before forwarding traffic Admin context will typically only need management routes Within VDC each Context requires;
the default route will have a next-hop of the North-side VRF HSRP address Route to server subnets with next-hop of South-side HSRP address Management route(s)
ip ip ip ip route route route route 0.0.0.0 0.0.0.0 10.80.199.109 default route 10.80.202.0 255.255.255.192 10.80.199.94 route to rservers 10.80.196.0 255.255.254.0 10.80.193.3 management route 147.149.163.128 255.255.255.128 10.80.193.3 management route
ACE Real Server Health Monitoring

- Out-of-band monitoring (Probes/Keepalives) - Probes can be used to - Detect the loss of a real server - Monitor a gateway or other remote device for failover purposes - Optional port and ip-address probe configuration - Multiple different native probe types including TCL support - Typically recommend a frequent simple probe (e.g. ping every 5 seconds) combined with a less-frequent more complicated probe (e.g. HTTP GET every 30 seconds). If either probe fails, the server will be declared down
Rservers, ServerFarms, Predictors and Probes

probe icmp ping interval 5 passdetect interval 120 receive 5 probe tcp tcpprobe port 80 interval 30 open 5 probe http httpprobe port 81 interval 30 passdetect interval 300 request method get url /index.shtm expect status 200 299 open 5 rserver host ip address probe ping inservice rserver host ip address probe ping inservice server1 10.1.4.101
server2 10.1.4.102
serverfarm host farm1 predictor leastconns probe tcpprobe rserver server1 inservice rserver server2 inservice serverfarm host farm2 probe httpprobe rserver server1 81 inservice rserver server2 81 inservice
Agenda
TCP Connection
Client
Server
SYN
Initialize
ACK Data
SYN_ACK
ACK
Use
ACK FIN
Data More Data
Close
ACK
ACK FIN
Agenda
ACE Load-Balancing Configuration

1. Create L3/L4 class map (define match criteria) 2. Create load-balancing policy map (define actions to perform) 3. Create a multi-match policy map to tie the L3/L4 class-maps and policy maps together 4. Activate the classification-action rules on either an interface or globally
class-map C1 match <criteria> policy-map type loadbalance P1 <action>
policy-map multi-match MMP1 match C1 policy P1 match C2 policy P2
interface vlanX service-policy input MMP1

ACE Load-Balancing Configuration

L3/L4 Class-map defaults to match-all, which means only one VIP address is allowed
bncmnace02/dev(config)# class-map fred bncmnace02/dev(config-cmap)# match virtual-address 1.1.1.1 tcp eq 80 bncmnace02/dev(config-cmap)# match virtual-address 1.1.1.1 tcp eq 443 Error: Only one match virtual-address is allowed in a match-all class-map and it cannot mix with any other match type bncmnace02/dev(config-cmap)#
match-any L3/L4 Class-map allows multiple VIPs

class-map match-any fred 2 match virtual-address 1.1.1.1 tcp eq www 3 match virtual-address 1.1.1.1 tcp eq https
Layer 3 & Layer 4 Load-balancing

L3 & L4 information is present in the first packet of the flow:
Source IP address Destination IP address IP Protocol Protocol ports
Load-balancing can be made on first packet of a flow
Layer 3/4 Flow Setup
SYN Identifies VIP (matches class-map) Selects Server Farm Makes Load Balancing Decision SYN
Layer 3/4 Flow Setup
SYN Identifies VIP (matches class-map) Selects Server Farm Makes Load Balancing Decision SYN
SYN_ACK
ACK Data
Agenda
Layer 7* Flow Setup

L7* load-balancing:
URL parsing Cookie parsing Generic HTTP header parsing SSL ID etc
Requires TCP termination and buffering of multiple packets before a LB decision can be made (this is why L7 load-balancing can never be as fast as L4 load-balancing)
* Layer 5 and above (SSL is Layer 5)
Sniffer Trace of HTTP Connection
Interesting information only arrives in the 4th packet
GET /css/cavendish/template.css
Layer 7 Flow Setup (1/3)
SYN Chooses seq# and replies w/ SYN_ACK SYN_ACK
SYN Chooses seq# and replies w/ SYN_ACK
SYN_ACK
ACK Starts buffering client packets Data (e.g. HTTP GET /) ACKs data received from client ACK
Buffers all packets until it has enough data for policy matching Elects serverfarm, makes balancing decision Sends previously buffered SYN to real server SYN
Buffers all packets until it has enough data for policy matching Elects serverfarm, makes balancing decision Sends previously buffered SYN to real server SYN SYN_ACK Does not forward SYN_ACK ACK
Empties buffer and sends data to server Data ACK Does not forward ACK Starts splicing the flows
Empties buffer and sends data to server Data (e.g. HTTP GET /) ACK Does not forward ACK Starts splicing the flows Data ACK
Agenda
ACE test topology

.100
3/21 54:be
VLAN 2004 10.1.2.0/24
.1 bt-fwsm-ace .5
VLAN 2001 10.1.1.0/24
Context landing
.4
b4:55
VLAN 2002 10.1.1.0/24
.6 bt-customer .1
VLAN 2006 10.1.4.0/24
3/22 3/23
.101
10Port81
0d:17
ef:5e
.102
11Port81
Sets Cookie serverid=server1
Agenda
ACE Layer 3 Policy

Destination IP address of incoming packet must match the VIP address(es) in the class-map Any protocol Any port
ACE Layer 3 Policy

rserver host server1 ip address 10.1.4.101 inservice rserver host server2 ip address 10.1.4.102 inservice serverfarm host farm1 rserver server1 inservice rserver server2 inservice class-map match-all classmap1 2 match virtual-address 10.1.1.100 any policy-map type loadbalance first-match policy1 class class-default serverfarm farm1 policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1
interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdown interface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all no shutdown interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown ip route 0.0.0.0 0.0.0.0 10.1.1.5 ip route 10.1.4.0 255.255.255.0 10.1.1.6
ACE Layer 4 Policy

Destination IP address of incoming packet must match the VIP address(es) Protocol(s) must match Port(s) must match
ACE Layer 4 Policy

serverfarm host farm1 predictor leastconns probe tcpprobe rserver server1 81 inservice rserver server2 81 inservice class-map match-all classmap1 2 match virtual-address 10.1.1.100 tcp eq www policy-map type loadbalance first-match policy1 class class-default serverfarm farm1 policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active
interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdown interface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all no shutdown interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown ip route 0.0.0.0 0.0.0.0 10.1.1.5 ip route 10.1.4.0 255.255.255.0 10.1.1.6
ACE Layer 7 Policy

Destination IP address of incoming packet must match the VIP address(es) Protocol(s) must match Port(s) must match Layer 5-7 information (e.g. HTTP URL, Cookie, Header, SSL session ID etc) must match Note: Regular expression matching is case-sensitive by default
ACE Layer 7 Policy

Typically used; when traffic differentiation is required (e.g. *.jpg sent to farm of Cache Engines, everything else sent to the Web servers) when traffic manipulation is required (e.g. Cookie insert, HTTP Header insert) Performance is less than L3/L4 due to; Delayed Binding (Layer 7 ME required (depends on persistent rebalance))
ACE Layer 7 Policy

Layer 7 Class-maps & Policy-maps can be used to;
Match on HTTP URL Match on HTTP headers (cookie, language, host, browser, etc) Match on string within HTTP payload (not header) Insert/Delete/Modify HTTP headers (e.g. Insert ClientIP, rewrite URL etc) Match RADIUS, RDP, RTSP and SIP fields Generic TCP/UDP data parsing Match on Source-IP address Set IP QoS (DSCP) values TCP Connection re-use
Layer 7 class-maps can use a match-all, match-any, or use nested class-maps (match A or B or [C & D])
ACE Layer 7 Policy

serverfarm host farm1 rserver server1 inservice rserver server2 inservice serverfarm host caches transparent rserver cache1 inservice rserver cache2 inservice interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdown interface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all no shutdown interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown ip route 0.0.0.0 0.0.0.0 10.1.1.5 ip route 10.1.4.0 255.255.255.0 10.1.1.6
class-map match-all classmap1 2 match virtual-address 10.1.1.100 tcp eq www class-map type http loadbalance match-any checkforstatic 2 match http url .*\.jpg 3 match http url .*\.pdf policy-map type loadbalance first-match policy1 class checkforstatic serverfarm caches class class-default serverfarm farm1 policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1
Agenda
ACE Route Health Injection

ACE can advertise the reachability of a VIP to the MSFC. If the VIP goes down, the route is withdrawn. Appears as a /32 static route, with the next-hop of the ACE Allows the MSFC to redistribute the route and advertise using routing protocol VRF-aware Default AD = 77
BNCMNSSW01#show ip route vrf bt-fwsm-ace Routing Table: bt-fwsm-ace <Snip> 10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks C 10.1.2.0/24 is directly connected, Vlan2004 C 10.1.1.0/24 is directly connected, Vlan2001 D 10.1.4.0/24 [90/3072] via 10.1.1.6, 5d23h, Vlan2001 S 10.1.1.100/32 [77/0] via 10.1.1.4, Vlan2001 B* 0.0.0.0/0 [20/0] via 10.1.2.0 (bt-sc1-fusion), 7w0d
VDC ACE RHI
VRF redistributes static routes into EIGRP and advertises northwards ACE RHI injects active VIPs into Firewall Block VRF
Agenda
Persistence Rebalance
HTTP 1.0 requires a separate TCP connection for each HTTP request HTTP 1.1 supports persistent TCP connections, allowing pipelining of multiple HTTP requests within the same TCP connection Processing Layer 7 (within ACE HTTP ME) information is more resource intensive than simply checking Layer 4 information By default, once the ACE has made a Layer 7 (check URL, Language etc) decision on the first packet of a flow, (which farm/server), all subsequent traffic will be sent to that server (fast-switched) Persistence rebalance disables this feature
Persistence refers to a persistent TCP connection (multiple pipelined HTTP requests) Rebalance refers to whether traffic should be re-balanced to another serverfarm
Persistent Rebalance (contd)

Only required if need to check (or manipulate) every HTTP packet within the same (persistent) TCP connection e.g.;
URL *.jpg & *.gif send to cache engines HTTP Header Language=French send to French farm HTTP Header Insert insert information into EVERY HTTP packet (rather than only the first one)
Persistence rebalance is disabled by default on ACE (enabled by default on CSM) HTTP parameter-map required to modify behaviour
Agenda
ACE Connection Handling

ACE can handle a maximum of 4 million concurrent connections Will continually monitor all connections to check whether the connection has closed, and resources can be freed and made available for new connections TCP is normally simple watch for FIN or RST Impossible to tell for UDP, or broken TCP connections
ACE Connection Handling

ACE idle timers TCP default = 1 hour UDP default = 2 minutes ICMP default = 2 seconds DNS, RADIUS etc LB may need to reduce the timeout so the connection entry does not stay up unnecessarily With default timers 33K DNS requests per second will utilise 100% of connections (within 2 minutes) Use connection parameter map to change the setting Value = 0 to 4294967294 seconds (136 years ) Set timeout to zero to disable the timeout (connection will stay up for ever)
Agenda
ACE Stickiness
Required when you need multiple sessions (concurrent or subsequent) from the same user to be sent to the same backend server. Many applications work by the Client initiating multiple connections e.g. HTTP sessions Without sticky, if ACE load-balances on round-robin, leastconnections etc, then connections from the same client are likely to be sent to different servers If no sticky entry exists (e.g. first time a client connects), then the Predictor configured on the serverfarm is used to select which server to send the traffic to. At this point, a sticky table entry is created, and can then be used for subsequent connections (until the entry times out)
ACE Stickiness
It is important to understand the application and the client profile before deciding which method to use N.B. Sticky resources are not allocated to a context by default (not included in the resource all designation), and need to be specifically assigned
ACE can stick on the following information;

Source/Dest IP address Layer 4 Payload* HTTP Content* HTTP Cookie HTTP Header RADI US attribut es* RTSP Header* SIP Header* SSL Session * Requires ACE A2.xID*
Source IP Stickiness
Advantages
Simple to configure and troubleshoot
Disadvantages
Proxy Servers in the path can present a single source IP address (SNAT) for many clients. Result is all users are sent to the same rserver Mega Proxies can change the SNAT IP address midsession
ACE test topology

.100
3/21 54:be
VLAN 2004 10.1.2.0/24
.1 bt-fwsm-ace .5
VLAN 2001 10.1.1.0/24
Context landing
.4
b4:55
VLAN 2002 10.1.1.0/24
.6 bt-customer .1
VLAN 2006 10.1.4.0/24
3/22 3/23
.101
10Port81
0d:17
ef:5e
.102
11Port81
ACE Source-IP Stickiness

serverfarm host farm1 rserver server1 inservice rserver server2 inservice sticky ip-netmask 255.255.255.0 address both group1 timeout 60 replicate sticky serverfarm farm1 class-map match-all classmap1 2 match virtual-address 10.1.1.100 any policy-map type loadbalance first-match policy1 class class-default sticky-serverfarm group1 policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdown interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown
Cookie Stickiness
Cookie can be
set by the rserver (which is learned by ACE) set by the ACE (Cookie-insert)
Cookie can be server-specific (sticky-serverfarm), or per-serverfarm (HTTP class-map) Advantages

Combats Proxy issues relating to source-IP stickiness
Disadvantages
Only supported with HTTP Client browser must support cookies
ACE test topology

.100
3/21 54:be
VLAN 2004 10.1.2.0/24
.1 bt-fwsm-ace .5
VLAN 2001 10.1.1.0/24
Context landing
.4
b4:55
VLAN 2002 10.1.1.0/24
.6 bt-customer .1
VLAN 2006 10.1.4.0/24
3/22 3/23
.101
10Port81
0d:17
ef:5e
.102
11Port81
ACE Cookie Match

serverfarm host farm1 rserver server1 inservice rserver server2 inservice policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdown interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown
Cookie name set by server
sticky http-cookie serverid cook_group serverfarm farm1 class-map match-all classmap1 2 match virtual-address 10.1.1.100 tcp eq www policy-map type loadbalance first-match policy1 class class-default sticky-serverfarm cook_group
Agenda
ACE SNAT
Source-NAT can be required for Client to Server, Server to Client, Server to Server NAT can be performed using either a pool of addresses, or statically with a one-to-one mapping (use where predictable IP is required)
Within the policy-map you must configure which NAT pool number and which egress interface is to be used
Caveats; The ACE will *not* NAT bridged traffic. It must hit a loadbalancing policy in order for SNAT to be implemented SNAT to the VIP address requires ACE 2.x software
ACE SNAT
e.g. RFC1918addressed server requires connectivity to the Internet
Dest IP = Internet Source IP = ACE NAT
ACE requires LB policy in order to catch traffic to NAT
1 Dest IP = Internet
Source IP = Web
ACE SNAT
rserver host gwnorth ip address 10.1.1.1 inservice interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all nat-pool 1 10.1.1.250 10.1.1.251 netmask 255.255.255.0 pat no shutdown
serverfarm host gateway_north_farm transparent rserver gwnorth inservice

class-map match-all SNAT-CLASS 2 match virtual-address 0.0.0.0 0.0.0.0 any
interface vlan 2002 description Server_VLAN bridge-group 1 policy-map type loadbalance first-match SNAT-POL access-group input nonip access-group input permit-all class class-default service-policy input SLB-NAT serverfarm gateway_north_farm no shutdown policy-map multi-match SLB-SNAT interface bvi 1 class SNAT-CLASS ip address 10.1.1.4 255.255.255.0 loadbalance vip inservice no shutdown loadbalance policy SNAT-POL nat dynamic 1 vlan 2001
ACE Server to Server SNAT

Some applications require server to server load-balancing. For example, load-balanced Web server to Application server traffic Some topologies (e.g. VDC) require extra configuration in order to ensure server-to-server load-balancing occurs correctly
Server to Server without SNAT (1/2)

1. Web Server initiates traffic to Application VIP 2. ACE loadbalances traffic to Application server B By default sourceIP is maintained
1
Dest IP = VIP Source IP = Web
2 Source IP = Web
Dest IP = App B
A
Application
B
Application
Server to Server without SNAT (2/2)

3. App Server replies to Web IP 4. MSFC routes to directly-connected subnet 5. Web Server sends TCP RST since the source IP (and SEQ info) does not match any open sessions
4 Dest IP = Web
Source IP = App B
Dest IP = Web Source IP = App B
Server sends TCP RST
A
Application
B
Application
Server to Server with SNAT (1/2)

1. Web Server initiates traffic to Application VIP 2. ACE loadbalances traffic to Application server B ACE configured to change source IP to a SNAT IP
1
Dest IP = VIP Source IP = Web
2 Source IP = ACE SNAT
Dest IP = App B
A
Application
B
Application
Server to Server with SNAT (2/2)

3. App Server replies to ACE SNAT IP MSFC routes to ACE 4. ACE changes he Source and Destination IP back to the VIP and Web, and traffic routed correctly
4 Dest IP = Web
Source IP = VIP Dest IP = ACE SNAT Source IP = App B
Application
Application
ACE Server to Server LB

serverfarm host farm1 predictor leastconns rserver server1 inservice rserver server2 inservice class-map match-all classmap1 2 match virtual-address 10.1.1.100 any policy-map type loadbalance first-match policy1 class class-default serverfarm farm1 policy-map multi-match mmp_ss1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active nat dynamic 1 vlan 2002 interface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all nat-pool 1 10.1.1.254 10.1.1.254 netmask 255.255.255.255 pat service-policy input mmp_ss1 no shutdown
interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown
ACE SSL
What is SSL Why terminate SSL on ACE SSL Termination Certificate Chains
What is SSL
Secure Sockets Layer Layer 5 protocol above TCP and below Applications, such as HTTP, FTP etc
ACE SSL components

SSL Server Certificate SSL key pair (private key and public key) Optional Certificate Chain
Without SSL Accelerators

Server terminates SSL session Certificates and keys are held on the server Load-balancer can only act at Layers 3-5, since the layers above are encrypted (cannot see URL or Cookie)
HTTPS
HTTPS SLB Server
Client
Benefits of SSL Accelerator

Manageability - One cert vs Many (cost, operations effort) Troubleshooting can sniff HTTP layer Stickiness can see HTTP Cookies Performance/Scalability
HTTPS
HTTP
Client
SLB
Server
SSL Certificates
Company Docs Application
Certificate Authority
Application
KEY Pair
Certificate Signing Request Public KEY Common name Domain name Location E-mail
Validation Process Certificate Signing Request Public KEY Common name Domain name Location E-mail
Public Key Private Key
Server Private Key
Certificate
Server Public Key
Certificate
Server Public Key
SSL Server
SSL Fundamentals: Key Exchange Packet Flow Overview

Client Hello Server Hello
Certificate
Server Public Key

Public Key Key Exchange
Server Public Key
Random Number Generator
RSA Encrypt Data
SAasdfkjw1340+jakjb//alkjt
RSA Encrypt Data
Private Key
Shared Secret Key Shared Secret Encrypt & Decrypt
Data Exchange SAasdfkjw1340+jakjb//alkjt SAasdfkjw1340+jakjb//alkjt
Shared Secret Key

Shared Secret Encrypt & Decrypt
Data
Data
Client Browser
Server
ACE SSL Termination

ACE SSL configuration is MUCH simpler (single termination point) than CSM/SSLM
The ACE requires the following in order to terminate SSL connections

SSL Server Key-pair (Private and Public Key) SSL Server Certificate Optionally SSL Certificate Authority Certificate Chain
ACE/context(config) # show crypto files Filename File File Expor Key/ Size Type table Cert ---------------------------------------------------mycert.pem 1275 PEM No CERT mykey.pem 283 PEM Yes KEY
SSL Termination
parameter-map type ssl sslparam cipher RSA_WITH_3DES_EDE_CBC_SHA cipher RSA_WITH_AES_128_CBC_SHA class-map match-all classmap1 2 match virtual-address 10.1.1.100 tcp eq https ! policy-map type loadbalance first-match policy1 class class-default serverfarm farm1 ! policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active ssl-proxy server sslproxy
cipher RSA_WITH_AES_256_CBC_SHA
! ssl-proxy service sslproxy key mykey.pem cert mycert.pem ssl advanced-options sslparam ! serverfarm host farm1 rserver server1 81
ACE
inservice
rserver server2 81 inservice
Encrypted Unencrypted
SSL Certificate Chains

Optional Typically required when the Certificate Authority that has signed the Server Certificate is not trusted by the Client ACE will send the complete certificate chain, and the client will check each certificate in turn to see if it trusts the signer (CA)
crypto chaingroup InternalCAcerts cert rootCA.pem cert ouCA.pem cert deptCA.pem ssl-proxy service secure_access key mykey.pem cert mycert.pem chaingroup InternalCAcerts
ACE Configuration spreadsheet

Requirement Worksheet
Resource-Class Context Name FT Group New Context Layer 3 LB Layer 4 LB Layer 7 LB L4 SSL L7 SSL SNAT
x x x
BVI
Routing Parameter-map Crypto chaingroup Ssl proxy service Probe Rserver
x
x
x x x
x x x x x x x
x x
x x
x x
x x
Server farm
Class-map Match-all virtualaddress (L3/4) http loadbalance (L7) Policy-map Type loadbalance multimatch Access-list
x
x
x
x
x
x x
x
x
x
x x
x
x
x x x
x x
x x
x x
x x
x x
For stickiness, apply a sticky-serverfarm to the LB policy-map, and apply the serverfarm to the sticky-group
ACE Documentation
Cisco ACE Documentation http://www.cisco.com/en/US/partner/products/ps6906 /tsd_products_support_model_home.html ACE Design Guidelines coming soon.. How to use the ACE Packet Capture feature
http://livelink.intra.bt.com/livelink/livelink.exe?func=ll&objId= 70435818&objAction=browse&sort=name&viewType=1
Questions?

Cisco ACE Training

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cisco ACE Training

Uploaded by

Copyright:

Available Formats

VDC Design ACE Training

British Telecommunications plc

What is the ACE Module?

Application Control Engine

What is the ACE Module?

The Evolution of L4 to L7 Services

Integrated Layer 4 and Layer 7 Rules

British Telecommunications plc

ACE Hardware Architecture

Control Plane SAN OS

CDE Switch 60Gbps

Switch Fabric Interface

Onboard SSL Offload (1K to 15K tps throughput)

ACE software versions

SanOS info has now (A2.x) been dropped for simplification;

ACE software versions (contd)

Multiple Virtual Systems (Dedicated Control and Data Path)

Traditional Device: Cisco Application Services Virtualisation:

British Telecommunications plc

ACE Multiple Contexts

Management station AAA

Admin Context + 250 Contexts (Licensed: five contexts in base code)

6500 with ACE

British Telecommunications plc

British Telecommunications plc

Bridged (Layer 2) Mode

Server Default Gateway: Upstream Router

British Telecommunications plc

Routed (Layer 3) Mode

Server Default Gateway: ACE IP

British Telecommunications plc

Server Default Gateway: Upstream Router Subnet C VLAN 30

ACE not in path PBR or SNAT required for return traffic

Routed, Bridged or One-Armed Mode?

One-armed (ACE is not inline for load-balanced traffic)

VDC ACE Topology

ACE Interface Configuration

Which slot is the ACE in?

British Telecommunications plc

Configuring ACE VLANs

British Telecommunications plc

Configuring ACE VLANs (contd)

British Telecommunications plc

Configuring ACE VLANs (contd)

British Telecommunications plc

Verify Cat6k Setup

Created by ---------ACE FWSM ACE

vlans ----7, 2001-2002 201-206,401-406,999-1000 2003

Cat6k>show svclc module Module Vlan-groups ------ ----------01 1,3

Cat6k>show firewall module Module Vlan-groups ------ ----------02 2,3

Accessing the ACE

Connect to the ACE from IOS:

British Telecommunications plc

Creating ACE Contexts

Verifying ACE Setup

ACE-Module/Admin# show context development

British Telecommunications plc

Accessing ACE Contexts

Access new context

[Prompt shows ACE hostname and current context]

ACE Resource Management

British Telecommunications plc

ACE Resource Management