Professional Documents
Culture Documents
Nikolai Pitaev
Engineer, Technical Marketing
BRKSPG-2381
“What’s in it for me?"
This session will help you to understand Cisco virtual Broadband
Network Gateway (BNG) Solution
In this session Out of scope
Introduction and Overview. Other Service Provider and Enterprise use cases.
Live Demo during the session if time permits. Troubleshooting and debugging deep dive
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
With our vBNG solution you can build
flexible
scalable
cost-effective
Broadband Aggregation
4
Agenda
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Real life deployment example of ASR1000 as physical BNG
Functions Services Scalability
ASR 1006 as BNG (aka BRAS) Local termination and L2TP 29.000 Dual Stack PPP Sessions
ATM and Ethernet Voice services 64K configured QinQ Subinterfaces
Total of appr. 500 systems in production QoS parameterization from RADIUS 16.000 Policy Maps
Hierarchical QoS 3 ISG Services 400 Concurrent LI taps
High-Availability / ISSU Lawful Intercept (LI)
1-second accounting accuracy
Residential
ATM BNG
Ethernet BNG
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Same Customer is using CSR1000V as virtual BNG
Implemented in a different country
Almost the same router config, same management interface as on ASR 1000
RT OSS
HGWs
L2 VLAN vBRAS vLNS – Retail ISP LNS –Retail ISP
attachments
x86 servers
vBNG vLNS
IPv6 IPv6 Core
tunnels vRouter LNS
VPEF CSR vLNS LNS
VMs LNS
Internet CSR LNS
VMs
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
CSR 1000V and XRv as vBNG
virtual network functions (VNF)
IOS-XE is a Swiss Army Knife
3.000+ Features
8 major Service Provider Use Cases
8 major Enterprise Use Cases
physical with ASR 1000
virtual with CSR 1000V
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
CSR 1000V is virtualized ASR 1001
Forwarding Plane Control Plane Infrastructure agnostic software
FFP Client / IOS • Familiar IOS XE software
Driver
Linux Container
Throughput Elasticity
FFP code
• Licensable throughput from 10 Mbps to 10 Gbps
vCPU vMemory vDisk vNIC
• Footprint options from 1 to 8 virtual CPUs
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Miercom tested CSR 1000V also as vBNG
Using just one or two vCPUs per VM, it delivers up to physical limit of 20
Gbps on x86 Server with two 10 GE ports and up to 5 Gbps on AWS.
Unlike classic routers a CSR 1000V setup has to be configured for optimal
performance on several levels. Major IO technologies like SR-IOV, fd.io
VPP, OVS-DPDK were tested as vSwitch.
Horizontal scaling – Performance of 3 x 2 vCPU VMs > 1 x 8 vCPU VM
vBNG Test:
• one VM with 2 vCPU on RHEL 7.2 with SR-IOV
• CPU: Intel® Xeon E5-2699 v3 @ 2.30GHz
• 8.000 Dual-Stack Sessions, 500K per Session
http://miercom.com/cisco
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Impact of CPU speed to VNF performance
Two servers with: Impact of Different server Core Speeds
• 3.2 GHz (16 cores) CSR 1000v, IMIX, SR-IOV, IOS XE 16.3
• 2.6 GHz (24 cores)
SR-IOV used to eliminate I/O overhead 7.367
3.2 GHz, 16 core
20
IP forwarding tested, not vBNG
6.001
2.6 GHz, 24 core
For 1 VM, performance increase 18.101
proportional to the CPU Cycle difference
!.# &.'
#.$
≈ $
0 5 10 15 20 25
1x2vCPU 3x2vCPU
For 3 VMs, not proportional
• Bottleneck switched from CPU to IO (2x10 GE)
• See horizontal scale with 3 VMs?
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
CSR1000V vBNG key numbers to remember
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
For Your
Reference
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
Bottlenecks exists on different levels
Differentiate between System
performance and VM performance Intra-VM Bottleneck
x86 Host
Guest Guest
User Kernel
VM1
Application .. VM n
Application
Host User
Virtual Switching
/Qemu
•
• Hypervisor performance vSwitch Bottleneck vSwitch
Kernel
KVM
Host
• Performance tuning pNIC Driver pNIC Driver
pNIC Bottleneck pNIC pNIC
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
vBNG runs at 8,5 Gbps with VM-FEX technology
Customer provided vBNG config, RADIUS profile and traffic definition
VM-Fex bypasses
Tested internally in Cisco lab on KVM with VM-FEX
bottlenecks and emulated
Throughput summary: unconstrained system
CSR 1000v demonstrated requested 20 Gbps with 3 VMs
Above 8.5 Gbps per VM on average.
Guest-OS Guest-OS
UCS Type UCS C240 M4S (2 Processors, 36 Cores)
UCS Manager UCS 6248 Fabric Interconnect with UCS Manager 2.2 (3f) CSR CSR CSR
Performance
NIC Type Cisco UCS VIC 1225 Virtio-net Virtio-net
Loss
layer-2 sorter / switch / classifier
SP Traffic Profile 1430B = 75%, 578B = 16.6%, 80B = 8.3% ~ (Avg Pkt Size = 1175B)
x86 machine
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Drop Rate Definition has significant impact to throughput
Throughput+as+a+funcCon+of+acceptable+
Typical definitions for Drop Rates Traffic+Loss+(%,+normalized,+KVM,+XE+3.13)+
• Non-drop Rate (NDR) = 0 packet loss 180%#
160%#
Normalized+Throughput+(%,+NDR+=+100%)+
• Partial Drop Rate (PDR) 0.01% or 0.05% 140%#
120%#
100%#
40%#
%#increase#in#Throughput#
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Optimize your vBNG System on 4 different levels
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Design your CPU mapping for better performance
CPU00 CPU01 CPU02 CPU03 CPU04 CPU05 CPU06 CPU07 CPU10 CPU11 CPU12 CPU13 CPU14 CPU15 CPU16 CPU17
Socket0 Socket1
Physical Physical
Interface 1 Interface 2
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
Same example, different design
CPU00 CPU01 CPU02 CPU03 CPU04 CPU05 CPU06 CPU07 CPU10 CPU11 CPU12 CPU13 CPU14 CPU15 CPU16 CPU17
Socket0 Socket1
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
For Your
Reference
CSR footprint Control Plane Data Plane PPE Data Plane HQF Data Plane Rx processing
/ Tx Processing
1 vCPU 0
2 vCPU 0 vCPU 1
4 vCPU 0 vCPU 1 & 2 vCPU 3
8 vCPU 0 vCPU 1-5 vCPU 6 vCPU 7
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
For Your
Reference
Control Control
Shared 1 1 1 Shared 1 2 2
Service 1 Service 1
Data 1 3 7 Data 1 2 6
Control Control
Shared 1 2 2 Shared 1 2 4
Service 1 Service 1
Data 1 2 6 Data 1 2 4
*Available in IOS XE RLS 3.16.02 and later BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
CSR1000V vBNG Roadmap and Positioning
Now: PPP over VXLAN Termination
Next: Performance improvement, PPP over L2TPv3-in-IPv6 Tunnels
On radar: Box-to-box High Availability
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
XRv 9000 scale and performance per VM
32.000 subscribers per VM, Geo-Redundancy
200 calls per second per VM, 100 CoA per second per VM
Performance:
UCS C240, 28 cores, Xeon E5-2697 v3 @ 2.6 Ghz, 128 GB RAM, 10x10Gig
32.000 IPoE Subscribers with H-QoS and ACL’s, 82 Gbps IMIX Throughput NDR
Expectation for 32.000 PPP Sessions: 5-10% less throughput, means 75 Gbps
Roadmap:
Dec'16 Apr-17 Jul-17 Sep-17
IPoE DEMO PPPoE Demo IPoE EFT PPPoE EFT
Disclaimer: numbers and dates are targeted numbers and subject to change till FCS
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
For Your
10G
10G
Ctrl Ctrl
Eth
Eth Eth
XRv Linux Kernel
vmxnet3
e1000
Linux
virtio
GE
WRL7 (3.14) Mg
bridge
mt
Eth
KVM, ESXi
(future: HyperV, AWS, bare metal, XEN,...) V
F
P
F
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
Cisco vBNG VNF Summary
CSR 1000V XRv 9000
IPoE – July 2017
Availability now
PPP – October 2017
2 vCPU VM: 28 vCPUs VM:
vBNG Scale 8.000 Sessions 32.000 Sessions
5 Gbps IMIX 80 Gbps IMIX
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Elastic Services Controller
for VNF Lifecycle Management
ESC is used for VNF lifecycle management
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
ESC enables scalable vBNG solution
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
Detailed ESC VNF lifecycle management For Your
Reference
Provision Configure
VM Service Service Custom Script
Predefined Action
Overloaded / Underloaded Action
One Event -> one action One Event -> multiple actions
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
ESC uses KPI thresholds for VM monitoring
Threshold Name Threshold Metric Value ESC Action Customized Action
Type
VM_ALIVE Rising/Falling ICMP Ping 3 successful Service Booted Verify CSR connectivity
Reachability pings Add to Radius
VM_OVERLOADED Rising Session Count >7000 Service Scale-Up (add VM) Adjust Radius Load-Balancing
VM_OVERLOADED_EMPTY Falling Session Count <1 Service Scale-Down Remove CSR from Radius
(remove VM) Load-Balancing
The CSR 1000V first boots in evaluation mode with throughput limited to 2.5 Mbps.
Smart Licensing is used to automatically download and install needed license.
Two Options:
1. Connection over Internet to Cisco Smart Licensing Server.
2. Install a Smart Software Manager Satellite (SSMS) in your network.
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Smart Licensing options
Router
Direct Deployment
Cisco Smart
Software Cisco Commerce
Firewall Workspace
Manager
Usage
Software
Unified
Proxy or SCH
Communications Transport Gateway
Offline Monthly
Router Inventory Update
Mediated Deployment
SSMS
Software
Unified
Communications
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
License sharing across different vBNGs saves money
Smart licenses can be shared among different CSR 1000V!
Example: 16 x CSR 1000V running as vBNG, each with 8.000 Broadband Sessions.
Option 1: 16 x L-CSR-BB-8K= would cost $24.000 * 16 = $384.000
Option 2: 1 x L-CSR-BB-128K-S= shared among 16 vBNGs would cost $128.000.
Result: saving of $256.000 on BB license, which is 66.67% saving!
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Bringing Subscribers over
Backbone to vBNG in central
locations
Use VxLAN or L2TPv3 Tunnels to bring subscribers to vBNG
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
PPP Sessions over VXLAN directly to vBNG
VxLAN Tunnel is established between aggregation Switch and vBNG:
vBNG extracts PPP packets out of the VxLAN Tunnel and terminates PPP.
Simple BDI Interface on vBNG:
interface BDI10
no ip address
vlan-id dot1q 2000
pppoe enable group global
!
pppoe enable group global
! BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
PPP over L2TPv3-in-IPv6 details with VPP
L2TPv3 Tunnel is established between aggregation Switch and fd.io VPP:
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Termination of L2TPv3 Tunnels directly on CSR1000V
L2TPv3 Tunnel is established between aggregation Switch and CSR1kv:
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Load Balancing across multiple vBNGs
Question: how load balance PPP sessions across multiple vBNGs?
Solution: PADO delay aka PPP Smart Server Selection Feature.
vBNG1 vBNG2
Example 1:
bba-group pppoe global-server-selection PADI
pado delay 512 ! <- this value can be changed by ESC
...
PADO PADO delayed
Example 2:
bba-group pppoe selected-server-selection
pppoe server remote-id delay 512 string contains TEST
pppoe server circuit-id delay 256 string "mac 1111.2222.3333" Client
...
Similar concept for IP sessions by delaying the offer timers of the DHCP server.
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
L2TP Load Balancing
Question: ONE L2TP Tunnel with 16.000 Sessions, but vLNS supports 8.000
Solution: LTS = L2TP Tunnel Switch
LAC vLTS vLNS1: 192.168.101.1
8.000 Sessions
16.000 Sessions
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
TCO Calculation Example
Example: calculation for 250.000 subscribers in 6 rack units
Throughput calculation:
§ Max throughput for PPP per VM without optimization is 2,5 Gbps
§ 2,5 Gbps / 500 Kbps per customer = 5.000 customers per VM
§ 250.000 customers / 5.000 customers per VM = 50 VMs
Result:
§ 50 VMs / 17 VM per UCS = 3 UCS systems are needed for 250.000 subscribers
§ 6 Rack Units total = 3 x 2 RU per UCS C240
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Live Demo
vBNG @ dCloud demo
270+ labs for Customers, Partners and Cisco Employees.
From scripted demos to fully customizable labs with administrative access!
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
vBNG orchestrated by ESC in OpenStack on dCloud
“All In One” Virtual Machine
ESC orchestrated
Manually started “dynamic” VMs: Tools
“static” VM:
CSR1kv as vBNG-1
FreeRADIUS
CSR1kv as PPP Client
CSR1kv as vBNG-2
VNC Server
ESC VM
...
SSH, Telnet, SCP
CSR1kv as vBNG-X
Wireshark
OpenStack
Ubuntu “Host”
dCloud Infrastructure
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Summary of the key steps in the dCloud vBNG lab
1. Manually start PPP client VM
2. Start ESC VM
3. Define orchestration rules (VM_Overloaded, VM_Underloaded, VM_Alive)
4. Test Scale_Up and Scale_Down cases based on defined rules
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
Summary of the key steps in our demo today
1. Manually start PPP Client VM
2. Start ESC VM
3. Define Orchestration Rules (VM_Overloaded, VM_Underloaded, VM_Alive)
4. Test Scale_Up and Scale_Down Cases based on defined Rules
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Summary: Bringing it all together
CSR 1000V benefits
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
Key benefits of our end-to-end vBNG solution
Elasticity – add multiple vBNGs within MINUTES, not weeks compared to physical model
No need for SP to change HW design or physically move links to a different port
Simplified and centralized hardware replacement and logistics
Engineering & Design Complex design, disaster radius, feature gaps yes
Capacity Planning Department Slow reaction on demand changes, CAPEX & OPEX reduction yes
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
Summary of the whole presentation
vBNG is one of major 8 CSR 1000V Service Provider Solutions.
vBNG is targeted for July (IP) and November (PPP) 2017 on XRv 9000
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
Call to action
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
Complete Your Online Session Evaluation
Give us your feedback and receive a
Cisco Live 2017 Cap by completing the
overall event evaluation and 5 session
evaluations.
Caps can be collected Friday 10 March Learn online with Cisco Live!
at Registration. Visit us online after the conference
for full access to session videos and
presentations.
www.CiscoLiveAPAC.com
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
With our vBNG solution you can build
flexible
scalable
cost-effective
Broadband Aggregation
57
Cisco Spark
Ask Questions, Get Answers, Continue the Experience
The Spark Room will be open for 2 weeks after Cisco Live
BRKSPG-2381 © 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Thank you