Professional Documents
Culture Documents
Cisco Confidential
Nexus 7000
5.0(3) 5.1(2) 5.1(3) 5.2
Complete Sync
Partial Sync
Nexus 5000
5.0(3)N1
5.0(3)N2 5.1(3)N1 5.2N1 Nexus 7000
5.0(3)U2
Andaman
5.1(3)U1
vPC basic components Hardware Specific Considerations vPC enhancements L3 and vPC Adding FEX Summary designs
Cisco Confidential
on Link Aggregation.
Eliminates the need for STP in the access-distribution
Physical Topology
Logical Topology
L2
Si
vPC domain
Non-vPC vPC Increased BW with vPC
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5
vPC peer a vPC switch, one of a pair vPC member port one of a set of ports (port
vPC
between vPC peer devices, must be 10GbE. Also carry multicast/broadcast/flooding traffic and data traffic in case of vpc member port failure
for state synchronization and configuration validation between vPC peer devices
Orphan portNon-vPC member port
Orphan Port
Cisco Confidential
IGMP bulk sync: On N7k: to be verified On N5k: starting from NXOS 5.0(3)N1(1a)
Multicast Optimization on Peer-link: On N7k: hidden comand as of NXOS 5.1(3) (but not supported) On N5k: starting from NXOS 5.0(3)N1(1a)
Autorecovery:
On the N7k: NXOS 5.2 On the N5k: NXOS 5.0(2)N2(1)
ARP synchronization: On N7k: NX-OS 4.2(6) and 5.0(2) (Bogota), fixed in 5.1(1) (Cairo) On N5k: under investigation for Goldcoast
vPC peer-switch: On N7k: 4.2(6), 5.x On N5k: under investigation for Goldcoast
Config-sync:
On the N7k: Freetown On the N5k: NXOS 5.0(2)N2(1)
vPC on FEX
On the N5k: NXOS 4.2(1)N1(1) On the N7k: NXOS 5.2
Cisco Confidential
vPC Peers
link/device failure
Port channel
Cisco Confidential
control traffic)
Carries multicast traffic (more details follow)
Cisco Confidential
Peer Keepalive can be carried over the OOB management network int mgmt 0
triggering recovery
Should not be carried over the Peer-Link Use the mgmt0 interface in the management VRF Can optionally be a dedicated link, 1Gb is adequate
dc11-5020-1(config)# vpc domain 20 dc11-5020-1(config-vpc-domain)# peer-keepalive destination 172.26.161.201 source 172.26.161.200 vrf management Note: --------:: Management VRF will be used as the default VRF ::--------
Cisco Confidential
10
and N7K)
Primary design requirement is to have a physically
communication over the vPC peer-link On Nexus 7000 when possible use dedicated VRF and front panel ports for peer-keepalive link (1G is more than adequate). 2nd best is to use the management interfaces 3rd option is to use an upstream L3 network for peer-keepalive
If using mgmt 0 interfaces do not connect the
In a dual supervisor configuration only one management port will be active at a given point in time!
Connect both mgmt 0 ports to the OOB network
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11
Adding FEX
Summary designs
Cisco Confidential
12
Cisco Nexus 7000 Series vPC works on all existing I/O modules Peer keepalive:
M1/F1 cards can be used for vPC Peer-link requires 10 GigE cards Peer-link should not span M1 and F1,
Cisco Confidential
13
N7K-M108X2-12L
N7K-F132XP-15
Cisco Confidential 14
M-Series Mode
M
Recommendation : for mixed chassis mode (F1/M1) with vPC peer-link on F1 ports, use at least 2 M1 LC. This will provide resiliency for L3 features (FHRP, SVI). (*) : command peer-gateway exclude-vlan <vlan list> needed for backup routing path over vPC peer-link
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15
NX-OS 5.1.3 introduces new behavior for handling vPC peer-gateway in mixed chassis
mode (M1/F1) : Topology with M1 peer-link : IP/ARP packets destined to the remote Active IP/MAC get routed locally Topology with F1 peer-link : IP/ARP packets destined to the remote Active IP/MAC use the tunneling mechanism
M-Series Mode
F-Series Mode
Knob Required for transit path/VLAN IP/ARP Tunneling over Peer link
Cisco Confidential
16
vPC Primary S1
F1
vPC Secondary S2
F1
vPC Primary
S1
M1
vPC Secondary S2
M1
vPC Peer-link
vPC Peer-link
vPC Primary S1
M1
vPC Secondary S2
F1
vPC Peer-link
Cisco Confidential
17
Adding FEX
Summary designs
Cisco Confidential
18
5k01
5k02
2 - Host subscribes to G1
Cisco Confidential
19
VPC Port-channels on upstream/downstream switches will be error-disabled by EtherChannel Misconfiguration Guard after ~90 seconds
http://www.cisco.com/en/US/tech/tk389/tk213/technologies_tech_note09186a008009448d.shtml
Cisco Confidential
20
disconnected
vPC secondary detects primary
5k02
vPC Secondary
suspends all its vpc member ports in order to avoid traffic drop
KEEP PEER KEEPALIVE
Cisco Confidential
21
dca-n7k2-vdc2
vPC supports standard 802.3ad port channels from upstream and or downstream devices Recommended to enable LACP channel-group 201 mode active
dca-n7k2-vdc2# sh run interface port-channel 201 version 4.1(5) interface port-channel201 switchport mode trunk switchport trunk allowed vlan 100-105
dc11-5020-1
dc11-5020-2
dc11-5020-1# show running int port-channel 201 version 4.1(3)N1(1) interface port-channel201 switchport mode trunk switchport trunk native vlan 100 switchport trunk allowed vlan 100-105 vpc 201
dc11-5020-2# show running int port-channel 201 version 4.1(3)N1(1) interface port-channel201 switchport mode trunk switchport trunk native vlan 100 switchport trunk allowed vlan 100-105 vpc 201
Cisco Confidential
22
dca-n7k2-vdc2
dc11-5020-1
dc11-5020-2
dc11-5020-1# show running int port-channel 201 version 4.1(3)N1(1) interface port-channel201 switchport mode trunk switchport trunk native vlan 100 switchport trunk allowed vlan 100-105 vpc 201
dc11-5020-2# show running int port-channel 201 version 4.1(3)N1(1) interface port-channel201 switchport mode trunk switchport trunk native vlan 100 switchport trunk allowed vlan 100-105 vpc 201
Cisco Confidential
23
MAC_C
5
CFS
via CFS
Copies of flooded frames are sent across the
56 3
N5K-1 N5K-2
Frames received on the vPC-Link are not forwarded out vPC ports
1. 2. 3. Host MAC_A send packet to MAC_C FEX runs hash algorithm to select one fabric uplink N5K-1 learns MAC_A and flood packets to all ports (in that VLAN). A copy of the packet is sent across the peer link N5K-2 floods the packet to any port in the VLAN except the vPC member ports to prevent duplicated packets N7K-1 and N7K-2 repeat the same forwarding logic N5K-1 updates the the MAC address learned on the vPC port on N5K-2 via CFS
CFS
4.
1
MAC_A
5. 6.
Cisco Confidential
24
MAC_C
a vPC if it exists 1. Host MAC_C send packet to MAC_A 2. N7K-2 forwards frame based on learned MAC address 3. N5K-2 forwards frame based on learned MAC address
N5K-1# sh mac-address-table vlan 101 VLAN MAC Address Type Age Port ---------+-----------------+-------+---------+----101 001b.0cdd.387f dynamic 0 Po30 101 0023.ac64.dda5 dynamic 30 Po201 Total MAC Addresses: 4
2
N5K-1 N5K-2
3
N5K-2# sh mac-address-table vlan 101 VLAN MAC Address Type Age Port ---------+-----------------+-------+---------+----101 001b.0cdd.387f dynamic 0 Po30 101 0023.ac64.dda5 dynamic 30 Po201 Total MAC Addresses: 4
MAC_A
Cisco Confidential
25
MAC_C
members of the vPC MAC address table is updated to forward frames for the vPC across the vPC Peer Link
Note: Po20 is the vpc peer-link
2
N5K-1 N5K-2
N5K-1# sh mac-address-table vlan 101 VLAN MAC Address Type Age Port ---------+-----------------+-------+---------+----101 001b.0cdd.387f dynamic 0 Po30 101 0023.ac64.dda5 dynamic 30 Po201 Total MAC Addresses: 4
N5K-2# sh mac-address-table vlan 101 VLAN MAC Address Type Age Port ---------+-----------------+-------+---------+----101 001b.0cdd.387f dynamic 0 Po20 101 0023.ac64.dda5 dynamic 30 Po201 Total MAC Addresses: 4
3
MAC_A
Cisco Confidential
26
2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential 27
configuration consistent: a manual process with an automated consistency check to ensure correct network behavior config-sync Two types of interface consistency checks
Type 1 Will put interfaces into suspend state to prevent invalid forwarding of packets
Cisco Confidential
28
dc11-5020-2# show vpc brief Legend: (*) - local vPC is down, forwarding via vPC peer-link <snip> vPC status ---------------------------------------------------------------------------id Port Status Consistency Reason Active vlans ------ ----------- ------ ----------- -------------------------- ----------201 Po201 down failed vPC type-1 configuration incompatible - STP interface port guard Root or loop guard 2010 Cisco and/or its affiliates. All rights reserved. inconsistent
Cisco Confidential
29
VLAN mismatch)
dc11-5020-1# sh run int po 201 version 4.1(3)N1(1) interface port-channel201 switchport mode trunk switchport trunk native vlan 100 switchport trunk allowed vlan 100-105 vpc 201 dc11-5020-2# sh run int po 201 version 4.1(3)N1(1) interface port-channel201 switchport mode trunk switchport trunk native vlan 105 switchport trunk allowed vlan 100-104 vpc 201
dc11-5020-1# show vpc brief vpc 201 vPC status ---------------------------------------------------------------------------id Port Status Consistency Reason Active vlans ------ ----------- ------ ----------- -------------------------- ----------201 Po201 up success success 100-104 2009 May 17 21:56:28 dc11-5020-1 %ETHPORT-5-IF_ERROR_VLANS_SUSPENDED: VLANs 105 on Interface portchannel201 are being suspended. (Reason: Vlan is not configured on remote vPC interface)
Cisco Confidential
30
c-nexus5010-1# show vpc consistency-parameters global Legend: Type 1 : vPC will be suspended in case of mismatch Name ------------QoS Network QoS (MTU) Network Qos (Pause) Type ---2 2 2 Local Value Peer Value
Global Parameters
---------------------- ----------------------([], [3], [], [], [], (1538, 2240, 0, 0, 0, (F, T, F, F, F, F) ([], [3], [], [], [], (1538, 2240, 0, 0, 0, (F, T, F, F, F, F)
STP Mode
STP Disabled STP MST Region Name STP MST Region Revision STP MST Region Instance to VLAN Mapping STP Loopguard
1
1 1 1 1
Rapid-PVST
None "" 0
Rapid-PVST
None "" 0
Disabled
Disabled
1
1 -
Enabled
Normal, Disabled, 1,50 50
Enabled
Normal, Disabled, 1 -
Cisco Confidential
31
sync
Any configuration that could cause an error in
the very beginning of the deployment, for ALL VLANs, the ones that exist as well as the ones that have not yet been created
Defining a region mapping is orthogonal to
vPC vPC vPC
creating a VLAN
Cisco Confidential
33
Adding FEX
Summary designs
Cisco Confidential
34
Inconsistency VLAN to MST Region mapping mismatch STP global settings (BA, Loop Guard, Root Guard)
Type
Impact
Recommendation Pre-provision and MAP all VLANs on the MST region Perform STP operations per port Operate change during maintenance window Leverage graceful conflict resolution Operate change during maintenance window and/or leverage graceful conflict resolution
New Enhancements
Global
1
Spanning-tree per interface settings, switchport type (trunk/versus access) Port-channel mode
Config Sync (5.0(2)N1(1) on N5K, Freetown for N7K) & Graceful Conflict Resolution (CSCtf84865,N7K 4.2(8)& 5.2, N5K 5.0(2)N2(1))
Per-vPC
Global
Minimum disruption Per-vPC
Cisco Confidential
35
tc-nexus5010-1# show vpc consistency-parameters global Name Type Local Value Peer Value
------------QoS
---2
---------------------- ----------------------([], [3], [], [], [], []) ([], [3], [], [], [], []) (1538, 2240, 0, 0, 0, 0) (F, T, F, F, F, F) (50, 50, 0, 0, 0, 0) (F, F, F, F, F, F)
(1538, 2240, 0, 0, 0, 0)
Network Qos (Pause) Input Queuing (Bandwidth) Input Queuing (Absolute Priority) Output Queuing (Bandwidth) Output Queuing (Absolute
2 2 2
2 2
Cisco Confidential
36
changes.
switch(config)# vpc domain 10 switch(config-vpc-domain)# [no] graceful
mst region vlans 1-5, 12 mst region vlans 1-5, 10
consistency-check
Requires 5.0(2)N2(1) on the Nexus 5k Requires 5.2 on the Nexus 7k
vPC vPC vPC
vPC primary
vPC secondary
Cisco Confidential
38
5.2
5.0(2)N2(1)
Check whether STP is enabled or disabled on per-VLAN basis. VLANs that have mismatched status will be suspended on both switches
Rest of VLANs wont be affected Prior to this change all VLANs are affected
Cisco Confidential
39
Config-sync allows administrators to make configuration changes on one switch and have the system automatically synchronize to its peers. This eliminates any user prone errors & reduces the administrative overhead of having to configure both vPC members simultaneously. Config-sync and Graceful conflict resolution are complementary features Config-sync traffic is carried over the peer keepalive link
vPC vPC
+ vlan 12
vPC
Cisco Confidential
40
Global Configurations:
synchronized?
Enabling Feature
vPC domain configuration FCoE configuration
Cisco Confidential
41
N5000-1# feature vpc vpc domain 10 peer-keepalive destination 10.29.170.8 N5000-1#sh run switch-profile Switch-profile Apple sync-peers destination 10.29.170.8 N5000-1(config-if)# config sync N5000-1(config-sync)# switch-profile Apple N5000-1(config-sync-sp)# int ethernet 100/1/3 N5000-1(config-sync-sp-if)# switch mode trunk N5000-1(config-sync-sp-if)# verify Verify Successful N5000-1(config-if)# config sync N5000-1(config-sync)# switch-profile Apple N5000-1(config-sync-sp)# commit Commit Successful
N5000-2# feature vpc vpc domain 10 peer-keepalive destination 10.29.170.7 N5000-2#sh run switch-profile Switch-profile Apple sync-peers destination 10.29.170.7
NOTE: Verify does not push the config to peer, user must issue commit for sync to take place If sync fails, then the config is in the BUFFER
Configuration is stored in a buffer until commit is applied. User can add/delete/move configuration. Once the config has been pushed via commit, it will no longer show up in buffer (it will show up in show runningconfig switch-profile X) If the commit fails due to mutex check or other reasons, the failed configuration still shows in the buffer, you have to explicitly remove it to continue
N5K-1(config-sync-sp-if)# sh switch-profile A buffer ----------------------------------------------------Seq-no Command ----------------------------------------------------1 interface Ethernet100/1/9 1.1 switchport mode trunk 1.2 switchport trunk allowed vlan 5-10 2 interface Ethernet100/1/10 2.1 switchport mode access
N5K-1(config-sync-sp)# ? buffer-delete Delete buffered command(s) buffer-move Move buffered command(s) N5K-1(config-sync-sp)# buffer-delete 1 N5K-1(config-sync-sp)# sh switch-profile A buffer ----------------------------------------------------Seq-no Command ----------------------------------------------------2 interface Ethernet100/1/10 2.1 switchport mode access
Cisco Confidential
43
Interface Ethernet1/11
Interface Ethernet1/11
switch-profile area
shut/no shut
This portion is synchronized
Cisco Confidential
44
A port-channel may consist of port ethernet 1/1 on n5k01 And erthernet 1/2 on n5k02 FEX A/A has the same FEX configured to both N5ks, so Preprovisioning has to be configured identically
Cisco Confidential
45
be disconnected completely from the vPC domain you can still operate the remaining one For this you need to leverage the commands reload restore and autorecovery
with the split brain scenario allowing a vPC peer to bring up new vPC ports even after a reload Autorecovery deals with the sequential loss of peer-link first, and peer-keepalive second, allowing the vPC secondary to bring up the vPC ports (which were down previously)
Cisco Confidential
46
Wait for per-port peer parameters and handshake to bring up vPC ports
Performs peer parameters consistency check on each VPC bringup Only after VPC port-channels are brought up. What if after a full DC outage (both Nexus down), only one switch is
coming up ?
Will not bring up VPCs if after a datacenter outage, only one VPC
Cisco Confidential
47
Switch1
Switch2
Switch1
Switch1
Switch3
Switch3
Switch3
Existing vPCs are brought up When adding a new vPC member port, the port goes up
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 48
1 Switch1 Switch2
Switch1
Switch2
Switch3
Switch1
Switch2
Switch3
Switch3
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 49
vPC 1
po1
vPC 1
po1
vPC peer-link
vPC 1
po1
po1
vPC 1
Cisco Confidential
51
Primary vPC
Secondary vPC
the vPC operationally primary switch and only such device sends out BPDUs on STP designated ports.
This happens irrespectively of where
BPDUs
device proxies STP BPDU messages from access switches toward the primary vPC
Cisco Confidential
52
ECMP
Secondary vPC
SW1
Primary Root
L3 L2
SW2
Secondary Root
The secondary vPC device does NOT source STP BPDUs on symmetrical vPCs
SW3 MAC_A
SW4 MAC_B
Cisco Confidential
53
Primary / Root
7k01
7k02
BA Inconsistent
with vPC
Bridge Assurance on peer-link is
Cisco Confidential
54
Primary
left# sh span vlan 101 VLAN0101 Spanning tree enabled protocol rstp Root ID Priority 8293 Address 0023.04ee.be01 This bridge is the root ... Bridge ID ... Interface ---------------Po1 Po100 Role ---Desg Root Sts --FWD FWD Cost --------1 2 Prio.Nbr -------128.4096 128.4195 Type --------------(vPC) P2p (vPC peer-link) Priority Address 8293 (priority 8192) 0023.04ee.be01
Secondary ROOT
ROOT
: 00:23:04:ee:be:01 : 00:1b:54:c2:42:43
right# sh span vlan 101 VLAN0101 Spanning tree enabled protocol rstp Root ID Priority 8293 Address 0023.04ee.be01 This bridge is the root ... Bridge ID Priority Address Role ---Desg Desg 8293 (priority 8192) 0023.04ee.be01 Sts --FWD FWD Cost --------1 2 Prio.Nbr -------128.4096 128.4195 Type --------------(vPC) P2p (vPC peer-link)
Cisco Confidential 55
In Peer-Switch mode bridge-ID comes from system-mac as opposed to local mac in normal mode
Dispute is default enabled (for both RSTP and MST on VPC) UDLD [normal mode] is recommended to take out bad links from channels BA + UDLD + Dispute (on all interswitch links when using Peer-switch) when all
Cisco Confidential
56
does not receive an LACP PDU from the peer. This behavior is different on the Nexus 7000 series where the default is to suspend a port if it doesnt receive LACP PDUs.
For server facing port-channels it is better to allow LACP ports to revert
to I-state if the server doesnt send LACP PDUs. By doing this the I-state port can operate like a regular Spanning-Tree port. Also this allows immediate server connectivity when it boots up before the full LACP negotiation has taken place.
For network facing ports, allowing ports to revert to I-state creates
configuration [no] lacp suspend-individual (which is the equivalent of the Catalyst IOS command port-channel standalone-disable.
57
Cisco Confidential
IGMP snooping shares the snooped reports with the peer vPC
reports received on one vPC switch is also forwarded to the vPC peer. Thus multicast forwarding state remains in sync on both the vPC switches.
Do NOT DISABLE IGMP Snooping!
If you need to support Firewalls / Clusters: Use static IGMP entries OR Create an IGMP querier!
Cisco Confidential
58
vPC Primary
vPC Secondary
vPC Primary
vPC Secondary
IGMP join/leave
Cisco Confidential
59
N5k02
Cisco Confidential
60
Cisco Confidential
61
N5k-1
N5k-2
avoid that. With the CLI multicast traffic is sent to vPC peer-link only when it is necessary, such as, there is singly connected host
Improving multicast convergence time
Cisco Confidential
62
see
CSCtc49559
1 2 3 4
feature
N5k01
N5k02
Cisco Confidential
63
S1 -Primary
Keepalive
S2-Secondary
Intended for devices that do not support port-channel. Other devices should be dually connected by vPCs (Orphanport CLI is available only on physical ports, not on portchannels) Configure single attached devices (like FW or LB) port as orphan-port When vPC peer-link goes down, vPC secondary peer device shuts all its vPC member ports as well as orphan ports
vPC peer-link
Active or Standby Active or Standby
vPC 1
po1 CE-1 Orphan port
S1(config)# int eth 1/1 S1(config-if)# vpc orphan-ports suspend S2(config)# int eth 1/1 S2(config-if)# vpc orphan-ports suspend
Cisco Confidential
64
Adding FEX
Summary designs
Cisco Confidential
66
HW Programmed to forward frames sent to the FHRP MAC address on BOTH Switches
HSRP Active HSRP Standby
the L3 hardware forwarding tables, creating a local forwarding capability on the HSRP standby device
Consistent behavior for HSRP, VRRP
and GLBP
No need to configure aggressive
Cisco Confidential
67
It recommended to not use HSRP link tracking in a vPC configuration Reason: vPC will not forward a packet back on a vPC once it has crossed the peer-link, except in the case of a remote member port failure Use an L3 point-to-point link between the vPC peers to establish a L3 backup path to the Core in case of uplinks failure A single point-to-point VLAN/SVI will suffice to establish a L3 neighbor
SVI VLAN 300
VLAN 100, 200,300
VLAN 100
VLAN 200
Cisco Confidential
68
Non-RFC compliant end hosts Device required to send packets to the MAC address returned in ARP response (HSRP virtual MAC) Some non-compliant devices use the MAC address of the sender device (Switch physical MAC) NAS devices (i.e. NETAPP Fast-Path or EMC IP-Reflect) have been found to do this vPC Peer Gateway - NX-OS 4.2(1) Allows a vPC peer to respond both the the HSRP virtual and the real MAC address of both itself and its peer
peer-gateway command tells the vPC to respond to the physical MAC address of its peer
L3 L2
VLAN 200
VLAN 100
Cisco Confidential
69
Not enabled by default After the peer-link comes up perform an ARP bulk sync over CFSoE to the peer switch Improve Convergence for Layer 3 flows
ARP TABLE
IP1 IP2 MAC1 MAC2 VLAN 100 VLAN 200 IP1 IP2
ARP TABLE
MAC1 MAC2 VLAN 100 VLAN 200
SVIs
P
S
IP1
MAC1
IP2
MAC2
Feature
VPC interaction with FHRP Peer-gateway vPC delay restore vPC exclude VLAN ARP synchronization PIM pre-built-SPT PIM dual DR
Function
Both active and standby peer function as gateway L3 forwarding when the DMAC is peers MAC Delay bringing up vPC ports CLI to specify SVI interfaces wont be suspended when peer-link fails Synchronize ARP between two peer switches Both N5k joins source tree as PIM last hop router Both N5k can be DR when it is first hop router
Availability
HSRP VRRP
Roadmap
Cisco Confidential
71
Adding FEX
Summary designs
Cisco Confidential
72
2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
Fairhaven
Cisco Confidential
74
Future
active
active
active
active
active
active
active
active
active
active
active
active
N
2010 Cisco and/or its affiliates. All rights reserved.
radar
Cisco Confidential
75
Redundancy model Single switch with dual supervisor for fabric, data control & management planes No SAN A and B isolation (VSAN isolation sufficient in the future?)
Cisco Confidential
76
Fabric links supported on N7K-M132XP-12 & N7K-M132XP-12L Port Channel only supported for Fabric Links
2010 Cisco and/or its affiliates. All rights reserved.
No support for DCB and FCoE (parent switch fabric ports not DCB capable yet)
Cisco Confidential 77
Redundancy model Single switch with dual supervisor, fabric, line card, data control & management planes
Cisco Confidential
78
24 FEX
Nexus 2000 Straight-through deployment
n5k01
n5k01
n5k02
FEX100 FEX101
FEX102
FEX120
FEX121
FEX122
FEX100 FEX101
FEX102
max 24 x 2
Active/Standby
Cisco Confidential
79
Peer Keepalive
FEX 2248
Cisco Nexus 2000 Series Straight-Through vPC Cisco Nexus 2000 Active-Active
vPC Primary
vPC Secondary
vPC Primary
vPC Secondary
up to 4 ports
up to 4 ports
Fabric Links
Fabric Links
up to 4 ports up to 4 ports
FEX100
HIF
up to 8 ports
FEX120
HIF
up to 8 ports up to 24 PC per FEX up to 24 PC per FEX
vPC 1
vPC 2
FEX100
HIF
FEX120
HIF
up to 8 ports
up to 8 ports
Cisco Confidential
80
Peer Keepalive
FEX 2232
Cisco Nexus 2000 Series Straight-Through vPC Cisco Nexus 2000 Active-Active
vPC Primary
vPC Secondary
vPC Primary
vPC Secondary
up to 8 ports
up to 8 ports
Fabric Links
Fabric Links
up to 8 ports up to 8 ports
FEX100
HIF
up to 8 ports
FEX120
HIF
up to 8 ports up to 16 PC per FEX up to 16 PC per FEX
vPC 1
vPC 2
FEX100
HIF
FEX120
HIF
up to 8 ports
up to 8 ports
SAN A
SAN B
FCoE traffic will NOT be load shared across both sets of fabric links
SAN A and B isolation is
LAN traffic
maintained
This may result in un-even sharing
traffic capacity
Cisco Confidential
82
Adding FEX
Summary designs
Cisco Confidential
83
root
vPC on the N7k Root
N7k02 2/10
logical equivalent
2/2
2/1
2/2
N5k02
Cisco Confidential
84
root
vPC on the N7k Root
N7k02 2/10
logical equivalent
2/2 Po10
2/1
2/2
Peer Link
N5k02
primary
secondary
x8
x8
x8
vPC peer link Running vPC only for server attach ports
5500 or 50x0
...
...
2248TPs
Cisco Confidential
86
logical equivalent
2/1
2/2
N5k02
primary
secondary
to aggregation
vPC is still useful to optimize traffic
SW02
2/10
and SW02
Peer-link is almost unutilized
Po51
2/1
2/2
Po10
2/1
2/2
N5k01
Peer Link
N5k02
Cisco Confidential
88
Define domains
Establish Peer Keepalive connectivity Create a Peer link Create vPCs Make Sure Configurations are
N7k01
N7k02
5 6 N2k01
7 8 N2k02
Cisco Confidential
89
N7k01
N7k02
across linecards
Connect the N5ks with redundant peer-links Create a single Port-channel leveraging LACP
1
LACP
2 3 4
there is a high percentage of single attached devices, you may want to size the peer-link to match the uplink bandwidth utilization
Cisco Confidential
90
If you use the peer switch functionality, then define Identical Priorities on the Aggregation Layer switches, to make them the root Do not use Bridge Assurance Keep the default STP priorities on the access layer switches IF using MST, Make Sure that VLAN range Configurations are Consistent With MST be aware of the NXOS VLAN range and of the Global Type-1 Inconsistencies, hence configure VLAN-to-region mappings from day 1 Use pathcost method long Configure STP port type edge or port type edge trunk
N7k01
N7k02
5 6 N2k01
7 8 N2k02
Cisco Confidential
91
balancers, filers
Configure regular L3 ECMP from the core to
N5k01 5 6 N2k01
N5k02 7 8 N2k02
1
2010 Cisco and/or its affiliates. All rights reserved.
3
Cisco Confidential 92
N7k02 2/10
connectivity for config-sync to work (you may want to use the same mgmt0 for vPC peer keepalive)
FEX A/A provides redundancy and
2/1
2/2
2/1
2/2
each HIF
Config-sync also helps with regular
N5k01
N5k02
port channels
FEX pre-provisioning is highly
recommended
Cisco Confidential
93
Thank You