VMware Validated Design

for Software-Defined Data Center 4.0

Logical Component Architecture

In a dual-region SDDC, the design instantiates two Platform Service Controllers and two vCenter Server instances in the appliance form factor. In a dual-region SDDC, two separate NSX Manager instances are deployed in Region A. One instance for the management pod and one instance for In a dual-region SDDC, a vRealize Log Insight cluster is deployed in each region and consists of three nodes, enabling
Core vSphere Management This includes a vCenter Server for the management pod and a vCenter Server for the shared edge and compute pods.
Each vCenter Server instance is connected to a load-balanced pair of Platform Services Controllers using an NSX Edge Services Gateway.
NSX the shared edge and compute pods, along with associated NSX Universal Controller Clusters.
In Region B the secondary NSX Manager instances automatically import the configurations of the NSX Universal Controller Clusters from Region A.
vRealize Operations
The design uses several VMware solutions for network, storage, and cloud management. You can monitor and
perform diagnostics on all of them by using vRealize Operations and solution management packs. vRealize Log Insight continued availability and increased log ingestion rates. vRealize Log Insight collects log data from Platform Service
Controllers, vCenter Server instances, ESXi hosts, and NSX components with syslog protocol. vRealize Log Insight also
To enable enhanced linked mode, the design joins the Platform Services Controller instances into a unified Single Sign-On domain integrates with vRealize Operations Manager to send notication events and facilitate root cause analysis.

Region A Region B Region A Region B Region A Region B

Management / Management /
Common vCenter Single Sign-On Domain Management /
vRealize
Management /
Compute
vRealize
Compute
vRealize
Automation
Management Stack Management Stack Compute
Automation
Compute Automation
(Ring Topology) vCenter Servers vCenter Servers
vCenter Servers vCenter Servers Proxy Agents
vCenter Server vCenter Server
Appliance Appliance
NSX vRealize Log Insight Cluster vRealize Log Insight Cluster NSX
NSX Remote Collectors Remote Collectors NSX
Event
Region A
Region B Management Stack Management Stack Master Worker Worker
Forwarding
via Ingestion Master Worker Worker
Clctr Clctr Clctr Clctr
NSX Manager Pairing Shared Node Node Node API Node Node Node
NSX Manager NSX Manager Storage
Node Node Node Node Shared
Storage vSAN vSAN
(Primary) (Secondary) Systems Systems

vRealize

Platform Services Platform Services SDPlatform Services Platform Services NSX Edge Management Stack Import of Management Stack NSX Edge Analytics Cluster
vRealize
Operations
vSAN NFS vSAN NFS Operations
Remote
Controller Controller Controller Controller Services Gateways NSX Universal NSX Controller Configuration Services Gateways
Collectors
Appliance Appliance Appliance Appliance from Primary NSX Manager Master Replica Data
(N/S Routing) Controller Cluster (N/S Routing) Node Node Node

Primary Storage Log Archives Primary Storage Log Archives

NSX Edge Services Compute Stack Compute Stack NSX Edge Services Region A Management Pod Region B Management Pod Region A Management Pod Region B Management Pod
NSX Edge Services NSX Edge Services Gateway w/ HA vCenter Server vCenter Server Gateway w/ HA
Gateway w/ HA Gateway w/ HA (One-Arm Load Balancer) Appliance Appliance (One-Arm Load Balancer)
(Load Balancer) (Load Balancer)

Compute Stack NSX Manager Pairing

Compute Stack VMware Validated Designs establish a Cloud Management Platform with vRealize Automation to provide the The design implements a single vRealize Automation tenant. Business groups can be created to fit your needs.
NSX Manager NSX Manager vRealize Automation, vRealize Orchestrator service catalog and self-service portal to deploy, update, and manage the workloads. vRealize Orchestrator vRealize Automation Within each business group the tenant administrators are able to manage users and groups, apply tenant-specific
(Primary) (Secondary) and vRealize Business for Cloud provides a repository of extensible workflows and integrations. vRealize Business for Cloud provides visibility into Business Groups & Reservations branding, enable notifications, configure business policies, and manage the service catalog.
Management Stack Compute Stack Management Stack Compute Stack the financial aspects of the cloud infrastructure, allowing cost to be tracked and optimized.
vCenter Server vCenter Server vCenter Server vCenter Server
Appliance Appliance Appliance Appliance
Business Business
Region A Management Pod Region B Management Pod Tenant
Admin Group Group
Tenant
Admin
vRealize vRealize vRealize vRealize vRealize
Sign In

Manager Manager
Orchestrator Automation Business Automation Business
https://my.sddc.local/vcac/org/company

vSphere Update vSphere Update NSX Edge Compute Stack NSX Import of Compute Stack NSX Edge VRO VRA IWS IMS DEM IAS SQL BUS IAS BUC
Manager Download Manager Download Services Gateways Universal Controller NSX Controller Configuration Services Gateways Edge Business Group Business Group Business Group Business Group Edge
Service Service (N/S Routing) Cluster
from Primary NSX Manager
(N/S Routing) Reservation Reservation Reservation Reservation Reservation Reservation
VRO VRA IWS IMS DEM IAS BUC IAS Fabric Fabric
Region A Fabric Group Region B Fabric Group
Admin Admin

IaaS Region A Data Center Infrastructure Fabric Region B Data Center Infrastructure Fabric IaaS
Region A Management Pod Region B Management Pod Region A Shared Edge and Compute Pod Region B Shared Edge and Compute Pod Admin Admin
Shared Edge/Compute Pod Additional Compute Pod(s) Shared Edge/Compute Pod Additional Compute Pod(s)
(Edge Resource Pool) (Edge Resource Pool)
Region A Management Pod Region B Management Pod

Core and Pod Architecture Pod and Clusters

Leaf-and-Spine Network Pods

The design uses a small set of common,
Host Connectivity
Universal Logical Switch Universal Logical Switch

Application Virtual Networks

standardized building blocks called pods.

Workload Virtual Networks

L2 L2
UDLR External UDLR & DLR

for SDDC Solutions

APP APP APP APP APP APP
L3 Networks APP APP APP
L3

OS OS OS OS OS OS OS OS OS
Universal Compute Transport Zone Management Pod ESXi Host Shared Edge and Compute Pod ESXi Host
Spine Spine Spine Universal Logical Switch Universal Logical Switch
Universal Management Transport Zone Local Compute Transport Zone

North/South
L2
10 GigE nic0 nic1 10 GigE 10 GigE nic0 nic1 10 GigE L2
UDLR UDLR & DLR

Routing
L3 L3 L3 L3
Spine Spine
Management Distributed Switch Compute Distributed Switch L3 L3
APP APP APP APP APP APP APP APP APP APP
Routed Uplinks (ECMP) MTU 9000 MTU 9000
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP OS OS OS OS OS OS OS OS OS OS
L3 L3
Layer 3 ToR Switch
Management Distributed Switch Compute Distributed Switch
VDP vCenter vCenter L2 L2

Core Platform
(Management)
ESXi ESXi ESXi ESXi ESXi ESXi ESXi ESXi L3
(Management) (Compute)

Services
MTU MTU
VLAN NFS VLAN NFS SRM PSC PSC
40 GigE 40 GigE L2 9000 9000 Edge
(Management) (Compute) (Management) NSX Controllers N/S NSX EDGE N/S NSX EDGE NSX Controllers
Leaf Leaf Leaf Leaf Leaf Leaf Resource
Leaf Leaf (Management) (Management) (Compute) (Compute)
MTU MTU Pool
VLAN Management VLAN Management VR NSX Manager NSX Manager
vSAN
9000 9000
L3 L3 L3
Any Supported Storage VLAN 1611 VLAN 1612 VLAN 1613 VLAN 1614
(Management) (Compute) (Management)
UDLR UDLR & DLR
IGMP IGMP IGMP IGMP IGMP IGMP MTU MTU
L2 IGMP IGMP VLAN vMotion 9000 VLAN vMotion 9000
L2 L2

Span of VLANs
plus NFS plus NFS MTU MTU Management Management
10 GigE 10 GigE VLAN VTEP (VXLAN) 9000 VLAN VTEP (VXLAN) 9000

Distributed
Management Pod
Shared Edge and Compute Cluster

Switches
North/South Uplink(s)

Span of VLANs
North/South Uplink(s)
(4 vSAN Ready Nodes) Management Cluster
& Compute Cluster n
MTU MTU vMotion vMotion
VLAN vSAN 9000 VLAN vSAN 9000
Minimum 4 vSAN Ready Nodes Recommended | vSAN Enabled Minimum 4 Nodes | vSAN Ready Nodes Recommended VLAN Trunk (802.1Q) vSAN vSAN
Server vSphere HA and DRS Enabled vSphere HA and DRS Enabled | Business Workload Requirements Region Dependent VXLAN VXLAN xxxx
ESXi Host VLAN External Management VLAN Uplink 01
Region Independent VXLAN VXLAN xxxx
Shared Edge and Compute Pod
Management Pod MTU
& Compute Pod n VLAN vSphere Replication 9000 VLAN Uplink 02
The management pod hosts the infrastructure components used to instantiate,
Management vMotion VXLAN vSAN
manage and monitor the SDDC. This includes the core infrastructure
Shared Edge and Compute Pod Workloads running in the SDDC do not have direct access to external networks.
172.16.11.0/24 172.16.12.0/24 172.16.13.0/24 172.16.14.0/24 VLAN Uplink 01
components, such as the Platform Services Controllers, vCenter Server instances,

Edge/Compute Cluster
To access external networks, tra c is routed through distributed routing ESXi-MGMT-01 ESXi-MGMT-02 ESXi-MGMT-03

Transport Zones
(4+ Hosts) VTEPs VTEPs VTEPs VTEPs ESX-COMP-01

Management Custer
NSX Managers, NSX Controllers for the management stack, vSphere Replication, to the NSX Edge Services Gateways in the shared edge and compute pod. DGW: DGW:
Site Recovery Manager, as well as the SDDC monitoring and automation Expansions beyond the initial shared pod are simply compute pods. 172.16.11.253 172.16.12.253 UDLR UDLR UDLR UDLR & DLR

Universal
solutions like vRealize Operations, vRealize Log Insight and vRealize Automation. VLAN Uplink 02

vSAN Any Supported

Additional Compute Pods
Storage
(Up to 19 2RU Hosts or 19 vSAN Ready Nodes) Management Stack Compute Stack The leaf switches of each rack acts as the Layer 3 interface for the corresponding subnet. The two 10GbE NICs on each host are connected across the top-of-rack leaf switches and teamed on the vSphere Distributed Switch via an active-active configuration.
Managed by Management Stack vCenter Server Managed by Compute Stack vCenter Server All pods are provided with externally accessible VLANs for access to the Internet and corporate networks. All port groups, except for the ones that carry VXLAN tra c, are configured for the 'Route based on physical NIC load' teaming algorithm.
The physical network architecture in the design is tightly coupled with the pod-and-core architecture and uses a VTEP kernel ports and VXLAN tra c use the Route based on SRC-ID' algorithm.
Management Pod Edge/Compute Pod
Layer 3 leaf-and-spine network model for an e cient, resilient, and distributed core. The vSphere Distributed Switch has a MTU of 9000 configured for Jumbo Frames along with with necessary VMkernel ports.

Distributed Logical Routing and Application Virtual Networks for Management, Operations and Automation Solutions Storage

Distributed Logical Routing vRealize Operations vRealize Automation, vRealize Orchestrator vSAN Ready Nodes NFS Storage
and Application Virtual Networks and vRealize Log Insight and vRealize Business for Cloud

Region A Region B
Caching
SSD PCIe NVMe Tier
Read and Write Cache NFS Storage Array NFS Storage Array
ECMP
NSX Edge
Region A Region B ECMP
NSX Edge
Region A Region B Region A Region B
ECMP ECMP Services Gateways Services Gateways
NSX Edge NSX Edge
Internet or Internet or Services Gateways Services Gateways BGP Universal Transit Network Universal Transit Network BGP
Peering Universal Logical Switch / VXLAN Segment 192.168.10.0/24 192.168.10.0/24 Universal Logical Switch / VXLAN Segment Peering
Enterprise Enterprise BGP Universal Transit Network Universal Transit Network BGP
WAN/MPLS WAN/MPLS Peering Universal Logical Switch / VXLAN Segment 192.168.10.0/24 192.168.10.0/24 Universal Logical Switch / VXLAN Segment Peering

Management Universal Distributed Logical Router

Management Universal Distributed Logical Router
Internet or Enterprise WAN/MPLS

Internet or Enterprise WAN/MPLS

Spine To Shared Edge and Compute Pod To Shared Edge and Compute Pod Spine Region Independent Application Virtual Network Region Independent Application Virtual Network
Switches Switches Region Independent Application Virtual Network Region Independent Application Virtual Network Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment Volume 1 Volume 2 Volume 1 Volume 2
To Additional Compute Pods To Additional Compute Pods
Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment Capacity
NSX Edge Services Gateway NSX Edge Services Gateway Data
NSX Edge Services Gateway NSX Edge Services Gateway
One-Arm Load Balancer
192.168.11.0/24 192.168.11.0/24
One-Arm Load Balancer
Persistence Export Export Export Export Export Export
One-Arm Load Balancer One-Arm Load Balancer
192.168.11.0/24 Tier (vRealize (vRealize (vSphere (vRealize (vRealize (vSphere
L3 L3 192.168.11.0/24
Top-of-Rack Top-of-Rack Automation) Log Insight) Data Protection) Automation) Log Insight) Data Protection)
Leaf Switches 172.16.11.0/24 172.17.11.0/24 Leaf Switches
L2 L2 APP APP APP APP
VRA VRA VRA VRA
OS OS OS OS
APP APP APP APP APP APP APP
BGP Peering BGP Peering
OS OS OS OS OS OS OS
NSXM VDP PSC VC VC PSC VDP NSXM
Master Replica Data Master Replica Data APP APP
IWS APP APP
IWS
VMware Validated Designs use vSAN Ready Nodes to ensure seamless compatibility and support. The design used NFS storage as a secondary storage tier for management and compute pods.
ECMP OS OS OS OS OS OS OS OS ECMP
Node Node Node
IWS IWS
NSX Edge NSX Edge Node Node Node OS OS OS OS The configuration and assembly for each node is standardized with all components installed the same manner to NFS is used as the target for vSphere Data Protection backups and vRealize Log Insight log archives in the management pod.
Services Gateways Services Gateways
BGP Universal Transit Network Universal Transit Network BGP vRealize Operations Replicated for Disaster Recovery eliminate system variability. vSAN enables both hybrid and all-flash architectures. NFS is also used to host the virtual machine templates in the compute pods.
Peering Universal Logical Switch / VXLAN Segment 192.168.10.0/24 192.168.10.0/24 Universal Logical Switch / VXLAN Segment Peering
APP APP
IMS APP APP
IMS
vSphere Data Protection is interchangable with any vSphere APIs for Data Protection compatible solution.
IMS IMS
OS OS OS OS

Region Dependent Application Virtual Network Region Dependent Application Virtual Network
Management Universal Distributed Logical Router
Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment APP APP APP APP
DEM DEM DEM DEM
OS OS OS OS
192.168.31.0/24
192.168.32.0/24

Reference
Region Independent Application Virtual Network Region Independent Application Virtual Network

Region Protection and Disaster Recovery

Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment APP APP APP APP
VRO VRO VRO VRO
OS OS OS OS
NSX Edge Services Gateway NSX Edge Services Gateway
One-Arm Load Balancer One-Arm Load Balancer APP APP APP APP
192.168.11.0/24 192.168.11.0/24 OS OS OS OS APP APP APP APP
Collector Collector SQL BUS SQL BUS
Collector Collector OS OS OS OS
APP APP APP APP APP
Reserved for Disaster Recovery
Node Node Node Node Region A Region A Replicated Region B Replicated Region B Networks Notable Acronyms
OS OS OS OS OS
vRealize Operations vRealize Operations
vRealize Automation / vRealize Orchestrator Replicated for Disaster Recovery Non-Replicated Non-Replicated
vRealize Business for Cloud
vRealize Automation vRealize Automation BUC vRealize Business Data Collector
Region Dependent Application Virtual Network Region Dependent Application Virtual Network External Transit Network(s) BUS vRealize Business Appliance
Universal Logical Switch / VXLAN Segment Universal Logical Switch / VXLAN Segment Region Dependent Application Virtual Network Region Dependent Application Virtual Network
vRealize Orchestrator vRealize Orchestrator
DEM vRealize Automation Distributed Execution Manager
Universal Logical Switch / VXLAN Segment 192.168.31.0/24 192.168.32.0/24 Universal Logical Switch / VXLAN Segment vRealize Business for Cloud vRealize Business for Cloud
192.168.31.0/24 192.168.32.0/24 vRealize Log Insight vRealize Log Insight IAS vRealize Automation IaaS vSphere Proxy Agent
APP
vRealize Operations vRealize Operations IMS vRealize Automation IaaS Manager Service
APP APP APP APP APP
OS OS OS IWS vRealize Automation IaaS Web Server
SRM SRM
OS OS OS
APP APP APP APP APP APP APP APP APP APP APP APP APP APP APP APP (using vSphere Replication) (using vSphere Replication) Universal Transit Network VXLAN
Cluster Master Worker Worker Cluster Master Worker Worker OS OS OS OS OS OS NSXM NSX Manager
OS OS OS OS OS OS OS OS OS OS VIP Node Node Node VIP Node Node Node PSC Platform Services Controller
IAS IAS BUC IAS IAS BUC
vRealize Log Insight
SQL Microsoft SQL Server Database
vRealize Log Insight
Application Virtual Networks for SDDC Management Solutions in Region A Application Virtual Networks for SDDC Management Solutions in Region B Region A Infrastructure Management Region B Infrastructure Management SRM Site Recovery Manager
Management Distributed Port Group UDLR Universal Distributed Logical Router
vSphere Update Manager Download Service, vSphere Update Manager Download Service,
vRealize Operations Analytics Cluster and Remote Collectors, Regional vRealize Log Insight Cluster, vRealize Operations Remote Collectors, Regional vRealize Log Insight Cluster, 192.168.11.51 > Active Node 192.168.11.57 > Active Node vSphere vSphere VDP vSphere Data Protection
Distributed vRealize Automation and Proxy Agents, and vRealize Business for Cloud Server and Collector. vRealize Automation Proxy Agents and vRealize Business for Cloud Collector. VRA VIP: 192.168.11.53 IMS VIP: 192.168.11.59 NSX NSX VR vSphere Replication
192.168.11.52 > Active Node 192.168.11.58 > Passive Node
Disaster Recovery vRealize Operations Analytics Cluster, Site Recovery Manager Site Recovery Manager VRA vRealize Automation Appliance
Distributed vRealize Automation, and vRealize Business for Cloud Server.
192.168.11.54 > Active Node 192.168.11.63 > Active Node vSphere Data Protection vSphere Data Protection Management Application Virtual Network VXLAN VRO vRealize Orchestrator Appliance
IWS VIP: 192.168.11.56 VRO VIP: 192.168.11.65 VTEP VXLAN Tunnel Endpoint
192.168.11.55 > Active Node 192.168.11.64 > Active Node
One region is designated as the primary region and the other as the secondary region. The SDDC management, automation and operations solutions are
deployed in the primary region and configured to migrate to the secondary region in the event of a disaster. All regions actively run business workloads.

Copyright 2017 VMware, Inc. All rights reserved. Refer to the design release notes for products and versions included in the design. @tenthirtyam | vmware.com/go/vvd-docs