You are on page 1of 32

Microsoft Azure Stack

Customer Deployment Worksheet


Aric Bernard
HPE | Pointnext Advisor and Professional Services

April 25, 2018


Understanding and Completing the CDW
(in 4 “easy” steps)

2
Step 1
Pick an Identity Provider

3
Azure Stack Identity Options

Azure Active Directory Active Directory Federation Services


– Connectivity to the Internet or ExpressRoute required – Connectivity to the Azure public cloud not required
– Connectivity to the Azure public cloud required – Fully disconnected scenarios supported
– First “tenant” is the administrative entity – Existing Active Directory required
– Identities must exist in the Azure public cloud – Existing AD FS instance or farm required
– Hybrid scenario enablement more likely – Single “entity” scenarios only
– Multi-tenant scenarios possible via two-way “trust” – Other federation options (e.g. PING) not yet supported
– Hosting scenario enablement more likely

4
Azure Stack Identity Requirements

Azure Active Directory Active Directory Federation Services


– Must have connectivity to the Azure public cloud – Must have connectivity to existing Active Directory
Global Catalog
– Must have an Azure Subscription
(and an Azure Active Directory) – Must have an AD account with LDAP read permissions
– Must provide Azure Active Directory Identity with – Must have connectivity to existing AD FS end-point
Global Administrator Role for deployment

5
Step 2
Pick a Billing Model

6
Azure Stack Billing Options

Capacity Model Consumption Model


– Pay upfront and at annual intervals (subscription) – No upfront software licensing fees
– Per core based licensing – Pay as you use compute and storage
– Select either IaaS only (compute and storage) – Pay as you use Windows licensing or BYOL
or add-on App Services
– Metered consumption of resources sent to Microsoft
– Bring your own licenses (BYOL)
– Use Microsoft, HPE, or a 3rd party as your Cloud Service
– Microsoft Enterprise Agreement required Provider (CSP)
– Receive Azure and Azure Stack consumption on a single
bill

7
Azure Stack Billing Option Retail Pricing

Capacity Model Consumption Model


Package Price Service Price
IaaS package $144/core/year Base virtual machine $0.008/vCPU/hour
(base virtual machine and ($6/vCPU/month)
Azure Storage) Windows Server virtual $0.046/vCPU/hour
App Service package $400/core/year machine ($34/vCPU/month)
(App Service, base virtual Azure Blob Storage $0.006/GB/month
machine, (no transaction fee)
and Azure Storage)
Azure Table and Queue $0.018/GB/month
Storage (no transaction fee)
Azure Standard $0.011/GB/month
Unmanaged Disk (no transaction fee)
Azure App Service (Web $0.056/vCPU/hour
Apps, Mobile Apps, API ($42/vCPU/month)
Apps, Functions)

8
Azure Stack Registration Requirements

Capacity Model Consumption Model


– Azure Subscription required – Azure Subscription required
– Must provide Azure Identity with Subscription Owner or – Must provide Azure Identity with Subscription Owner or
Billing Manager privileges Billing Manager privileges
– Must provide an Enterprise Agreement number – Enables Marketplace syndication
– Offline Marketplace syndication is possible

9
HPE GreenLake Flex Capacity for Microsoft Azure Stack
Lower cost, elastic capacity, enterprise support for Hybrid IT

Pay only for what you use1


– Aligns costs with usage monthly via advanced metering Increase capacity
– No upfront payment requirements
– Save the costs of overprovisioning

Infrastructure capacity that never runs out


Requested Pay only for
– Scalable—add capacity in minutes, not months what you use1
capacity
– Avoid long procurement cycles
– Manage capacity jointly with account team
Applies to servers, software-defined and traditional storage,
networks, converged, and software
– HPE and multivendor support Decrease capacity
– Hybrid-ready with certain Microsoft Azure public cloud services
Business application
Add HPE ProLiant with Microsoft Azure Stack to an existing
Flexible Capacity contract 1. Subject to a minimum commitment for hardware and software

For HPE and Channel Partner internal use only 10


Step 3
Pick your names and get some certificates

11
Domain Names

– Internal domain name


– Used only by and for the Azure Stack infrastructure
– External domain name
– Represents the external name known internally to the “enterprise” or externally to the Internet
– Region name
– Prefix used to distinguish between two regions
– Prepended to the external domain name
– Certificates are tied to this name

12
Certificates
Subject Alternative Names
Required certificate subject and
Deployment folder Scope (per region) SubDomain namespace1
subject alternative names (SAN)1,2
Public Portal portal.<region>.<fqdn> Portals <region>.<fqdn>
Admin Portal adminportal.<region>.<fqdn> Portals <region>.<fqdn>
Azure Resource Manager Public management.<region>.<fqdn> Azure Resource Manager <region>.<fqdn>
Azure Resource Manager Admin adminmanagement.<region>.<fqdn> Azure Resource Manager <region>.<fqdn>
ACSBlob *.blob.<region>.<fqdn> Blob Storage blob.<region>.<fqdn>
ACSTable *.table.<region>.<fqdn> Table Storage table.<region>.<fqdn>
ACSQueue *.queue.<region>.<fqdn> Queue Storage queue.<region>.<fqdn>
KeyVault *.vault.<region>.<fqdn> Key Vault vault.<region>.<fqdn>
KeyVaultInternal *.adminvault.<region>.<fqdn> Internal Keyvault adminvault.<region>.<fqdn>
Admin Extension Host *.adminhosting.<region>.<fqdn> Admin Extension Host adminhosting.<region>.<fqdn>
Public Extension Host *.hosting.<region>.<fqdn> Public Extension Host hosting.<region>.<fqdn>
ADFS2 adfs.<region>.<fqdn> ADFS <region>.<fqdn>
Graph2 graph.<region>.<fqdn> Graph <region>.<fqdn>

1 <fqdn> represents the external domain name of the Azure Stack instance
2 only required when ADFS is used as the identity provider
3 an astericks (*) indicates a wild card
13
Certificates
Requirements
– Certificates must be issued from either an internal CA or a Public CA.
– Network access must exist to the certificate authority used to sign the certificates
– The use of self-signed certificates are not supported
– A single wild card certificate covering all name spaces in the Subject Alternative Name (SAN) field can be
used
– The certificate signature algorithm must be stronger than SHA1
– The certificate format must be PFX, as both the public and private keys are required for Azure Stack
installation
– The certificate pfx files must have a value "Digital Signature" and "KeyEncipherment" in its “Key Usage"
field
– The certificate pfx files must have the values “Server Authentication (1.3.6.1.5.5.7.3.1)” and “Client
Authentication (1.3.6.1.5.5.7.3.2)” in the "Enhanced Key Usage" field
– The certificate's "Issued to:" field must not be the same as its "Issued by:" field
– The passwords to all certificate pfx files must be the same at the time of deployment

14
Certificates
Requirements for SQL, MySQL, and App Service Resource Providers
Required certificate subject and subject
Scope (per region) SubDomain namespace1
alternative names (SAN)1,2

*.dbadapter.<region>.<fqdn> SQL, MySQL dbadapter.<region>.<fqdn>

*.appservice.<region>.<fqdn>
appservice.<region>.<fqdn>
*.scm.appservice.<region>.<fqdn> App Service3
scm.appservice.<region>.<fqdn>
*.sso.appservice.<region>.<fqdn>
appservice.<region>.<fqdn>
api.appservice.<region>.<fqdn> App Service4
scm.appservice.<region>.<fqdn>
appservice.<region>.<fqdn>
ftp.appservice.<region>.<fqdn> App Service4
scm.appservice.<region>.<fqdn>
appservice.<region>.<fqdn>
sso.appservice.<region>.<fqdn> App Service4
scm.appservice.<region>.<fqdn>

1 <fqdn> represents the external domain name of the Azure Stack instance
2 an astericks (*) indicates a wild card
3 Requires one certificate with multiple wildcard SANs – multiple wildcard SANs on a single certificate might not be supported by all CAs
4 The App Service resource provider explicitly requires the use of separate certificates for these endpoints
15
Step 4
Provide your network information

16
Network
Environmental Parameters
– Time Synchronization (NTP) servers
– DNS Forwarders – comma separated IP addresses
– Syslog Server – IP address (optional)
– Network subnets – 5 (or more) subnets dedicated to the Azure Stack deployment
– Route exchange – BGP or static routes are supported between TORs and uplink switches
– Autonomous System Number (ASN) for uplink switches if BGP is selected
– ASN for TORs
– ASN for Azure Stack (aka SLB/MUX)

17
Subnet Information

– BMC Network /26


– Storage & Internal VIPs Network /24*
– Infrastructure Network /24
– External Network /26 minimum*
– Switch Infrastructure Network /26

*Virtualized Networks instantiated by the SDN


BMC Network

– Each iLO is connected – Deployment VM


– Access to HLH iLO through TOR – Only required during deployment

– HLH is connected – Access to DNS forwarders and time servers*

– Access to HLH RDP through TOR – Access to the Azure public cloud**

– Access to DNS forwarders and time servers*

– OneView appliance
– Optional connectivity to OneView web console through TOR
– Access to DNS forwarders and time servers*

Hardware Lifecycle Host Azure Stack Hosts TOR Switch BOR Switch

Hyper-V Internet
NIC Software Gateway
Team Hyper-V
NIC
Team
BMC Switch NAT/tProxy
BMC
*Minimal traffic and throughput required
BMC
**For Azure AD connected deployments

BMC Network
19
Storage & Internal VIPs
Network
– Private Network
– Never leaves the rack
– Storage Replication
– Internal VIPs
– Virtual IPs for Azure Stack infrastructure
– Used internally by Azure Stack components

Internal Virtual IPs Subnet


Storage Subnet

Hardware Lifecycle Host Azure Stack Hosts TOR Switch BOR Switch

Hyper-V Internet
NIC Software Gateway
Team Hyper-V
NIC
Team
BMC Switch NAT/tProxy
BMC

BMC 20
Infrastructure Network

– Mostly private network


– Azure Stack infrastructure VMs
– Shared by all scale units in a region
– Most external communications traverse SLB via Azure Stack on the external network
– Exceptions
– Emergency Recovery Console (ERCS) / Privileged Endpoint (PEP)
– Used for certain highly privileged operations
– Azure Stack registration
– Access to DNS forwarders and time servers*
Internal Infrastructure Subnet
– Access to the Azure public cloud**

Hardware Lifecycle Host Azure Stack Hosts TOR Switch BOR Switch

Hyper-V Internet
*Minimal traffic and throughput required NIC Software Gateway
Team Hyper-V
**For Azure AD connected deployments NIC
Team
BMC Switch NAT/tProxy
BMC

BMC 21
External Network

– Used by Azure Stack for:


– “Public” facing endpoints: portals, APIs, etc.
– Tenant workloads with a “public” IP address
– Typically public address space
– Pool is owned by the Azure Stack network controller (Software BGP ASN in CDW)
– Allocated address are advertised as /32 route
Public Virtual IPs Network
– Connected Solutions
– Access to DNS forwarders and time servers
– Access to the Azure public cloud**
– Expected to have Internet access*
Hardware Lifecycle Host Azure Stack Hosts TOR Switch BOR Switch

Hyper-V Internet
*True of most ‘connected’ scenarios NIC
Team
Software Gateway
Hyper-V
**For Azure AD connected deployments NIC
Team
BMC Switch NAT/tProxy
BMC

BMC
22
Inbound/Outbound Protocol Requirements*

Inbound Outbound
Endpoint (VIP) DNS host A record Protocol Purpose URL Protocol
AD FS Adfs.<region>.<fqdn> HTTPS login.windows.net HTTP
Portal (administrator) Adminportal.<region>.<fqdn> HTTPS Identity1,2 login.microsoftonline.com
HTTPS
Azure Resource Manager (administrator) Adminmanagement.<region>.<fqdn> HTTPS graph.windows.net
Portal (user) Portal.<region>.<fqdn> HTTPS https://management.azure.com
Azure Resource Manager (user) Management.<region>.<fqdn> HTTPS https://*.blob.core.windows.net
Marketplace syndication HTTPS
Graph Graph.<region>.<fqdn> HTTPS https://*.azureedge.net
Certificate revocation list Crl.<region>.<fqdn> HTTP https://*.microsoftazurestack.com
DNS *.<region>.<fqdn> TCP,UDP Patch & Update https://*.azureedge.net HTTPS
1
Key Vault (user) *.vault.<region>.<fqdn> HTTPS Registration https://management.azure.com HTTPS
Key Vault (administrator) *.adminvault.<region>.<fqdn> HTTPS https://*.microsoftazurestack.com
Usage HTTPS
HTTP https://*.trafficmanager.com
Storage Queue *.queue.<region>.<fqdn>
HTTPS Insight Remote
https://api.support.hpe.com HTTPS
HTTP Support2
Storage Table *.table.<region>.<fqdn>
HTTPS
HTTP
Storage Blob *.blob.<region>.<fqdn>
HTTPS
SQL Resource Provider sqladapter.dbadapter.<region>.<fqdn> HTTPS
MySQL Resource Provider mysqladapter.dbadapter.<region>.<fqdn> HTTPS *Additional port information available at:
https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-integrate-endpoints
*.appservice.<region>.<fqdn> TCP
1Required by Infrastructure network
*.scm.appservice.<region>.<fqdn> TCP
App Service
api.appservice.<region>.<fqdn> TCP 2Required by BMC network
ftp.appservice.<region>.<fqdn> TCP,UDP

23
Switch Infrastructure
Network
– BMC, TOR, and BOR switches
– Inter-switch communications
– Access to DNS forwarders and time servers
– Access to Syslog server

Hardware Lifecycle Host Azure Stack Hosts TOR Switch BOR Switch

Hyper-V Internet
NIC Software Gateway
Team Hyper-V
NIC
Team
BMC Switch NAT/tProxy
BMC

BMC

24
Switch Infrastructure Network
All together

Public Virtual IPs Network

Internal Virtual IPs Subnet


Infrastructure Networks Storage Subnet
Internal Infrastructure Subnet

Hardware Lifecycle Host Azure Stack Hosts TOR Switch BOR Switch

Hyper-V Internet
NIC Software Gateway
Team Hyper-V
NIC
Team
BMC Switch NAT/tProxy
BMC

BMC

BMC Network

Switch Infrastructure Network

25
High-level Integration (initial release)
Border Device Rack

–In-Rack switching BOR1 BOR2

– 2x Top of Rack (TOR)


–Redundant connection from each scale unit
node /30 /30 /30

– 1x Base Management Controller (BMC)


–Connection from each scale unit node iLO Azure Stack Rack

– Acquired as part of the validated solution TOR1 VPC/MLAG TOR2


from the OEM PEER

–HPN, Cisco, … NIC1 NIC2

–Up-stream switching
– Redundant pair of switches (Border/BOR)
/30 iLO /30
– Part of the customers existing network
BMC
BGP Routing Integration Option
Border Device Rack

BOR1 BOR2

–Border Gateway Protocol (BGP) ASN 647x1

– Routing protocol often used between


Autonomous Systems Fault Domain /30 /30 /30

–Autonomous System Dynamic BGP advertisement to Azure Stack Rack


TORs
– Collection of connected Internet IP
routing prefixes under the control of a TOR1 VPC/MLAG
PEER
TOR2

single administrative entity


– Represented by a 16-bit number (ASN) Dynamic BGP advertisement to
BORs and SLB
Software Load Balancer

–Explicitly used within the rack


Internal VIPs
ACLs deny private networks
External VIPs
(Internal-VIPs and Storage)
routing /30 /30
–Optionally used with the BOR
ASN 648x1

switches Dynamic BGP advertisement


BMC

to TORs
ASN 647x1
BGP Routing Integration Option
Border Device Rack

BOR1 BOR2

#TOR Example ASN 647x1


bgp 647x1
router-id 10.1.80.12
peer 10.1.80.0 as-number 646x1
peer 10.1.80.11 as-number 647x1 Fault Domain /30 /30 /30
peer 10.1.81.0 24 as-number 648x1
peer 10.1.81.0 24 connect-interface LoopBack0
peer 10.1.81.0 24 ebgp-max-hop 2
Dynamic BGP advertisement to Azure Stack Rack
# TORs
address-family ipv4 unicast
bestroute as-path-neglect VPC/MLAG
TOR1 TOR2
balance 8 PEER
balance as-path-neglect
network 10.1.80.0 255.255.255.254
network 10.1.80.8 255.255.255.254 Dynamic BGP advertisement to
Software Load Balancer
network 10.1.80.12 255.255.255.255 BORs and SLB
network 10.1.80.16 255.255.255.254 Internal VIPs
network 10.1.81.0 255.255.255.0 ACLs deny private networks
External VIPs
network 10.1.82.0 255.255.255.128 (Internal-VIPs and Storage)
peer 10.1.80.0 enable routing /30 /30
ASN 648x1
peer 10.1.80.11 enable
peer 10.1.81.0 24 enable
BMC
Dynamic BGP advertisement
to TORs
ASN 647x1
Static Routing Integration Option
Border Device Rack

BOR1 BOR2

–Static Routes
– Infrastructure (inbound)
Fault Domain /30 /30 /30

– Switch Management (inbound) BOR Static Routes to TORs


 Infrastructure
– Public VIPs (inbound)  BMC Management
 Switch Management
Azure Stack Rack
 Point-to-Point Links
– BMC (inbound)  Publi-VIPs TOR1 VPC/MLAG TOR2
PEER

– Default Route (outbound)


–BGP explicitly used within the rack
TOR Static Default Route
Software Load Balancer
(0.0.0.0/0) to BORs
Internal VIPs
Dynamic BGP advertisement
External VIPs
to SLB
/30 /30
ASN 648x1

BMC
Dynamic BGP advertisement
to TORs
ASN 647x1
Static Routing Integration Option
Border Device Rack

BOR1 BOR2

# TOR Example
ip route-static 0.0.0.0 0 10.1.80.0
#
bgp 647x1
router-id 10.1.80.12 Fault Domain /30 /30 /30
peer 10.1.80.11 as-number 647x1
peer 10.1.81.0 24 as-number 648x1 BOR Static Routes to TORs
 Infrastructure
peer 10.1.81.0 24 connect-interface LoopBack0
 BMC Management Azure Stack Rack
peer 10.1.81.0 24 ebgp-max-hop 2  Switch Management
#  Point-to-Point Links
address-family ipv4 unicast  Publi-VIPs VPC/MLAG
TOR1 TOR2
bestroute as-path-neglect PEER
balance 8
balance as-path-neglect
network 10.1.80.8 255.255.255.254 TOR Static Default Route
Software Load Balancer
network 10.1.80.12 255.255.255.255 (0.0.0.0/0) to BORs
network 10.1.80.16 255.255.255.254 Internal VIPs
network 10.1.81.0 255.255.255.0 Dynamic BGP advertisement
External VIPs
network 10.1.82.0 255.255.255.128 to SLB
peer 10.1.80.11 enable /30
ASN 648x1 /30
peer 10.1.81.0 24 enable

BMC
Dynamic BGP advertisement
to TORs
ASN 647x1
External Connectivity
Existing network Border Device Rack

– No support for a configurable proxy server Core BOR

– Transparent or pass-thru proxy supported


FW/Proxy DMZ

Edge

Azure Stack Rack

Internet TOR

BMC
Tenant Connectivity Provider Network
Express Route

–Tenants create virtual networks Internet

– VxLAN vNETs Provider Edge Switch

– vNET IP ranges can overlap Azure Public Cloud Customer Network Border Device Rack

–Internet (NAT)
–Public IP (optional inbound) Express Route Edge Internet Edge Customer Edge Switch Border Switches

–Site-to Site (S2S)


Azure Stack Rack
– vNET Site-to-Site VPN Endpoint

– Public IP address Top-of-Rack Switches

– Virtual Network Gateway Internal Router

– Local Network Gateway Virtual Network Gateway Virtual Network Gateway

– Site-to-Site (S2S) Connection 10.3.1.0/24 10.1.1.0/24 10.2.1.0/24


Virtualized Tenant Subnet Corporate Subnet Virtualized Tenant Subnet

–ExpressRoute ASN - 64512 ASN - 64522

You might also like