Understanding Networking with Hyper-V

White Paper
Published: June 2009
For the latest information, please visit:
http://www.microsoft.com/windowsserver2008/en/us/hyperv.aspx
Contents
Introduction .................................................................................................................................2
Vocabulary..............................................................................................................................2

Virtual Networking.......................................................................................................................4
Hyper-V Virtual Network Manager..........................................................................................4
Virtual Network .......................................................................................................................5
Private Virtual Network ...........................................................................................................5
Internal Virtual Network ..........................................................................................................6
External Virtual Network .........................................................................................................7
Summary ................................................................................................................................8
Virtual Network Adapters........................................................................................................9
Virtual Legacy Network Adapter ....................................................................................... 10
Virtual High Speed Network Adapter................................................................................ 10
Virtual LAN Identification .................................................................................................. 11
SCVMM Preferred Location and Tag .................................................................................. 14

Hyper-V Network Configuration Scenarios for Live Migration ................................................. 16

Customer Scenarios ............................................................................................................ 16
Current Landscape and Challenges.................................................................................... 16
Different network access needs for Live Migration ............................................................. 16
Live Migration Networking Scenario Recommendations..................................................... 17

Windows Server 2008 R2 Hyper-V MAC Address Management ............................................ 19

Hyper-V Dynamic MAC Address ......................................................................................... 20
Hyper-V MAC Address Pool Duplication ............................................................................. 20
Detecting MAC Address Duplications ................................................................................. 21
Adjusting the MAC Address Pool ........................................................................................ 21

Virtual Network Setup .............................................................................................................. 22

New Virtual Network ............................................................................................................ 22
Virtual Network ................................................................................................................. 22
Setup Details ....................................................................................................................... 29
Dedicated Virtual Network for Each Virtual Machine ....................................................... 30
Networking with iSCSI Shared Storage............................................................................ 32
Physical Machine Network Communication with External Physical Machine and Virtual
Network............................................................................................................................. 33
Virtual Machine Network Communication on a Private Virtual Network .......................... 34
Virtual Machine Network Communication with Management OS on the Internal Virtual
Network............................................................................................................................. 35
Guest OS Network Communication with Management OS on External Network Switch 38
Dedicated External Virtual Network.................................................................................. 39

Hyper-V Networking Best Practices......................................................................................... 43

Best Practices...................................................................................................................... 43
Hyper-V in the DMZ............................................................................................................. 43

Updates in Windows Server 2008 R2...................................................................................... 44

Conclusion ............................................................................................................................... 46

Understanding Networking with Hyper-V 1

Introduction
This document covers networking functionality in Windows Server 2008 Hyper-V RTM. The
end of the document contains information and updates related to Windows Server 2008 R2
Hyper-V.
In general terms, server virtualization allows for a single physical computer to host and run,
simultaneously, two or more different OS environments which function as independent entities
with individual identities, application stacks, and so on. Hyper-V is a next-generation, 64-bit
hypervisor-based virtualization technology that offers reliable and scalable platform
capabilities. Together with System Center, it offers a single set of integrated management
tools for both physical and virtual resources.
Implementing Hyper-V not only reduces costs, improves utilization, optimizes infrastructure,
but allows businesses to rapidly provision new servers.
This document covers the ins and outs networking within Hyper-V to provide you a
compressive understanding of Hyper-V and its architecture.
Note: If you are running the version of Hyper-V that shipped with Windows Server 2008, the
physical host needs to be updated to the RTM version of Hyper-V, which can be downloaded
here:
Hyper-V Update for Windows Server 2008 x64 Editions (KB950050)

Vocabulary
The following list defines the vocabulary and acronyms in Hyper-V:
x APIC: Advanced Programmable Interrupt Controller is a device which allows priority
levels to be assigned to its interrupt outputs.
x Hypercall: The Hypercall is the interface for communication with the hypervisor.. Guests
communicate with the hypervisor via Hypercalls.
x Management Operating System (OS): Manages machine-level functions such as
device drivers, power management, and device hot addition/removal. The management
OS is the only OS that has direct access to physical memory and devices.
x Guest Operating System (OS): A virtual machine that hosts a guest operating system -
All access to physical memory and devices by Guest OS is provided via the Virtual
Machine Bus (VMBus) or the hypervisor.
x IC: Integration component s allow guest OSs to communication with the hypervisor and
management OS more effectively.
x Legacy Device: A virtualized device that mimics an actual physical hardware device so
that guests can use the typical drivers for that hardware device.
x Synthetic (High Speed) Device: A virtualized device not attached to physical hardware
so that guests need a driver (virtualization service client) to that high speed Bus. The
driver uses the VMBus to communicate with the Hypervisor.
x Physical Hard Disk: A physical disk in the physical machine can be used by virtual
machines directly.
x Enlightenment: An improvement to a guest OS to make it aware of VM environments
and increases overall performance.

Understanding Networking with Hyper-V 2

x Virtualization Stack: A collection of software components in the management OS that
work together to support VMs. The virtualization stack works with and sits above the
hypervisor. It also provides management capabilities. The following items are part of that
stack.
x VMBus: Is a channel-based communication method used for inter-guest communication
and device enumeration on systems with multiple active virtualized guests. Access to the
VMBus is installed with the Hyper-V Integration Components.
x VSP: Virtualization Service Provider Resides in the management OS and provide high
speed device support to guest OSs over the Virtual Machine Bus (VMBus).
x VID: The virtualization Infrastructure Driver provides guest management services, virtual
processor management services, and memory management services for virtual
machines..
x VMMS: The Virtual Machine Management Service is responsible for managing the state
of all VMs in guest OSs.
x VMWP: The Virtual Machine Worker Processes a user mode component of the
virtualization stack. The worker process provides virtual machine management services
from the Windows Server 2008 instance in the management OS to the guest OSs. The
Virtual Machine Management Service creates a separate worker process for each running
VM.
x VSC: Virtualization Service Client is a high speed device that resides in a guest OS.
VSCs utilize hardware resources that are provided by Virtualization Service Providers
(VSPs) in the management OS. They communicate with the corresponding VSPs in the
management OS over the VMBus.
x WinHv: The Windows Hypervisor Interface Library (WinHv) is a bridge between a
management OS systems drivers and the hypervisor which allows drivers to call the
hypervisor using standard Windows calling methods.
x WMI: The Virtual Machine Management Service exposes a set of Windows Management
Instrumentation (WMI)-based APIs for managing and controlling virtual machines.
x MSR: Memory Service Routine

x I/O stack: Input/output stack

Understanding Networking with Hyper-V 3

Virtual Networking
In the Hyper-V architecture, guest operating systems do not have direct access to hardware
resources. In Hyper-V the management OS controls the I/O (input / output) of all physical
devices. Guest OSs have high speed hardware devices installed on the VM, which are
presented to the guest OS in the same manner as if it was physical hardware. These high
speed devices run in a Virtualization Service Client (VSC). When hardware requests are
made within the guest OS, they are redirected by the VSC via a high-speed interconnect,
unique to each VM, known as a Virtual Machine Bus (VMBus), which then sends it to the VSP
(Virtualization Service Provider) in the management OS. The management OS then manages
the requests to the physical hardware. The correlating physical hardware response is then
redirected back to the guest OS in the same manner.
The diagram below illustrates the Hyper-V architecture on Windows Server 2008 R2.

Hyper-V Virtual Network Manager

The Virtual Network Manager in Hyper-V allows for the creation and control of virtual network
switches. There are three types of virtual networks in Hyper-V: external, internal, and private.
Each network type will be discussed in the following section.
There is no limit to the number of internal and private virtual networks that can be created
within Hyper-V. On the other hand, external virtual networks are bound to physical network
adapters, so it is only possible to create as many external virtual networks as there are
physical network adapters in the management OS.

Understanding Networking with Hyper-V 4

Virtual Network
In Hyper-V, virtual networks are logical segmentations which virtual network cards (NICs) of
virtual machines can be connected to. Think of each virtual network as an independent
network switch used to facilitate, or segment, communication between VMs, the
management OS, and/or the physical network.
The following sections will explain the types of virtual networks available within Hyper-V and
the differences between them.

Private Virtual Network

A private virtual network is a logical, completely isolated, network segment within Hyper-V and
not bound to any physical network adaptor. This means that only VM-to-VM network
communication occurs with connected to this private virtual network. When it is created, the
Hyper-V Networking Management Service running on the management OS creates a virtual
switch and allows for the connection of virtual machines to this switch. The management OS
does not participate in this private virtual network and, as such, cannot communicate with
virtual machines connected to it. Figure 1 below provides a visual representation of a private
virtual network.

Understanding Networking with Hyper-V 5

 Figure 1: Private Virtual Network

Internal Virtual Network

An internal virtual network is similar to private virtual network in that it is not bound to any
physical network adaptor. However, the primary difference with an internal virtual network is
that the management OS does participate, thus resulting in VM-to-Management OS and VM-
to-VM network communication.
Figure 2 below provides a visual representation of an internal virtual network.

Understanding Networking with Hyper-V 6

 Figure 2: Internal Virtual Network

External Virtual Network

Unlike a private or internal virtual network, an external virtual network is bound to the physical
NIC by way of miniports which may exist in the form of multiple miniports for a single physical
NIC, a single miniport representing multiple physical NICs, or a single miniport representing a
single physical NIC.
The Hyper-V Networking Management Service running on the management system creates:
x A virtual network
x A virtual network adapter on the Management OS that is connected to the virtual network.

The physical network adapter is then bound to the virtual switch and allows both virtual
machines and the management OS to connect to the physical (i.e. external) network. The
diagram below provides a visual representation of an external virtual network.

Understanding Networking with Hyper-V 7

 Figure 3: External Virtual Network

Summary
In conclusion, when selecting a virtual network switch, consider the following.
x Private: Virtual machines connected to this type of network can communicate among
themselves. The management OS has no network connectivity with the virtual machines.
x Internal: Virtual machines connected to this type of network can communicate among
themselves and the management OS. There is no connectivity with the physical network.
x External: An external virtual network binds to miniports which may exist in the form of
multiple miniports for a single physical NIC, a single miniport representing multiple
physical NICs, or a single miniport representing a single physical NIC, allowing both
virtual machines and the management OS to access the physical network.

Understanding Networking with Hyper-V 8

NOTES:
An existing network connection to the management OS is lost as soon as an external virtual network is
created that binds with the miniports created on the physical NIC of the management OS. After the
creation of the virtual switch and a virtual NIC, the management OS is connected to the physical
network through the virtual NIC and the virtual switch.
An external virtual network that is bound to a wireless adapter cannot be created.
The virtual network adapter that appears under Network Connections has the same name as the virtual
network switch with which it is associated.
It is possible to create an internal virtual network; this exposes a virtual network adapter to the
management OS without needing to associate a physical network adapter to it.
Hyper-V only binds the Microsoft Virtual Network Switch Protocol to a physical network adapter when a
virtual switch is associated with the physical network adapter in question.
Having at least two network adapters installed on the physical machine is recommended. In a Hyper-V
environment in which the management OS has only one physical network adapter and the virtual
machines are connected to an External Virtual Network, the management OS can still be accessed via
the new virtual network adapter created on the management OS using its static or dynamic IP address.
This is useful in a test or demonstration environment.

Virtual Network Adapters

Hyper-V supports two types of devices for virtual machines: high speed and legacy.
High-speed devices package requests made by virtual machine devices and forward them
over the VMBus, an in-memory pipeline that forwards the device requests to the VSP. This in
turn forwards the packet to the appropriate physical device. High-speed devices are included
in the integration service and are available for supported guest operating systems, including
Windows Server 2008, Windows Server 2003, Windows XP, Windows Vista, XEN-enabled
Linux, and Red Hat Linux.
A list of supported guest operating systems is available at:
http://www.microsoft.com/windowsserver2008/en/us/hyperv-supported-guest-os.aspx.
Legacy devices use software to emulate the device using additional processing power.

NOTES:
Each virtual machine can have a total of 12 virtual network adapters. Eight network adapters may be
high-speed adapters and four network adapters may be legacy adapters.

Understanding Networking with Hyper-V 9

Virtual Legacy Network Adapter

A legacy network adapter emulates a physical network adapter (multiport DEC 21140) and
hence works without installing a virtual machine driver because the driver is already available
on most operating systems. A legacy network adapter also supports network-based
installations because it includes the ability to boot to the Pre-Boot Execution Environment
(PXE).
Notes:
x The 64-bit edition of Windows Server 2003 and the Windows XP Professional x64 Edition
do not include a driver for the legacy network card.
x If the guest operating system does not support the installation of integration services, it
will require the use of a legacy network adapter.
x After using a legacy adapter to perform a network install with PXE, switching the
networking to a network adapter for performance reasons is recommended. If the guest
operating system supports integration services, switching the guest operating system to a
network adapter after the installation has been completed is recommended.

Virtual High Speed Network Adapter

This device is designed to work with virtualization and is optimized for that environment,
making its performance better than with legacy devices. To use a network adapter in a guest
operating system, integration services must be installed.

Understanding Networking with Hyper-V 10

NOTES:
Counters associated with both adapter types are available in the Performance Monitor MMC in the
management OS.
Hyper-V Virtual Network Adapter\Bytes Received/sec and Hyper-V Virtual Network Adapter\Bytes
Sent/sec are two useful counters for the high-speed network adapter.
Hyper-V Legacy Network Adapter\Bytes Received/sec and Hyper-V Legacy Network Adapter\Bytes
Sent/sec are for the legacy network adapter.
VM Switch and VSP in Hyper-V take measurements across virtual networks that are created and
bound to physical adapters. When looking at performance across these counters, be aware that multiple
VMs can share a virtual network. Therefore, counter results may be aggregated across multiple virtual
machines.
Each virtual machine can have a total of 12 virtual network adapters. Eight network adapters may be
high-speed adapters and four network adapters may be legacy adapters.
Each virtual network adapter can have a static or dynamic MAC address.

Virtual LAN Identification

Virtual LAN Identification specifies an identification number (VLAN ID) that can be used to
isolate network traffic from the operating system running on the management OS or other
guest operating systems sharing the same virtual switch. A physical network adapter must
support a virtual LAN configuration; no configuration on the physical network driver side
should be required. In particular, setting a VLAN ID in the physical adapter should not be
required.
A VLAN ID is a number that uniquely identifies a virtually segmented network. A network card
configured with that VLAN ID is identified as belonging to a particular VLAN.
The VLAN ID is encapsulated within the Ethernet frame, which is how multiple VMs using the
same physical NIC can communicate simultaneously on different VLANs.

Understanding Networking with Hyper-V 11

The image below illustrates setting the VLAN ID to the virtual NIC in the management OS in
Hyper-V.

Understanding Networking with Hyper-V 12

The image below illustrates setting the VLAN ID to a virtual network adapter in Hyper-V.

The diagram below illustrates using a single physical NIC in the host that is connected to an
802.1q trunk on the physical network carrying three VLANs (5/10/20). The design objectives
in this example are as follows.
x An 802.1q trunk carrying three VLANs (5/10/20) is connected to a physical adapter in the
host.
x A single virtual switch is created and bound to the physical adapter.
x The VLAN ID of the virtual NIC in the management OS is set to 5, allowing the virtual NIC
in the management OS to communicate on VLAN 5.
x The VLAN ID of the virtual NIC in Guest OS 1 is set to 10, allowing the virtual NIC to
communicate on VLAN 10.
x The VLAN ID of the virtual NIC in Guest OS 2 is set to 20, allowing the virtual NIC to
communicate on VLAN 20.

Understanding Networking with Hyper-V 13

NOTES:
A physical network adapter that supports VLAN tagging is required, and this feature needs to be
enabled.
Not setting the VLAN ID on the physical network adapter is recommended; it should be set on either
the virtual switch or the virtual machines.
The management OS VLAN ID is set in the virtual switch dialog.
The virtual machine uses the VLAN ID on the individual virtual network adapter. The virtual machine
isnt away of VLAN and this information is stripped in the switch.

SCVMM Preferred Location and Tag

SCVMM (System Center Virtual Machine Manager) uses the Preferred Location and
Preferred Tag on virtual network adapter in VMs, Templates, or Hardware Profiles, which are
used to place VMs on the desired Hyper-V host that matches a connection with the specified
networking requirements.
The preferred location is a VMHostNetworkAdapter property detected by the DHCP server
connected to the Active Directory tree and the property is left blank if cannot be autodetected.
The following list describes the characteristics of the Location property:
x VMs will inherit the location property when attached to a virtual network.
x VMs that are not attached to a virtual network, SCVMM will assign it to the Internal
Network.
x SCVMM allows the location to be replaced with any non-empty value if a change is
required or there is no autodetect location.
The following describes the characteristics of the Tag property:
x The tag property is a virtual network setting that helps differentiate between virtual
networks connected at the same location which offers more granular settings when
selecting the best virtual network.

Understanding Networking with Hyper-V 14

NOTES:
x When using static IP addresses for the hosts, the location is not detected as it is detected from the
DHCP response.
x When denying the host access to an External Virtual Network, the location cannot be detected since
there is no IP stack on the Host partition.

Understanding Networking with Hyper-V 15

Hyper-V Network Configuration Scenarios for Live
Migration
As customers are deploying Windows Server 2008 R2 Hyper-V in different environments
using different hardware configurations available it is important for Microsoft to provide
guidance on how to plan and configure network for optimal performance and security. This is
especially critical while deploying for Live Migration. This section outlines common customer
scenarios that need some guidance, current situation and recommendations for some
commonly found server configurations.

Customer Scenarios
The following is a list of possible scenarios that your organization might be facing.
1. New to Windows Server 2008 R2 Hyper-V and would like to evaluate how Live Migration
works and would like to use my existing hardware before investing in recommended
hardware configuration.
2. A Hyper-V infrastructure has been invested into the enterprise and needs to deploy
Windows Server 2008 R2 Hyper-V to take advantage of Live Migration and enable
maintenance host without downtime, dynamic server utilization, load balancing of server
and effective power management scenarios.
3. Planning on deploying Windows Server 2008 R2 Hyper-V in production environment and
would like to understand network configuration requirements before purchasing necessary
equipment for optimal performance.

Current Landscape and Challenges

Microsoft realizes that many customers understandably, try before deploying, using servers
which often come with limited network ports, as low as 2 NICs, or have the need to combine
multiple network interfaces for NIC teaming solutions. Most customers are still running in
1Gbps infrastructure and given current economic situation (as of Spring of 2009) many are
unlikely to go to 10gbps infrastructure soon. In most cases customers are likely to have 4 or
more NICs in the Hyper-V servers but there are the above mentioned scenarios where no
guidance we will risk the loss of functionality, causing failure or suboptimal performance from
the initial attempt.
As more customers are deploying mission critical workloads on a single Hyper-V server, there
are an increasing number of customers deploying Hyper-V in a failover scenario.
It is also conceivable that due to nature of different network traffic patterns used by different
parts of Hyper-V customers want to optimize using same network interfaces to minimize
bandwidth waste.

Different network access needs for Live Migration

Live Migration requires failover cluster and shared storage that provides the lowest latency
possible. Microsoft recommends using Cluster Shared Volume (CSV) technology on iSCSI or
SAN to provide shared access for optimal performance.
The table below provides different network access needs in a Hyper-V environment when
deployed in cluster:

Network Access Purpose Nature of Traffic Recommended

Type Pattern Access

Understanding Networking with Hyper-V 16

Storage Access to storage High bandwidth and Private
typically through iSCSI latency sensitive
or Fiber Channel (FC).
FC does not need a
network interface
VM access Workloads running Varies Public, can be teamed
inside VMs would for link aggregation
typically communicate purpose
with client/servers
external to
server/clients
Management Managing Hyper-V Low bandwidth Public, can be teamed
host, this is used by for fail-over purpose
Hyper-V Manager,
MOM, SCVMM etc.
CSV/Cluster This network is used to Normally low Private
send the heart beat of bandwidth, highly
the cluster to ensure sensitive to latency;
that health is some time needing high
maintained. CSV uses bandwidth
this network to send
meta-data between the
owner and non-owner
nodes, when the
storage path is
interrupted it will use
this network for
accessing storage or
during backup of CSV
volume.
Live Migration Live Migration process High bandwidth and Private
uses the network to highly latency sensitive
transfer memory pages
and VM state.

Live Migration Networking Scenario Recommendations

The following is a list of networking recommendations:
1. Configurations shouldnt violate recommendations of keeping Cluster, CSV and Live
Migration on private networks.
2. In scenarios where you are limited by number of NICs deviating from the best practice of
not sharing network adapters between VMs and the management OS, using a VLAN is
highly recommended to isolate traffic between VM access and the management OS.
3. Using VLANs in Hyper-V is recommended whenever possible if a network adapter is
shared between different network access needs.
4. Bandwidth (BW) throttling may need to be tuned to suit specific environment. BW
proposed below only provides control over outgoing traffic from a given physical network
interface, and does not guarantee end to end Quality of Service (QoS). To implement
QoS, you need control of network traffic between two endpoints and the entire network
infrastructure in between.
The table below is the expected scenario configurations:
Network Storage VM Management CSV/Cluster Live Notes
Access Access* Migration

Understanding Networking with Hyper-V 17

Configurati
on
2x1gb FC or vNIC1/publi vNIC1/public NIC2/private NIC2/privat Unsupported,
iSCSI: c e iSCSI
NIC2/pri requires
vate dedicated
NIC.
3 x1gb iSCSI vNIC1/publi vNIC1/public, NIC2 /private NIC2 3 x1gb
c 10% Max BW /private
3x1gb FC vNIC1/publi vNIC1/public, NIC2/private NIC3/privat 3x1gb
c 10% Max BW e
4x1gb FC vNIC1/publi NIC2/public NIC3/private NIC4/privat Recommend
c e ed
5x1gb iSCSI:NI vNIC2/publi NIC3/public NIC4/private NIC5/privat Recommend
C1/privat c e ed
e
4x1gb iSCSI: vNIC2/publi vNIC2/public, NIC3/private NIC4/privat 4x1gb
NIC1/pri c 10% Max BW e
vate
2x10gb FC vNIC1/publi vNIC1/public, NIC2/private NIC2/privat
c 1% Max BW e, 20%
Max BW
2x10gb iSCSI: vNIC2/publi vNIC2/public, NIC1/private NIC1/privat
NIC1/pri c 1% Max BW e, 20%
vate, Max BW
30%
Max BW
2x10gb, FC vNIC1- NIC2- NIC3- NIC3- Recommend
1x1gb 10g/public 1g/public, 10g/private 10g/private ed
, 20% Max
BW
2x10gb, iSCSI: vNIC2- NIC3- NIC1/private NIC1/privat Recommend
1x1gb NIC1- 10g/public 1g/public e, 20% ed
10g/priva Max BW
te/30%
Max BW
*VM access needs will vary depending upon Network IO needs and VM density
The following is a list is the Bandwidth throttling configuration recommendations:
1. iSCSI throttling using Windows QoS on pacer/NIC on a TBD destination port iSCSI uses.
2. LM throttling using Windows QoS on pacer/NIC on a TCP port 6600 from the host.
3. Management throttling using Windows QoS on pacer/NIC from a given IP address on
which Management OS is bound to the vNIC.

Understanding Networking with Hyper-V 18

Windows Server 2008 R2 Hyper-V MAC Address
Management
In Windows Server 2008 R2 Hyper-V VMs can be created with a static or dynamic MAC
address. Dynamic addresses are automatically assigned from the MAC address pool that is
created in the management OS when you install Hyper-V
The Virtual Network Manager in Windows Server 2008 R2 contains a new feature that allows
the manual configuration for the range of the MAC addresses that Hyper-V will issue when
VMs are assigned to dynamic MAC address generation. The MAC address range allows for
an increase in the ports available for VMs and ensures that other Hyper-V servers are not
duplicating the same address range.

When a VM is initial started for the first time Hyper-V will assign the next available MAC
address from the pool of MAC addresses given to Hyper-V. All virtual network adapters and
virtual network switches will receive a MAC address from Hyper-V. By default there are only
256 MAC addresses available under the dynamic address pool.
Once a dynamic MAC address has been assigned the VM will continue to use this MAC
address unless the VM is deleted.

Understanding Networking with Hyper-V 19

When all 256 MAC addresses in the pool have been issued Hyper-V will look through the
MAC address pool and search for an available address by starting at the first address in the
pool. If the MAC address is still being used then Hyper-V will move to the next address. If the
search for a MAC address goes through a complete pass without finding an available
address, Hyper-V will display an error that there are no available MAC addresses.

Hyper-V Dynamic MAC Address

Physical network adapter as assigned MAC Address when the vendor produces it from the
pool that has been assigned to them, When using dynamic MAC address assignment in
Hyper-V it is possible to have duplicate MAC address as a result, you will receive errors with
in communication on the network. This section covers how Hyper-V allocates dynamic MAC
addresses and potential issues you might encounter.
Hyper-V has an algorithm to deal with duplicate MAC addresses on a Hyper-V server host,
but not in environment where multiple Hyper-V servers have been deployed.
When Hyper-V is installed it creates two registry keys called MinimumMacAddress and
MaximumMaxAddress and by default offers a range of 256 MAC Addresses. While this
amount is potentially enough on a single server it is very possible that in environments with
large scale Hyper-V servers could have conflicting MAC Address Pools.
All MAC address used by Hyper-V will always begin with 00-15-5D when using dynamic
assignment. The last three hex values in the MAC Address are created by using the last two
octets of the IP address on the network adapter is that resolved first in the management OS
and is then converted to hex.
For example if the first network adapter to be resolved is assigned the IP address of
10.10.1.100 then 10.100 will be converted to hex, which in this case equals 01-64.
MinimumMacAddress is set to 00 and MaximumMacAddress is set to FF. This would give the
Hyper-V management OS a MAC address range of 00-15-5D-01-64-00 through 00-15-5D-01-
64-FF.

Hyper-V MAC Address Pool Duplication

The MAC address pool is produced when the Hyper-V role is installed. If you install the
Hyper-V roles and then create an image of the Hyper-V server you are also creating a
duplicate MAC address pool. Sysprep will not reset the registry values. To resolve the
potential issue described above it is recommended that one of the follow solutions be used.
If you plan on creating an image of your Hyper-V servers:
1. Dont install the Hyper-V role, Sysprep the machine and then install the Hyper-V roles as
part of the mini-setup. This option will create unique values for the MAC address pool
each time the Hyper-V role is installed on a new image. For more information please see
the Automated Installation Kit (AIK) for Windows Vista SP1 and Windows Server 2008.
2. After installing the Hyper-V roles and before you Sysprep the Hyper-V server, delete the
two keys created by Hyper-V under HKLM\Software\Microsoft\Windows
NT\CurrentVersion\Virtualization. This option will cause Hyper-V to generate new unique
keys when the VMMS service is started,
a) MinimumMacAddress
b) MaximumMacAddress
Using option one will force unique values for the MAC address pool as the Hyper-V role is
installed. Using option two will force the keys to be recreated when the VMMS service starts
during boot.

Understanding Networking with Hyper-V 20

Detecting MAC Address Duplications
When deploying VMs from multiple host, if you experience unpredictable behavior when
pinging between VMs it is possible to have duplicate MAC addresses.
In general duplicate MAC addresses can result in host unreachable, request timeouts, and /
or errors with domain login or SMB network access. It is also possible to get mixed results
such as, pings not returning and then suddenly starting to work.
There are two ways to check the MAC address of a VM in Hyper-V:
1. In Hyper-V select the VM and then in the Actions pane click Settings. In the Settings
dialog click the network adapter in question and then review the MAC address.
2. In the VM, from and administrative command prompt, type IPCONFIG /ALL and then
review the MAC address.

Notes:
For more information to automate this process using a script or PowerShell please see the WMI
documentation for msvm_virtualswitchmanagementservice and CreateInternalEthernetPort method at
http://msdn.microsoft.com/en-us/library/cc136938(VS.85).aspx.

Adjusting the MAC Address Pool

In Windows Server 2008 R2 Hyper-V you can adjust the MAC address pool on the server by
click the Virtual Network Manager in the Actions pane of the Hyper-V MMC. It is important to
note that changing the address pool does not resolve any current duplication issues, this
change should happen in the deployment stage before creating any VMs in Hyper-V.
Based on the example dynamic MAC address pool given earlier if we changed the third octet
of MinumumMacAddress to 00-15-5D-01-60*-00 and MaximumMacAddress to 00-15-5D-01-
6F*-FF, this would give you a total of 4096 MAC addresses.
* In the original example both of these values were set to 4, effectively disabling that
bit.
If VMs have already been created before the MAC address pool has been changed there are
two ways to change the dynamic MAC address on a VM.
1. Change the VM to a static MAC address and assign a static MAC address to the
virtual network adapter.
2. Remove and then add a new virtual network adapter in the VM, which will assign a
new dynamic MAC address.

Understanding Networking with Hyper-V 21

Virtual Network Setup
If additional network intensive operations such as iSCSI are required, consider additional
network adapters. One network adapter will be used for managing the management OS, and
one or more network adapters should be purposed for virtual machine networking. Virtual
machines should be exposed to the production network, and the dedicated management
network adapter should be exposed to the backend management network.
This section walks through the networking experience on Windows Server 2008 Hyper-V and
highlights various changes when selecting the network types discussed above.

New Virtual Network

This section provides guidance on the initial experience of setting up Hyper-V and assumes
that Windows Server 2008 x64 is installed on a physical machine capable of running Hyper-V.
The properties of a physical network connection on Windows Server 2008 with Hyper-V not
installed appear similar to those on the image below.

Virtual Network

When installing the Hyper-V role on Windows Server 2008, a virtual network for each physical
adapter installed on the host can be created. For this white paper, this remains unchecked,
and the installation of Hyper-V is finished. After Hyper-V has been installed, a more in-depth
look at networking will be provided.

Understanding Networking with Hyper-V 22

After installing the Hyper-V role, the Local Area Connection now has the Microsoft Virtual
Network Switch Protocol installed. Do not configure or check this setting. Hyper-V performs
the required actions when installing an external virtual network. This can be seen in the image
below.

Understanding Networking with Hyper-V 23

Now that Hyper-V has been installed, a virtual network will be configured. On the Start Menu,
click Administrative Tools and then click Hyper-V Manager. In the Actions pane of
Hyper-V Manager, click Virtual Network Manager. In the Virtual Network Manager window,
select Internal and then click Add.

Understanding Networking with Hyper-V 24

Giving the virtual network a friendly name allows the administrator to easily identify the
network when assigning adapters to virtual machines. This name is also bound to the virtual
network adapter created on the management OS in Network Connections.

Understanding Networking with Hyper-V 25

Following the same steps as above, an external virtual network is now added. The external
network virtual switch must be bound to a physical adapter on the host. Only one virtual
switch may be bound per physical adapter.

Understanding Networking with Hyper-V 26

When applying changes for an external network switch, network connectivity may be lost
temporarily; this will disrupt network operations in progress. If the physical network adapter
has static settings, these settings will be overwritten and will need to be reapplied to the newly
created virtual NIC in the management OS. This is important when working on the
configuration via Remote Desktop.
It is important to note that these settings should now be reapplied to the new virtual network
named External in this case.
The virtual network switch acts like a physical layer 2 switch. A physical switch is limited
because more ports cannot simply be added to the switch. Each time a virtual machine is
created with a virtual adapter connected to the virtual network switch, a port is effectively
added to this virtual switch.
Three network adapters now exist: the physical adapter now attached to the external network,
the newly created virtual adapter named external, and the internal adapter.

Understanding Networking with Hyper-V 27

The image below illustrates that the status of Local Area Connection 4 is identified with the
status of an unidentified network. This means that the network adapter has not received an IP
address from DHCP or has not been configured with static IP address information.
All of the bindings are now disabled on the physical adapter except for the Microsoft Virtual
Network Switch Protocol. Hyper-V has enabled these bindings to the new external virtual
adapter.
The TCP/IP protocol from the management OS physical network adapter is only bound to the
virtual network adapter on the host when a virtual external network switch is created in Hyper-
V. This will not happen when a virtual internal virtual network switch is created. When a virtual
internal virtual network switch is created, a virtual network adapter is still created on the
management OS but that adapter is not bound to any physical adapter and, hence, both
adapters have their own TCP/IP protocols. The internal virtual network adapter cannot
communicate with the physical network adapter since internal virtual network switches are
contained only to the physical machine and the virtual machines. When a private virtual
network switch is created in Hyper-V, no virtual network adapter is created on the
management OS.
The image below illustrates how the protocols are automatically configured on the physical
network adapter and an external virtual network switch.

Understanding Networking with Hyper-V 28

The image below illustrates how the protocols are automatically configured on the physical
network adapter and an internal virtual network switch.

NOTES:
If two internal (or private) virtual networks are created in Hyper-V and two virtual machines are created
on separate IP subnets, they cannot communicate with each other. The virtual switch operates at layer
two of the ISO/OSI Network Model. To achieve routing at higher levels, a router must be used, the same
as would occur in a physical environment. ISA 2006 or RRAS can also be used to achieve this
functionality.

Setup Details
When installing the Hyper-V roles on Windows Server 2008, the option exists to select which
physical network adapter on the management OS will be used by Hyper-V as virtual external
network switches. This option is not available when installing Hyper-V on Server Core, and
needs to be configured via the Virtual Network Manager. It is recommended that the physical
machine have a minimum of two physical network adapters, one for management and the
second dedicated to virtual machines.

Understanding Networking with Hyper-V 29

Dedicated Virtual Network for Each Virtual Machine

In this scenario, the physical server has four network adapters and storage is not iSCSI; it
may be directly attached, SAS, or Fiber Channel. The first physical network adapter is left
unchecked and assigned to the management OS for management. The remaining three
physical network adapters are checked and assigned to virtual switches for virtual machine
networking.

Understanding Networking with Hyper-V 30

The image below illustrates the architecture for this configuration.

Understanding Networking with Hyper-V 31

Networking with iSCSI Shared Storage

In this scenario, the physical server has four network adapters and iSCSI storage. The first
physical network adapter is left unchecked and is assigned to the management OS for
management. The second network adapter also remains unchecked for iSCSI. The remaining
two physical network adapters are checked, assigning them to virtual switches for virtual
machine networking. This allows the physical server to maintain the relationship between
itself and the iSCSI target when used as storage for virtual machines.
It is a recommended best practice that the network adapter dedicated to iSCSI is on either
another network or a subnet than the network adapter for the management network adapter.

Understanding Networking with Hyper-V 32

The image below illustrates the architecture for this configuration.

Physical Machine Network Communication with External Physical Machine and Virtual
Network

Assume that the Ping command is run from the physical machine running Hyper-V. Ping
sends an IP packet to its destinationin this case the other physical machineand waits for a
response. The following are the detailed steps that this packet takes in the above illustration.
x Ping uses the Windows networking stack to determine where the IP protocol is bound.
The only choice in this scenario is the virtual NIC.
x The IP packet is then sent down to the networking stack bound to the virtual NIC.
x The virtual NIC acts like a physical NIC and places the packet on the virtual wire to the
virtual networking switch.
x The packet reaches the virtual switch port.
x The virtual network switch does what a physical switch would do and routes the packet to
its destination, in this case the external virtual port on the virtual switch.
x This switch knows about the Microsoft Virtual Network Switch Protocol and places the
packet onto the physical NIC.
x The packet is then placed onto the physical network with its destination as the other
server.
x Once the packet reaches the destination, it follows the same path in reverse.

A second adapter in the management OS can cause connectivity issues, such as:
x Multiple DNS entries, delayed or incomplete NetBIOS resolution, and routing confusion;

Understanding Networking with Hyper-V 33

x Delayed NetBIOS resolution or incomplete network browsing because of multiple
management OS network adapters (multiple browsing lists associated with each network
adapter).

Virtual Machine Network Communication on a Private Virtual Network

In this scenario, two virtual machines (Guest OSs) are shown on the host and they
communicate with each other via a private virtual network. The image below illustrates the
virtual network topology.
On the management OS, all of the TCP/IP bindings are still enabled because an internal or
external virtual network has not yet been created.

The NYC-DC-01 virtual machine has been configured with the IP address of 192.168.1.1 and
the management OS has been configured with the IP address of 192.168.1.7. The image
below illustrates that the NYC-DC-02 virtual machine can communicate successfully with
NYC-DC-01 but has no access to the management OS or the public network.

Understanding Networking with Hyper-V 34

Virtual Machine Network Communication with Management OS on the Internal Virtual
Network

Since an internal virtual network on Hyper-V only allows network communication with other
virtual machines and the management OS, it is automatically routed to the virtual network
created on the management OS.
The virtual network adapter of the virtual machine is connected to the internal virtual network
switch and its IP address is set to 192.168.1.2. The management OS internal virtual network
adapter is configured with the IP address of 192.168.1.7. This can be seen in the illustration
below.
Local Area Connection 3 is the virtual internal adapter on the management OS.

Understanding Networking with Hyper-V 35

The virtual machine can now communicate successfully with the management OS.

The physical network adapter on the management OS cannot be communicated with.

Understanding Networking with Hyper-V 36

To enable Guest OS network access to the management OS, configure exceptions in
Windows Firewall used by the virtual machines. Click Start and then click Control Panel. In
the Control Panel, double-click Windows Firewall. In the Windows Firewall window, click
Turn Windows Firewall on or off. In the Windows Firewall settings dialog, click Advanced.

Understanding Networking with Hyper-V 37

The disabled firewall was on the internal virtual network connection. The virtual machines are
being routed to the network using the wireless network adapter, which is still protected by
Windows Firewall.

Guest OS Network Communication with Management OS on External Network Switch

In Hyper-V, the default network behavior for a virtual machine to communicate with the
management OS is as follows.
x The packet leaves from the virtual network adapter on the Guest OS and goes to the
external virtual network switch.
x Once the packet has left the external virtual network switch, it goes to the physical
network adapter on the management OS.
x Once the packet has left the physical network adapter, it goes to the physical network
switch.
x The packet returns to the physical network adapter on the management OS, in this case
the second physical network adapter.

After the networking algorithm in the TCP/IP stack has learned the least cost path to the
management OS, it behaves as follows.
x The packet leaves the Guest OS and goes to the external virtual network switch.
x From the external virtual network switch, the packet is received at the virtual network
adapter on the management OS.

Understanding Networking with Hyper-V 38

Dedicated External Virtual Network

The dedicated virtual network is a modified form of the external virtual network offered by
Hyper-V. This type of virtual network allows VMs to communicate with other VMs on the same
machine, as well as with VMs on other systems. They can also access the external network,
although these VMs do not have direct access to the management OS as in the external
virtual network configuration. Removing this direct path eliminates many of the drawbacks of
the external virtual network type discussed above.
The VMs still have access to the management OS through the external network if the
management OS virtual network adapter is connected to the virtual switch, or if the
management OS has another network adapter not dedicated to Hyper-V. Unlike the other
three virtual network types discussed above, dedicated virtual networks are not directly
configurable with Hyper-V Virtual Network Manager. The dedicated virtual network type
discussed here is created by first creating an external virtual network and then modifying the
virtual network adapter added to the management OS.
WMI can be used to implement a dedicated virtual network without causing an additional
virtual network adapter to appear in the management OS. Please see the following link for
more information about the Virtualization WMI Provider.

Start by ensuring that an external network has been created in the Virtual Network Manager.

Understanding Networking with Hyper-V 39

Ensure that the virtual machine is connected to the external network adapter and has
connectivity.

Understanding Networking with Hyper-V 40

On the management OS, disable the virtual network adapter. The Guest OS can still
communicate with the external network via the physical network adapter.

Connectivity to the management OS has been lost; another network adapter for management
has to be installed to restore this functionality.

Understanding Networking with Hyper-V 41

NOTES:
If the physical network adapter is disabled via a remote desktop session, remote connectivity will be
permanently lost unless a second adapter is installed.

Understanding Networking with Hyper-V 42

Hyper-V Networking Best Practices
This section contains a list of best practices for Hyper-V networking.

Best Practices
x Have at least two physical NICs. If additional services are required, add additional
physical network adapters as needed.
x If only communication between virtual machines is needed and not with the physical
machine or the external network, only create a private virtual network.
x If only communication between virtual machines and the physical machine is needed,
create an internal virtual network.
x If the virtual machines need to communicate with the entire network or the Internet, create
an external network.
x If separate communication is needed between the virtual machines and the physical
server machines while maintaining communication with an external network, use an
external network without a virtual network adapter in the management OS.
x If two internal or private virtual networks are created in Hyper-V and two virtual machines
are created on a separate IP subnet, they cannot communicate with each other. The
virtual switch operates at layer 2 of the ISO/OSI Network Model. To achieve routing at
higher levels, a router needs to be used, , the same as would be done in a physical
environment. ISA 2006 or RRAS may be used to achieve this functionality.
x When using an internal virtual network, create an exception to enable the virtual machines
to communicate with the physical server.
x When using virtual machines to communicate with the management OS, ensure that they
are on the same IP subnet.
x Each virtual machine can have a total of 12 virtual network adapters. Eight network
adapters can be assigned to a high-speed adapter and four network adapters can be
assigned to a legacy adapter.
x If the virtual machine experiences high traffic volume, it is recommended that a dedicated
physical network adapter be assigned to the virtual machine external network switch.
x Whenever possible, use high-speed devices in the virtual machines by enabling the
integration services.

Hyper-V in the DMZ

When considering using Hyper-V for server consolidation in a DMZ it is recommended not to
run VMs of vastly differing trust levels on the same physical host in production environments
(i.e. do not consolidate all DMZ boxes on one physical host). Instead, the recommendation is
to consolidate all the front-end boxes on one physical server and do the same for the back-
end, depending on the workloads.

Understanding Networking with Hyper-V 43

 Updates in Windows Server 2008 R2
Hyper-V in Windows Server 2008 R2 leverages several new networking technologies to
improve overall VM networking performance.
TCP Offload allows a VM to pass its network processing load onto the NIC of the host
computer. This works the same as in a physical TCP Offload scenario. Now, Hyper-V simply
extends this functionality into the virtual world, benefitting both CPU and overall network
throughput performance, and it is fully supported by Live Migration. Please view the Windows
Server 2008 R2 Reviewers Guide and the Windows Server Enterprise Networking
Offloading Technologies Presentation for a detailed overview on this subject.
Virtual Machine Queues (VMQ) is a technology that enables the NIC to deliver packets
directly to targeted virtual machines. This improves overall network performance and reduces
the cycles per byte costs for packet processing.
The VMQ interface supports:
x Classification of received packets in NIC hardware by using the destination MAC address
to route the packets to different receive queues.
x NIC ability to use DMA to transfer packets directly to a virtual machine child partitions
shared memory. For more information about shared memory, see NDIS 6.20 Memory
Management Interface.
x Scaling to multiple processors by processing packets for different virtual machines on
different processors.
Like TCP Offloading, Windows Server 2008 also introduces support for jumbo frames. Hyper-
V in Windows Server 2008 R2 extends this capability to VMs. Therefore, as in physical
network scenarios, jumbo frames add the same basic performance enhancements to virtual
networking. These enhancements include up to six times larger payloads per packet, which
improves overall throughput and reduces CPU utilization for large file transfers.
For more detailed information on Windows Server 2008 R2 please visit the following links:
x Windows Server 2008 R2 Reviewers Guide
x Top 10 IT Pro Tasks Made Easier by Windows Server 2008 R2
x Top 10 Reasons to Upgrade to Windows Server 2008 R2
x Windows Server 2008 R2 TDM Whitepaper
x Direct Access Technical Overview
x Direct Access Executive Overview
x Windows 7 Networking Executive Overview
x Windows 7 Networking Enhancements for Enterprises
x BranchCache Executive Overview
x Windows Server 2008 R2 Overview
x Windows Server 2008 R2 Active Directory Updates
x Building an Efficient Branch Infrastructure Using Windows Server
x Developing Applications for More Than 64 Logical Processors in Windows Server 2008 R2
x Enabling Test Automation Using Windows Server 2008 Hyper-V
x Extending Terminal Services and Hyper-V VDI in Windows Server 2008 R2
x Microsoft IIS 7.0 and Beyond

Understanding Networking with Hyper-V 44

x Optimizing Application for Remote File Access Over WAN
x Presentation Virtualization Graphics Remoting (RDP) Today and Tomorrow
x Windows Server 2008 File and Storage Solutions
x Windows Server 2008 R2 Streamlined Management
x Windows Vista PKI Enhancement in Windows 7 and Windows Server 2008 R2

Understanding Networking with Hyper-V 45

Conclusion
Hyper-V in Windows Server 2008 allows for large-scale and flexible network deployments, all
of which can be accomplished in a virtual manner, reducing the TOC of networking hardware
that might otherwise have to be purchased.
To properly leverage many of these features, it is often necessary to perform a thorough
analysis of the current physical network and gain a significant understanding of the Hyper-V
architecture. To learn more about the Hyper-V architecture and features mentioned in this
article, please visit http://technet.microsoft.com/en-us/library/cc753637.aspx.

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the
date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment
on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.
This white paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS
DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this
document may be reproduced, stored in, or introduced into a retrieval system, or transmitted in any form or by any means (electronic,
mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft
Corporation.
Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in
this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not
give you any license to these patents, trademarks, copyrights, or other intellectual property.
2007 Microsoft Corporation. All rights reserved.
The example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted
herein are fictitious. No association with any real company, organization, product, domain name, e-mail address, logo, person, place,
or event is intended or should be inferred.
All other trademarks are property of their respective owners.

Understanding Networking with Hyper-V 46