Creating a Windows 2003 Load Balanced Cluster using VMWare Server

(used to be GSX but is now the ree version!

- Summary:
We wanted to utilize the RAM capabilities of Windows 2003 !" #the !"-bit
$ersion of Windows% to allow for many user sessions in a &erminal Ser$er
en$ironment' (nfortunately) the $endor did not yet support a !"-bit *S' We had
already purchased 3 ser$ers so we decided to run two +! #32-bit% Windows 2003
&erminal Ser$er ,M-s per machine' We then o.ered to load balance these $irtual
machines across separate hosts so that the end user would always /et the most out
of each machine' &hese systems needed to be as hi/hly a$ailable as possible and
we thou/ht we co$ered our trac0s pretty well' 1 thou/ht 1 would document our
process to help others since 1 had 2uite a bit of trouble /ettin/ e$erythin/ to run in
the 3rst place'
"#$ M%S" &M'%(")*" step we learned 4 since we ha$e a 51S5* networ0)
there M(S& be a static AR6 entry for the 5lustered 16-s MA5 address on the 51S5*
switch' &he cluster must run in a 7Multicast7 mode and the way 51S5* implements
multicastin/ re2uires this static entry or else pac0ets to and from the cluster will be
i/nored when tra$ersin/ a 51S5* /ateway' &he AR6 entry only needs to be added
to the /ateway of the subnet where the clustered nodes reside 4 so if there is a
redundant /ateway the 51S5* /ateway) then both switches need the AR6 entry'
&his may be true for other networ0 types as well) but is de3nitely true for 51S5*
networ0s'
82uipment:
- 3 9ell 2:;0-s
- 8ach ser$er has " <15s #2 *n-board =roadcom <15s and 2 sin/le port 1ntel <15s
all purchased from 9ell%
- 8ach Ser$er has >!?= of RAM
- 8ach ser$er has ">"!?= 2'; inch SAS dri$es
- @ost setup:
8ach ser$er was setup the sameA 1 chan/ed the names to protect the
innocent :-%' &he systems are loaded with Windows 2003 !" Standard on each
ser$er' &hese systems all had local stora/e so we simply made a RA19; $irtual dis0
out of 3 dri$es with the "
th
bein/ a hot spare' &hen made a 5: dri$e for the *S
#about >2?= in size% and the rest of the space as a bi/ 9: dri$e where the $irtual
machines will sit' We also put a 2"?= pa/e 3le on the 9: dri$e' We loaded ,MWare
Ser$er >'0'"-;!;2+ #thou/h 1-m sure the current $ersions wor0 similarly% on the 9:
dri$e and pointed our default /uest directory to a folder called 9:B,irtual Machines'
*nce the *S was loaded) we setup the networ0 cards' @ere is where some
special considerations are needed' &he ,MWare documentation says that each
/uest will need a li$e <15 and a <15 dedicated to the load balancin/' &he
instructions also say that the load balanced <15 should be mapped to a separate
<15 #either host only or eternal%' With those points in mind) we teamed the 2 on
board <15s and treated each of the 1ntel <15s as a separate attached <15' 8ach of
the =roadcom <15s was then plu//ed into one of 2 ed/e switches in the rac0 #for a
primary and secondary connection% and each 1ntel <15 was plu//ed into primary or
secondary switch as well #one 1ntel <15 to each%'
&he 16 scheme for all machines must be on the SAM8 S(=<8& for all this to
wor0' &hat will mean lots of 16s in this situation' 1 /i$e eamples below with labels
and then 1 will describe what we did:
- >0'0'0';0 C Dull 16 for ,M@*S&0>
- >0'0'0';> C >
st
support 16 for ,M@*S&0>
- >0'0'0';2 C 2
nd
support 16 for ,M@*S&0>
- >0'0'0';3 C Dull 16 for ,M@*S&02
- >0'0'0';" C >
st
support 16 for ,M@*S&02
- >0'0'0';; C 2
nd
support 16 for ,M@*S&02
- >0'0'0';! C Dull 16 for ,M@*S&03
- >0'0'0';E C >
st
support 16 for ,M@*S&03
- >0'0'0';+ C 2
nd
support 16 for ,M@*S&03
- >0'0'0'!0 C ,irtual System Dull 16 for ,M?(8S& 0> #on ,M@*S&0>% actual
networ0 <15
- >0'0'0'!> C Foad =alanced <15 16 for ,M?(8S&0> #on ,M@*S&0>%
- >0'0'0'!2 C ,irtual System Dull 16 for ,M?(8S& 02 #on ,M@*S&0>% actual
networ0 <15
- >0'0'0'!3 C Foad =alanced <15 16 for ,M?(8S&02 #on ,M@*S&0>%
- >0'0'0'!" C ,irtual System Dull 16 for ,M?(8S& 03 #on ,M@*S&02% actual
networ0 <15
- >0'0'0'!; C Foad =alanced <15 16 for ,M?(8S&03 #on ,M@*S&02%
- >0'0'0'!! C ,irtual System Dull 16 for ,M?(8S& 0" #on ,M@*S&02% actual
networ0 <15
- >0'0'0'!E C Foad =alanced <15 16 for ,M?(8S&0" #on ,M@*S&02%
- >0'0'0'!+ C ,irtual System Dull 16 for ,M?(8S& 0; #on ,M@*S&03% actual
networ0 <15
- >0'0'0'!: C Foad =alanced <15 16 for ,M?(8S&0; #on ,M@*S&03%
- >0'0'0'E0 C ,irtual System Dull 16 for ,M?(8S& 0! #on ,M@*S&03% actual
networ0 <15
- >0'0'0'E> C Foad =alanced <15 16 for ,M?(8S&0! #on ,M@*S&03%
- >0'0'0'E; C Shared #load balanced% 16 address for cluster
- ourclustername'ourdomainname'netCour uni2ue 9<S name for our cluster
As you can see there are a lot of repeats) so the same thin/ usually occurs on
a per machine basis' *n the 3rst machine #,M@*S&0>%) we assi/ned the full 16 of
>0'0'0';0 to the team created from the 2 on-board =roadcom <15s' &his team was
/i$en the /ateway information) had the subnet mas0 info) had 9<S ser$er entries)
was set to re/ister this 16 with 9<S) and e$en had the W1<S entries' We net
assi/ned the >
st
support 16 >0'0'0';> 16 to the 3rst 1ntel <15' We did <*& add a
/ateway) only the 16 and subnet mas0' &his <15 also had <* 9<S entries) did <*&
re/ister with 9<S) had <* W1<S entries) and we disabled <et=1*S o$er &56G16 on
the W1<S tab as wellA this secondary information is only needed by the 3rst) li$e 16'
&he second 1ntel <15 was con3/ured the similarly to the 3rst 1ntel <15 with the 2
nd

support 16 of >0'0'0';2 and its subnet mas0) but all other settin/s were disabled Hust
li0e the 3rst 1ntel <15'
We also had to ma0e some adHustments to the 7Ad$anced Settin/s7 #in the
networ0 connections folder under the 7Ad$anced7 menu% on the 7Adapters and
=indin/s7 tab' We made sure that the 7&eamed7 $irtual adapter was listed at the
top in the 75onnections7 order) followed by each of the team members) and then
3nally the 1ntel <15s at the bottom' Iust an DJ1 4 we do bac0ups of our ,MWare
machines by bac0in/ up open 3les on the host each ni/htA if there is e$er a
problem) we simply restore the whole $irtual machine itself' &hese chan/es help
the bac0up see the host correctly' =ac0in/ up the host only also sa$es us on bac0up
licenses since our bac0up $endor counts a $irtual machine and a full client if we
load an a/ent on it' We repeated the *S and 16 settin/s across the other 2 host
machines #,M@*S&02 and ,M@*S&03%'
- ,MWare Software setup:
<et in line is to setup the ,MWare software and settin/s on $irtual machines'
Dirst we made sure the ,MWare networ0 settin/s will utilize what we want to do as
far as routin/ $irtual <15 information out of speci3c <15s' Dor what 1-m callin/ the
7,irtual System full 167) 1 want to ma0e sure that on each /uest system) this 16 is
mapped to the created teamed <15 on the host' 1n ,MWare) on the menus) we went
to 7@ost7) 7 ,irtual <etwor0 Settin/sK7' &hen on the 7@ost ,irtual <etwor0
Mappin/7 tab) we chan/ed the ,Mnet0 to map to the teamed adapter #for the
=roadcom-s it was called the 7=AS6 ,irtual Adapter7%' We 0ept the other defaults
but we set ,Mnet2 to the >
st
lintel <15 and ,Mnet3 to the 2
nd
1ntel <15' <ow we can
start buildin/ the /uest machines'
- ?uest Machine setup:
We built a base ,M and we set the followin/ parameters:
- Memory to ma #3'!?=%
- >20?= hard dri$e #set to /row as needed%
- *nly > <15 for now #after clonin/ we will add a second $irtual <15%A settin/s left
default
- 2 processors
- Set for 2003 Standard
- *ther parameters set after clonin/
<et we loaded Windows 2003 +! #32bit% standard as ,M?(8S&0> and
loaded all patches) $irus scannin/) etc' We then shut it down and copied it ! times
#now we can sa$e the base ,M if there are any problems and use it to build more
later%'
&o create ,M?(8S&0> and each subse2uent ,M) we will needed some etra
twea0in/ before each was added to the domain) terminal ser$ices was loaded) and
networ0 load balancin/ was loaded' We need to ma0e this system uni2ue and not
Hust a copy' We could run sysprep) but that is tedious because you ha$e to re-run all
initial setup parameters' 1nstead we ran the 7<ewS197 utility from
www'systeminternals'com #bou/ht by Microsoft''' the direct lin0 to the utility is
http:GGtechnet'microsoft'comGen-usGsysinternalsGbb+:E">+'asp% 4 we /a$e it a
random S19 and a new name on each system and let it reboot' <et we had to ta0e
care of the <etwor0 adapters
When the system came bac0 on) we went into de$ice mana/er and
uninstalled the $irtual ,M <15' (nfortunately) the 7<ewS197 utility does not assi/n
uni2ue ?(19s to the <15-s so if we did not ta0e this step) e$ery cloned machine
would appear to be usin/ the same <15 which will cause problems later when
settin/ up networ0 load balancin/ in windows' After uninstallin/ the $irtual <15 in
de$ice mana/er) we shut down the ,M'
<et) we went into the properties of each ,M and remo$ed the eistin/
$irtual 8thernet adapter 4 it has to be remo$ed and re-added so the /uest will thin0
that 7new hardware7 has been added' &hen) we added one bac0) ma0in/ sure that
its networ0 connection is 7custom7 and used ,Mnet0 #remember ,Mnet0 should be
mapped to the teamed <15 of the host%' After) we added a second $irtual 8thernet
adapter to the $irtual system chan/in/ the 7<etwor0 5onnection7 to use ,Mnet2
#that means traLc from this second adapter will be brid/ed to /o out of the 3rst
1ntel <15 that was setup earlier%' &here were 2 /uest machines on this host so we
needed to do somethin/ similar with the second machine' When the second /uest
was shutdown) remo$e and re-added the 3rst $irtual 8thernet adapterA this 3rst
adapter needed its networ0 connection customized to point to the ,Mnet0 adapter'
&he second /uest also needed a second adapterA howe$er) this second adapter
needed to be customized to point to ,Mnet3 #the traLc from this adapter would
now be brid/ed to /o out of the second) non-teamed 1ntel <15 in the host%' We
made sure we ran the <ewS19 on the second /uest machine after all that as well'
&hese steps made sure that a 7new7 adapter would be installed in the /uest
#assi/nin/ a uni2ue ?(19 in the *S% and allow for the traLc to Mow correctly in the
<etwor0 Foad =alanced cluster 4 1 emphasize it because it messed us up as we tried
to /et thin/s wor0in/'
We made all the similar chan/es on the rest of the /uests across the other 2
host machines and turned on all the /uest machines' <ow we could wor0 on the
,M?(8S&s and /et their 16-s setup and enable networ0 load balancin/'
- 8nablin/ networ0 load-balancin/ for terminal ser$ices across all ! $irtual
machines:
Dirst) we assi/ned the 16 to the /uest machine' 1n our eample) we assi/ned
>0'0'0'!0 to the 3rst adapter #which we renamed 7> 4 Actual networ07 to help us
out%' &his adapter /ot the full /amut of settin/s includin/ 16) subnet) /ateway) 9<S
ser$ers) re/istration) and W1<S ser$er settin/s' &he second <15 on this /uest
#which we renamed 72 4 F=<7% would /et the 16 address of >0'0'0'!> and the subnet
settin/s but nothin/ else' We would also turn o. 9<S re/istration and disable
<et=1*S o$er &56G16 on this second <15' We assi/ned the 16-s as outlined across the
/uests and made sure all 16s are pin/able #the host) each of its 2 1ntel <15 16s) and
each of the 2 16-s on each /uest%'
Since this is a &erminal Ser$er Foad =alanced 5luster) we had to do all the
stu. needed for terminal ser$ices on each ?(8S& machine' We added them to the
domain) loaded &erminal Ser$ices on each /uest) and pointed them to the licensin/
ser$er in the domain' We tested them each to be sure that &erminal Ser$ices was
wor0in/ 3ne'
<ow we could setup the <etwor0 Foad =alancin/ cluster for &erminal Ser$ices
across the ! ,M?(8S& machines' We lo//ed locally to the 3rst /uest machine
#,M?(8S&0>% and started the <etwor0 Foad =alancin/ Mana/er in the
Administrati$e &ools folder' We ri/ht clic0ed on the 7<etwor0 Foad =alancin/
5lusters7 icon and chose 7<ew 5luster7' 1n the dialo/) added the 16 for the cluster
#>0'0'0'E; for our eample%) the subnet mas0) the 7Dull 1nternet <ame7
#ourclustername'ourdomainname'net%) and 4 M%S" &M'%(")*"L+ 4 chose
7Multicast7 as the 5luster operation mode' =ecause of all the brid/in/ and
switchin/ used with ,MWare) (nicast Hust didn-t seem to wor0 ri/ht' 5lic0ed net)
lea$in/ the net spot at it defaults #unless additional cluster 16-s may be needed 4
we didn-t need them%'
5lic0ed <et a/ain) and we came to the 6ort rules' Since this was a &erminal
Ser$er cluster we only needed a rule for port 33+: #DJ1 4 all of this can be adapted
to use for port +0 or ""3 for web ser$ices as well) or it can be left to all ports if you
ha$e an app that uses a wide ran/e%' We remo$ed the default rule and added a
new rule' 1n the 7Add7 dialo/) we left 7All7 chec0ed) chan/ed the port ran/e to /o
from 33+: to 33+:) chan/ed 6rotocols to &56) and set the Dilterin/ mode to
7Multiple host7 with sin/le aLnity' About that last settin/ 4 we 0ept the aLnity to
sin/le so that if a user /ot disconnected from their &erminal session) they could re-
connect to the same session on the same host node 4 the same settin/ is useful for
websites that need to maintain persistent connections' 1f thatNs not important) then
you can chan/e aLnity to 7<one7' We clic0ed o0 and net when done'
Fater we added the hosts that would be a part of the cluster' We typed the
name of the 3rst /uest machine #,M?(8S&0>% and clic0ed connect' &here should
be 2 <15-s listed #in our eample they would be 7> 4 Actual <etwor07 and 72 4 F=<7'
1f there are not 2 <15s) it meant that to Windows) one of the <15-s has the same
?(19 as the other and would screw up the <F= con3/uration' &he <15 would need
to be uninstalled on the /uest machine) the /uest machine shut down) the $irtual
8thernet adapter remo$ed from the ,MWare settin/s) and re-added in ,MWare
before proceedin/' When re-stared) the &56G16 settin/s could be re-added and then
we could see 2 <15s%' Drom the 2 <15S) we selectedGhi/hli/hted the Foad =alance
<15 #72 4 F=<7 in our eample% and clic0ed net'
1n &he 7@ost 6arameters7 dialo/) we assi/ned a uni2ue 76riority7 number #this
would be di.erent for each /uest%' Also) we made sure it was pullin/ in the correct
16 information that was already assi/ned to the 72 4 F=<7 <15 on the ,M?(8S&0>'
All the rest was default and clic0 3nish' &he app went throu/h the motions and
added the >
st
node to the cluster' When it was /reen in the mana/er) the net
nodes were added simply by ri/ht-clic0in/ on the new cluster name and choosin/
7Add host to cluster7 and /oin/ throu/h some of the same motions for each
successi$e node'
Afterwards) node priority twea0in/ could be done on the port parameters' We
left ours at default #82ual% since they were all the same hardware essentially' =ut if
the port parameters on an indi$idual host were opened) an adHustment could be
made to the 7Foad Wei/ht7 4 lower numbers mean the system would more li0ely be
connected to o$er hi/her numbered nodes'
@ope this helpsK sorry for bein/ lon/ winded and chan/in/ /rammatical
tense and person from time to time but 1 can be lon/ winded' 1 Hust li0e detail in
what 1 am doin/ so 1 wanted to /i$e the same'
=rian Joun/
Sr' Ser$er Analyst
bdyoun/O/mail'com