Tutorial Asterisk Load-Balancing with UltraMonkey Page |1

Tutorial
Load-Balancing Asterisk Servers with
UltraMonkey

Author: Dr. Daniel Krahenbuhl (CEO Intuit Innovations)

Date: 15. March 2009

www.intuitinnovations.com Dr. Daniel Krahenbuhl

Tutorial Asterisk Load-Balancing with UltraMonkey Page |2

The following tutorial will guide you in installing a enterprise Asterisk deployment which you
can scale up to many 10’000 concurrent calls. I assume that you have Linux, mySQL, Apache2
and general network know-how. We deployed this kind of architecture in various live
environments without any issues.

The tutorial only covers the server’s in “green” color.

This is the setup that we normally use to deploy high-end Asterisk systems.

We are not discussing the setup of Asterisk Real Time Database, Nagios and MySQL cluster in
this tutorial.

www.intuitinnovations.com Dr. Daniel Krahenbuhl

Tutorial Asterisk Load-Balancing with UltraMonkey Page |3

Enable LVS
In this first step you need to enable IPVS on the load balancing servers (directors). IPVS stands for IP
Virtual Server and implements transport-layer load balancing inside the Linux kernel, so called Layer-4
switching.

echo ip_vs_dh >> /etc/modules

echo ip_vs_ftp >> /etc/modules
echo ip_vs >> /etc/modules
echo ip_vs_lblc >> /etc/modules
echo ip_vs_lblcr >> /etc/modules
echo ip_vs_lc >> /etc/modules
echo ip_vs_nq >> /etc/modules
echo ip_vs_rr >> /etc/modules
echo ip_vs_sed >> /etc/modules
echo ip_vs_sh >> /etc/modules
echo ip_vs_wlc >> /etc/modules
echo ip_vs_wrr >> /etc/modules

Then you need to load the new modules / drivers into the kernel.

modprobe ip_vs_dh
modprobe ip_vs_ftp
modprobe ip_vs
modprobe ip_vs_lblc
modprobe ip_vs_lblcr
modprobe ip_vs_lc
modprobe ip_vs_nq
modprobe ip_vs_rr
modprobe ip_vs_sed
modprobe ip_vs_sh
modprobe ip_vs_wlc
modprobe ip_vs_wrr

If you get errors during this process you most probably you kernel wasnot compile with IPVS support.
You need to compile a new kernel or install a kernel image with IPVS support.

If you install Debian 4 (Netinstall) you will have no problems and your kernel will support IPVS.

www.intuitinnovations.com Dr. Daniel Krahenbuhl

Tutorial Asterisk Load-Balancing with UltraMonkey Page |4

2.Install UltraMonkey on the Load Balancers (on both load balancers)

Ultra Monkey is a project to create load balanced and highly available services on a local area network
using Open Source components on the Linux operating system; the Ultra Monkey package provides
heartbeat (used by the two load balancers to monitor each other and check if the other node is still
alive) and ldirectord, the actual load balancer.

UltraMonkey is a project to build load balanced and high available services on a local area network using
100% open source components on the Linux operating system. UltraMonkey installs heartbeat (used to
build high availability between your two load balancer with constant monitoring if the other node is still
alive) and ldirector, the actual load balancer.

You need to adjust your sources.list to be able to install UltraMonkey with apt

vi /etc/apt/sources.list

add this two lines at the end of the file

deb http://www.ultramonkey.org/download/3/ sarge main

deb-src http://www.ultramonkey.org/download/3 sarge main

Now you need to update your sources

apt-get update

and then we can install UltraMonkey

apt-get install ultramonkey

If you see this warning like this

¦ libsensors3 not functional ¦
¦
¦
¦ It appears that your kernel is not compiled with sensors support. As a
¦
¦ result, libsensors3 will not be functional on your system.
¦
¦
¦
¦ If you want to enable it, have a look at "I2C Hardware Sensors Chip
¦

You can ignore it! Smile.

www.intuitinnovations.com Dr. Daniel Krahenbuhl

Tutorial Asterisk Load-Balancing with UltraMonkey Page |5

During the Ultra Monkey installation you will be asked a few question. Answer as follows:

Do you want to automatically load IPVS rules on boot?

<-- No

Select a daemon method.

<-- none

1. Enable Packet Forwarding on the Load-Balancers (LB1 & LB2)

The load balancers must be able to route traffic to the Asterisk servers. Therefore we must
enable packet forwarding on the load balancers. Add the following lines to /etc/sysctl.conf:

vi /etc/sysctl.conf

# Enables packet forwarding

net.ipv4.ip_forward = 1

Then modify the kernel parameter during runtime with the following command:
sysctl –p

2. Configure heartbeat And ldirectord

Now we have to create three configuration files for heartbeat. They must be identical on LB1 and LB2

vi /etc/ha.d/ha.cf
logfacility local0
bcast eth0 # Linux
mcast eth0 225.0.0.1 694 1 0
auto_failback off
node loadb1
node loadb2
respawn hacluster /usr/lib/heartbeat/ipfail
apiauth ipfail gid=haclient uid=hacluster

Important: As node we must use the output of

uname –n

www.intuitinnovations.com Dr. Daniel Krahenbuhl

Tutorial Asterisk Load-Balancing with UltraMonkey Page |6

In our sample here we broadcast the heartbeat over the eth0 interface. You could also use a dedicated
eth for the heartbeat if you add/use a additional network card on both load balancers. It is also possible
to do the heartbeat over a serial cable (null modem cable) between the two servers.

Now you need to configure the haresources file which defines the virtual or floating IP address for your
load balancers as well as it starts the necessary services on the load balancers.

loadb1 \
ldirectord::ldirectord.cf \
LVSSyncDaemonSwap::master \
IPaddr2::192.168.0.105/24/eth0/192.168.0.255

Important: This file must be exactly the same on both machines. It defines:

 Who is the master (in our case loadb1)

 The floating IP (192.168.0.105)
 And starts the ldirectord (load balancing software)

The first word (loadb1) is the result of uname –n on the LB1 server.

Now you need to setup the authentication between the two load balancers.

vi /etc/ha.d/authkeys

auth 3
3 md5 somerandomstring

somerandomstring is your password which the two heartbeat daemons on loadb1 and loadb2 use to
authenticate against each other. Use your own password here.

The /etc/ha.d/authkeys file must be readable by root only, therefore you do this:

chmod 600 /etc/ha.d/authkeys

As you know by now ldirectord is the actual load balancer application. You are going to configure your
two load balancers (LB1 andLB2 in an active/passive setup, which means we have one active load
balancer, and the other one is a hot-standby and becomes active if the active one fails. To make that
work, you must create the ldirectord configuration file /etc/ha.d/ldirectord.cf which again must be
identical on LB1 and LB2.

www.intuitinnovations.com Dr. Daniel Krahenbuhl

Tutorial Asterisk Load-Balancing with UltraMonkey Page |7

vi /etc/ha.d/ldirectord.cf

checktimeout=10
checkinterval=2
autoreload=no
quiescent=yes
logfile="/var/log/ldirectord.log"

virtual=190.255.200.17:506

real=190.255.200.21:5060 gate
real=190.255.200.22:5060 gate
real=190.255.200.23:5060 gate
service=sip
scheduler=rr
persistent=600
protocol=udp
checktype=connect

In the virtual line we enter our virtual / floating IP address, and in the real lines we list the IP addresses
of our Asterisk servers. The other parameter you can find on http://linux.die.net/man/8/ldirectord

Let’s create the system startup link for heartbeat and remove those of ldirectord because ldirectord we
want to start by the heartbeat daemon.

update-rc.d heartbeat start 75 2 3 4 5 . stop 05 0 1 6 .

update-rc.d -f ldirectord remove

Great Job! You can fire your load balancer up for the first time.

/etc/init.d/ldirectord stop
/etc/init.d/heartbeat start

www.intuitinnovations.com Dr. Daniel Krahenbuhl

Tutorial Asterisk Load-Balancing with UltraMonkey Page |8

3. Test the Load Balancers (LB1 & LB2)

Let’s see if the load balancers working as expected. LB1 should have the virtual IP assigned and
ldirectord running, LB2 should not have the virtual IP assigned as well as ldiretord should not be running.

ip addr sh eth0

LB1 should show this:

2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:16:3e:40:18:e5 brd ff:ff:ff:ff:ff:ff
inet 190.255.200.15/24 brd 192.168.0.255 scope global eth0
inet 190.255.200.17/24 brd 192.168.0.255 scope global secondary eth0

LB2 should show this:

2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:16:3e:50:e3:3a brd ff:ff:ff:ff:ff:ff
inet 190.255.200.16/24 brd 192.168.0.255 scope global eth0

Now you need to check if ldirectord is running on LB1 and stopped on LB2

ldirectord ldirectord.cf status

Output on LB1:

ldirectord for /etc/ha.d/ldirectord.cf is running with pid: 2345

Output on LB2:

ldirectord is stopped for /etc/ha.d/ldirectord.cf

www.intuitinnovations.com Dr. Daniel Krahenbuhl

Tutorial Asterisk Load-Balancing with UltraMonkey Page |9

Now we want to see the routing output tables of LB1 and LB2.

ipvsadm -L –n

Output on LB1:

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn

UDP 190.255.200.17:5060 rr persistent 600

-> 190.255.200.21:5060 Route 1 0 0
-> 190.255.200.22:5060 Route 1 0 0
-> 190.255.200.23:5060 Route 1 0 0

Output on LB2:
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn

Now we want to check the master / slave status.

/etc/ha.d/resource.d/LVSSyncDaemonSwap master status

Output on LB1:
master running
(ipvs_syncmaster pid: 1591)

Output on LB2:
master stopped

If all this tests are successful you can now configure the Asterisk Servers.

www.intuitinnovations.com Dr. Daniel Krahenbuhl

Tutorial Asterisk Load-Balancing with UltraMonkey P a g e | 10

4. Configure the Asterisk servers (AS1, AS2,AS3)

First we need to install iproute on all Asterisk servers.

apt-get install iproute

Change the following in /etc/sysctl.conf:

vi /etc/sysctl.conf
# Enable configuration of arp_ignore option
net.ipv4.conf.all.arp_ignore = 1

# When an arp request is received on eth0, only respond if that address is

# configured on eth0. In particular, do not respond if the address is
# configured on lo
net.ipv4.conf.eth0.arp_ignore = 1

# Ditto for eth1, add for all ARPing interfaces

#net.ipv4.conf.eth1.arp_ignore = 1

# Enable configuration of arp_announce option

net.ipv4.conf.all.arp_announce = 2

# When making an ARP request sent through eth0 Always use an address that
# is configured on eth0 as the source address of the ARP request. If this
# is not set, and packets are being sent out eth0 for an address that is
on
# lo, and an arp request is required, then the address on lo will be used.
# As the source IP address of arp requests is entered into the ARP cache
on
# the destination, it has the effect of announcing this address. This is
# not desirable in this case as adresses on lo on the real-servers should
# be announced only by the linux-director.
net.ipv4.conf.eth0.arp_announce = 2

# Ditto for eth1, add for all ARPing interfaces

#net.ipv4.conf.eth1.arp_announce = 2

Then run this:

sysctl –p

www.intuitinnovations.com Dr. Daniel Krahenbuhl

Tutorial Asterisk Load-Balancing with UltraMonkey P a g e | 11

Then you need to modify the /etc/network/interfaces.

vi /etc/network/interfaces

auto lo:0
iface lo:0 inet static
address 190.255.200.17
netmask 255.255.255.255
pre-up sysctl -p > /dev/null

Then bring up lo:0

ifup lo:0

That’s it you can now point your IP phone to the load balancers virtual IP.
In our case 190.255.200.17

Make sure you add the port to the register server parameter in your IP phone.

Like: 190.255.200.17:5060

www.intuitinnovations.com Dr. Daniel Krahenbuhl