Hosted Private Cloud

Open source cloud computing with openQRM

by Rene Buest

2013 NewAgeDisruption.com

INSIGHTS

Abstract
Companies have recognized the benefits of the flexibility of their IT infrastructure. However, the recent past has reinforced the concern to avoid the path to the public cloud for reasons of data protection and information security. Therefore alternatives need to be evaluated. With a private cloud one is found, if this would not end in high up-front investments in own hardware and software. The middle way is to use a hosted private cloud. This type of cloud is already offered by some providers. However, there is also the possibility to build it up and run themselves. This INSIGHTS report shows how this is possible with the open source cloud computing infrastructure solution openQRM.

2013 NewAgeDisruption.com

Why a Hosted Private Cloud?

Companies are encouraged to create more flexible IT infrastructure to scale their resource requirements depending on the situation. Ideally, the use of a public cloud is meeting these requirements. For this no upfront investments in own hardware and software are necessary. Many companies dread the way into public cloud for reasons of data protection and information security, and look around for an alternative. This is called private cloud. The main advantage of a private cloud is to produce a flexible self-service provisioning of resources for staff and projects, such as in a public cloud, which is not possible by a pure virtualization of the data center infrastructure. However, it should be noted that investments in the IT infrastructure must be made to ensure the virtual resource requirements by a physical foundation for building a private cloud. Therefore, an appropriate balance needs to be found that allows a flexible resource obtaining for a self-service, but at the same time must not expect any high investment in the own infrastructure components and without to waive a self-determined data protection and security level. This balance exists in hosting a private cloud at an external (web) hoster. The necessary physical servers are rented on a hoster who is responsible for their maintenance. In order to secure any physical resource requirements, appropriate arrangements should be made with the hoster to use the hardware in time. Alternatives include standby server or similar approaches. On this external server-/storage-infrastructure the cloud infrastructure software is then installed and configured as a virtual hosted private cloud. For example, according to their needs this allows employees to start own servers for software development and freeze and remove them after the project again. For the billing of the used resources, the cloud infrastructure software is responsible, which provides such functions.

openQRM Cloud
Basically, an openQRM Cloud can be used for the construction of a public and private cloud. This completely based on openQRMs appliance model and offers fully automated deployments that can be requested by cloud users. For this openQRM Cloud supports all the virtualization and storage technologies, which are also supported by openQRM itself. It is also possible to provide physical systems over the openQRM Cloud. Based on the openQRM Enterprise Cloud Zones, a fully distributed openQRM Cloud infrastructure can also be build. Thus, several separate data centers may be divided into logical areas or the company topology can be hierarchically and logically constructed safely separated. Moreover openQRM Enterprise Cloud Zones integrates a central cloud and multilingual portal including a Google Maps integration, so an interactive overview of all sites and systems is created.

2013 NewAgeDisruption.com

Structure of the reference environment

For the construction of our reference setup a physical server and multiple public IP addresses are required. There are two options for installing openQRM: Recommended: Configuration of a private class C subnet (192.168.xx/255.255.255.0) in which openQRM is operated. openQRM required an additional public IP address for access from the outside. Option: Install openQRM in a virtual machine. In this variant openQRM controls the physical server and receives the virtual machines from the physical host for subsequent operations of the cloud.

For the assignment of public IP addresses cloud NAT can be used in both scenarios. This openQRM Cloud function will translate the IP addresses of the private openQRM Class C network into public addresses. This requires pre-and postrouting rules on the gateway / router using iptables, configured as follows: o o o iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o br0 -j MASQUERADE iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o eth0 -j MASQUERADE More information on pre-and post-routing with iptables can http://www.karlrupp.net/en/computer/nat_tutorial

be

found

at

For the configuration of complex network environments, the IP management plugin is recommended. This enterprise plugin allows to set any network- and IP address configurations for the managed servers. In the openQRM Cloud, it also provides a mapping of networks to cloud users and groups and also supports the automated VLAN management. In addition, two bridges are needed: One of the public interface with a public IP address. One for the private interface dpe for which DHCP is configured. The data in the cloud are later stored in the local storage of the physical server. For this purpose, there are two variants: Recommended: o KVM-Storage LVM Deployment (LVM logical volume deployment) o Requires one or more dedicated LVM volume group (s) for the virtual machines. For more complex setups a central iSCSI target or a SAN is recommended. Option: o KVM-Storage BF Deployment (blockfile deployment) o Create a directory on the Linux server as /var/lib/kvm-storage/storage1 /var/lib/kvm-storage/storage2 (The storage directories can be set arbitrarily on the plugin configuration.) o For more complex setups, a central NAS for the configured mount points should be used.

At the end iptables must be configured according to the rules above and the desired own safety. After that the installation of openQRM follows. Packages for popular Linux distributions are available at http://packages.openqrm.com. After openQRM has been installed and initialized the configuration follows.

2013 NewAgeDisruption.com

Basic configuration of openQRM

The first step after initialization is editing the /usr/share/openqrm/plugins/dns/etc/openqrm-plugin-dns.conf, by changing the default value to the own domain. Configure domain for the private network # please configure your domain name for the openQRM network here! OPENQRM_SERVER_DOMAIN="oqnet.org" After that we activate and start the plug-ins via the web interface of the openQRM server. The following plugins are absolutely necessary for this: DNS Plugin o Used for the automated management of the DNS service for the openQRM management network. DHCPD o Automatically manages the IP addresses for the openQRM management network. KVM Storage o Integrates the KVM virtualization technology for the local deployment. Cloud-Plugin o Allows the construction of a private and public cloud computing environment with openQRM.

2013 NewAgeDisruption.com

Further additional plugins are recommended: Collectd o A monitoring system including long-term statistics and graphics. LCMC o Integrates the Linux Cluster Management Console to manage the high availability of services. High-Availability o Enables automatic high availability of appliances. I-do-it (Enterprise Plugin) o Provides an automated documentation system (CMDB). Local server o Integrates existing and locally installed server with openQRM. Nagios 3 o Automatically monitors systems and services. NoVNC o Provides a remote web console for accessing virtual machines and physical systems. Puppet o Integrates Puppet for a fully automated configuration management and application deployment in openQRM. SSHterm o Allows secure login via a web shell to the openQRM server and integrates resource

Plugins which offer more comfort in the automatic installation of virtual machines as cloud templates are: Cobbler o Integrates cobbler for automated deploying of Linux system in openQRM. FAI o Integrates FAI for the automated provisioning of Linux systems in openQRM. LinuxCOE o Integrates LinuxCOE for the automated provisioning of Linux systems in openQRM. Opsi o Integrates Opsi for the automated provisioning of Windows systems in openQRM. Clonezilla/local-storage o Integrates Clonezilla for the automated provisioning of Linux and Windows systems in openQRM.

2013 NewAgeDisruption.com

Basic configuration of the host function for the virtual machines

Case 1: openQRM is installed directly on the physical system
Next, the host must be configured to provide the virtual machines. For that an appliance type KVM Storage Host is created. This works as follows: Create appliance o Base > Appliance > Create Name: e.g. openQRM Select the openQRM server itself as resource Type: KVM Storage Host

This gives openQRM the information that a KVM storage is to be created on this machine.

Case 2: openQRM is installed in a virtual machine running on the physical system

Using the "local server" plugin the physical system is integrated into openQRM. To this the "openQRM-local-server" integration tool is copied from the openQRM server on the system to be integrated, e.g.

scp /usr/share/openqrm/plugins/local-server/bin/openqrm-local-server [ip-address of the physical system]:/tmp/ After that, it is executed on the system to be integrated: ssh [ip-address of the physical system]: /tmp/openqrm-local-server integrate -u openqrm -p openqrm -q [ip-address of the openQRM server] -i br0 [-s http/https] (In this example "br0" is the bridge to the openQRM management network.) The integration via "local server" creates in openQRM automatically: a new resource a new image a new kernel a new appliance from the sub-components above Next, the appliance of the currently integrated physical system must be configured to provide the virtual machines. For this the appliance is set as type KVM Storage Host. That works as follows: Edit the appliance o Base > Appliance > Edit Type: Set KVM Storage Host

This gives openQRM the information that a KVM storage is to be created on this machine.

2013 NewAgeDisruption.com

Basic configuration of the storage function

Now, the basic configuration of the storage follows. For this purpose, a storage object of a desired type is created. This works like this: Create storage o Base > Components > Storage > Create Case 1, select the resource of the openQRM server Case 2, select the resource of the integrated physical system Name: e.g. KVMStorage001 Select deployment type o This depends on the selected type at the beginning: KVM-Storage LVM deployment or directory (KVMStorage BF deployment)

2013 NewAgeDisruption.com

Preparation of virtual machine images

In order to provide virtual machine (VM) later over the cloud portal as part of finished products, an image for a VM must first be prepared. This works as follows: Creating a new virtual machine with a new virtual disk and install an ISO image on it. o Plugins > Deployment > LinuxCOE > Create Templates o The created images are automatically stored in an ISO pool which each virtual machine within openQRM can access.

Subsequently a base for the master template is created. This serves as a basis to provide users a product over the order process. Create a new appliance o Base > Appliance > Create Create a new resource o KVM-Storage virtual machine Create a new VM Make settings Select an ISO image Create o Select created resource Create a new image o Add image as KVM-Storage volume o Select KVM-Storage o Select volume group on KVM-Storage o Add a new logical volume o Select an image for the appliance o Edit to set a password The previously chosen password of the ISO is overridden. Select kernel o From the local disk o (LAN boot is also possible) Start appliance o The automatic installation can now be tracked over VNC. o Further adaptations can be done itself. o Please consider Misc > Local-Server > Help >Local VMs Local-Server for local virtual machines

2013 NewAgeDisruption.com

Cleaning up
The created appliance can now be stopped and deleted afterwards. The important point was to create an image that can be used as a master template for the cloud. The created image using the appliance includes the basic operating system which was created from the ISO image.

Configuration of the openQRM Cloud

We have now finished all preparations to start configuring the openQRM cloud. We find the necessary settings at Plugin > Cloud > Configuration > Main Config. All parameters which are adapted here have a direct impact on the behavior of the whole cloud.

Basically an openQRM Cloud can be run with basic settings. Depending on the needs and the own specific situation, adaptations can be make. The area description in the right column of the table are helpful.

2013 NewAgeDisruption.com

However, there are parameter which are need to consider regardless of the own use case. These are: Automatic provisioning (auto_provision) o Determines if systems are automatically provisioned by the cloud or if an approval of a system administrator is needed.

Provisioning of physical systems (request_physical_systems) o This parameter defines if besides virtual machines even physical hosts can be provisioned by the cloud.

Cloning of images (default_clone_on_deploy) o By default the cloud rolls out copies (clones) of an image.

High-availability (show_ha_checkbox) o Enables to operate the openQRM cloud including the high-availability of the provided resources.

Billing of the used resources (cloud_billing_enabled) o openQRM has an extensive billing system to determine own prices for all resources to get a transparent overview of the running costs.

Cloud product manager (cloud_selector) o Enables the product manager to provide users various resources over the cloud portal.

Currency for the settlement of resources (cloud_currency) o Determines the local currency with which the resources are to be settled.

Exchange ratio for resources in real currency (cloud_1000_ccus) o Determines how many 1000 CCUS (Cloud Computing Units) correspond to a previously fixed real currency.

Resource allocation for groups (resource_pooling) o Determines from which host an appointed user group receive their virtual machines.

2013 NewAgeDisruption.com

Creating products for the openQRM Cloud

To provide our users the resources over the cloud portal we have to create products first which define the configuration of a virtual machine. The settings for that we find at Plugin > Cloud > Configuration > Products.

The Cloud product management is used to create various products which users can choose later to build own virtual machines itself over the cloud portal. Products which are available for us are: Number of CPUs Size of local disks Size of RAM Kernel type Number of network interfaces Pre-installed applications Virtualization type If a virtual machine should be high-available

2013 NewAgeDisruption.com

Over the status line by using +/- each product can be activated or deactivated to show or hide it for the user in the cloud portal. Please note: Products which are deactivated but are still active within a virtual machine continue to be billed. To create a new CPU product we select the CPU tap and define in the area Define a new CPU product our wanted parameter.

The first parameter defines how many CPUs (cores), here 64, our product should have. The second parameter determines the value of the product and how many costs occur per hour during its use. In this example, 10 CCUs per hour for 64 CPUs occurs.

With the arrow keys the order on how the single products are displayed in the cloud portal can be determine. The default value is above one. Please note: In the cloud portal standard profiles in the sizes small, medium and big exist. According to the order the profiles are automatically be determined under the respective products. That means that small is always the first value, medium the second and big the third.

2013 NewAgeDisruption.com

openQRM also allows to order virtual machines with pre-configured software stacks. For this openQRM uses Puppet (Plugins > Deployment > Puppet). Thus, for example, it is possible to order the popular LAMP stack. If we have configured our product portfolio, its the users turn to order virtual machines. This is done via the cloud portal.

2013 NewAgeDisruption.com

openQRM Cloud-Portal

To create a new virtual machine (VM) we click on the tap New. An input mask follows on which we can create our VM based on the products the administrator has determined and approved in the backend.

We choose the profile Big and a LAMP server. Our virtual machine now consists of the following products: Type: KVM-Storage VM RAM: 1 GB CPU: 64 cores Disk: 8 GB NIC: 1

In addition the virtual machine should be high-available. This means, if the VM fails, automatically a substitute machine with exactly the same configuration is started to work on with.

For this configuration we will have to pay 35 CCUs per hour. This is equivalent to 0.04 euros per hour or 0.84 per day or 26.04 per month. If we want to order the virtual machine we select send.

2013 NewAgeDisruption.com

Below the tap Orders we see all current and past orderings we have made with our user. The status active i n the first column shows that the machine is already started.

Parallel to this we receive an e-mail including the ip-address, a username and a password, we can use to log into the virtual machine.

2013 NewAgeDisruption.com

The tap Systems confirms both information and shows further details of the virtual machine. In addition we have the opportunity to change the systems configuration, pause the virtual machine or to restart. Furthermore the login via a web-shell is possible.

If the virtual machine is not needed any more it can be paused. Alternatively it is possible that the administrator disposes this due to an inactivity of the system or at a specific time.

Creating a virtual machine with the Visual Cloud Designer

Besides the ordinary way of building a virtual machine, the openQRM Cloud portal enables the user to do that conveniently via drag and drop. Here the Visual Cloud Designer helps, which can be find behind the tap VCD.

Using the slider on the left below Cloud Components it is possible to scroll between the products. Using the mouse allows to assemble the Cloud Appliance (virtual machine) in the middle with the appropriate products.

2013 NewAgeDisruption.com

Our virtual machine Testosteron we assembled in this case with KVM-Storage, Ubuntu 12.04, 64 CPUs, 1024 MB Ram, 8 GB disk, one NIC, and software for a webserver and the high-availability feature.

With one click on Check Costs, openQRM tells us that we will pay 0.03 EUR per hour for this configuration.

2013 NewAgeDisruption.com

To start the ordering process for the virtual machine we click request. We get the message that openQRM starts rolling out the resource and we will receive further information into our mailbox.

The e-mail includes, as described above, all access data to work with the virtual machine.

In the cloud portal under systems we already see the started virtual machine.

2013 NewAgeDisruption.com

Creating a virtual machine with the Visual Infrastructure Designer

Besides the provisioning of single virtual machines the openQRM cloud portal also offers the opportunity to provide complete infrastructures consisting of multiple virtual machines and further components, at one click. Thus, we use the Visual Infrastructure Designer. This can be found in the cloud portal behind the tap VID.

Using the VID it is possible to build and deploy a complete WYSIWYG infrastructure via drag and drop. For this purpose, it is necessary to create ready profiles with pre-configured virtual machines at first, which include for example webserver, router or gateways. These can be deployed afterwards.

2013 NewAgeDisruption.com

Contact
New Age Disruption research | analysis | strategy | advisory
Rene Buest Dipl.-Informatiker (FH) M.Sc. in IT-Management and Information Systems Koernerstrasse 30 24103 Kiel, Germany Phone: +49 (0)431 28 93 42 52 Mobile: +49 (0)173 36 49 468 E-Mail: hello@newagedisruption.com Web: http://newagedisruption.com Twitter: @ReneBuest CloudUser: http://clouduser.de

Image source cover: Paul-Georg Meister / PIXELIO

2013 NewAgeDisruption.com