AbstractInitiative

Performance Analysis & Tuning / Capacity Management / Cluster (HA) & Grid Computing / LDAP

www.AbstractInitiative.com

Viral Grid Computing

Contents
Summary ....................................................................................................................................................... 3
The Workflow................................................................................................................................................ 3
Differences .................................................................................................................................................... 3
Philosophy ..................................................................................................................................................... 6
Scalability ...................................................................................................................................................... 7
More Complex Architectures ........................................................................................................................ 7
Workloads ..................................................................................................................................................... 8
Permission, Trademarks, and Copyrights ..................................................................................................... 8

Abstract Initiative, LLC All Rights Reserved 2|Page

Summary
This computing model has been around for a long time. I know this because the
principal architects here at Abstract Initiative, LLC have been doing it for a long time -
around 10 years in some form or another.

You've seen it on news websites, possibly on television. It's cheap, powerful, efficient, &
most importantly incredibly effective. Thousands of desktop computers infected with
some sort of computer virus receiving instructions from some IRC chat room and acting
on a task or tasks en masse.

The Workflow
The basic workflow is this:

1) A job source fills a queue with atomic tasks. (The queue may be well satisfied
with a database and an application to insert/update/delete records - a simple
queue.)
2) Preconfigured worker nodes pull tasks from the queue, update their status
(running/run time/output/errors/completion/etc.)
3) A "queue manager" process checks for errors/failed jobs/etc. and depending on
the desired queue policies can resubmit jobs that do not complete correctly, or
can stall/isolate failed jobs if necessary.
4) An application reports on queue status, performance, errors, job completion.

This scenario works very well for repetitive, high volume batch-type work - but the
model can easily be adjusted for online/transactional type work as well. (The type of
work commonly performed by MPI environments - with a few changes to the scheduling
model.)

Differences
Historically, in "HPC" or "grid" computing there are one or more "manager" or "head"
nodes from which jobs are initiated. The jobs are then initialized and sent to the
compute slices. This model isn't much different, except for the reliance on a message
passing middleware and possibly some sort of distributed, shared memory mechanism.
These mechanisms usually require non-standard programming and architectural
knowledge on the part of the application developers and can sometimes restrict the
possible programming languages to those supported by the MPI or NUMA/Shared
Memory middleware.

Because of moving the tasks previously managed by an MPI suite

(MPI_send/MPI_recv/etc.) to simple communications between the compute slices and

Abstract Initiative, LLC All Rights Reserved 3|Page

the "manager" node(s), a significant layer of complexity is removed. Because this model
does not provide for a shared memory mechanism, attention must be paid at the
workload architecture level to atomicity of units of work, and some facility must be
created to reassemble the "pieces" into the final product.

There is no specific operating system, programming language, high level network

transfer protocol, queue software, hardware platform, or specific vendor that one needs
to use to implement this sort of computing platform. This decision can be made for the
specific workload, or for your infrastructure. We are effectively simplifying what was
previously a complex computing operation.

A common infrastructure in HPC/Grid computing is to interconnect all of the nodes with

each other. This helps with shared memory access and utilization, data migration, and
in some cases fencing - In the event of node failure. Because we are breaking our
workload into smaller, discreet bite-sized pieces we shouldn’t need this additional
infrastructure. The atomicity of the units of work should effectively render this
functionality useless.

In the images below, the inter-node interconnects are red arrows. In real life, with a
large number of compute slices, these red arrows become expensive and heavily used
critical infrastructure.

Abstract Initiative, LLC All Rights Reserved 4|Page

Abstract Initiative, LLC All Rights Reserved 5|Page
In addition to the hardware infrastructure simplicity, there are more benefits to Viral Grid
Computing with regards to batch work. Another common piece of the puzzle is some
sort of job scheduling software. There are many out there with broad ranges in price
and functionality, but this type of architecture is capable of negating the need for this
added complexity.

Philosophy
The source of this philosophy arguably comes from 2 sources:

1) The UNIX(R) philosophy of "Do one thing, do it well" as evidenced by the large
number of small, single-function commands that specialize in specific tasks. As
well as:

Abstract Initiative, LLC All Rights Reserved 6|Page

 2) The "virus" methodology of feeding work to an army of single or limited function
"muscle" nodes (sometimes known as zombies).

Scalability
The Queue: Depending on how the work queue(s) is/are implemented you may be able
to scale the "manager" node(s) significantly in a horizontal fashion. RDBMS
technologies like Oracle RAC(R), MySQL NDB, or other "cluster-able" database
architectures can scale to the physical or logical limits of the RDBMS software. This can
also be leveraged to increase the availability of the "manager" nodes or to increase the
concurrent processing power thereof.

Worker Nodes: Since the work is pulled, worked, and returned by the worker nodes, and
in the simplest examples of this architecture - your logical limits are

1) Physical resource limitations (rack space, network/storage bandwidth,

network/storage ports, electricity, etc.)
2) The amount of available work and the amount of time/resources required to
complete that work.
3) The capabilities of the code written to complete each unit of work, and the ability
to reassemble the completed pieces (if necessary).

Reporting/Management Application: This is merely a user-queue interoperability

mechanism. Its scalability depends on the complexity of the workload(s) and the
capabilities and functionality of this application.

More Complex Architectures

Because this is nothing more than a multi-server queue (M/M/C in Kendall notation) it
can also be expanded into a multi-queue/multi-server queuing network (open or closed).
If you think of the base architecture as people lining up at a bank to get to a teller, the
more complex architectures can be seen as:

1) Going to the bank.

2) Then going to the DMV.
3) Then a short shopping spree at your local shopping mall.

The high-level description of this queuing network could simply be named "errands".
There is no rule that says you can't have more than one queue, and that your worker
nodes (compute slices) can't be multi-talented. You simply have to allow for this in your
workload design.

Abstract Initiative, LLC All Rights Reserved 7|Page

The same thing can be done with MPI but for the most part this is fairly uncommon
because of the complexities of MPI architectures and the development time and effort
required for implementation. To implement this, the management application simply
needs to include the logic to migrate a job/task to another queue upon completion (or
assembly of completed jobs) from the previous queue.

Workloads
Almost every IT shop has a workload or two that would benefit greatly from this sort of
architecture. An easy way of picking the right workload is to find one that has a large
number of discreet units of work, or a number of stages.

One type of application that comes to mind is a payroll system, for example. In this case
paycheck amounts, accrued vacation & sick time, and items of that sort. Traditionally a
payroll system will work through these calculations in a serial manner. To make this
more efficient, simply take the information needed to make these calculations and split
them up (by job band, by business unit, or whatever) and put it in a queue for a compute
node to process. Instead of processing each payroll instance one at a time, you can
send them out in blocks, and reassemble them upon completion. This can be done with
MPI as well, but vendor support for this may vary. You may not even need to make
many changes to an existing payroll system to do it this way (licensing costs may be
another concern) - by simply having a compute node per-business unit (or whatever
your distinction may be) and using the database that stores this information as an
informal “queue”.

Permission, Trademarks, and Copyrights

The contents of this document were assembled from documents originally authored,
except for the “Summary” section and mention of specific vendor’s RDBMS products, by
Brian Horan between August 1998 and November of 2008, used with written permission
and copyright 2002-2009 Brian Horan.

UNIX is a registered trademark of The Open Group.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names
may be trademarks of their respective owners.

MySQL is a trademark of Sun Microsystems, Inc. in the United States, the European
Union and other countries. Abstract Initiative, LLC is independent of Sun Microsystems,
Inc.

Abstract Initiative, LLC All Rights Reserved 8|Page