Platform Cluster Manager: HPC Kit

Platform HPC Version 2.1 Release date: December 2010

Copyright

1994-2011 Platform Computing Corporation.

Although the information in this document has been carefully reviewed, Platform Computing Corporation (Platform) does not warrant it to be free of errors or omissions. Platform reserves the right to make corrections, updates, revisions or changes to the information in this document. UNLESS OTHERWISE EXPRESSLY STATED BY PLATFORM, THE PROGRAM DESCRIBED IN THIS DOCUMENT IS PROVIDED AS IS AND WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT WILL PLATFORM COMPUTING BE LIABLE TO ANYONE FOR SPECIAL, COLLATERAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING WITHOUT LIMITATION ANY LOST PROFITS, DATA, OR SAVINGS, ARISING OUT OF THE USE OF OR INABILITY TO USE THIS PROGRAM. Wed like to hear from you You can help us make this document better by telling us what you think of the content, organization, and usefulness of the information. If you find an error, or just want to make a suggestion for improving this document, please address your comments to doc@platform.com. Your comments should pertain only to Platform documentation. For product support, contact support@platform.com. Document redistribution and translation Internal redistribution This document is protected by copyright and you may not redistribute or translate it into another language, in part or in whole.

You may only redistribute this document internally within your organization (for example, on an intranet) provided that you continue to check the Platform Web site for updates and update your version of the documentation. You may not make it available to your organization over the Internet. LSF is a registered trademark of Platform Computing Corporation in the United States and in other jurisdictions. ACCELERATING INTELLIGENCE, PLATFORM COMPUTING, PLATFORM SYMPHONY, PLATFORM JOB SCHEDULER, PLATFORM ISF, PLATFORM ENTERPRISE GRID ORCHESTRATOR, PLATFORM EGO, and the PLATFORM and PLATFORM LSF logos are trademarks of Platform Computing Corporation in the United States and in other jurisdictions. UNIX is a registered trademark of The Open Group in the United States and in other jurisdictions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. Microsoft is either a registered trademark or a trademark of Microsoft Corporation in the United States and/or other countries. Windows is a registered trademark of Microsoft Corporation in the United States and other countries. Intel, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Other products or services mentioned in this document are identified by the trademarks or service marks of their respective owners.

Trademarks

Third-party license agreements

http://www.platform.com/Company/third.part.license.htm

Contents
About this kit .......................................................................................................................... 5 Install a kit .............................................................................................................................. 6 Install a kit ....................................................................................................................... 6 Install kit using CLI .......................................................................................................... 6 Install kit using One Step Install on the Platform HPC Console ...................................... 9 Remove a kit ........................................................................................................................ 10 Remove kit using CLI .................................................................................................... 10 Remove kit using the Platform HPC Console ............................................................... 10 Included components .......................................................................................................... 12 Enable components ...................................................................................................... 13 Related configurations and commands ............................................................................... 14 Related documentation and important links ........................................................................ 15

Platform Cluster Manager: HPC Kit 3

4 Platform Cluster Manager: HPC Kit

About this kit

About this kit

The HPC 2.1 Kit is based on OS provided packages. It contains tools, libraries, and utilities for high performance computing clusters.

Important:

This version of the HPC kit for Platform HPC 2.1 is no longer dependent on kernel versions. The BLCR package was dropped as it depends on kernel versions. Linpack, Scalapack, and BLACS packaged with openmpi component only are compiled with and work with OpenMPI, not supporting mpich2 and mvapich2. This kit for Platform HPC 2.1 has a reduced set of packages that builds on RHEL 5.5.

This version of the HPC kit integrates with the OFED kit, Platform LSF kit, and Platform HPC Console kit.

Note:
The OS based HPC kit cannot work with the Platform OFED kit.

Visit http://my.platform.com/products/platform-hpc for the latest kit documentation.

Platform Cluster Manager: HPC Kit 5

Install a kit

Install a kit
Install this kit by using the kusu-kitops or kusu-kit-install on the command-line interface (CLI), or by using the One Step Install feature of the Platform HPC Console. kit_install_overview.xml

Install a kit
Install this kit by using the kusu-kitops or kusu-kit-install on the command-line interface (CLI), or by using the One Step Install feature of the Platform HPC Console.

Install kit using CLI

On the command-line interface (CLI), you can install the kit via the kusu-kit-install or kusu-kitops tool.

Note:
Cluster management tools had been renamed in this release. The old command names (such as kitops, repoman, cfmsync, ngedit, etc.) still exist but will be deprecated in future release.

Install kit using kusu-kit-install

The command kusu-kit-install is a kit installation and deployment tool that assumes reasonable defaults to seamlessly integrate kusu-kitops, kusu-repoman, kusu-cfmsync, and kusu-ngedit functions for easy deployment.

Note:

The kusu-kit-install tool and its library files are packaged with a man page and documentation files as rpm. This rpm is a dependency of the component-base-installer component that is only meant for the head node and not for compute nodes. This is auto installed on the head node with the rest of the base kit during cluster deployment. If a kit is already present in the system, it still has to be added to the repositories. If a kit is already added to a repository, its components have to be associated with the matching node groups.

Follow these steps to install a kit using kusu-kit-install on the CLI: 1. Download the kit ISO. 2. Add the kit using kusu-kit-install tool.

6 Platform Cluster Manager: HPC Kit

Install a kit Run command:

kusu-kit-install /home/foo/kit-foo.iso

or
kusu-kit-install /dev/cdrom

Default repository:
kusu-kit-install source

where source refers to the location of the kits to be installed; can be /tmp/kit.iso (file), /dev/ cdrom (mount point), or /data/kit (directory) Customize repository:
kusu-kit-install -r repo_names source

where repo_names refer to comma separated list of repository names Examples of repo_names: rhel5.5-5-x86_64,\ sles10.3-10-x86_64 Installation status appears once the process is completed.

Synopsis and options

Synopsis
kusu-kit-install [-h|-v] [-r repo_names] source cdrom (cdrom device), or /data/kit (directory)

where source is the location of the kits to be installed (for example: /tmp/kit.iso (file), /dev/

where repo_names refer to comma separated list of repository names

Options

-r, --repolist = repo_names (optional)

Adds kits to the specified repositories.

Note:
Multiple repository names must be separated by commas (,). The default repository used is the native repository (with repo id = 1000).

Examples:
kusu-kit-install -r rhel5.3-5-x86_64, rhel5.2-5-x86_64 /mnt kusu-kit-install -r 'rhel5.3-5-x86_64, rhel5.2-5-x86_64'/mnt

-h, --help Prints the help option and exits. -v, --version Prints the version information and exits.

Install kit using kusu-kitops

The kusu-kitops tool is a kit operations tool for adding and removing kits. Follow these general steps to install a kit using the kusu-kitops command.

Platform Cluster Manager: HPC Kit 7

Install a kit 1. Import the kit information to the database, and copy files from the media. To do this, use the kusukitops command:
kusu-kitops -a -m path_to_kit

where path_to_kit can be an ISO, URL, FTP path, or NFS path.

Note:
Command option definitions:

-a: Adds a new kit. -m: Points to the ISO (media) containing the kit files.

The kit and its components are registered in the database, and the kit contents are copied to this directory on the head node:
/depot/kits/kit_id

Note:
To check the kit id, use the command:
kusu-kitops -l

2. Add the kit to a repository on the head node. To do this, use the kusu-repoman command: a) Determine which repository on the head node to which you want to add the kit. To do this, list the repositories currently in the system.
kusu-repoman -l

b) Add the kit to the desired repository.

kusu-repoman -r "repository name" -a -i kit_id

c) Refresh the repository.

kusu-repoman -r "repository name" -u

Note:
Command option definitions:

-l: Lists the repositories currently in the system. -r: Selects the repository by name. -a: Adds the kit to the repository. -i: Points to id of the kit you are adding to the repository. -u: Rebuilds the repository metadata with the newly added kits.

3. Associate kit components with node groups. Some kits may automatically create associations with node groups. In this case, you can add components immediately by running kusu-cfmsync -p. If components are not automatically associated, run kusu-ngedit to launch the Node Group Editor tool, and then complete the following: a) Within the Node Group Editor, navigate to the Repository page. b) Ensure the node group is using the repository that contains the new kit, and then select Next. c) Navigate to the Components page. d) Select the components you want to install on all nodes in the node group.

8 Platform Cluster Manager: HPC Kit

Install a kit e) Save any changes you have made. 4. From the Node Group Editor, you can update existing nodes in the node group with the new kit components. The last page in kusu-ngedit asks if you want to push the kit package to the nodes using cfm (Configuration File Manager utility). You can also choose to run the kusu-cfmsync utility manually at a later time, or simply reboot the nodes to force a reinstall of the new packages.

Note:
Any new nodes added to the node group after saving your changes within kusu-ngedit will automatically have the kit package installed.

Install kit using One Step Install on the Platform HPC Console
The One Step Install integrates with the kusu-kit-install command to perform a one-step kit addition with acceptable defaults. Follow these steps to install a kit using the One Step Install function of the Platform HPC Console. 1. Download the kit ISO. 2. Open a browser and access the Platform HPC Console:
http://primary_host_name:8080/platform

3. Navigate Clusters>Cluster Inventory then click Kits. 4. Click Add. The One Step Install appears on the Installation Options. 5. Select location where kit ISO is stored. 6. Select repository. Select Default Repository to use default repository. Select Custom Selection repository to select the current and snapshot repository.

7. Select the kit ISO. 8. Click Add Kit. A message displays once kit installation is completed.

Platform Cluster Manager: HPC Kit 9

Remove a kit

Remove a kit
Remove kits via any of these two options: 1. Remove kit via the command-line interface (CLI). 2. Remove kit via the Platform HPC Console.

Important:
If the kit has a support component enabled on the head node group, disable other components (whether on the head nodes node group or other node groups) first before the support component.

Remove kit using CLI

Follow these general steps to remove a kit via CLI. 1. Disassociate all components from all node groups using kusu-ngedit.

Tip:
Use kusu-kitops to check which node groups are associated with the kit to be removed.

2. Run kusu-cfmsync -f -p. 3. Remove kit from all repositories:

kusu-repoman -r all repos affected -e -i kit_id

Tip:
Use kusu-repoman -l to check which repositories contain the kit to be removed.

4. Refresh the repository:

kusu-repoman -r repository name -u

5. Completely remove kit from the cluster:

kusu-kitops -e -i kit_id

Remove kit using the Platform HPC Console

You can also use the Platform HPC Console to remove a kit. 1. Log into the Platform HPC Console. 2. Navigate Clusters>Cluster Inventory then click Kits. Kits page showing a list of existing kits in the PCM library displays. A check mark indicates OS kits, and if a kit is optional (removable). 3. In each node group, remove all components of the kit. 4. Click Repository and remove the kit from each repository then rebuild the repository. 5. Select the kit(s) to be removed from the PCM library.

10 Platform Cluster Manager: HPC Kit

Remove a kit

Caution:
A kit is made up of several components which could be associated with a node group. Disassociate the kit from the node group, remove it from the repository, then rebuild the repository first before removing the kit.

6. Click Delete. Delete confirmation prompt appears. 7. Confirm deletion of selected kit(s).

Platform Cluster Manager: HPC Kit 11

Included components

Included components
The Platform HPC kit is made up of several components. Some of these components contain subcomponents that you can choose to enable or disable, while others serve as wrappers containing required pieces (for example, complete library and tool sets).

Note:
Except for component-hpc-source-code, all components in the Platform HPC kit are associated with the installer and compute (package-based install) node groups by default.

componentbenchmarkingtools

By default, this component is enabled and associated with both the installer and package-based compute node groups. Subcomponents include:

bonnie++ v1.03e: This is a disk benchmarking tool. When enabled, this

component lets you test the disk and file system performance of your Platform Cluster Manager cluster. iozone v3_327: This is a disk benchmarking tool with graph-presentation capabilities. When enabled, this component lets you test the disk and file system performance of your PCM cluster.

componentThis is a high performance message-passing library. Enable this component if your linear-OpenMPI- application requires this library. By default, this component is enabled and libraries associated with the installer, compute-packaged,compute-imaged, and computediskless node groups.

Important:
This component depends on OpenMPI library which is provided in Platform OFED kit (component: component-OFED-OpenMPI-libraries) or Mellanox kit (component: component-MellanoxMPI-gcc). To prevent dependency problems, make sure that you have enabled the OpenMPI library before enabling component-linear-OpenMPI-libraries.

Subcomponents include:

Linpack (latest HPL package): This is a collection of Fortran subroutines that analyze and solve linear equations and linear least-squares problems. ScaLAPACK v1.8.0: This is a scalable LAPACK library for linear algebra computations on distributed memory MIMD parallel computers. You can use this to validate the HPC kit functionality. BLACS v20000224: This is an MPI library used for basic linear algebraoriented computations that can be used across a range of distributed memory platforms. You can use this to validate the Platform HPC kit functionality.

12 Platform Cluster Manager: HPC Kit

Included components componentMPICH2libraries

This component includes the MPICH2 v1.2.1. This is a portable implementation of a high performance MPI (Message Passing Interface). Enable this component if your application requires this library. By default, this component is enabled and associated with both the installer and package-based compute node groups.

component-hpclibraries

By default, this component is enabled and associated with both the installer and package-based compute node groups. Subcomponents include:

netCDF v4.0.1 and netcdf-devel: Set of software libraries and data formats for

managing array-oriented data. Enable this component if your application requires this library. HDF5 v1.8.4: This is a set of data models, file formats, APIs, libraries, and tools used to manage large and complex data collections. When enabled, this component provides for the management of extremely large and complex data collections. Enable this component if your application requires this library. FFTW v3.2.3 and FFTW-devel: A fast-fourier transform library. When enabled, this C-subroutine library computes discrete Fourier transform (DFT) in one or more dimensions of arbitrary input size and of real and complex data. Enable this component if your application requires this library. ATLAS v3.9.21: This is a library used for creating automatically tuned linear algebra software. When enabled, provides C and Fortran77 interfaces to certain implementations. Enable this component if your application requires this library.

component-hpcsource-code

This component includes all SRPMs in other components. By default, this component is not associated with any node group.

Enable components
Use the Node Group Editor (kusu-ngedit) to individually enable or disable components within a node group. 1. Run kusu-ngedit. 2. After selecting the node group on which this kit is installed, navigate to the Components window and select platform hpc. 3. Select (or clear) the desired kit components.

Platform Cluster Manager: HPC Kit 13

Related configurations and commands

Related configurations and commands

Use available OpenMPI, Linpack, ScaLAPACK, and BLACS tools, test drivers, and commands to test the functionality of Platform HPC kit component libraries. Detailed information on each of these analytical libraries can be found on their respective web sites:

OpenMPI: http://www.open-mpi.org/ Linpack: http://www.netlib.org/linpack/ ScaLAPACK: http://www.netlib.org/scalapack/ BLACS: http://www.netlib.org/blacs/

Environment Modules utility

The Platform Cluster Manager base kit includes a utility for changing your environment to allow different versions of the same library to run concurrently on the same machines. When you install Platform HPC kit components, a helper script is installed to run this provided utility. For more information on this utility, visit http://modules.sourceforge.net/.

Set up InfiniBand interfaces

Consult your vendor or operating system documentation for details on how to set up networking.

Define public FQDN hostname in hosts file

When running mpdboot from the Platform Cluster Manager head node, define public FQDN hostname in hosts file as it has two NICs. Full name is the FQDN hostname in public interface. Example:
Run $ mpdboot -n 4 -f hosts & [1] 31255

14 Platform Cluster Manager: HPC Kit

Related documentation and important links

Related documentation and important links

Platform documentation

Visit the Platform Cluster Manager download site for all the latest Platform Cluster Manager product and kit documentation, including installation and user guides: http://my.platform.com/products/platform-hpc Guides for various Platform Cluster Manager kits are also installed on your local system:
http://primary_install_host_name/kits/kit_name/version/

Kit component support and information

Visit the following web sites for more information about tools and commands for the various kit component libraries and utilities:

hpc-libraries:

ATLAS: http://math-atlas.sourceforge.net/ HDF5: http://www.hdfgroup.org/HDF5/ FFTW:

Homepage: http://fftw.org/ Documentation: http://fftw.org/#documentation NetCDF: http://www.unidata.ucar.edu/software/netcdf/ Benchmarking-tools: bonnie++: http://www.coker.com.au/bonnie++/ iozone: http://www.iozone.org/ BLACS:

Homepage: http://www.netlib.org/blacs/ Papers available for download: http://www.netlib.org/blacs/BLACS/Papers.html Environment Modules Project: http://modules.sourceforge.net/ MPICH2:

Homepage: http://www.mcs.anl.gov/research/projects/mpich2/ Documentation: http://www.mcs.anl.gov/research/projects/mpich2/documentation/index.php? s=docs MVAPICH: Homepage: http://mvapich.cse.ohio-state.edu/ MVAPICH2 user guide: http://mvapich.cse.ohio-state.edu/support/user_guide_mvapich2-1.2.html FAQ: http://mvapich.cse.ohio-state.edu/support/faq.shtml OpenMPI:

Homepage: http://www.open-mpi.org/ Documentation (FAQ): http://www.open-mpi.org/faq/ ScaLAPACK Project:

Platform Cluster Manager: HPC Kit 15

Related documentation and important links Homepage: http://www.netlib.org/scalapack/ Scalapack User forum: http://icl.cs.utk.edu/lapack-forum/ Linpack: http://www.netlib.org/linpack/

Technical support
Contact Platform Computing for technical support in one of these ways:

Web Portal eSupport You can take advantage of our web-based self-support available 24 hours per day, 7 days a week (24x7) by visiting http://my.platform.com. The Platform eSupport and Support Knowledgebase site enables you to search for solutions, submit your support request, update your request, enquire about your request, as well as download product manuals, binaries and patches. Email Support: support@platform.com Telephone Support: contact information available at http://www.platform.com/services/support

16 Platform Cluster Manager: HPC Kit