Porting UNIX Applications to the NonStop Server

Business white paper

Introduction ........................................................ 2 Platform architecture ............................................ 2 Scalability .......................................................... 3 Middleware and frameworks ................................ 4 Performance ....................................................... 4 Open Systems Services ........................................ 5 Database ........................................................... 5 Languages ......................................................... 6 Fault tolerance .................................................... 6 Summary ........................................................... 7

Introduction
The NonStop Server, introduced as a fault-tolerant platform to support mission-critical on-line transaction processing, has evolved to become an open platform consisting almost entirely of standard hardware components. The original application programming interfaces (APIs) to the operating system have been complemented with a full set of POSIX interfaces, known collectively as the Open System Services (OSS). These APIs allow developers to get access to the NonStop Kernel (NSK) operating system in a programming environment similar to UNIX. The NonStop Server also supports todays use of layered software in place of more direct calls to the operating system. Middleware components provide frameworks that make application design and implementation faster, easier and more reliable. These include the following, all of which are supported on the NonStop Server: Tuxedo, a transaction processing environment (now owned by Oracle) CORBA, a set of standards developed by the industry consortium Object Management Group (OMG) Open source Java frameworks: Tomcat, Spring, Struts and JEE On the database side, the NonStop supports a POSIX/UNIX file system accessible by OSS users and, for more sophisticated requirements, it supports two relational database products. SQL/MP, NonStops first RDBMS, provides many standard ANSI 1989 features. The newer product, SQL/MX, complies closely with ANSI SQL 1999 and includes many ANSI 2003 features. This, combined with multiple independent assessments of its low total cost of ownership (TCO) and other attributes, makes the NonStop Server an attractive platform for the development and execution of many kinds of applications. While the NonStop Server is an open platform that conforms to standards, its extraordinary achievements in scalability, availability and performance are based on an architecture that might also seem extraordinary to users of other platforms. This paper discusses some of these differences and how they might affect the port of an application from UNIX to NonStop. This may also be useful when contemplating new application development on the NonStop.

Platform architecture
Most applications are designed to take advantage of the architecture of the platform on which they will execute, and most of the UNIX applications are designed to execute on Symmetrical Multi-Processor (SMP) servers. SMP servers consist of multiple processors, all sharing the same memory space for both code and data. This has several implications for the design of an application, particularly in the way that data is exchanged between processing entities and how context data may be stored between invocations of a processing entity. The NonStop Server is based on a different concept the Massively Parallel Processing (MPP) architecture. Here each processor has its own separate memory, and code and data are private to each. This approach provides fault isolation and thus improves availability. A failure in one processor and its memory, caused by either a hardware or software malfunction, will not affect the other processors or their memory; so the system can continue to serve applications and their clients. Because memory is not shared on an MPP machine, CPUs (processors and their memory) must communicate with each other quickly and often. On the NonStop, this communication occurs through ServerNet, a very high-speed switching fabric that interconnects all the CPUs to each other and to all the I/O controllers. Consistent with its MPP hardware architecture, the NonStop Kernel (NSK) is a message-based operating system. In other words, all communication between processes (one instance of a compiled program: The unit of work scheduled by the operating system) occurs through messages that are sent and received through an NSK programming interface. This is true even if the processes are executing in the same CPU. An analogy with UNIX is to consider that all processes communicate through socket calls to one another never through data structures held in memory. It is always about messaging on a NonStop Server. This difference in platform architecture has major implications for a potential port from UNIX to NonStop. How does the application share data and state among different processing units? If shared memory is used to communicate between threads or processes, then this aspect of the application will probably need to be re-designed and re-implemented.

On the NonStop Server, transaction routers can be used for applications in which the servers are context-free or for which the context is stored on disk. As described earlier in the Architecture discussion, context management cannot occur through shared If context is saved in memory, then this will also memory on an MPP computer, because there is no most likely need to be changed. shared memory. Context-free servers expect the input There are several alternative solutions of this problem, transaction message to carry any context data that is necessary. Alternatively, this message could contain including persisting context to disk or a process a pointer to the context data on disk. Either of these or simply forcing the transaction back to the same methods allows any server to process any request, process instance each time. The choice made here without the server process needing to know about can affect some aspects of scalability, as the next the clients previous message. section shows. In summary, the first aspect to be researched for a potential port to NonStop is to look at how and where the application exploits the SMP architecture, especially its reliance on shared memory for communication between processes and context storing. Simple load balancing can be achieved with the communication controller when messages arrive through the IP protocol: A round-robin algorithm allocates IP Connect requests to listeners on each CPU. This allows the Oracle WebLogic Server (WLS), for example, to run on the NonStop Server without changes but with good linear scalability as transaction rates increase. In other cases, the application may need the assistance of the NonStop TS/MP product. This middleware component employs the concept of the Server Class, a set of executing copies of the same compiled program, which can be addressed as one logical entity by other processes. TS/MP load balances the messages sent to a Server Class across the set of processes defined for it, which are typically spread across the CPUs. As the transaction load increases, more processes can be added to the Server Class, and TS/MP continues to balance the processing load. Many UNIX applications scales use a different concept they rely on spawning another processing thread within the program or process. These threads spread the processing load by running on different CPUs in an SMP system sharing the same data structures. Spawning of threads will not provide scale on a NonStop Server. While NonStops OSS supports threads within a single process, they all execute in the same CPU. That is, the NonStop Kernel supports User-Level threads but not Kernel-Level threads.

Similarly, how is context stored by the application? Often a transaction spans several interactions with a client, or a request needs to be made to another server during a transaction. The context of that transaction (that is, its state and the value of certain variables) needs to be preserved between process invocations. On UNIX systems, context is typically saved in memory and retrieved later by another process or thread to continue the transaction. This strategy will typically not work on a NonStop Server, and so a different design will be needed.

handle the incoming transaction rate, more copies of the process are needed, and preferably these would run in different CPUs. In order to scale, some kind of load balancing is required to allocate transactions across these process instances.

Scalability
Many are attracted to the NonStop Server by its reputation for linear scalability. HP has demonstrated in many benchmarks and customer installations that the throughput of a NonStop Server, measured in transactions per second (TPS) is linear, as more CPUs are added. In other words, if two CPUs can process 100 TPS, then four CPUs will be able to handle 200 TPS. This is attributable to its MPP architecture and the NonStop Kernel, but it also relies on adherence to certain design patterns and principles. Some of these, like loose coupling and high functional cohesion, are simply best practices, regardless of platform architecture. Others are more closely related to the MPP architecture itself. Applications that fail to exploit the MPP architecture through use of these design patterns may also fail to achieve their desired levels of scale. Typically, an OLTP (on-line transaction processing) application will receive a message from a transaction source, process that message (that is, apply business logic) and then either respond to the source or forward the transaction to another process. When one process of a given type is not enough to

Another key topic to research for a potential port from UNIX to NonStop is understanding how the application scales to handle an increasing transaction load. If it relies solely on SMP threading techniques, then this indicates significant re-work to move from a thread-based model to a process-based model, integrating TS/MP into the application in the place of threading. It will be discussed later that, this may also have some significant benefits in providing fault tolerance for the application.

In the case of WLS, the same code downloaded from Oracle to run on Windows or UNIX is also used on NonStop; no changes are required to enable it to run on the NonStop. Because WLS is designed to execute and scale successfully for a cluster, it proves a very good fit for the MPP architecture of the NonStop Server, and as noted earlier, it also scales very well in most circumstances. WMQ was ported to the NonStop Server by IBM. Much of its code base is shared between NonStop (using OSS) and the various UNIX platforms it supports. A few components were closely integrated into the TS/MP product to enable WMQ to scale throughout the NonStop Server. IBM also implemented NonStop process pairs in its port to provide a very high-level of fault tolerance in several key processes. Fault tolerance is discussed later in the paper.

Middleware and frameworks

As noted earlier, many applications are now implemented to take advantage of the advanced services of software products that could be termed Middleware or, more recently, Frameworks. HP supports many of these for its NonStop Servers. Tuxedo was one of the earliest transaction processing middleware products in widespread use on UNIX systems. When HP ported it to the NonStop Kernel, Tuxedo was re-architected to exploit the TS/MP load balancing product and facilitate scalability. NonStop Tuxedo is used by a number of large customers, particularly in the finance industry. Existing applications that use Tuxedo also use Tuxedo APIs, so in general they will port to NonStop with little or no change except for interactions with the database, which will be discussed later in this paper.

Performance
Many customers and Independent Software Vendors (ISVs), who initially find porting an application to NonStop to be a fairly straightforward task, discover eventually that its performance is not what they expect. This can have many causes, including those we discussed earlier. However, a successful port to NonStop involves more than a consideration of load balancing.

The NonStop Server is intended to host mission-critical applications and so includes many operating system features that provide fault tolerance More recently, the focus has been on Java and the and, above all, data integrity. One implication of various frameworks that have emerged from the fault tolerance is the assumption that applications open source community, many of which are hosted are intended to run for many months, if not years, in Apache Tomcat. This is a container for Java Servlets and JavaServer Pages and has been ported without interruption. Thus, initialisation and release to NonStop by HP as the NonStop JavaServer Pages of resources may take much longer than will be seen on UNIX systems. This is particularly true for process product (NSJSP). As in the case of Tuxedo, this creation/deletion and file open/close. framework is also built on top of the TS/MP product to provide scalability on NonStop Server systems. It is not uncommon for a UNIX application process Other Java frameworks that can be deployed to Tomcat have also been successfully tested on the NonStop. Spring, Axis2, JavaServer Faces and Hibernate, for example, are fully supported by HP on NonStop. No changes are required to allow these products to run on the NonStop, but HP does supply an additional component for Hibernate, a dialect file which allows syntax generation for the SQL/MX database. to spawn a process when a new processing entity is needed. While TS/MP can do this within a Server Class (a dynamic server), it is more usual to create a number of static servers when the application is first started. Static servers are always available to handle the workload and consume virtually no resources when idle. Most importantly, static servers lower the need to start processes on-the-fly.

Other examples of frameworks successfully ported to Similarly, UNIX applications are often implemented NonStop include Oracle WebLogic Server (WLS) and to open and close files for each use. This is an expensive operation on the NonStop and must be IBMs WebSphere MQ (WMQ). avoided except when necessary. Typically, all files

are opened during initialisation and then left open. (Data integrity is preserved by locking, isolation levels and other system-provided options). Spawning of new processes and frequent opening and closing of files degrade application and should be avoided on the NonStop Server. Thus, some re-design may be needed if the application to be ported has either or both of these characteristics.

design is good design, and its benefits accrue even when there is no prospect of an application port. For more details on specific differences in APIs and features that may be encountered when porting from UNIX to OSS, consult the Open Systems Services Porting Guide found in the NonStop Technical Library at http://docs.hp.com/

Open Systems Services

The HP NonStop Open System Services (OSS) environment on the NonStop Server is based upon standards available in the open systems community. These standards include the POSIX standards, XPG4 interface specifications and the ISO/ANSI C compiler language standard. OSS supports all the required application program interfaces defined within the 1990 edition of the POSIX.1 standard, which defines the APIs, system call interface and the C run-time routine interfaces for portable programs. OSS also supports many required shell and utility interfaces defined within the 1990 edition of the POSIX.2 standard and a subset of interfaces from later editions of the standard. OSS interfaces and functions greatly resemble those of UNIX systems because of their common relationship to the POSIX standards. Vendors frequently extend the common standard to differentiate from their competitors. NonStop, like other vendors, has implemented extensions. OSS adds to the UNIX API by offering access to Guardian fundamentals and interoperability with the native Guardian environment. Interoperability is achieved through extensions to the standard API, additional OSS function calls and access to Guardian procedure calls. For example, a program running under OSS can communicate with processes running under NSKs Guardian environment and vice versa. The widespread use of extensions means that even UNIX applications do not port from one platform to another without some changes, even though they might be minor. Extensions must always be considered when porting an application from one standard platform to another. The key to a successful port is good program design, including functional cohesion and the use of abstraction layers to remove references to platform specific services from the business logic. This isolates any platform-related changes to a few modules that require platform awareness, instead of permeating the application with platform specific services. Good

Database
Databases on the NonStop Server can be accessed on-platform only by NonStop SQL/MX or NonStop SQL/MP. SQL/MX, the newer product, can have access to SQL/MP tables as well as its own native MX tables. NonStop databases can also be accessed through ODBC (open database connectivity) and JDBC (Java database connectivity) remote connections. The NonStop Server does not support the Oracle database product or any of the other common databases for UNIX platforms, such as MySQL (except through ODBC or JDBC remote connections). Any port of an application to NonStop that involves database activity will require changes to the database and stored procedures. This topic warrants its own paper, however some observations are summarised below: Data types: Changes will most likely be required in the data types defined for many columns in the database. Some of these changes are very straightforward, only require a change all type of editing, but others may need some study, and few may just prove impossible. For example, SQL/MX does not support the BLOB or CLOB data types unless it is accessed through JDBC. Logical and physical table design: Some changes will also be required in the logical and physical layout of tables, partitions and indices to help optimise data access. These modifications may be related to the changed hardware configurations or to the desire to exploit NonStop features (parallelism, index only reads, multi-dimensional access method (MDAM) and so on). DML syntax: In addition, some changes will also be required in the actual SQL statements contained within the application, as there are minor differences in the syntax between SQL/MX and other databases, especially Oracle. DML isolation levels: It is possible that some inspection and modification of isolation level settings for reading through locks and for transaction management will be required.

Stored procedures: As there is no common standard, each vendors implementation of stored procedures is different. For example, Oracle implements stored procedures using a proprietary language called PL/SQL, while SQL/MX uses Java. Stored procedures must always be changed in a port, and these changes often comprise the bulk of the work. The work of an application port, like the application itself, very often centres on the database and any related stored procedures.

The highest level of fault tolerance is known as NonStop process-pairs. This uses NonStop Kernel services to provide a Primary and Backup process architecture. Execution state and data are checkpointed from the primary to the backup at key points in the processing cycle. Should the CPU host the Primary process fail, the NonStop Kernel automatically hands control to the Backup process, which takes over at the Primarys point of failure and continues processing the transaction. This is a complex programming model, and most customers choose not to implement their applications in this way. Instead, they rely on the services provided by the TS/MP product, mentioned earlier in the context of scalability, to provide fault tolerance. In addition to facilitate scalability through load balancing, TS/MP limits the effect of a CPU failure to transactions being processed in the CPU at its time of failure. Applications running under TS/MP use one or more Server Classes to process transactions: A Server Class consists of a configurable set of processes, each executing the same compiled program but typically running in different CPUs. Transactions are routed from clients to TS/MP using the logical TS/ MP Server Class name and from TS/MP to server processes using real process names. If a CPU fails, all of its processes fail too, and any transactions that are in-flight will be lost (except where process-pairs are involved). Any uncommitted database activity belonging to a failed process will be backed out by the Transaction Management Facility (TMF) to help ensure database integrity. Without TS/MP, that would be the end of the story. However, with TS/MP, the application is still available in all the other CPUs, and load balancing still occurs. If a transaction is re-submitted, or new ones arrive, these will be serviced by the remaining processes in the Server Class as if there had been no failure. The TS/MP software layer provides load balancing and contributes to fault tolerance on the NonStop platform. In summary, some re-design will be needed for a ported application to attain a level of fault tolerance above what the basic platform infrastructure provides. Most customers find integration with the TS/MP framework sufficient for their needs and do not use the more complex NonStop process-pairs approach.

Languages
Language issues rarely arise when porting to the NonStop Server. The original application programming language used on the NonStop was a proprietary ALGOL derivative called TAL (Tandem Application Language). This was soon supplemented with support for the COBOL and FORTRAN languages, and now most development on the NonStop Server is done in C, C++ or Java. Previous incompatibilities with run-time libraries for C++ have been resolved.

Fault tolerance
One of the appeals of the NonStop Server is its record of fault tolerant processing. The NonStop Server is designed in such a way, so that no single component failure can bring down a system. In a NonStop system, peripherals, including disks, communication controllers and the data paths to them, are duplicated in different CPUs, like the software that controls them. All key parts of the operating system are also duplicated in different CPUs. The operating system is replicated across all CPUs, so even if one fails, the operating system continues to run, all system services are available and access to all peripherals is maintained. Applications that are ported to the NonStop Server automatically benefit from the fault tolerance and data integrity of the platform. If a disk fails, it is transparent to the application. Even failure of a CPU is transparent, though end users may see an error code and need to retry an operation. This means, an application executing in the failed CPU will not recover unless some specific steps are taken in its design and implementation. Various levels of software fault tolerance can be implemented on the NonStop Server, but some re-design will be needed to attain more fault tolerance than that provided by the system hardware and software.

Summary

The NonStop Server provides a high degree of fault tolerance out of the box, but this is ideally supplemented in applications by either NonStop This paper has examined UNIX to NonStop application process-pairs or, more often, use of TS/MP and porting from the aspects of platform architecture, Server Classes. scalability, middleware and frameworks, performance, Open System Services, database, languages and fault Experience has shown that applications can usually tolerance with the major findings that: be ported from UNIX to NonStop very easily The NonStop Server, like other platforms, is open, standards based and yet unique. The NonStop Server is differentiated by its underlying MPP architecture and heritage of highly available and scalable computing with dependable data integrity. Most candidate UNIX applications exploit SMP architecture and must be modified to avoid the use of shared memory for context saving and communication between processes/threads. The NonStop Server is an MPP machine that is message based. Context is carried in transaction messages, so servers are context-free. Scalability depends on load balancing among existing processes, instead of spawning of new processes. Load balancing can be achieved simply through the use of certain communications controllers or more robustly with NonStops TS/MP middleware. NonStop supports many commonly used middleware and framework options, including Tuxedo, Java frameworks like Apache Tomcat, Spring, Axis2, JavaServer Faces and Hibernate, and commercially offered products such as Oracle WebLogic Server (WLS) and IBMs WebSphere MQ (WMQ). NonStop means exactly that: Processes should be designed to run forever. Process spawning and frequent opening and closing of files degrade performance on NonStop Servers. Vendor extensions provide competitive differentiation but create issues even for UNIX to UNIX ports. Consult the Open Systems Services Porting Guide for information about features and API extensions that affect porting to the NonStop Server. The NonStop Server supports two on-platform relational database products, SQL/MP and SQL/MX and implements stored procedures using Java. As a result, the bulk of a porting effort may be attributable to this area. Programming languages generally present no issues for a port from UNIX to NonStop. with demonstrated functional execution, due to the high degree of compatibility between UNIX and NonStops OSS and C/C++ compilers. However, this apparent ease can lead to disappointment when other factors were not considered from the outset. These factors fall into two general categories: Application design and database access. UNIX applications that are closely tied to the SMP architecture will likely encounter impediments to scalability and compromised performance without significant rework. However, applications that are built on the middleware and frameworks mentioned above or that were designed with loosely coupled components will port much more easily and effectively. These are better candidates to take advantage of the scalability, fault-tolerance and data integrity of the NonStop Server platform. Where database services are required by the application, some level of conversion will be required. This aspect of the port can prove to be the most significant in terms of effort, and, in a small minority of cases, can make the port infeasible, either technically or financially. These can usually be identified very quickly. This overview examined a number of architectural, design and software considerations related to successfully porting applications from UNIX to NonStop Server platforms. It included examples of successful ports and described potential problem areas how to identify them and how to mitigate their effects. Perhaps, most importantly it provides information about the MPP architecture and its compatible design patterns.

Get connected
www.hp.com/go/getconnected

Get the insider view on tech trends, alerts and HP solutions for better business outcomes

Technology for better business outcomes

To learn more, visit www.hp.com/go/nonstop
Copyright 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. 4AA2-9867EEW, November 2009