Real-world Enterprise SOA implementations Using JCAPS

White Paper

Version 1.0

Date 08/09/2007

Change Description Prepared / Revised By First Draft Ranga Srinivasan SOA & Middleware HCL Technologies

Confidential Information Information contained within this white paper is confidential and proprietary to HCL and should not be disclosed to anyone other than the recipients.

2007. HCL Technologies Ltd. No information may be reproduced or retransmitted without explicit permission from the company.

Table of Contents Real-world Enterprise SOA implementations Using JCAPS Page 2 of 15

Table of Contents
1. Introduction .....................................................................................................3 1.1. Target Audience ...........................................................................................4 1.2. JCAPS Highlights ..........................................................................................4 1.3. SOA and JCAPS ............................................................................................4 2. Application Integration through eGate and eWays ...........................................5 2.1. eWays - Adapters .........................................................................................5 3. SOA through Business Process Management eInsight ...................................6 4. Version Control ................................................................................................7 5. Deployments, Management and Monitoring .....................................................7 6. Adopt Best Practices ........................................................................................7 6.1. Set Up Integration Competency Center (ICC)...................................................8 6.2. Development Methodology Standards .............................................................8 6.3. Set Up Integration Testing Methodology..........................................................9 6.4. Messaging Standards and Common Information Model......................................9 6.5. Set up Generic Integration Services (Logging, Auditing and Error Handling) ........10 6.6. SOA Naming Standards.................................................................................10 7. Case Studies.....................................................................................................11 7.1. Case Study 1 ...............................................................................................11 7.2. Case Study 2 ...............................................................................................13 8. Conclusion........................................................................................................15

Company Confidential HCL Technologies Ltd

Introduction Real-world Enterprise SOA implementations Using JCAPS Page 3 of 15

1. Introduction
Business enterprises have grown in size and markets they serve and their application systems have also become more complex. The need for application programming interfaces between these systems to share the data stored in these heterogeneous systems has escalated. Enterprise Application Integration (EAI) provides the means of integrating these disparate applications within the enterprise. Service Oriented Architecture (SOA) enables reuse of services. Emerging EAI tools help build integration as services and maximize reuse and the return on investment (ROI). Some of the important driving factors for middleware and integration are: Growing adoption of packaged applications Base of business critical legacy systems Multiple platforms, protocols and technologies Internet enabling of applications through portals, self-service websites and so on The solution to these requirements is using an EAI product which shields business applications from the integration complexities by adding a layer that allows them to communicate seamlessly. As shown in figure 1, EAI system acts as an Information backbone for the organization. Once in place, the EAI backbone becomes reusable and strategic asset, readily meet the future needs, typically to construct complex business processes. EAI also reduces the cost of change in adding new applications, supporting new processes all without impacting the users or developers.

Figure 1: A typical EAI Sys System, where EAI acts as a backbone for all the application integration

Sun Java Composite Application Suite (Sun JCAPS) provides an open source platform which enables enterprises to maximize their IT assets through easy integration of their IT systems such as ERPs, CRMs and custom applications, automating the business processes and easy partner integration. The scope of this paper is to highlight HCLs real world SOA implementation using Sun JCAPS in a financial industry and beverages industry.

Company Confidential HCL Technologies Ltd

Introduction Real-world Enterprise SOA implementations Using JCAPS Page 4 of 15

1.1. Target Audience

This document is intended for Sun JCAPS technical analysts/architects and developers.

1.2. JCAPS Highlights

The various types of integration provided by Sun JCAPS are:
Application Integration

This provides a framework of collection of technologies (Sun JCAPS eGate, JMS and eWays) that together provide near real-time integration through application programming interfaces e.g. integration to SAP ERP system through BAPI eway.
Data Integration

Sun JCAPS eTL provides an integrated access to variety of relational and non-relational database sources and also to mainframe and flat file data. In this approach enterprises still maintain separate applications but opt to use EAI tools to move data from one data source to another.
Process Integration

Business Process integration (Sun JCAPS eInsight) works at the higher abstract Business Flow level within the organization. The pre-condition for this is that, the applications within the enterprise must be already communicating with each other. Process Integration interconnects the Business Processes within the organization thereby actually managing the flow of information between the applications. With the help of the Business Process Integration tools the business managers can define, monitor and change business processes through a graphical modeling tool. Sun JCAPS eBAM provides business activity monitoring. Required key Business events are collected from the message bus (eGate or eInsight business processes). These business events can be aggregated or summarized based on specific key performance indicators (KPIs) and presented on a dashboard.
Business-to-Business Integration

B2B Integration works across the businesses to implement the real e-business by integrating customers, suppliers and partners. Sun JCAPS eXchange enables partner integration in the Sun JCAPS Suite.

1.3. SOA and JCAPS

Sun JCAPS enables services to be defined and reused across the tiers. Sun JCAPS conforms to most of the SOA web services standards. SOA being loosely coupled Architecture offers better scalability and eliminates proprietary technologies by using product independent industry standards/specifications. Enterprises can deploy new applications faster by using exposed services rather than building them from scratch.

Company Confidential HCL Technologies Ltd

Application Integration through eGate and eWays Real-world Enterprise SOA implementations Using JCAPS Page 5 of 15

2. Application Integration through eGate and eWays

JCAPS provides rich integration through eWay adapters and necessary data transformation. The data transformation can be achieved through JAVA/XSLT. Integration applications are deployed at runtime on the J2EE applications servers (Sun SeeBeyond Integration server and Sun Java System Application Server and BEA Web Logic Application Server). Deploying applications onto BEA Weblogic application server can not be done through eDesigner. Application services are built using eDesigner graphical interface. eDesigner is the common IDE for all the product suites in JCAPS. eDesigner provides easy to use drag and drop features to build the business rules, build the graphical representation of the application. Services can be built using Java (JCDs) can be exposed as web services. The web services are enabled through HTTP/s protocol. Integration applications can use messaging to communicate across application. Messaging allows applications to be loosely coupled. Java Message System Server (Sun SeeBeyond JMS Server and Sun Java System Message Queue, IBM Websphere JMS implementation support) provides reliable messaging in JCAPS. Sun SeeBeyond JMS server follows JMS 1.1 specification and supports point-to-point (p2p) messaging via JMS queues implementation and multiple listeners through JMS topics (pub-sub). The JMS Server supports many of the performance features which can be tuned for performance: subscriber pooling (processing messages concurrently by only one receiver even though multiple receivers are registered on the same queue; ServerSessionPoolsize and Concurrency set to Connection Consumer) Throttling ( property which alerts the message producers and stops reading further messages as the messages are piled not subscribed) The web services have a slight overhead compared to straight event driven message based architecture. Wherever synchronous response is not needed and communicating systems can support JAVA/JMS, it is best to use the event driven, asynchronous JMS. Web service can be considered an option if there is an opportunity for multiple applications use this service synchronously.

2.1. eWays - Adapters

Sun JCAPS provides many adapters for easy integration with external applications ERPs(SAP, Oracle App), CRM (Siebel) and database (Oracle, SQL*Server) When processing large volumes (of over 5-8 million records) of data through the JCD and database eWays accessing the data through JDBC, performance issue is one of the main concern. Of course, care should be taken to analyze the database indexes for performance. In many instances, we have seen eTL tools do the heavy duty extraction and transformation and upload in fraction of the time taken through the first approach. With MQ eways, appropriate version of the MQ is must as we have seen connections remain open (observed with MQ 5.3) and patches to the MQ are needed to ensure open connections closed and released to the connection pool. SAP eWays use BAPIs, RFC and IDocs for communicating with SAP. We have also use SAP Business Transaction Event mechanism to have the SAP publish the data as trigger data for JCAPS. JCAPS would in turn look at the trigger data and process as needed. For posting transactions to SAP, one can use the BAPI/Idoc.

Company Confidential HCL Technologies Ltd

SOA through Business Process Management eInsight Real-world Enterprise SOA implementations Using JCAPS Page 6 of 15

Communicating with Siebel CRM can be done through Siebel EAI or through HTTP eWays. Typically large volumes can not be pushed through Siebel EAI or HTTP, so appropriate message structure (XSD) need to be in place to batch the messages and push.

3. SOA through Business Process Management eInsight

JCAPS eInsight enables easy modeling and orchestration of business processes. Business rules can be created graphically. At runtime, the business processes instances can be persisted into a database. Any failed step in the business process can be corrected and restarted. The business processes can be exposed as web services. Legacy services can be wrapped and can be exposed as web services. Event based decisions need to be incorporated into the business process if the business process is exposed as web services with multiple operations. The business processes can be enabled with human workflow as well. For these the user activities need to be defined as sub business processes. The sub-business processes can be created manually with Receive and Reply activities or through WSDLs as well. JCAPS eInsight allows exceptions to be caught at the business process level and named exception level. Named exceptions are exceptions configured from WSDLs or JAVA activities. Named exceptions configured from the java processes/web services activities are caught and handled properly from the tool. The eInsight modeling tool creates the X-path expressions for the rules and are Business Process Execution Language (BPEL) Compliant. However, the BPEL rules are not easily ported to other tools. The transaction management is handled through XA. The activities within the business process behave properly i.e. rolling back in case of failure and committing in case of success for example on activities comprising databases and message servers. The transactions processed through invoking web services in the business process do not behave properly. The WSTransaction support is not yet present. Some of the emerging WS- standards (e.g. Ws-Routing, WS-Notification) are yet to find space in JCAPS eInsight. So, work-around need to be in place to handle the same as well. For example, a requirement could be any service that makes a change to the core data should publish data through WS-Notification Broker standard. To overcome this support for this standard in JCAPS, we publish the input message as a JMS Topic after successful return code from the service. The message published is consumed by interested applications. The web services exposed from the business processes are by default are exposed to HTTP protocol. JCAPS is yet to support web services through JMS protocol (SOAP/JMS). To handle the routing capability, we have approached using the canonical schema where in messages are sent across to the bus. Router process at the local ESB picks up the message and if the message is to be processed outside local ESB (derived based on routing rules stored in persistent database), the message would be routed to the central ESB. If has to be processed through local ESB, message would be sent to the relevant process as an event.

Company Confidential HCL Technologies Ltd

Version Control Real-world Enterprise SOA implementations Using JCAPS Page 7 of 15

4. Version Control
JCAPS allows all the components (JCDs, Business Processes, CMAPS and deployment profiles) to be developed in eDesigner. The JCAPS artifacts are stored in Sun SeeBeyond JCAPS repository. JCAPS itself provides version control capabilities for all the artifacts in the repository enabling maintaining multiple versions. Enterprises prefer to have single source control system. JCAPS does not integrate with commercial/open source version control software yet, although, there are plans for such in the future release. In our experience, on a daily basis, the projects are exported from development or testing environment and checked into Subversion/CVS. This would be fall-back in case of any unforeseen issues with the code, one can get the latest project from Subversion and import them and build on.

5. Deployments, Management and Monitoring

Dedicated environments for development, testing (system, integration and user acceptance) needs to be in place. This includes the development of artifacts (JCAPS repository, JCAPS Projects and related configurations) and runtime instances. Appropriate change control mechanism is must for migrating from one environment to the other environment. Application administrators need to be the sole administrators to manage the environments. The default Sun SeeBeyond JMS server does not support clustering. But Sun JCAPS has an additional Sun Java Message Service Grid (JMS Grid) which supports JMS clustering for highavailability. The load balancing can be achieved with or without the JMS Grid. JCAPS Sun SeeBeyond Integration server can be deployed in clustered fail-over environments. For the Sun SeeBeyond Integration server, it needs the scripts at the operating system (OS) level for fail-over. Sun SeeBeyond Integration server does provide load balancing when eInsight business processes are deployed across multiple Sun SeeBeyond Integration servers. JCAPS developed projects can also be deployed under other J2EE capable application servers such as Web Logic or Sun Application Server. These provide application level clustering. JCAPS did away with having a repository running for managing and monitoring compared to earlier versions. JCAPS eManager connects to the given application server/integration server for easy management and monitoring. Business process instances can be mentored through the eManager. Application server configuration and management of the application can also be handled with eManager. However, lot of enterprises would have their own management software to manage the application servers. Sun SeeBeyond integration server and Sun application server can work with JMX API. Enterprises management software can integrate with Sun SeeBeyond Integration server and Sun Application server to manage through the standard JMX API.

6. Adopt Best Practices

Any new tool in the systems landscape introduces certain challenges and new learning for the IT staff and introduces new processes. It is best to set up set of best practices. To set up best practices involve people with enough great credentials (been there and done that Sun and its partner HCL) into the team from the inception stage of the project.

Company Confidential HCL Technologies Ltd

Adopt Best Practices Real-world Enterprise SOA implementations Using JCAPS Page 8 of 15

6.1. Set Up Integration Competency Center (ICC)

The Integration competency center (ICC/CoE) helps enterprises to evangelize the integration methodologies and standards to other teams in the organization. Typically ICC guides the other teams in the enterprise by helping the development teams with the integration skills, coming up with relevant training materials, methodologies and templates. ICC builds certain common services for the enterprise. These services could be business services or technical services. This would be available for use by the other teams in the enterprise.

6.2. Development Methodology Standards

It is very important to standardize on the methodology to be adopted for the development. This provides the necessary templates and processes to be adopted during the development and avoids developing from scratch. We at HCL have applied 4-D (define, Design, Develop and Deploy) methodology to the projects. 4-D Methodology is focused on implementation best practices for integration projects. 4-D Methodology has the flexibility to complement and co-exist with a customers or a consulting partners established methodologies.

Company Confidential HCL Technologies Ltd

Adopt Best Practices Real-world Enterprise SOA implementations Using JCAPS Page 9 of 15

Figure 2: 4D Methodology

For projects requiring one of type integrations, prototyping becomes important and adequate time needs to be arranged.

6.3. Set Up Integration Testing Methodology

Test plans should be created for entire testing cycles unit testing, integration testing and user acceptance testing. It is best to identify the test data as well during design. Test cases should be documented, results recorded and any bugs fixed should be regression tested. HCLs CrossFit supports customizable testing process and enables the test automation. CrossFit is extensible via cartridges to interface with custom applications and standards. It addresses the testing needs of distributed systems that are typical in enterprise application integration scenarios.

6.4. Messaging Standards and Common Information Model

Standard messaging across applications enables services to look at the part of the message instead of going through the entire message to be processed, thus enhancing the performance. Each industry there could be set of specific standards or one could come up a custom standard for the message. Some common messaging used across the industry is: XML based canonical data model relevant to the domain (e.g. ebXML) SWIFT HL7 ANSI X12/EDIFACT Common Information model helps in defining the information model and its relationship among different entities. For example, due to different business scenarios, a Customer entity in System A is represented by set of X attributes; similarly a Customer entity in System B could be represented by set of Y attributes. Common Information model brings about identifying the attributes for a Customer and its relationships and representing them in the XML schema. Common Information model provides the following: Data exchange between applications is standardized. This could be specific to the industry domain (e.g. SWIFT/HL7). Data exchange between disparate applications become seamless as any new application needing integration can just exchange information adhering to the common format Definition of data model to adhere to common information model is major task during the implementation. Frequent changes to applications and interfaces to CIM would make the handling of the messages complex as versioning need to be in place. In many cases one can come up with a normalized information model as below:
<Header Data> <TransactionSourceSys> <TranscationType> <TransactionSouceIP> <ServiceDetails> <.> <ServiceMessageDest> <TransactionDetails> <TransactionEntiry>

Company Confidential HCL Technologies Ltd

Adopt Best Practices Real-world Enterprise SOA implementations Using JCAPS Page 10 of 15

<TransactionTimezone> <> <Payload>

The above information model helps applications to easily route the data to target applications for processing further.

6.5. Set up Generic Integration Services (Logging, Auditing and Error Handling)
Transactions processed through the integration bus can error due to data formatting issues, business validation issues and connectivity with external systems issues etc. Auditing and Error handling are critical processes that help in reconciling the transactions. Auditing, Logging and Error handling are provided by common services across the enterprise. At HCL, we have developed Generic Integration services that provide logging, auditing and error handling functions. It is a best practice build this functionality or get from Sun or its preferred partners (HCL). This provides a jump-start capability for integration projects. Auditing captures the flow of the events in the integration bus. Certain key information (created user, source system, transaction details, timestamp etc) of the transactions are captured and stored in persistence storage. Certain other integration products provide out-of-the-box functionality on auditing as the messages are processed through the service bus. Sun JCAPS does not have such functionality. Errors captured while processing the transactions are logged into files or database for later analysis. Errors can also be played back or reprocessed. Time bound errors may not be able to be reprocessed as their validity expires after sometime. Appropriate coding techniques and model elements (e.g. try/catch in JCD and Scope and Exception handling in eInsight business process model) need to be in place to capture the errors with proper stack trace and error text to aid in resolving the error.

6.6. SOA Naming Standards

Establish naming standards for the components developed and publish to the team This would help the team building the components and artifacts for easier maintainability and any future enhancements and modifications.

Company Confidential HCL Technologies Ltd

Case Studies Real-world Enterprise SOA implementations Using JCAPS Page 11 of 15

7. Case Studies
7.1. Case Study 1
Implementation of Trade Promotion Management

One of the leading confectionery companies in the world has strong regional presence in the beverages market in Americas and Australia. With origins stretching back over 200 years, this company employs 60,000 people is trying to integrate disparate systems due to mergers acquisitions and various partners. A current process to measure the trade promotions given to its partners is not effective. By implementing the TPM, this company expects to measure the trade spent accurately and target effective marketing programs.
Proposed Solution

Siebel solution is implemented for the Trade Promotion Management (TPM) across North America A Common Information Model (CIM) has been defined for applications to communicate within the enterprise. An EAI solution based on SeeBeyond JCAPS has been conceived to extract the data from different source systems Informatica was used to cleanse the data, remove the duplicates and harmonize the records and update the CIM golden copy. Middleware SeeBeyond JCAPS (older version of Sun JCAPS) extracted the data from the cleansed datastore and updated the Siebel Trade Promotion Management (TPM) system

Company Confidential HCL Technologies Ltd

Case Studies Real-world Enterprise SOA implementations Using JCAPS Page 12 of 15

Figure 3: SOA / ESB Solution

After the trade promotions are set in Siebel TPM, relevant incentives for trade effectiveness are to be sent back to retailers and other suppliers. This includes getting the vendor data from source systems for issuing payment. Also, to get the TPM system to be capable of measuring the trade effectiveness the invoice and other performance information had to be brought in from source systems. Business transaction Event in SAP was enabled to send the event triggers on payments, invoices. Appropriate real-time data exchanged between the source systems (SAP) and Siebel TPM. Services were exposed for invoices and vendors wrapping the SAP BAPIs. Other JCAPS services were built to handle payment and deduction data to be posted to SAP.

Company Confidential HCL Technologies Ltd

Case Studies Real-world Enterprise SOA implementations Using JCAPS Page 13 of 15

7.2. Case Study 2

SOA Based Solution for Financial Services Firm

One of the leading expert in the securities exchange industry in the Nordic region, client provides technology and services to companies in the securities industry around the globe. HCL is partnering with this client on creating the product for handling trading platform back-offices support, removal of legacy systems. The solution is based on HCL One Foundation architecture as depicted below.

Some of the issues implementation:

faced

with

the

exisitng

Time to market new products Conforming to regulatory compliance Extending services to application service provider (ASP) and Business Service Provider (BSP) model Changes to existing system difficult
Proposed Solution

Build a SOA and ESB based Securities Processing System Solution to handle ASP and BSP model First phase to get the data synchronization with operation systems to synchronize with SPS data store (based on HCL One Foundation)

Company Confidential HCL Technologies Ltd

Case Studies Real-world Enterprise SOA implementations Using JCAPS Page 14 of 15

The system is entirely built on services based architecture. The presentation services were built on .Net thick client. The business services are built on j2ee/Spring/Hibernate framework. JCAPS business process (eInsight) exposed as a service receives the request from the .net client and invokes the business web services and returns the response to the client. JCAPS MQ and database eWays are used to retrieve the data from source systems and synchronize with the One Foundation data store by invoking the business services.

Company Confidential HCL Technologies Ltd

Conclusion Real-world Enterprise SOA implementations Using JCAPS Page 15 of 15

8. Conclusion
JCAPS has all integration capabilities enterprises need to embrace the integration and enterprise-wide SOA/BPM initiatives. JCAPS is a open source and based on J2EE standards. JCAPS is yet to support quite a few of the web services standards. JCAPS is proven integration tool in the market across many of the verticals and does require serious attention for the enterprises to look into.

Company Confidential HCL Technologies Ltd