Business Service Management solutions

Working with
Atrium Integration engine (AIE)
and
Atrium Integrator (AI)
value

Introduction
2

This session will describe Atrium Integration Engine (AIE) and Atrium
Integrator (AI), explain the difference between the two, and work
through an example of importing data to the CMDB using each.

Outline
3

AIE and AI defined

AIE and AI differences
AIE architecture and overview
AIE data exchange example
Exporting AIE data exchanges
Troubleshooting AIE
AI architecture and overview
AI example

Objectives/Results
4

Objectives
Explain AIE and AI purpose and architecture
Explain differences between AIE and AI
Demonstrate AIE and AI to import data into the CMDB
Results
Students will understand AIE and AI and have a template to use to
create data imports
Skills developed
AIE and AI proficiency
Atrium CMDB administration

2011 World Wide Remedy Users Group. All Rights Reserved.

BMC Atrium Integration Engine

The legacy method to import data from external sources
into the AR System or CMDB Forms (replaced in CMDB
7.6.4 by Atrium Integrator)

BMC Atrium Integrator

6

AI is the new tool to import data from external sources into the
CMDB. Introduced with Atrium 7.6.4 and recommended for all
new data imports.

Differences between AIE and AI

7

AI is based on the open source Pentaho ETTL Suite

(www.pentaho.com) AIE is built using ARS forms and
DB connectors written in C
AI connects to data stores using a wider variety of
connectors - JDBC, ODBC, JMS, native databases, web
services, and complex XML AIE is limited to SQL Server,
Oracle, DB2 and flat file
AI cant populate ARS forms AIE can

AIE Architecture
8

AIE has the following

main components:
Data Exchange
application
AIE service
Event Request
interface
Adapter
Development Kit

Data Exchange Application

9

Data Exchange Identifies:

The external data store is the source or the target for a data
transfer;
Which adapter to use;
How to connect to the external data store; and
Identifies the method used to execute the exchange.
Data Transfer Exchange of data
Data Mapping Defines what to transfer

AIE Service
10

AIE service:
Obtains the defined data exchange from the Data Exchange
application; and
Completes the transfer of data by communicating with the
adapter specified in the data exchange definition.
Adapters provides access to external data and responds to calls
from BMC Atrium Integration Engine during a data exchange.

Event Request Interface

11

Provides the execution of a data exchange on an eventdriven basis.

Each event-driven request is created as an entry in the
Application Pending form.
To trigger an event-driven request, create an entry into
the Application Pending form, by using AR System
workflow to create the entry, or by invoking the aiexfer
utility.

Adapter Development Kit

12

The Adapter Development Kit is a software

development kit that lets you build your own
adapters.
Adapters provides access to external data and
responds to calls from BMC Atrium Integration
Engine during a data exchange.
Set of C++ objects that are used by the AIE Engine to manage the data
exchange.

AIE Block Diagram

Remedy Link
for MS SQL
Adaptor

MS SQL
Server
3rd Party
Data

SQL Server Example

AIE

CMDB
Master
Dataset

CMDB
3rd Party
Dataset

CMDB
Sandbox
Dataset

CMDB
Reconciliation
Engine

AIE Mapping
Rules

Data
Cleansing
Workflow

CMDB
Reconciliation
Rules

AR System Server

Remedy
Asset Mgmt

AIE Consoles
14

Atrium Core Console AIE console is for

monitoring data exchange progress and status
Home Page AIE Console is for creating data
exchanges, mappings and configuring AIE

Atrium Core Console AIE Console

Home Page AIE Console

16

A
C

Design the AIE Process

before Building It
17

Use the same process to build the AIE Mappings.

Define Data Mapping

18

Whether using AIE or AI you will save a lot of time if you

plan your data mapping in advance
To do these next steps, it is much easier to have access
to the source database so you can explore and sample
the data, while you build the mappings

Define Data Mapping

19

First you want to map tables to CI Classes. Find things

that are obvious matches, but also be sure you
understand the CDM very well, and also the source
database
Your best friend here is the CMDB Data Modeling Guide
(DMG)! Read it, know it, live it!
DMG helps you understand classes and relationship classes to
use

Define Data Mappings

20

Next map CI attributes to source data columns, or

hardcoded values.
For best practice product categorization, refer to the
Product Catalog DML guide
For relationships, make sure to follow best practice
relationship naming
See the CMDB Mapping Your Data XLS file
End result will be a spreadsheet that defines your
mapping

Data Mappings Spreadsheet

21

Ensure You Have an

Account to Target Source
22

Ensure you have an account in your source database and

that you have created the necessary permissions in that
database.
If your external data store is DB2 UDB, Oracle, or SQL
Server, data mapping is easier and faster if you first
populate the database field menus in the CI Class
Mapping Information window with values from your
database.
Next steps will check connection and also populate the
database field menus

Steps to Create a Data Mapping

23

Step 1: With BMC Remedy User, log into the

server where BMC Atrium Integration Engine
is installed. The IT Home page appears if
ITSM is installed.
Step 2: In the Quick Links bar, click AIE
Console. The Data Exchange Console
should appear.

AIE Console
Data Exchange Console
24

Check the Connection

25

Test Connections button (replaced by Get Results button) triggers

the test.

Failed Connection
26

Successful Connection

Load the Data Fields into

the CI Mapping Form
28

First time to connect to the Database or Flatfile.

Loading the fields makes it easier to build the CI
Mapping.

Database Field Menus Console

29

Set up the Database

Connection Fields
30

Server Name Overall name of the database to access.

Database Name Name of the database schema to access.
Login Name User login that has read/write access to the
database.
Password Users password.

Load and Select the Tables

from the Target Database
31

When the connection is successful, select the Load Table & View
Names button to load the DB schemas tables.
Tables will appear in the Available Tables and Views table field.

Update the Database Field

Menus
32

The more tables brought over, the longer it will take to update the
Database Field Menus.
Existing entries will be removed and new entries will be added to the form.

Results of Database Field

Menus Being Loaded

Return to the Data

Exchange Console

Alternate: Checking Connection

35

For some connections (for example, when using AD

Authentication) the AIE Console Connections and
Database Menus functions will not work
Use rlshelper.exe (SQL Server) or rlohelper.exe (Oracle)
instead located in AtriumCore\aie\service\bin
If using AD Authentication, log to the AIE server as the
user with credentials

Alternate: Checking Connection

36

Double-click rlshelper.exe (rlohelper.exe) and fill in

the Database Connetion tab
If using Windows Authentication, check the Enable
Windows Auth checkbox and leave the database
user name and password blank
Click Test SQL and AR Connections

Alternate: rlshelper.exe
37

Alternate: Loading Database

Field Menus
38

If the rlshelper.exe connects, congratulations you

have verified connectivity both to the database and
AR System!
Click on the Main tab, enter the Limit tables and
views to the DBO role (usually its dbo) and click
Load table/view Names
Select those tables or views you wish to use in data
mappings and click Add
Then click Create Menu
You can now return to the AIE Console to proceed
with class mapping

CI Class Mapping
39

CI Class Mapping
40

After Saving Initial Build

of the CI Mapping
41

Establishing the Primary

Key Field Match
42

AR System Form

Remedy Data Import

Target Source Form

Requires a unique, numeric,

5 to 15 Digit setting

Request ID
AIE Import
AR System Form

Target Source Form

Serial Number

Serial Number

Model Number

Model

Asset ID

Token ID

Selecting the Primary Key

Mapping Fields
43

Identify the Data Fields to

Import and Store
44

Defining the Rule Options

45

constant| - Interprets the value that follows the pipe as

a constant.
Each set of curly brackets ({}) encloses a field mapping
rule that is evaluated independently.
The first rule to return a response value is considered the value
to be returned.
function| - Performs the function that follows the pipe.
sql| - Runs an SQL command to obtain a value.

Defining the Rule Options

46

targetsql| - Runs an SQL command on the target to

obtain a value.
process| - Runs a process to obtain a single character
value in response.
targetprocess| - Runs a process on the target to obtain a
value.
stmt| - Defines an if/then/else statement to obtain a
value.
function|calc_tokenid - Calculates a unique
alphanumeric value for all the fields listed.

Examples of Rules
47

Constant Examples:
constant|Hardware - Returns the character string Hardware.
constant|15 - Returns the number 15.
constant|15 - Returns the character string 15.

Examples of Rules
48

{PO Number} {Notification Number} {constant|

Unknown Value}
The PO Number is returned if not null. If the PO Number is not
found, the Notification Number is returned if not null. If the
Notification Number is not found, the Unknown Value string is
placed in the field.

Examples of Rules
49

function|calc_token_id ($field1$,$field2$)
Calculates the token ID.
function|concat("abc", "def")
Returns the value that results from appending the string:
abcdef
sql|select categoryid from categories where name =
$category$
Returns category IDs from the categories table where the
name equals the value obtained for the $category$ field.

49

Customizing the Rules

50

You can append rules to the rule

control on the:
Data Field Mapping;
Response Field Mapping;
Delete Field Mapping; and
Query tabs.
Use the AIE:Rules form to
customize this control.
Currently function | is the only valid
prefix.

Set up the Response Section

(Optional)
51

Set up Delete Field Mapping

Options
52

Establish a Query to
Improve Performance
53

Setting up the Options

Features for the Map
54

Final Stage The Data

Exchange
55

A data exchange is a set of rules that defines how

data is transferred between an external data store
and the CMDB or an AR System Form.
Best Practice for CMDB Process:

First, set up a single Parent AIE Data Exchange.

EX: BMC_ComputerSystem
Second, set up the Children AIE Data Exchange.
EX: BMC_Product; BMC_Monitor; BMC_DiskDrive
Finally, set up the relationship AIE Data Exchange.
EX: BMC_HostedSystemComponent

How to Build a Relationship

Map?
56

Relationship Map Main Tab

57

Relationship Map
Primary Key Mapping
58

Relationship Map
Attribute Field Mapping
59

Data Exchange Processing Steps

60

Generate the Data Exchange

61

1. Define the Data

Exchange Parameters
62

After Saving the Data

Exchange More Options
63

2. Selecting the Desired

CI Data Map
64

3. Manage When the

Exchange Executes
65

4. Setting up the
Connection Requirement
66

5. Troubleshooting Options
67

Activating the Data Exchange

68

Running the Data Exchange

69

Exporting Data Exchange

70

You can export data exchanges directly to a server:

AIE 7.5.00 to AIE 7.5.00
AIE 7.5.00 to AIE 7.1.00.
Cannot export data from AIE 7.1.00 to AIE 7.5.00
Not supported on BMC Atrium Integration Engine Version 7.1.00.

You can also export data exchanges to a file (ARX format)

Exporting the Data Exchange

71

Steps to Export Data from AIE

to File
72

Enter path to a folder to hold the export file in the Output

File: C:\
Click Export. The AIE_Configuration.arx file will be
created in the folder named.
NOTE: you must edit the ARX file in a text editor and
remove the data for these forms:
EIE:BackupLoadFlag
EIE:ApplicationSettings
EIE:VendorConfiguration

Editing AIE_Configuration.arx
73

Editing AIE_Configuration.arx
74

In the EIE:DataExchange section, change the instance

name to the target servers instance name
In the EIE:Data section, change any paths for flat files
that will be different on the target server

Importing
AIE_Configuration.arx
75

The last step is to import the ARX file. Import it to the

target server using one of these commands:
AIE 7.5+/ARS 7.5+
<AR install>\ARSystem\dataimporttool\dataimporttool.exe -x
"pramteke-pun-01" -u "Demo" -p "" -o C:
\AIE_Configuration.arx" -l "D:\log\logfilename

AIE 7.5+/ARS 7.1

<AR install>\arimportcmd.exe -x "pramteke-pun-01" -u "Demo"
-p "" -o C:\AIE_Configuration.arx" -l "D:\log

Steps to Export Data from AIE

to AIE
76

1. On the Data Exchange Console, choose AIE >

Export Configuration in the menu bar.
The AIE menu is not available for other BMC Atrium Integration Engine

consoles, such as CI Class Mappings, Relationship Class Mappings, and AR

System Form Mappings.

2. In the Source Server Details, specify the

following information:
Server Name: The server on which the data resides
(Optional) TCP Port: The TCP port if you are not using the default port
Admin Login Name: The BMC Remedy AR System administrator user
(Optional) Admin Password: The BMC Remedy AR System password

Steps to Export Data from AIE

to AIE
77

3. In the Target Server Details, specify the

following information.
Server Name: The server to which data is to be transferred
(Optional) TCP Port: The TCP port if you are not using the default port
Admin Login Name: The BMC Remedy AR System administrator user
(Optional): The BMC Remedy AR System password
(Optional) Data File Location: The location on the destination server to

which the data file is to be saved.

For Windows, specify <aie_install_dir>//service//data//. For example:
C:\\Program Files\\BMC Software\\AtriumCore\\vm1-w23-prem11\\aie\
\service\\data\\
For UNIX, specify <aie_install_dir>\opt\service\data.
Errors might occur if you use the source location directory.

Steps to Export Data from AIE

to AIE
78

4. For the Export All Exchanges option, accept the

default option of Yes or select No-Only Selected Ones.
5. If you select the No-Only Selected Ones option, a list
of data exchanges appears in the table.
6. Select the data exchanges you want to export.

Steps to Export Data from AIE

to AIE
79

7. Click Export. The label of the Export button changes

to Get Result.
8. Click Get Result. One of the following messages
appears.

Export in progress.
Export completed.
Export failed.

9. A detailed log is available in the result field.

Troubleshooting AIE Issues

that May Occur
80

AR System server accidentally stopped when running the

AIE service on UNIX
AIE service stopped after password change
AIE service fatal error
SQL adapter not working in UNIX or Linux environment
BMC Atrium Integration Engine RPC port error
New attributes do not appear on the Data Field Mapping
window

1. AR Server Stops When

AIE Runs in UNIX
81

ISSUE: Using the same terminal window as the AR

System server to start and stop the BMC Atrium
Integration Engine service, the AR System server
disconnects abruptly.
ERROR MESSAGE: 390600: Another copy of the server is
already running on the same RPC socket 390600: AR
System server terminated -- fatal error encountered
FIX: User a different terminal to start the AR System
Server and AIE.

2. AIE Service Stopped after

Password Changed
82

FIX:
1.In Remedy User, open up the Object List.
2.Open User Form - Search for your username.
3.Reset your password.
4.From the command prompt, navigate to the service\bin subdirectory
of your BMC Atrium Integration Engine installation.
5.Run the aiecfedit utility using specific command lines (found in your
book).
6.On the BMC Atrium Integration Engine Console, click the Configuration
Integration Engine Application Console link.
In the Admin Password field, specify the newly reset password.
7.Save the changes and restart the BMC Atrium Integration Engine
service.

3. AIE Service Fatal Error

83

ERROR MESSAGE: In the <AIE service

instance>_eiemain.dbg log file, A fatal setup error
occurred. The service cannot run until it is corrected.
(EIEMSG 3317).
FIX:

Examine dataexchange.dbg file for Size errors:

CMDBGetListIntances Error: (120149) Size: 5379225, Type: calloc,

Source File: .\aroscinstanceutil.cpp, Line Number: 2012

Ensure ChunkSize parameter in the aie.cfg file is less than

50000 if it is a size error.

4. SQL Adaptor Not Working

in UNIX or LINUX
84

To communicate with SQL server, the BMC Atrium

Integration Engine uses the SQL OLEDB APIs, which are
proprietary to Microsoft.
These APIs are not supported on UNIX or Linux.

5. BMC Atrium Integration

Engine RPC Port Error
85

In case of an RPC port error, confirm whether BMC

Atrium Integration Engine is using any port numbers
reserved for BMC Atrium CMDB.
If any of the BMC Atrium CMDB APIs fail, verify that you are
able to call the same function with the cmdbdriver program.
If the functions fails using the cmdbdriver program, check for
errors in the AR System server and BMC Atrium CMDB log file.

6. New Attributes do not

Appear on the Data Mapping
86

FIX

Use the Object List to locate AIE:FieldNames form.

Run a search on the form.
Select the BMC Atrium CMDB class to which you added attributes.
Delete the new attributes from the AIE:FieldNames form.
Save the AIE:FieldNames form and open the specific mapping from
the CI Class Mapping Console.
The attribute list on the Data Field Mapping table is refreshed.

Atrium Integrator Architecture

87

AI jobs are created using the Pentaho ETTL suite

with many input connectors and a CMDB output
connector

87

2011 World Wide Remedy Users Group. All Rights

Reserved.

AI Components - Pentaho
88

Pentaho Data Integration tool

Spoon graphical design tool for advanced editing of jobs to
add complex steps
Carte Server web server engine that can be used to run jobs
remotely
Repository stores job definitions

AI Components AI Console
89

Atrium Integrator Console application in the Atrium

Core Console
Integration Job Builder Wizard a BMC wizard used to build
jobs
Manage Data Stores tool used to create and store data store
connections
Job Scheduler schedules jobs
Job Monitoring monitor job progress

Accessing AI Console
90

Access AI Console through the

Atrium Core Console http://
<midtier server>/arsys/home
Log in and from the Home Page
select Atrium Core Console, then
the Integrator icon

Design the AI Process before

building it
91

Preparing Data Transfer Job

92

Create a dataset to receive the data (this is done exactly

the same as for AIE)
Create datastore connections
Optional: add attributes to CMDB to hold datastore keys
Dont forget indexes in the CMDB for attributes used as
keys!

Creating a Datastore Connection

93

You must create both a source datastore connection and

target connection (i.e., the CMDB)
In AI Console, click Manage Datastore Connections,
select type as database, enter in required parameters and
click Test Connection
For the target connection select type as CMDB and
provide required information

Manage Datastores
94

Creating a Job
95

Next create a job using the Job Builder Wizard

Click the Create Job icon in the AI Console and click
anywhere in the wizard screen
Enter in Job Name and Description
Select the datastore Type and Name of your source
datastore
Select the CMDB datastore for your target and enter the
CMDB dataset ID
Click Next

Job Builder Wizard - General

96

Creating a Job CI Type

97

From the Select CI Type from, select CI List to see

individual CI classes, or Template to see groupings of CI
and Relationship types by subject area (Application,
Desktop, Virtual, etc.)
Choose your CI types carefully it is much easier to add
them using the wizard than later using Spoon. Plan in
advance what types of CI you will need.
Click Next

Job Builder Wizard CI Types

98

Job Builder Wizard CI Type Templates

99

Creating a Job CI Relationships

100

From the Available CI Types Relationships, select the

relationship types you will need. Again, careful planning
in advance is needed to be able to select the correct
types.
Click Next

Job Builder Wizard CI Relationships

101

Job Builder Wizard - Mapping

102

The CI mapping page appears next, presenting you

with a page per CI Type
Select the table or view to map to, select attributes
or constants and add to the mapping
You must select the field or fields that will be the
Source Key for the mapping
NOTE: you can only select one table per CI type. If
data exists in multiple tables in the source
datastore, create joins
On the Transformation Options page, select a last
modified date and update options if possible

Job Builder Wizard Mapping

103

Job Builder Wizard Relationship Mapping

104

For each relationship type, select how the two CI types

are related in the source datastore.
Key columns in both parent and child table
Key columns in a third association table
The Name attribute of the relationship will be set
according to best practices (see Data Modeling Guide)
Click Next

Job Builder Wizard Relationship Mapping

105

Job Builder Wizard - Schedule

106

The Schedule page appears next. Enter in the time and

interval for the job to run.
Click next and review the job on the Summary page.

Edit Job in Spoon

107

The Job Builder Wizard allows you to quickly build a

job, which can then be edited in Spoon
Certain tasks and data transformations must be built
in Spoon and arent available in the Wizard
Access Spoon on a machine where the Spoon client
is installed or click the Launch Pentaho icon in the
AI Console
Leave the default Repository of NGIEREPD and use
the default login of admin password admin

Edit Job in Spoon

108

Select the Repository menu, the Explore Repository

Expand the folder for your Job SCCM to CMDB
Double-click the SCCM to CMDB job
The job opens

Edit Job in Spoon

109

Edit Spoon Sub Job

110

The data transformation job is broken into steps.

The Set Variables step shows the delta and response
settings selected in the Job Builder Wizard
Double-clicking the Sub_<job name> step shows
the mapping of the delta/response variables.
Right-clicking and selecting Open shows the
transformation contents of the step
For a full list of transformation steps see
documentation. Typical steps

11 Right-click Transfer Calbro data_Computer system and select Open

transformation.

Edit Job in Spoon

system
Transformation
Steps
The computer
transformation is displayed. Allen
Allbrook makes edits to
this transformation in Editing the job in Spoon on page 45.

111

Figure 2-17: Computer system transformation in Spoon

Running the Job in Spoon

112

Save any changes then select the Run icon in Spoon. In

the Execute job window, if Spoon is running on the AI
server, select Local execution (select the AI server if not)
Logging details are displayed in the bottom half of the
window. Debugging information is displayed in the Spoon
logging window as well

Running and Monitoring Job in AI Console

113

You can also view, run or monitor the job in the Atrium
Core Console > Atrium Integrator Console

50

! To monitor the data transfer job

1 From the job catalog of the Atrium Integrator console, select the Transfer Calbro
data job.

AI Console Job

The History tab displays the start and end times for each run of this job, as shown
in Figure 2-23 on page 50.

114

Users Guide

Figure 2-23: History tab of the Atrium Integrator console

Edit and Extend the AI

Wizard Templates
115

You can add relationships types to the AI Wizard:

Create a custom relationship by opening

CMDB:DataModelReference_RelationshipName in New
mode. Enter the name of the relationship class (ex.
BMC_HostedSystemComponent).
In the Value attribute add a name for the relationship (ex.
SYSTEMOS) (see Data Modeling Guide), and the same
value in CDM 1.x Equivalent Class
Enter in the Source CI and Destination CI class names

Edit and Extend the AI

Wizard Templates
116

You can add templates to the AI Wizard:

Open NGIE:BSMTemplate in New Mode and select the

template you want to modify

From the Config items field select the alias names of the
Cis you want to add to the template

Issues and Problems with AI

117

Recommended to apply Atrium CMDB SP2 (or latest SP)

Sometimes editing the AI job in Spoon may break it, and
it wont work from the AI console in Atrium Core Console
(but it may work when run from Spoon)
The AI Wizard limits you to mapping one table to one
class (not many:one)

Questions/Discussion
118

Davin Lindner-Green
Davin.lindner-green@raptek.com
www.raptek.com
BMC Level 1 Support Partner
BMC Software Solutions Partner
BMC Education Partner
Remedy On Demand - Hosted ITSM Solutions

2011 World Wide Remedy Users Group. All Rights Reserved.