mySAP BI SAP BW 3.

0 Openess and Connectivity

Rainer Uhle
BW Product Management SAP AG

Topics

BW Connectivity - Openess BW Extraction from R/3 BW Dealing with XML Formats

BW DataBase Connect SAP BW & Ascential DataStage ETL BW Open Hub Service

SAP AG 2002, SAP Skills 2002

I1_Uhle

The Organizational Challenge

Provide each organizational unit or better each role with the needed reliable, consolidated, integrated, actual and historical information
e.g. at business unit level provide a: local/ subsidary view

regional view global view

at headquarter level across business units provide regional view global view

SAP AG 2002, SAP Skills 2002

I1_Uhle

Business Roles and Related Content are Key

BW

SAP AG 2002, SAP Skills 2002

I1_Uhle

Open Data Warehouse Architecture

SAP AG 2002, SAP Skills 2002

I1_Uhle

Open Data Warehousing with SAP BW

T Transparency and T T T T T T

manageability of all loading processes Flexible set of ETL capabilies Open to ETL-tools Close integration of Ascential DataStage Application level ETL from SAP systems XML feeds (open standards) and flat files DB connect for legacy data

SAP AG 2002, SAP Skills 2002

I1_Uhle

BW Architecture: Layers & Accessiblity

Operational Data Store Operational Reporting Near Real-Time / Volatile Granular Built with ODS Objects

Data Warehouse Non volatile Granular Historical foundation Integrated Built with ODS Objects
7

Multidimensional Model

Multidimensional analysis Aggregated view Integrated Built with InfoCubes

SAP AG 2002, SAP Skills 2002

I1_Uhle

Accessiblity

XML/A XML for Analysis Based on HTTP/SOAP Any platform

ODBO OLE DB for OLAP Based on COM Windows platform only

BAPI OLAP BAPIs Based on RFC Access library on any SAP platform

SAP AG 2002, SAP Skills 2002

I1_Uhle

Open Hub Service - Details

BW Server Relational Table
InfoCubes

External DataMart ... ...

T

Flat File

Controlled distribution of consistent data Target: file or DB table Central monitoring Select filter criteria and columns Scheduling Full or delta mode

T T

Master Data

ODS Objects

SAP AG 2002, SAP Skills 2002

I1_Uhle

Topics

BW Connectivity - Openess BW Extraction from R/3 BW Dealing with XML Formats

BW DataBase Connect SAP BW & Ascential DataStage ETL BW Open Hub Service

SAP AG 2002, SAP Skills 2002

I1_Uhle

10

Types of Extractors
BW Extractors

Application Specific Extractors

Generic Extractor

BW Content DataSources
FI HR LO Cockpit CO ...

Generated DataSources
LIS CO-PA FI-SL ...

BW Content DataSources

Generated DataSources

Database View SAP Query

Business Content DataSources Business Content DataSources

T T T T T T T T T T

Build in Extractors Build in Extractors High coverage (Applications, Industries) High coverage (Applications, Industries) Transaction -- and Master Data Transaction and Master Data Ready to run Ready to run Reduce implementation efforts and costs Reduce implementation efforts and costs
I1_Uhle 11

Creation of own DataSources Creation of own DataSources

T T T T

Possibility to generate own Possibility to generate own DataSources in Application DataSources in Application Generic extractor tool for customer Generic extractor tool for customer specific tables or applications specific tables or applications

SAP AG 2002, SAP Skills 2002

Logistic Extraction for BW Delta Mode

R/3 OLTP-System
Dialoguesession Backgroundprocessing

Update Queue
Process 1 Store Document in DB Save Document Process 2 Store Session Data (Document + Context) for BW needs

Document Data

Delta Queue

-MANAGEMENT
-------------------------------------------------------

Delta-Extraction

SAP AG 2002, SAP Skills 2002

I1_Uhle

12

Logistic Extraction Batch Set Up Scenario

R/3 OLTP-System
Extract Structures
MC11VA0HDR MC11VA0ITM MC11VA0SCL
LIS Communication Structure

MC11V_0ITM MC11V_0SCL
.....

MC11V_0ITM_SETUP

Document Data

MC11V_0SCL_SETUP

Setup-Tables

SAP AG 2002, SAP Skills 2002

I1_Uhle

13

Extracting Transactional Data

R/3 OLTP-System

Business Information Warehouse

Full Upload with Delta Initialization

Setup-Tables

Cubes

Data-/InfoSource

-MANAGEMENT
----------------------------------------------------

Delta Upload

ODS/PSA

Delta-Extraction

SAP AG 2002, SAP Skills 2002

I1_Uhle

14

Service API 3.0B Zero Downtime Service

Zero-downtime during initial load / Early-delta initialization In logistics, you can achieve zero-downtime during the initial load: data can already be written to the delta queue at the same time as data is transferred from the setup tables. A flag in the scheduler (as of 3.0A SP7) controls this feature.

Zero-downtime prior to and during initial load using a mirror source system The Service API zero downtime service allows you to fill the setup tables in a mirror system, while posting continues in the original source system and and fills the Delta queue. After the setup tables have been filled and the initial load from the mirror system into the BW has been completed, the status information is synchronized with the original source system. Delta transfer takes place from the original source system now into BW.

SAP AG 2002, SAP Skills 2002

I1_Uhle

15

Topics

BW Connectivity - Openess BW Extraction from R/3 BW Dealing with XML Formats

BW DataBase Connect SAP BW & Ascential DataStage ETL BW Open Hub Service

SAP AG 2002, SAP Skills 2002

I1_Uhle

16

Scenario for Integrating XML Data Sources

Push of document-type data in delta queue of corresponding DataSource within the BW server BW

External application
document-type document-type data data
CRM

InfoSource

SOAP SOAP rfc rfc Service Service push XML data

BW DataSource

File DataSource

Delta

pull

Init Full

mass data

Delta queue FTP (non XML format)

SAP AG 2002, SAP Skills 2002
I1_Uhle 17

File system

SOAP Body for HTTP Post Request

Example for HTTP/SOAP body (DataSource Test)
<?xml version="1.0" ?> <SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/"> <SOAP:Body> <rfc:_-BIC_-QITEST_RFC xmlns:rfc="urn:sap-com:document:sap:rfc:functions"> <DATASOURCE>TEST</DATASOURCE> <DATA> <item> <VENDOR>Smith</VENDOR> <MATERIAL>soap</MATERIAL> <DATUM>20010815</DATUM> <UNIT>KG</UNIT> <AMOUNT>1.25</AMOUNT> </item> <item> <VENDOR>Fritz</VENDOR> <MATERIAL>body lotion</MATERIAL> <DATUM>20010816</DATUM> <UNIT>ml</UNIT> <AMOUNT>150</AMOUNT> </item> </DATA> </rfc:_-BIC_-QITEST_RFC> </SOAP:Body> </SOAP:Envelope>

SAP AG 2002, SAP Skills 2002

I1_Uhle

18

Data Flow: Generic Staging

SOAP compliant HTTP Service

BW Server
Country DE City Hamburg

Ext. Application
XML parsing

Delta queue Transformation (XSL) PSA

Not available so far

Marketplaces

<Country>DE</Country> <City>Hamburg</City>

Java services www

Commercial layer, xCBL

ODS, InfoCubes, Master Data

SAP AG 2002, SAP Skills 2002

I1_Uhle

19

File DataSource as Entry Point for SOAP Processing

Create BW-DataSource with SOAP-Interface

Field names given from XML data format have to be added manually

SAP AG 2002, SAP Skills 2002

I1_Uhle

20

Pushing Data into the Queue via HTTP

<?xml version="1.0" ?> <SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/"> <SOAP:Body> <rfc:_-BIC_-QIURGETXML_RFC xmlns:rfc="urn:sap-com:document:sap:rfc:functions"> <DATASOURCE>URGETXML</DATASOURCE> <DATA> <item> <VENDOR>Otto</VENDOR> <MATERIAL>Seife</MATERIAL> <DATUM>20010104</DATUM> <UNIT>KG</UNIT> <AMOUNT>1234567</AMOUNT> </item> <item> <VENDOR>Fritz</VENDOR> <MATERIAL>Rasierwasser</MATERIAL> <DATUM>20010213</DATUM> <UNIT>ml</UNIT> <AMOUNT>100</AMOUNT> </item> </DATA> </rfc:_-BIC_-QIURGETXML_RFC> </SOAP:Body> </SOAP:Envelope>

SAP AG 2002, SAP Skills 2002

I1_Uhle

21

DeltaQueue Entries after http Request

SAP AG 2002, SAP Skills 2002

I1_Uhle

22

Topics

BW Connectivity - Openess BW Extraction from R/3 BW Dealing with XML Formats BW DataBase Connect

SAP BW & Ascential DataStage ETL BW Open Hub Service

SAP AG 2002, SAP Skills 2002

I1_Uhle

23

DB Connect (BW3.0B) - Scenario

SAP BW

Application ORACLE DataBase Unix

Purchasing

DB-Connect

MS SQL DataBase

NT

A purchasing application runs on a legacy system based on an ORACLE database. To analyze the data from the purchasing application, the data needs to be loaded into the BW System (possibly installed on a different database, e.g. MS SQL-Server). DB-Connect can be used to connect the DBMS of of the purchasing application and extract data from the tables or views.
SAP AG 2002, SAP Skills 2002
I1_Uhle 24

DBConnect as part of the Open Data Warehouse Architecture

SAP AG 2002, SAP Skills 2002

I1_Uhle

25

DB Connect - Architecture
SAP BW External Database Server
Database Database Catalog Catalog
List of Tables + Views Field Details

DataSource

InfoSource

Data

Staging Engine

Direct access to external RDBMS Read metadata and data Access to DBMSs supported by mySAP Technology
SAP AG 2002, SAP Skills 2002
I1_Uhle 26

DB Connect Architecture
SAP BW 3.0 Application Server SAP DB-Interface SAP DB MultiConnect DB Connect Implementation SAP Basis 6.10 / 6.20 BW SAP DBSL for ORA DB-Client for ORA SAP DBSL for MSSQL DB-Client for MSSQL MSSQL Database Management System
27 28

Oracle Database Management System

SAP AG 2002, SAP Skills 2002

I1_Uhle

DB Connect Supported Source DBMS

DB Connect enables data to be extracted from the following database systems supported by mySAP technology

DB2 UDB DB2/400 DB6 Informix MS SQL Server 7.0 & MS SQL Server 2000 Oracle 8.1.7.3+ SAP DB

SAP AG 2002, SAP Skills 2002

I1_Uhle

DB Connect - Prerequisites

T SAP BW 3.0B T Table names and field names have to be in capital letters T Usage of views on top of tables is recommended T The SAP specific part of the database interface, the DBSL

(database shared library) for the source DBMS, has to be installed on the application server. It can be downloaded from the SAP Service Marketplace. It is licensed and delivered to customers solely for use as part of the SAP BW. source MS SQL

T In some cases you need a Windows based AppServer, e.g. for T The database specific DB-Client for the source DBMS has to be
installed on the application server. It has to be licensed by the customer from the DB vendor and is delivered as part of the database access license.

SAP AG 2002, SAP Skills 2002

I1_Uhle

29

Topics

BW Connectivity - Openess BW Extraction from R/3 BW Dealing with XML Formats BW DataBase Connect SAP BW & Ascential DataStage ETL

BW Open Hub Service

SAP AG 2002, SAP Skills 2002

I1_Uhle

30

Complementary Software Partners (CSP)

Open interface to the staging engine

For 3rd party extraction tools and For customers (for own extraction programs) Exchange meta data Import of transcation data Import of master data

SAP AG 2002, SAP Skills 2002

I1_Uhle

31

SAP - Ascential Partnership

DataStage is SAPs preferred data integration technology. SAP signed Reseller Agreement with Ascential SAP resells DataStage and DataStage Load PACK for SAP BW. SAP and Ascential working together in Sales & Marketing, Engineering & Support Ascential Engineer at SAP in Germany .

SAP AG 2002, SAP Skills 2002

I1_Uhle

32

SAPs Offering

Development

API

R/2
Legacy Systems

Non SAP Non SAP

SAP BW

Business

API

^ ^
VSAM ADABAS
RDBMS (DB2)

File
E-Business Applications
Oracle S I IEEB EELL S B PeopleSoft

Data DataS Sttag age E e ETL TL C Com ompo pone nents nts

PlugIns

BW Load Pack BW Load Pack

XML Pack

XML Pack

ts s onen t nen omp o mp TL C o LC ET ge ttage E S a Datta S Da a

Appl. Spec. Extr.Pack FileInterface ODBC FileInterface ODBC

SAP AG 2002, SAP Skills 2002

I1_Uhle

33

SAP BW Load PACK Capabilities

DataStage Load PACK for SAP BW

T T T T T T T T

Meta Data Browser allows you to browse and select SAP BW Source System, InfoSource and Transfer Structure meta data RFC Server listens and responds to SAP BW request for data Uses standard SAP Business APIs Extracts meta data from SAP BW Validates meta data match Sequence, launch and monitor all job steps from SAP BW SAP Certified Solution National Language Support (multi-byte)

SAP AG 2002, SAP Skills 2002

I1_Uhle

34

Testing

Productive

BW Service API for SAP BW Service API for SAP

Service

SAP BW Load PACK Release 3.0 Enhancements

Available June 2002

Enhanced Meta Data Integration - Browse, Select, Create & Change SAP BW Meta Data Objects from DataStage
certified

T T T T T

InfoCatalogs Source Systems, including generate Source System and assignment of program ID for launching RFC servers InfoSources, including master and transaction data, and Source System assignment InfoObjects, including characteristics, hierarchies, key figures and units InfoPackages, including enhanced DataStage parameter & value selection from SAP BW, and invoking InfoPackage from DataStage

SAP AG 2002, SAP Skills 2002

I1_Uhle

35

BW Certification Levels for CSP Partners

Certification Level 1.2 2.0 4.0

Staging BAPIs

Staging& Metadata BAPIs

no significant BAPI development

Staging& Metadata& ..... BAPIs BAPIs

BAPIs

BAPIs

BAPIs

BW 1.2
SAP AG 2002, SAP Skills 2002
I1_Uhle

BW 2.0
36

BW 3.0

BW 4.0

DataStage Packs General Requirements

SAP BW
Foreign Appl. Foreign Appl. Access via Application Extract Packs

DataStage Job BW Load Pack

BAPI

RDBMS

Operational Data
RDBMS

Direct Access

MetaData Business Objects Delta Capabilities Business Content

SAP AG 2002, SAP Skills 2002

I1_Uhle

37

Example for Application Packs

Siebel PlugIn (Siebel Releases 6.3/2000; 7.0/2001)

EIM capabilities are used to access the underlying datamodel via the Business Object level (220 Business Objects?)
S Step 1: Set up a EIM configuration file S Step 2: Start EIM to slect the data needed S Step 3: ODBC Extraction of that EIM Table

Direct Access capabilities give access the underlying datamodel directly via DDIC information from Siebel (list of tablenames)
S Browsing Tables, selecting Columns and Fields generates SQL Statements that can be

enhanced easily to views, joins, projections on the relevant subset of DB tables

Delta Capabilities are given last date of jobrun can be passed to EIM as selection criteria for step 2 (last update ge last date of jobrun, last update field very konsistent and reliable within Siebel Applicatipons)

SAP AG 2002, SAP Skills 2002

I1_Uhle

38

EIM Plug-in Operations

Operating system
DATABASE

3. Extract data

Siebel OLTP Tables

Siebel EIM Tables

2. Run EIM
EIM Component
EIM Configuration File

1. Config file created.

SAP AG 2002, SAP Skills 2002

I1_Uhle

39

EIM Tab

SAP AG 2002, SAP Skills 2002

I1_Uhle

40

EIM Configuration file

[Siebel Interface Manager] TABLEOWNER = SIEBEL PROCESS = ds_auto_process [ds_auto_process] TYPE = Export BATCH = 12 TABLE = EIM_ORDER_ITEM USE INDEX HINTS = False CLEAR INTERFACE TABLE = True EXPORT ALL ROWS = False EXPORT MATCHES = (LAST_UPD > '1970-01-01 00:00:00' and LAST_UPD <= '200110-24 09:29:29')

EIM Component

EIM Configuration File

SAP AG 2002, SAP Skills 2002

I1_Uhle

41

Direct Access to Siebel: Selecting a Table

SAP AG 2002, SAP Skills 2002

I1_Uhle

42

Direct Access to Siebel: Selected Columns

SAP AG 2002, SAP Skills 2002

I1_Uhle

43

Direct Access to Siebel: Delta Tab

SAP AG 2002, SAP Skills 2002

I1_Uhle

44

Direct Access to Siebel: SQL Tab

SAP AG 2002, SAP Skills 2002

I1_Uhle

45

Example for Application Packs

PeopleSoft PlugIn (PS Releases 7.5; 8.0; 8.3)

T T T T

Add Business View functionality accesses the underlying datamodel via the Business Object level (using Panels and Panel Groups) Add Table View functionality accesses the underlying datamodel directly via DDIC information from PS (list of tablenames)
Browsing Tables, selecting Columns and Fields generates SQL Statements that can be enhanced easily to views, joins, projections on the relevant subset of DB tables Delta Capabilities are given automatic search for effective Date fields within PS (Type: Timestamp) automatic creation of a delta-file with the date of the last jobrun

SAP AG 2002, SAP Skills 2002

I1_Uhle

46

Ready fr Success

FI, MM,APO, . .run on mySAP.com

Web application used for your B2B procurement PeopleSoft is in use for HR

Local offices use Excel for budget planning Siebel is in use for CRM Purchasing runs on a legacy sys.

SAP AG 2002, SAP Skills 2002

I1_Uhle

47

Topics

BW Connectivity - Openess BW Extraction from R/3 BW Dealing with XML Formats BW DataBase Connect SAP BW & Ascential DataStage ETL BW Open Hub Service

SAP AG 2002, SAP Skills 2002

I1_Uhle

48

Open Data Warehouse Architecture

SAP AG 2002, SAP Skills 2002

I1_Uhle

49

Information Distribution - Scenarios

Analytical Application
SAP BW SAP BW

Sales Data Mart

Regional Data Mart

Legacy Application

SAP BW

Open Hub Service

BW Open Hub Service

BW as an Information Hub Controlled Distribution of consolidated and integrated data

SAP AG 2002, SAP Skills 2002

I1_Uhle

50

Open Hub Service - Details

z z z z z z
(Data + Metadata)

Controlled distribution of consistent data Target: file or DB table Central monitoring Select filter criteria and columns Scheduling Full or delta mode

ODS Objects

SAP AG 2002, SAP Skills 2002

I1_Uhle

51

Functional Details

T T

Asynchronous data transfer for mass data New meta object InfoSpoke describes

T T

Data sources: IC, ODS-Object, attributes, texts, hierarchies Full and Delta (Delta for InfoCubes and ODS objects ) Logical Target system, e.g. Data Mart US Distribution targets: File, BW table Format: CSV Selection criteria Selected columns Transformation via BAdI (optional)

Scheduling Integrated in process chains (process type InfoSpoke)

Monitoring by application log

SAP AG 2002, SAP Skills 2002

I1_Uhle

52

Model: Design-Time

BW
Variante Scheduler
Periodicity, Starting condition Selections for data source

Process
Reference to Data Source Reference to Transformation Reference to Data

Extern

Data Source
Metadata, Selectabilty, Data

Transfomation

Data Target
Metadata, Data

SAP AG 2002, SAP Skills 2002

I1_Uhle

53

Model: Run-Time

BW
Request
Selections for data source

Request Broker
Reference to Data Source Reference to Transformation Reference to Data

Log/Monitor

Extern

Data Source
Data

Data Target
Datapackage(in) Datapackage(out)
Data

Transformation
SAP AG 2002, SAP Skills 2002
I1_Uhle 54

Definition of InfoSpoke (1)

A
InfoSpoke Header

Source Destination Description

B
Destination

Destination Type Description

SAP AG 2002, SAP Skills 2002

I1_Uhle

55

Definition of InfoSpoke (2)

C
Field List

Take over Data

Source fields

SAP AG 2002, SAP Skills 2002

I1_Uhle

56

Definition of InfoSpoke (3)

D
Selection criteria

Selections for all

characteristics of Data Source

SAP AG 2002, SAP Skills 2002

I1_Uhle

57

Open Hub Monitor - Overview

SAP AG 2002, SAP Skills 2002

I1_Uhle

58

Open Hub Monitor Single Request

SAP AG 2002, SAP Skills 2002

I1_Uhle

59

Summary

T Openness to extract data from BW T All BW data objects are supported for extraction T Delta tracking for each target system T Scheduling and central monitoring
SAP BW can serve as an enterprise information hub and feed data to downstream systems

SAP AG 2002, SAP Skills 2002

I1_Uhle

60

Outlook

T T T T

Support XML as export format Enable validation of the extract, e.g. check records or generate audit/control report Push the data to a program (RFC-Server) Extract based on a query

SAP AG 2002, SAP Skills 2002

I1_Uhle

61

Open Hub Pricing Issues

What the price list says about Open Hub:

Open Hub has to be licensed whenever data is being extracted and transferred from mySAP BI into 3rd party target systems. The communication between multiple SAP BWs does not require Open Hub.

Note: The term Open Hub here has a broader scope, than the software functionality Open Hub Service

SAP AG 2002, SAP Skills 2002

I1_Uhle

62

Warehouse Management with SAP BW

T DBMS independent (Oracle, MS, T T T T T T T T

IBM, Informix) Scalable architecture Mass volume capable Consistent meta data (including business definitions) Extensive monitoring and error tracking Data Quality checks Multi-lingual Master Data Open Hub Services Openess to multiple sources

SAP AG 2002, SAP Skills 2002

I1_Uhle

63

Copyright 2002 SAP AG. Alle Rechte vorbehalten

T T T T T T T T T T T T
Copyright 2002 SAP AG. Alle Rechte vorbehalten. Weitergabe und Vervielfltigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die aus-drckliche schriftliche Genehmigung durch SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen knnen ohne vorherige Ankn-digung gendert werden. Die von SAP AG oder deren Vertriebsfirmen angebotenen Softwareprodukte knnen Softwarekomponenten auch anderer Softwarehersteller enthalten. Microsoft, WINDOWS, NT, EXCEL, Word, PowerPoint und SQL Server sind eingetragene Marken der Microsoft Corporation. IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, Informix und Informix Dynamic ServerTM sind Marken der IBM Corporation in den USA und/oder anderen Lndern. ORACLE ist eine eingetragene Marke der ORACLE Corporation. UNIX, X/Open, OSF/1 und Motif sind eingetragene Marken der Open Group. Citrix, das Citrix-Logo, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, MultiWin und andere hier erwhnte Namen von Citrix-Produkten sind Marken von Citrix Systems, Inc. HTML, DHTML, XML, XHTML sind Marken oder eingetragene Marken des W3C, World Wide Web Consortium, Massachusetts Institute of Technology. JAVA ist eine eingetragene Marke der Sun Microsystems, Inc. JAVASCRIPT ist eine eingetragene Marke der Sun Microsystems, Inc., verwendet unter der Lizenz der von Netscape entwickelten und implementierten Technologie. SAP, SAP Logo, R/2, RIVA, R/3, SAP ArchiveLink, SAP Business Workflow, WebFlow, SAP EarlyWatch, BAPI, SAPPHIRE, Management Cockpit, mySAP, mySAP.com und weitere im Text erwhnte SAP-Produkte und Dienstleistungen sowie die entsprechenden Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und anderen Lndern weltweit. MarketSet und Enterprise Buyer sind gemeinsame Marken von SAP Markets und Commerce One. Alle anderen Namen von Produkten und Dienstleistungen sind Marken der jeweiligen Firmen.

SAP AG 2002, SAP Skills 2002

I1_Uhle

64