Copyright 2013

Wellesley Information Services, Inc.

All rights reserved.
Lessons for Using SAP
Data Services to Build a
Data Mart That Rapidly
Delivers Consistent
Information to Reports,
Dashboards, and Mobile
Devices
Don Loden
Decision First Technologies
1
In This Session
We will discuss an end-to-end solution of design, development,
and delivery for a Business Intelligence (BI) data model where
content is delivered to both a laptop dashboard and iPad
The focus will be on:
Back-end data design for optimum storage and delivery
Front-end user consumption components
Throughout this process, maintaining absolute data
synchronization is at the forefront of criticality while enriching
end-user experience and timeliness of the data follow suit
2
What Well Cover
Key learning points
Design approach
Data sources, development, and synchronization
Presentation of dashboard
The security model integration
Best practices for implementation
Wrap-up
3
Learning Points
Architect data delivery mechanisms to ensure compatibility and
precision between two different user interfaces:
SAP BusinessObjects Dashboards (formerly Xcelsius)
Mobile iPad application
Combine the following:
SAP BusinessObjects Enterprise (BOE) Web service data
Complex user security and multi-variable calculations
Provide drill-through functionality with stacked column bar
charts in SAP BusinessObjects Dashboards
Use advanced features of universe design, SAP BusinessObjects
Web Intelligence, BOE security to apply row-level security of data
against hierarchical user data
4
What Well Cover
Key learning points
Design approach
Data sources, development, and synchronization
Presentation of dashboard
The security model integration
Best practices for implementation
Wrap-up
5
Design Approach (High Level)
Deliver the sales force sales data from merging a manufacturing
ERP and third-party pricing data to a dimensional data model to
be consumed by SAP BusinessObjects Dashboards
Collapse the dimensional data model down and be able to
simplistically store the same data on an iPad
Be able to deliver a finite list of products to recommend based on
specific business rules
6
Design Approach (Recommended Products
Mockup)
Recommend Products Report
Semi-Cartesian product of: Product Type, Style, Ranking,
Brand, Back, Price, New Product Indicator, Weight, Color,
Density, Texture, Pattern
Summed the Sales Dollars
Sorted based on mandated criteria
7
Design Approach (Source System Analysis)
Use the Data Services Profiler to analyze the ERP products, sales,
customers, sales force hierarchy (Source System Analysis [SSA])
Also look into the new Pricing system
Needed to ensure 1:1 match from ERP into Pricing
The Data Services Profiler provides such information as the
number of distinct values, min values, max values, patterns, nulls,
and zeros
Output from the SSA provides guidance for final data model
design
8
Design Approach iPad Dashboard Data
Model Design
Consolidated data model (non-star) to three tables:
Dealer Sales
Recommended Products
Benchmark Inputs (estimates of expected purchases from that
customer)
9
Design Approach Laptop Dashboard Data
Model Design
Dimensional data model for the reporting data mart
Conducive to dashboard design for dropdowns, prompts, and
inputs (distinct lists from dimensions)
Star schema (fact[s] in the center with surrogate keys to
dimensions)
3 Fact Tables: Transactions (sales, prices, etc.)
14 Dimensions: Means to slice (customers, products, etc.)
Utilize the following SAP BusinessObjects technologies:
Universe
Web Intelligence
Web Services (BI Services and QaaWS)
Dashboards
Data Services
10
See SAP Data Services Profiling in Action!
11
What Well Cover
Key learning points
Design approach
Data sources, development, and synchronization
Presentation of dashboard
The security model integration
Best practices for implementation
Wrap-up
12
Data Sources
Manufacturing ERP
Sales Force
Sales (stated invoices)
Customers (dealers)
Products
Entitlements (who can buy what and lowest available prices)
Pricing System
Agreed upon pricing for customers and products
Different than entitlements (above)
Active Directory
To look up salesman user names and enforce row-level security
13
Development Process
Dual-layer approach for data delivery to iPad model and
Dashboard model:
Source data, perform transformations along the way, generate
surrogate keys, and place it into a STAGE environment
Heavy lifting done in the STAGE build
Intense processing and time consuming to rebuild
Allowed back-end ETL development to occur and not impact
the Data Mart
Changes to the Data Mart can be pushed out at night
From there, provide data to DATA MART environment
(replication of STAGE tables)
Simultaneously, consolidate the STAGE tables into the three
needed for the iPad model
14
Development Change Data Capture
Needed a way to determine changes in data
iPad supported three data tables
And one Notification table
Rudimentary Type II slowly changing dimensions in data
warehousing
Needed a way to generate notifications amongst data
This record is new, generate a notification for ADD
This record changed, generate a notification for CHANGE
This record isnt in the input dataset, generate a DELETE
15
Development Change Data Capture (cont.)
Wanted to leverage the transforms within Data Services:
Table Comparison transform: Looks into a table and
determines if a record has changed
If the record that the Table Comparison is looking for is not
found, the input record is marked as INSERT
If the record that the Table Comparison is looking for is
found, the Table Comparison begins to look to see if
anythings changed
If there is no change, the record is discarded
If there is a change, the record is flagged as UPDATE
If the record is not present on the input, but does exist in the
target, the record (in the target) is marked DELETE
16
Development Change Data Capture (cont.)
Delete all data from the Notification Table
Well see why shortly
The source STAGE tables come together in the Query transform
(join on SKEYs, filter, etc.)
The Table Comparison looks into the History table:
First execution, the History table is empty (no history)
Subsequent executions, there is a historical reference to
compare against
17
Development Change Data Capture (cont.)
If the record is not found in the History table, its flagged as
INSERT
A copy also goes down to the Map_Operation transform
The Map_Operation transform manipulates these INSERT,
UPDATE, and DELETE statements
18
Development Change Data Capture (cont.)
If a record is found in the History table, the Table_Comparison
begins to compare the data values (price, style, etc.)
If theres no change, the record is discarded
If there is a change, the record is:
UPDATED in the History table (making the History table be
current)
The Map_Operation changes the UPDATE to INSERT
(because the Notification table is empty each execution)
19
Development Change Data Capture (cont.)
Each newly inserted record into the Notification table also has the
change code (ADD, CHANGE, DELETE) on the record:
Now we can publish these notifications to the iPad with the data
Op Code Val1 Val2 Val3
ADD Y0 CLR Y
CHANGE Y1 C2 N
CHANGE Y9 BLU N
DELETE Y11 RD Y
20
Development Change Data Capture (cont.)
The data needed to be delivered(and the notification) to the iPad
by a certain time each day (5 am EST)
21
Walk Through the SAP Data Services ETL Code
22
Development Performance
Large data sets
5M products
450K customers
80M entitlements (semi-Cartesian product developed)
8M recommended products
Not all of this data will fit on an iPad
Devise sorting and ranking to pick top qualities of products
to recommend for a salesmans customers
23
Development Performance (cont.)
Make use of Data Services database pushdown capabilities
The database engine can crank through a local data set better
than the ETL engine pulling that set across the network
Database I/O is the biggest bottleneck
Ensure use of physical hardware
For ETL engine
For database servers (MS SQL 2008R2)
Ensure fresh indexes (as needed)
Index rebuild tasks
Can be performed by database server job or in SAP
24
Development Flow Diagram
25
Ensuring Data Synchronization
Ensure the iPad model (three tables) matches that of the Data Mart
Since both were built off the same data set (STAGE), place
audits on both models to ensure sales figures and prices were
in alignment
Could be done with SAP Data Services Auditing or manually
Manual tables and processes provide more control
Do not interfere with performance
Do not update data in the event that there was a mismatch (use
stale data)
26
What Well Cover
Key learning points
Design approach
Data sources, development, and synchronization
Presentation of dashboard
The security model integration
Best practices for implementation
Wrap-up
27
Presentation of the Dashboard Opening
Screen
28
Presentation of the Dashboard
Sales by Construction
29
Presentation of the Dashboard (cont.)
Some special considerations to support migration between
environments
Ensure to use Lifecycle Manager to manage components within
SAP BusinessObjects Enterprise (BOE)
This will maintain the CUIDs for objects
30
What Well Cover
Key learning points
Design approach
Data sources, development, and synchronization
Presentation of dashboard
The security model integration
Best practices for implementation
Wrap-up
31
The Security Model Integration
Could pull the salesman ID (e.g., a number) out of the ERP
Lookup to MS Active Directory
Use stored procedure within MS SQL Server called from SAP
Data Services
Leverage single sign-on (SSO) within SAP BusinessObjects (for
laptop dashboard)
32
The Security Model Integration (cont.)
Security handled for the iPad elsewhere
Need to provide the user name for the salesman within each of
the iPad tables (for filtering)
Need to restrict who can see what (e.g., one salesman should not
be able to see anothers sales/pricing, etc.)
Leverage row-level security model
Inner join amongst data model
Dimensions held sales force hierarchy (salesman reports to
regional manager who reports to district manager, etc.)
NET_ID = @Variable(BOUSER)
33
What Well Cover
Key learning points
Design approach
Data sources, development, and synchronization
Presentation of dashboard
The security model integration
Best practices for implementation
Wrap-up
34
Best Practices for Implementation
Align business requirements with tool capabilities and reduce
complexity in dashboard design
Optimize integration of data sources and data model to enhance
dashboard performance
Flattening data to make all of the data available with one read
Aggregating the data to provide a very high level of granularity
for presentation in a dashboard
Data delivery mechanism must ensure compatibility and precision
between various different user interfaces
Audits are important!
Use advanced features of universe design, WebI, and BOE
Administration to apply row-level security of data against
hierarchical user populations
35
What Well Cover
Key learning points
Design approach
Data sources, development, and synchronization
Presentation of dashboard
The security model integration
Best practices for implementation
Wrap-up
36
Wrap-Up
The data model(s) should drive the technology being used
SAP BusinessObjects universes work best on a star schema
Consolidating that model into a few tables to be stored on a
mobile device, the iPad
When dealing with large data sets, ensure that database
pushdown from the ETL is maximized
Separate back-end and front-end development tasks (if possible)
Helps to not pull the rug out from underneath the front end
during data reloads
If possible, build in the security model such that a user of such
secure data wouldnt need another housekeeping task of
managing a user name and password
Leverage the use of MS Active Directory and SSO within BOE
37
Where to Find More Information
http://scn.sap.com/docs/DOC-26314
Bashir Arwan, BusinessObjects 4.0 Single Sign On (SSO) with
WIN AD (SAP Community Network, 2012).
Configuration of SSO for BOE
http://help.sap.com/businessobject/product_guides/boexir3/en/xi3
_ds_performance_optimization_guide_en.pdf
Data Services Performance Optimization Guide (Business
Objects, 2008).
Documentation with Data Services to ensure that data flows
are achieving the best/most database pushdown
38
Where to Find More Information (cont.)
http://blog.tech-cats.com/2007/09/querying-active-directory-
through-sql.html
Boyan Kostadinov, Querying Active Directory Through SQL
Server (Aaquagua! Blog, 2007).
Technical configuration of MS SQL Server to query AD
https://scn.sap.com/message/9875562
OpenDocument functionality for launching Web Intelligence
reports from dashboards thread on SAP Community Network
39
7 Key Points to Take Home
Back-End Considerations
Ensure as much database pushdown as possible (for
performance)
Make use of default records in all dimension tables
Avoid expensive outer joins
Make every query an inner join!
Provide the same base model, STAGE, for both the dashboards
reporting (the Data Mart, which was a replication of the STAGE
model) and the iPads delivery (three consolidated tables)
Single sign-on (SSO) can remove the headache with
maintaining SAP BusinessObjects Enterprise users
40
7 Key Points to Take Home (cont.)
Front-End Considerations
Push down data aggregation, calculations, formulas to simplify
code, isolate integration components, and facilitate ease of
maintenance
Excel logic Web Services logic Web Intelligence
Universe Database
Minimize dashboard navigation, dynamic visibility, and drill-
through to enhance usability and reduce overall integration
complexity
When conducting Quality Assurance and User Acceptance
Testing, Lifecycle Manager (LCM) effectively migrates SAP
BusinessObjects Enterprise dashboard components by
maintaining adherence to object CUIDs between environments
41
Your Turn!
How to contact me:
Don Loden
don.loden@decisionfirst.com
@donloden
Please remember to complete your session evaluation
42
Disclaimer
SAP, R/3, mySAP, mySAP.com, SAP NetWeaver

, Duet

, PartnerEdge, and other SAP products and services mentioned herein as well as their
respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and
service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor controlled by SAP.




















Wellesley Information Services, 20 Carematrix Drive, Dedham, MA 02026
Copyright 2013 Wellesley Information Services. All rights reserved.