DSAG TT 2013 Hahne Consulting V12 PDF

SAP BW Near-line
Storage (NLS)
Dr. Michael Hahne
Hahne Consulting GmbH
February 27, 2013

Disclaimer
This presentation outlines SAP’s general product direction and should not be relied on in
making a purchase decision. This presentation is not subject to any license agreement or
any other agreement with SAP. SAP has no obligation to pursue any course of business
outlined in this presentation or to develop or release any functionality mentioned in this
presentation. This presentation and SAP's strategy and possible future developments are
subject to change and may be changed by SAP at any time for any reason without notice.
This document is provided without a warranty of any kind, either express or implied,
including but not limited to, the implied warranties of merchantability, fitness for a particular
purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in
this document, except if such damages were caused by SAP intentionally or grossly
negligent.
©  26.02.13
2012 SAP AG. All rights reserved. Copyright Hahne Consulting GmbH 2013 22
Agenda –
SAP BW Near-line Storage (NLS)
n  What is Near-line Storage (NLS)?

n  History, Strategy, Definition, Differentations …
n  What is the NLS Interface in SAP Business Warehouse (BW)?

n  Architecture, Functionality, Modeling Objects, Designtime vs. Runtime ...
n  What is an Implementation for the SAP BW NLS Interface?

n  Architecture, Partner Solutions, Differentiators, License Model ...
n  What‘s new with the SAP owned BW NLS Implementation for Sybase IQ?
n  Benefits, Realization, Availability, Piloting, Licensing, Roadmap ...
©  26.02.13
Data-Aging Strategies resp. Information Life Cycle
Definition of Near-line Storage (NLS) for SAP BW
Storage Type / Near-Line Storage Classic Archive

Online Database
Data Category (read only) (read only)
Frequently read /
changed data
(actual)

Infrequently read
Near-line Storage (NLS)
data (mature) n alternative storagetype with SQL based direct access
capabilities for reporting, ETL, and look-ups
n  extracted NLS partitions are deleted in RDBMS (NLS
Very rarely read data
(aged)   
storage and Online Storage together consistently
reflect the BW data persistency of an InfoProvider)
n  NLS storage is much more cheaper than DB storage
n  NLS partitions of an InfoProvider are write-protected
(!)
n  NLS is primarily partitioned by time/age
©  26.02.13
Design Aspects for a Classic BW Environment –
Near-line Storage (NLS) vs. BW Accelerator (BWA)
BW
InfoMarts (InfoCube)
ADK Archive
BWA Acceleration Archiving Near-line Storage
Acquisition
RDBMS
Access - very frequently frequently not frequently rarely
©  26.02.13
Multi Temperature Data
Data volume Performance
•  Data is read and/or written frequently

hot •  In memory
•  No restrictions, all features available
•  Infrequent access
Non-Active Data Concept
warm •  On disk, no need to keep in memory all the time
•  No restrictions, all features available
NLS Management for read-

•  Sporadic access only data
cold •  Not stored in HANA DB; stored in Near-line Storage
•  Restricted to NLS capabilities
Providing lower TCO by optimized data volume management
©  26.02.13
Near-line Storage – Implementation Challenges
n  Analysis and Reporting are operating on a combination of Online-, and

Near-line data whereby data consistency is an indispensable requirement.
n  Transitionprocesses touching several storage types have to

guarantee consistency:
n  Archiving and Deleting of online data has to be one Logical Unit of Work (LUW).
n  Rollback mechanisms for single archiving steps have to be available.
n  The ‘Archive’ is getting the attidude of a database.
For this reason the processing of an

Data Archiving Process (DAP)
is structured in different phases:
©  26.02.13
Agenda –



©  26.02.13
NLS Interface – Design and Runtime Objects
NLS - Interface Data Archiving Process

DAP – Design Time (DAP)
BW n  Define partitioning
criteria
n  Generate DAP
InfoProvider
Younger …
DAP – Scheduling /
Q1/2011 Monitoring / Run-
Q4/2010 Time:
NLS
n  Describe partition Implementation
Q3/2010 to be extracted
n  Generate request
n  InfoProvider
Manage Menu
n  Process Chain
Q2/2010
n  Archive / Extract (DAP Q1/2010
menu, Process Chain
n  Restore (DAP menu)
DB
DBMS
Add. Features
n  Query access to NLS
n  DTP access to NLS
n  Look up during Transforms
©  26.02.13
Data Management with Near-line Storage
Implementation Aspects
1 Create a Data Archiving Process 5n  Look-up during Transformation

2 Create and schedule archiving requests
n  Query Settings
6
3 Restore archiving requests
4 Load data to subsequent Data Targets 7 Multiprovider
n  Settings
Reporting Layer SAP Sales InfoCube

MultiProvider
(Architected Data Marts) 7 Nearline Storage
6
4
Data Propagation Layer Corporate
DTP
Memory
LSA
DTP
Nearline Storage
2 3Nearline Storage
DTP
DAP
5 1
Data DTP InfoSource
Acquisition
Layer PSA
InfoPackage DataSource
©  26.02.13
10
SAP BW - Key facts about NLS Interface
n  NLS should be a part of an Information Lifecycle Management (ILM) strategy

n  Based on well-established SAP / SAP BW archiving concepts
n  Data consistency guaranteed before deleting the data from source
n  Data archived in NLS can be incorporated into reporting
n  Saves storage costs and other system resources
n  High compression rate (up to 95% depending on implementation)
n  Increases retention period for analysis data
n  Included in the query statistic data collection (RSRT)
n  NLS Interface Features

n  Supports archiving of InfoCubes and DataStore Objects
n Mainly time-based archiving, yet can also be based on other characteristics
n Lock of the archived data slice in the original InfoProviders
n NLS Process Steps in Process Chain Handling
n Copes with changes in the meta data to the BW objects of the archived data
n Deletion of NLS partitions supported
n Direct load from ADK archive files into NLS supported
©  26.02.13
11
Advantages of a Data Management Strategy
Performance
Faster Data Loads
Faster Queries
Costs
Less storage costs
Less administrative effort
controlled system growth
Availability
of data:
faster rollups and change runs etc.
of system:
less downtime for backups, upgrades etc.
Enterprise Data Warehousing

„Data Marts on Demand“: more flexibility, faster rollout
©  26.02.13
2012 SAP AG. All rights reserved. Improve Performance and lower Costs for SAP BW 12
12
Agenda –



©  26.02.13
13
The Near-line Storage Solution for SAP BW
Based on the NearLine Storage Interface

§ 
Development Partners can implement their
Solutions for Archiving and NLS into SAP BW
§  3rd Party NLS Solutions

  are implemented within the SAP BW ABAP Stack in partner
specific namespaces
  have to pass a certification process
  can offer specific Application Area in the SAP Support Portal
  have to be licensed in addition to SAP licenses
  can have a different release cycle compared to
SAP NetWeaver BW
NLS
Partner
Solution
©  26.02.13
14
SAP Defined Certification Category:
Nearline Storage
Certified Partners
(January 2013)
(see also http://www.sap.com/ecosystem/customers/directories/SearchSolution.epx )
©  26.02.13
15
Informatica ILM Nearline for SAP BW
(Former SAND Nearline)
•  Reduce production storage costs by moving infrequently accessed data to a nearline data store.
•  Does not require any database administration.
•  Data is stored in encrypted files on standard file system, simplifying backup and restore processes as
well as replication to other environments.
•  Reduce non-compliance risk with continued access to data in immutable, compressed files -
compression rates as high as 98%, with average of 90%+ compression rate.
•  Contact: rsobral@informatica.com
©  26.02.13
16
SAP NetWeaver BW Nearline Storage on DataVard
OutBoard™
  DataVard‘s OutBoard™ is the all-in-SAP nearline storage solution

  Built in ABAP, runs in your existing SAP landscape, no additional database or hardware required
  OutBoard™ is priced per productive system landscape. No matter how much data you load onto it.
  Compresses all InfoProviders, PSA and Changelogs up to 95%
  Contact: outboard@datavard.com
OutBoard™
SAP
cluster
tables Business Warehouse
Near-Line Storage
OutBoard™
§  All SAP-certified
RDBMS External /
SAP NLS
Interface
§  Big data ready Remote
(e.g. Apache DB
Hadoop) Storage
Mgmt.
§  File
§  Apache Hadoop
File / HANA or DB
Cloud
§  Corporate Cloud
§  End of lifecycle Deletion

OutBoard™ Storage Management
©  26.02.13
17
SAP NetWeaver BW Nearline Storage on PBS CBW
  With CBW PBS provides an add-on solution for the data retrieval of archived/nearlined BW
data.
  On the basis of the SAP ADK files, the CBW solution creates aggregates on the archive data
and (optionally, if needed) index structures to enable a fast retrieval option for the archived data
directly from the end-user's original query.
  PBS CBW NLS IQ for Sybase Analytics Server IQ utilizes the Sybase Analytics Server IQ as a
nearline database and offers extremely fast query response times.
  Contact: klaus.zimmer@pbs-software.com
©  26.02.13
18
SAP NetWeaver BW Nearline Storage on IBM DB2
  Takes full advantage of DB2 compression

  Fully exploits DB2‘s self managing capabilities
  Included in SAP DB2 OEM licnese at no extra cost
  SAP NLS (Near Online Storage) solution for DB2 has been
downported to NW 7.0 EhP1 and will be released for SAP
customers with SPS06 end 2009.
The solution is also contained in NW 7.0 EhP2 and NW 7.20
  Contact: KARL.FLECKENSTEIN@DE.IBM.COM
Install Near-line Storage data

base
©  26.02.13
19
SAP owned MaxDB based Sample Solution of NLS
Demo and verification implementation of the interface
Not fo
SAP BW r p ro d
uctive
use
MaxDB schema for

user SAPR3 with
NLS objects
Table structure
Because the implemention of the Nearline Interface is completely delegated to partners

a test and reference implementation is necessary.
See SAP Notes 1154026 and 1025273 (with attachment specifying the NLS API)
©  26.02.13
20
Overview SAP BW NLS Partner Solutions
n  Different storage types supported

n  BW Database inbound
n  Underlying BW DB (from Datavard)
n  Databases
n  IBM DB6 (from IBM)
n  Sybase IQ (from PBS)
n  Sybase IQ (from DataVard)
n  MaxDB ( for SAP’s T&E implementation)
n  Informatica Nearline (former SAND DNA) (from Informatica)
n  Apache Hadoop (from Datavard)
n  File System
n  ADK storage systems (from PBS)
n  File or Apache Hadoop (from Datavard)
n  Different license models

n  # of systems based
n  Volume based
n  User based
n  Mixed approach
©  26.02.13
21
Data analysis and assistance for ROI analysis
n  Sizing of NearLine-Storage solutions:

n  Hardware sizing of the NearLine-Storage solution has to be done by the vendor
n  Different NearLine-Storage technologies on the market
n  From database solutions, to file-based solutions, to column-based storage solutions
n  System Analysis e.g. with DataVard Fitnesstest

n  Deliver a thorough analysis of BW objects distribution
n  Can help on estimating the data volume that may be archived / transferred to NLS for
the largest InfoProviders within the system
n  Considers only “technical facts” (and not the customer’s “business requirements”)
©  26.02.13
22
BWFT Result: Best case 1,7 TB savings
3500
3000
2500
OutBoard
2000 Other data
ODS data
Temporary data
1500
Cube data
Master data
1000
500
0
Heute mit OutBoard und ERNA
©  26.02.13
23
ERNA - Features Overview
Scope of Housekeeping
Application (e.g. BW) System (Netweaver)

n  PSAs n  Application log
n  Change Logs n  Batch log
n  Request logs & tables (RSMON* and RS*DONE) n  IDoc tables (EDI40, EDIDS)
n  Unused DIMIDs, Unused SIDs n  qRFC, tRFC
n  Unused Masterdata n  Job-Tables (TBTCO, TBTCP etc.)
n  Cube & Aggregate compression n  Change & Transportsystem
n  NRIV buffering n  Spool data (TST03)
n  Table buffering n  Table Change Protocols
n  BI-Statistics n  Batch Input Folders
n  Process Chain Log n  Alert Management Data (SALRT*)
n  Errorlog for DTPs (RSBERRORLOG) n  ...
n  ...
Central access for scheduling and monitoring

Process Automation
Automated recommendations
©  26.02.13
24
Agenda –



©  26.02.13
25
Motivation
Main aspects of NLS
•  Separation of frequently accessed (hot) data from less frequently accessed

(cold) data keeping the HANA in memory costs in a reasonable range
•  Complementary to active / non-active classification within HANA
•  Data in NLS is highly compressed in cost-efficient storage with fewer backups

reducing operational costs
©  26.02.13
26
Design Aspects for BW powered by HANA –
Near-line Storage (NLS) vs. SAP HANA DB
BW on HANA data volume performance

RAM
•  Active Data
Temporary
•  in HANA DB
data
•  no restrictions, all features available
•  active + not-active partitions

hot Processing •  in HANA
data •  no restrictions, all features available
Displacement Strategy •  (mostly) not-active data

warm Filer •  in HANA
Persistent •  no restrictions, all features available
data
cold Sybase NLS

IQ •  NLS
Persistent •  in Sybase IQ
data •  restricted to [BW] NLS capabilities
©  26.02.13
27
SAP BW powered by HANA and Sybase IQ NLS
Architecture Overview
BI Clients
An SAP owned BW NLS Implementation for Sybase

SAP Netweaver BW 7.3x IQ offers a fully integrated solution from one provider
Transient InfoCube/DSO Main aspects

MultiProvider
Provider
•  Deliver an ABAP based implementation of the BW
NLS interfaces
Near-Line SDK
•  Deliver an Sybase IQ DBSL ‘light’ that covers all
needs for the above mentioned NLS implementation
PBS BW NLS4IQ new •  Sybase IQ to deliver reliable and performing
OEM SAP Native execution of the DBSL driven loads and queries
SAP •  Planned availability Q1/2013
HANA
Sybase
IQ
No longer
required
©  26.02.13
28
BW NLS Implementation for Sybase IQ
(DBSL based solution)
BW BW Porting
InfoProvider Nearline Adapter (ABAP based )
General Porting
DBSL for
Sybase ASE (/w IQ ext.)
(Kernel based; Open SQL + Native IQ
SQL via ADBC)
NLS Interface ODBC Driver from IQ DB Client
HANA DB
©  26.02.13
29
Preliminary Results from ongoing
Performance Tests
Environment
  BW 7.30 on HANA Rev. 42 (Online DB)
IBM M-Box – Intel Xeon X7560 – 64 Cores + HT – 2,27 GHz – 512 GByte RAM
  Sybase IQ 15.4 (Near-line Storage)
Nehalem BWA-Blade – Intel Xeon X5670 – 12 Cores + HT – 2,93 GHz – 72 GByte RAM
Setup
  SPO partioned by 10 InfoCubes with 10 Mio. rows each
  Single target table on IQ side partitioned by Archiving Request (i.e. Time-/Dataslice)
Results
  64 Mio. rows/h (26 GByte/h) for single relocation run (1 Cube)
  238 Mio. Rows/h (97 GByte/h) for 4-fold parallel relocation run (4 Cubes)
  Still potential for improvements in single process mode and scalability (e.g. IQ 16.x)
©  26.02.13
30
Example for 3(4)-fold Archiving Process
on single Object
©  26.02.13
31
Further Information – SAP Public Web
SAP Public Web

SAP NetWeaver Business Warehouse – SCN Space
http://scn.sap.com/community/data-warehousing/netweaver-bw
BW Powered by HANA Overview on saphana.com

https://www.saphana.com/docs/DOC-1522/
SAP NetWeaver BW 7.3 powered by SAP HANA – Release Notes on SAP Help
http://help.sap.com/nw73bwhana/#section1
SAP NetWeaver BW 7.3 Documentation – Section “Using the SAP HANA Database”
http://help.sap.com/saphelp_nw73/helpdata/en/AF/
81F38183CF46B9B919F82D7BB7EC47/frameset.htm
©  26.02.13
32
How to get started…
•  Analysis Workshop / Fitnesstest

–  Objective: identify „Quick Wins“ / low hanging fruits
–  Benchmarking and Recommendations in areas of
•  System Performance and Robustness
•  Data Management
•  Housekeeping
–  ROI in a couple of months
•  Proof of Concept
–  Test installation
–  Proof of Concept
•  Project and usage
–  Analysis and design
–  Implementation
–  Operation
–  Check-up with Fitnesstest
©  26.02.13
33
Thank You!
Hahne Consulting GmbH

Dr. Michael Hahne
Managing Director
michael@hahneconsulting.de
+49 671 9203662

DSAG TT 2013 Hahne Consulting V12 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DSAG TT 2013 Hahne Consulting V12 PDF

Uploaded by

Copyright:

Available Formats

SAP BW Near-line

Dr. Michael Hahne

Hahne Consulting GmbH

February 27, 2013

n What is Near-line Storage (NLS)?

n What is the NLS Interface in SAP Business Warehouse (BW)?

n What is an Implementation for the SAP BW NLS Interface?

Storage Type / Near-Line Storage Classic Archive

BWA Acceleration Archiving Near-line Storage

Access - very frequently frequently not frequently rarely

Data volume Performance

• Data is read and/or written frequently

NLS Management for read-

Providing lower TCO by optimized data volume management

n Analysis and Reporting are operating on a combination of Online-, and

n Transitionprocesses touching several storage types have to

For this reason the processing of an

n What is Near-line Storage (NLS)?

n What is the NLS Interface in SAP Business Warehouse (BW)?

n What is an Implementation for the SAP BW NLS Interface?

NLS - Interface Data Archiving Process

1 Create a Data Archiving Process 5n Look-up during Transformation

Reporting Layer SAP Sales InfoCube

n NLS should be a part of an Information Lifecycle Management (ILM) strategy

n NLS Interface Features

Enterprise Data Warehousing

n What is Near-line Storage (NLS)?

n What is the NLS Interface in SAP Business Warehouse (BW)?

n What is an Implementation for the SAP BW NLS Interface?

Based on the NearLine Storage Interface

§ 3rd Party NLS Solutions

(see also http://www.sap.com/ecosystem/customers/directories/SearchSolution.epx )

 DataVard‘s OutBoard™ is the all-in-SAP nearline storage solution

§ End of lifecycle Deletion

 Takes full advantage of DB2 compression

Install Near-line Storage data

MaxDB schema for

Because the implemention of the Nearline Interface is completely delegated to partners

n Different storage types supported

n Different license models

n Sizing of NearLine-Storage solutions:

n System Analysis e.g. with DataVard Fitnesstest

Application (e.g. BW) System (Netweaver)

Central access for scheduling and monitoring

n What is Near-line Storage (NLS)?

n What is the NLS Interface in SAP Business Warehouse (BW)?

n What is an Implementation for the SAP BW NLS Interface?

Main aspects of NLS

• Separation of frequently accessed (hot) data from less frequently accessed

• Data in NLS is highly compressed in cost-efficient storage with fewer backups

BW on HANA data volume performance

• active + not-active partitions

Displacement Strategy • (mostly) not-active data

cold Sybase NLS

An SAP owned BW NLS Implementation for Sybase

Transient InfoCube/DSO Main aspects

InfoProvider Nearline Adapter (ABAP based )

NLS Interface ODBC Driver from IQ DB Client

SAP Public Web

BW Powered by HANA Overview on saphana.com

• Analysis Workshop / Fitnesstest

Hahne Consulting GmbH

You might also like

n  What is Near-line Storage (NLS)?

n  What is the NLS Interface in SAP Business Warehouse (BW)?

n  What is an Implementation for the SAP BW NLS Interface?

•  Data is read and/or written frequently

n  Analysis and Reporting are operating on a combination of Online-, and

n  Transitionprocesses touching several storage types have to

n  What is Near-line Storage (NLS)?

n  What is the NLS Interface in SAP Business Warehouse (BW)?

n  What is an Implementation for the SAP BW NLS Interface?

1 Create a Data Archiving Process 5n  Look-up during Transformation

n  NLS should be a part of an Information Lifecycle Management (ILM) strategy

n  NLS Interface Features

n  What is Near-line Storage (NLS)?

n  What is the NLS Interface in SAP Business Warehouse (BW)?

n  What is an Implementation for the SAP BW NLS Interface?

§  3rd Party NLS Solutions

  DataVard‘s OutBoard™ is the all-in-SAP nearline storage solution

§  End of lifecycle Deletion

  Takes full advantage of DB2 compression

n  Different storage types supported

n  Different license models

n  Sizing of NearLine-Storage solutions:

n  System Analysis e.g. with DataVard Fitnesstest

n  What is Near-line Storage (NLS)?

n  What is the NLS Interface in SAP Business Warehouse (BW)?

n  What is an Implementation for the SAP BW NLS Interface?

•  Separation of frequently accessed (hot) data from less frequently accessed

•  Data in NLS is highly compressed in cost-efficient storage with fewer backups

•  active + not-active partitions

Displacement Strategy •  (mostly) not-active data

•  Analysis Workshop / Fitnesstest