Professional Documents
Culture Documents
2.
3.
4.
SID Handling
5.
1.
MASTER DATA
1. Mass Lookups during Master Data Load
Data loads into a Master Data bearing Characteristic require database look-ups to find out if records exist on the database with the
same key as the ones being loaded. In releases prior to SAP BW 7.3, this operation was performed record-wise, i.e. for every record in
the data-package, a SELECT was executed on the database table(s). Obviously, this resulted in a lot of communication overhead
between the SAP Application Server and the Database Server, thereby slowing the Master Data loads down. The effect is pronounced
on data loads involving large data volumes.
The issue of overhead between the SAP Application Server and the Database Server has now been addressed by performing a masslookup on the database so that all records in the data-package are looked-up in one attempt. Depending on the DB platform it can bring
up-to 50% gain in load runtimes.
Starting NW 7.30 SP03, this flag will be renamed to New Records Only. The renaming has been done to
align with a similar feature supported by activation of DSO data. (See blog Performance Improvements for DataStore
Objects )
As mentioned above, the Master Data Load performs a look-up on the database for every data-package to ascertain which key values
already exist on the database. Based on this information, the Master Data load executes UPDATEs (for records with the same key
already existing in the table) or INSERTs (for records that dont exist) on the database.
With the Insert-Only feature for Master Data loads using DTPs, users have the opportunity to completely skip the look-up step, if it is
already known that the data is being loaded for the first time. Obviously, this feature is most relevant when performing initial Master
Data loads. Nevertheless, this flag can also be useful for some delta loads where it is known that the data being loaded is completely
new.
Lab tests for initial Master Data loads indicate around 20% reduction in runtime with this feature.
The Insert-Only setting for DTPs loading Master Data can be found in the DTP Maintenance screen under the UPDATE tab as
shown below.
Note :
If the Insert-Only flag is set, and data is found to exist on the database, the DTP request will abort. To recover
from this error, the user simply needs to uncheck the flag and re- execute the DTP.
Although the new Master Data Deletion has be available for some time now (since BW 7.00 SP 23), it was never the default version in
the system. This implied that the BW System Administrators needed to switch it ON explicitly. With BW 7.30 however, the New Master
Data Deletion is the default version and no further customizing is necessary to use it.
All further information about this functionality is documented in the SAP note:1370848 underhttps://websmp130.sapag.de/sap(bD1lbiZjPTAwMQ==)/bc/bsp/spn/sapnotes/index2.htm?numm=1370848
4. SID Handling
This feature relates to the handling of SIDs in the SAP BW system and while it is certainly relevant for Master Data loads, it is not
restricted to it. The performance improvements in SID handling are relevant for all areas of SAP BW where SIDs are determined, for
example Activation of DSO Requests, InfoCube Loads, Hierarchy Loads and in some cases, even Query processing.
In BW 7.30, SIDs are determined en-masse meaning that database SELECTs and INSERTs that were done record-wise previously
have been changed to the mass SELECTs (using the ABAP SELECT FOR ALL ENTRIES construct) and mass INSERTs. The system
switches to this mass-data processing mode automatically when the number of SIDs to be determined is greater than a threshold value.
The default value of this threshold is 500.
The threshold value is customizable of course and that can be done in the SAP IMG for customizing under the transaction SPRO by
following the path: SAP Netweaver -> Business Warehouse -> Performance Settings -> Optimize SID-Determination for MPPDatabases.
Note: As the threshold value corresponds to the minimum number of SIDs
to be determined in one step, setting the threshold to a very high value
(For example: 100000) causes the system the system to switch back to the
classical behavior.
The fact that the data from the navigational attributes is available as part of the source structure allows the
data to be used in custom logic in Transformations (example : Start Routines).
Secondly, the data from the navigational attributes is read by performing database joins with the
corresponding Masterdata tables during extraction. This helps in improving the performance of scenarios where a lot of
look-ups are needed and/or a lot of data is to be looked-up.
To use this feature in Transformations, the navigational attributes need to be switched ON in the source InfoProvider in the InfoProvider
maintenance screen as below -
Once this is done, the selected navigational attributes are available as part of the source structure of Transformations as shown below
This feature of the DTP is used to combine several data packages in a source object into one data package for the DataTarget. This
feature helps speed up request processing when the source object contains a large number of very small data packages.
This is usually the case when memory limitations in the source systems (for example: an SAP ERP system) results in very small datapackages in the PSA tables in BW. This DTP setting can be used to propagate the data to subsequent layers in BW in larger chunks.
Also, InfoProviders in BW used for operational reporting using Real-time Data Acquisition contain very small data packages. Typically,
this data is propagated within the DataWarehouse into other InfoProviders for strategic reporting. Such scenarios are also a use-case
for this feature where data can be propagated in larger packets.
As a prerequisite, the processing mode for the DTP needs to be set to Parallel Extraction and Parallel Processing. Also note that only
source packages belonging to the same request are grouped into one target package.
Below is a screenshot of the feature in the DTP Maintenance.
Remember that you can double-click on the objects in the first column to view details. You can look up for example
that Ive configured to stop RDA requests after 13 errors.
Everything looks fine. So lets start the RDA daemon. It will execute all the InfoPackages and DTPs assigned to it at a
frequency of one per minute. But wait whats this?
The system asks me whether Id like to start a repair process chain to transfer missing requests to one of the data
targets. Why? Ah, okay Ive added a DTP for the newly created HybridProvider but forgotten to transfer the requests
already loaded from the DataSource. Lets have a closer look at these repair process chains while they are taking care
of the missing requests.
On the left hand side, you can see the repair process chain for my HybridProvider. Besides the DTP, it also contains a
process to activate DataStore object data and a subchain generated by my HybridProvider to transfer data into the
InfoCube part. On the right hand side, you can see the repair process chain for my airline attributes which contains an
attribute change run. Fortunately, you dont need to bother with these details the system is doing that for you. But
now lets really start the RDA daemon.
Green traffic lights appear in front of the InfoPackages and DTPs. I refresh the RDA monitor. Requests appear and show
a yellow status while they load new data package by package. The machine is running and I can go and work on
something else now.
About a day later, I start the RDA monitor again and get a shock. What has happened?
The traffic lights in front of the InfoPackages and DTPs have turned red. The RDA daemon is showing the flash symbol
which means that is has terminated. Dont panic! Its BW 7.3. The third column helps me to get a quick overview: 42
errors have occurred under my daemon, 4 DTPs have encountered serious problems (red LEDs), and 4 InfoPackages
have encountered tolerable errors (yellow LEDs). I double-click on 42 to get more details.
Here you can see in one table which objects ran into which problem at what time. I recognize at a glance that 4
InfoPackages repeatedly failed to open an RFC connection at around 16:00. The root cause is probably the same, and
the timestamps hopefully indicate that it has already been removed (No more RFC issues after 16:07). I cannot find a
similar pattern for the DTP errors. This indicates different root causes. Finally, I can see that the two most recent
runtime errors were not caught and thus the RDA daemon has terminated. You can scroll to the right to get more
context information regarding the background job, the request, the data package, and the number of records in the
request.
Lets have a short break to draw a comparison. What would you do in BW 7.0? 1) You could double-click on a failed
request to analyze it. This is still the best option to analyze the red DTP requests in our example. But you could not find
the tolerable RFC problems and runtime errors.
2) You could browse through the job overview and the job logs. This would have been the preferable approach to
investigate the runtime errors in our example. The job durations and the timestamps in the job log also provide a good
basis to locate performance issues, for example in transformations.
3) You could browse through the application logs. These contain more details than the job logs. The drawback however
is that the application log is lost if runtime errors occur.
These three options are still available in BW 7.3 they have even been improved. In particular, the job and application
logs have been reduced to the essential messages. Locating a problem is still a cumbersome task however if you dont
know when it occurred. The integrated error overview in the RDA monitor, BW 7.3 allows you to analyze any problem
with the preferred tool. Let me show you some examples.
Unless you have other priorities from your business users Id suggest starting with runtime errors because they affect
all objects assigned to the daemon. RDA background jobs are scheduled with a period of 15 minutes to make them
robust against runtime errors. In our example, this means the RDA daemon serves all DataSources from the one with
the lowest error counter up to the one which causes the runtime. The job is then terminated and restarted 15 minutes
later. The actual frequency is thus reduced from 60/h to 4/h, which is not real-time anymore. Lets see what we can do
here. Ill double-click on 10 in the error column for the request where the problem has occurred.
I just double-click on the error message in the overview to analyze the short dump.
Puh This sounds like sabotage! How can I preserve the other process objects assigned to the same daemon from this
runtime error while I search for the root cause? I could just wait another hour of course. This RDA request will then
probably have reached the limit of 13 errors that I configured with the InfoPackage. Once this threshold is reached, the
RDA daemon will exclude this InfoPackage from execution. The smarter alternative is to temporarily stop the upload
and delete the assignment to the daemon.
The overall situation becomes less serious once the DataSource has been isolated under Unassigned Nodes. The
daemon continues at a frequency of onc per minute although there are still 32 errors left.
Note that most of these errors namely the RFC failures can be tolerated. This means that these errors (yellow LEDs)
do not hinder InfoPackages or DTPs until the configured error limit is reached. Assume that Ive identified the root
cause for the RFC failures as a temporary issue. I should then reset the error counter for all objects that have not
encountered other problems. This function is available in the menu and context menu. The error counter of an
InfoPackage or DTP is reset automatically when a new request is created. Now lets look at one of the serious
problems. Ill therefore double-click on 2 in the error column of the first DTP with red LED.
When I double-click on the error message, I see the exception stack unpacked. Unfortunately that does not tell me
more than I already knew: An exception has occurred in a sub step of the DTP. So I navigate to the DTP monitor by
double-clicking the request ID (217).
Obviously, one of the transformation rules contains a routine that has raised the exception 13 is an unlucky number.
I navigate to the transformation and identify the root cause quickly.
In the same way, I investigate the exception which has occurred in DTP request 219. The DTP monitor tells me that
something is wrong with a transferred fiscal period. A closer look at the transformation reveals a bug in the rule for the
fiscal year variant. Before I can fix the broken rules, I need to remove the assignment of the DataSource to the
daemon. When the corrections are done, I schedule the repair process chains to repeat the DTP requests with the fixed
transformations. Finally I re-assign the DataSource to the daemon.
The RDA monitor already looks much greener now. Only one DataSource with errors is left. More precisely, there are
two DTPs assigned to this DataSource which encountered intolerable errors, so the request status is red. Again, I
double-click in the error column to view details.
The error message tells me straight away that the update command has caused the problem this time rather than the
transformation. Again, the DTP monitor provides insight into the problem.
Of course GCS is not a valid currency (Should that be Galactic Credit Standard or what?). I go back to the RDA
monitor and double-click on the PSA of the DataSource in the second column. In the request overview, I mark the
source request of the failed DTP request and view the content of the problematic data package number 000006.
Obviously, the data is already wrong in the DataSource. How could this happen? Ah, okay Its an InfoPackage for Web
Service (Push). Probably the source is not an SAP system, and a data cleansing step is needed either in the source
system or in the transformation. As a short-term solution, I could delete or modify the inconsistent records and repeat
the failed DTP requests with the repair process chain.
Thats all. I hope you enjoyed this little trip to troubleshooting real-time data acquisition, even though this is probably
not part of your daily work yet. Let me summarize what to do if problems occur with RDA. Dont panic. BW 7.3 helps
you to identify and resolve problems faster than ever before. Check the error column in the RDA monitor to get a quick
overview. Double-click wherever you are to get more details. Use the repair process chains to repeat broken DTP
requests.
Disclaimer
http://www.sdn.sap.com/irj/sdn/index?rid=/webcontent/uuid/b0b72642-0fd9-2d10-38a9-c57db30b522e
Introduction
If you remember older releases of SAP NetWeaver BW hierarchies could only be loaded through the old 3.x data flow. In this case you
needed the so called direct update functionality of the corresponding InfoSource for uploading the hierarchy. This InfoSource 3.x was
connected to an 3.x DataSource through update rules.
First, hierarchy DataSources were available only for flatfile and SAP source systems. Besides, end users could
only create own hierarchy DataSources for the flat file system.
Second you could not take full advantage of the new data flow, even some old data flow features (e.g. the start
routine) could not be used. Furthermore, to change the structure of hierarchies during runtime you had to implement
complex scenarios (e.g. with the help of the analysis process designer APD). The direct update functionality didn't
allow you to load the hierarchy to a DSO or an other arbitrary object and manipulate it according to the end users'
needs.
Third, monitoring was often unclear because the framework was not optimal for segments.
First you are able to use any BW object as source for a hierarchy, you are not limited to a DataSource for
hierarchies. This leads to simpler scenarios if you want to transform your hierarchy according to your needs. You just
have to connect your hierarchy through a transformation and a data transfer process.
Within this transformation you are able to use all features of a transformation, for example start, end or expert
routines. You are not limited as you were in the 3.x data flow.
You can use any DataSource as a source for your hierarchy, you are not restricted to hierarchy DataSources
any more. This makes hierarchy extraction of SAP source systems possible, too.
Last but not least you are now able to take full advantage of all capabilities of the new data flow. You can
distribute the data loaded from one DataSource to several hierarchies and you can use an arbitrary number of
InfoSources in between the DataSource and your hierarchy. A very useful feature is the automatic filling of the fields
CHILDID, NEXTID and LEVEL through the framework if they are not filled by the source (e.g. if only the PARENTID is
provided).
The hierarchy structure contained of the five fields NODEID, IOBJNM, NODENAME, LINK and PARENTID. The NODEID was the
internal number for your node, the field IOBJNM contained the InfoObject name for your node. The NODENAME contained the value in
compounded format (if compounded InfoObjects were used). This is the case if you use e.g. cost center hierarchies which are
compounded to the controlling area.
The new framework now uses more fields to describe the hierarchy structure. Whereas the first five fields are the same, the new
structure now contains fields for every InfoObject of a compounded structure. For example, if you use the cost center which is
compounded to the controlling area, the new structure contains both InfoObjects.
This new rule type is called "hierarchy split". To use the new rule type you have to map the field NODENAME to every InfoObject of
your structure, in this case the controlling area and the cost center. The hierarchy split automatically transforms the value of
NODENAME to the corresponding values of controlling area and cost center.
Please remember that this functionality only works if you haven't changed the length of the corresponding InfoObjects. For example, if
you changed the length of the controlling area to 5 digits instead of 4 you cannot use this feature.
After choosing the correct type of DataSource you can edit the properties of your DataSource. The main properties can be found in the
tab "extraction" which is shown in the following figure.
In the top of the screen you can provide information on full/delta loading and the direct access, both you will know of other
DataSources. The interesting parts are highlighted. If you select "hierarchy is flexible", the DataSource will be created in the new
structure, if you leave it blank the DataSource will consist of the old structure. Second you are now able to create the hierarchy header
directly from the DataSource. In older releases you had to maintain the hierarchy header in the InfoPackage. I think the other
parameters are known, if not just feel free to ask me.
Within the transformation you have to map the corresponding segments to each other. The interesting one is the mapping of source
segment "hierarchy structure" to the correspondent target segment.
You can see that there are a lot of 1:1 rules within the transformation except the mapping of field NODENAME to 0COSTCENTER and
0CO_AREA. Here, the new rule type "hierarchy split" has been set.
Maybe you have noticed that the key field 0OBJECTID has not been mapped. This field will be used to load more than one hierarchy to
have a unique key. Please be aware that in this case each data package has to contain exactly one complete hierarchy.
Now let's have a look at the corresponding data transfer process (DTP). There are two major changes when loading data through a
hierarchy DataSource:
1.
If you select full extraction mode in the DTP there's a new flag called "only retrieve last request". This flag is
helpful if you don't want to clear your PSA before loading the new hierarchy. Only the newest request in the PSA will be
selected and loaded.
2.
In the update tab of your DTP (see figure) you can decide how to update your hierarchy (full update, update
subtree, insert subtree). Furthermore you can activate the hierarchy from within the DTP, no separate process step in a
process chain is necessary to activate the hierarchy!
Now you are able to load a hierarchy from a hierarchy DataSource to an InfoObject.
First change catching the eye is a new folder Process Chains in the Modelling View. Seems a small change, but there comes
something important with it search.
At last you are able to search for process chains by name directly in the workbench. Plus, you might have noticed: The display
components are ordered hierarchical like any other folder in RSA1. But not only the display components, but also the chains itself have
a hierarchical structure, displaying the metachain subchain relationship. So you can directly navigate to the lowest subchain in your
metachain hierarchy if you only remember the name of the top-level master.
But this is not what we were looking for, we spoke about monitoring. So lets go to the Administration area of RSA1
Ok, same tree, but that does not really make a difference. Lets look at the Monitors.
Chains
If RSPCM was used in this system before upgrade, I will be asked the first question, else the second one. What does it mean? You
might know, that RSPCM requires you to choose the chains, which shall be monitored. So I understand the second question, this will
simplify the setup. But what about the first question? Well, RSPCM became user-dependent with 7.30. Each user has his own unique
selection of process chains to monitor.
Good thing for most users. But stop what if I have a central monitoring team where all administrators shall look at the same list? Does
each of them need to pick his or her chains himself? No:
Besides the known possibilities to add a single chain resp. all chains of a display component there is the option to Select Include.
And then I am on the ususal RSPCM list and can add chains. But now, how does the new RSPCM list look like? Here it is:
The first five columns look pretty familiar. But there is something hidden there as well! Lets click on one of the red chains:
The system is extracting the error messages from the single processes of the failed process chain run and directly displays them
together with the corresponding message long text. So no more searching for the red process and navigating and searching the log.
One click and you know whats wrong (hopefully ;-). If not, you still can go to the log view of the process chain by the Log Button on the
popup.
And look the button alongside, called Repair, looks interesting. No more error analysis, just press repair any failure. THAT was what
you were looking for all the years, isnt it? Sorry, it is not that simple. The button will repair or repeat any failed process in the chain run
like you could do already before via context menu. And whether this is a good thing to try still depends on the particular error. But you
should be a little faster now finding it out and doing it.
What about the other two buttons? Lets delay this a little and first go back to the overview table. There are some new columns:
First, there is a column to switch on and off the Time Monitoring because you might not be interested in the run time or start delay of
any chain in RSPCM, especially considering that you can still schedule RSPCM to batch to perform an automatic monitoring of your
chains and alert you in case of red or yellow lights. Considering performance, it does not make much of a difference, because the run
time calculations of previous chains are buffered in an extra table. After all, the performance of RSPCM also greatly improved by
following setting:
You can choose to refresh only those chains, which have yellow status or not refresh the status at all but display the persisted status
from the database. For the red and green runs, it is expected, that the status does not change frequently. If you nevertheless expect a
change of such chains, you can also force the status refresh in the transaction itself by pressing the Refresh All button in the
screenshot above.Now, how come the system judges a run time of 48 seconds Too Long? Did somebody set a maximal run time for
each chain? Let us click on the red icon.
Seems like this chain runs usually not longer than about 35 seconds. Considering this, 48 seconds is too long. More specifically, the
chain takes usually 25.5 plusminus 3.9 seconds. The 48 seconds are about 22.5 seconds more than usual. Especially, these 22.5
seconds are much more than the usual deviation of 3.9 seconds.
So, the system uses some simple statistics to judge on the run time of a chain run. If you like, I can also give you the mathematical
formulas for the statistics, but I guess this is not so interesting for most of the readers ;-)
To stabilize this statistics, the system even removes outliers before calculating the average run time and average deviation. If you are
interested in the filtered outliers, you can press the Outlier button:
This indeed is an outlier, dont you think? Lets get back in time a little bit and look at older runs of this chain by pressing the + button
twice, which will add twenty additional past values.
Now our outliner is not the biggest anymore. Even more interesting, the chain seemed to stabilize its runtime over the past weeks.
Considering the previous values as well, our current run is no longer extraordinary long. Is it a false alarm then? No, because if the run
time turns back to former instabilities, you would like to get notified. If that instable behaviour however becomes usual again, the
system will automatically stop to consider this run time extraordinary and thus stop alerting you, because it always considers the last 30
successfull executions for calculating the statistics for the alerts.
So let us check, what process made this current run longer. We press button Display Processes.
Of course a DTP. You would not have expected something different, would you? I now could click on the DTP load to open the DTP
request log and start analyzing what went wrong, but at this point rather lets go back to the RSPCM overview list and look at the
Delay column.
Today it is the 14.5.2010, so an expected start on 27.4.2010 indeed I would judge heavily delayed. But how does the system come to
the conclusion that the chain should start on 27.4. (plusminus six and a half days)? The answer comes when clicking on the LED.
Obviously this chain was executed only five times in the past, and it had been started once a week (in average). Seems like the
responsible developer lost interest in this chain meanwhile. If he continues not to start the chain, the system will stop complaining about
the delay and turn the icon grey. Now it is important to note, that this chain is started either immediately by hand or by event. If it would
be scheduled time periodic in batch, the delay is not calculated in this statistical manner, but the delay of the TRIGGER job is observed
and the alert is raised after 10 minutes of delay.
And now also the question about the two buttons in the status popup is answered: They open up the shown run time and start time
popups.
I hope you have fun using these new functions and appreciate that it is not neccessary to laboriously customize any thresholds in order
to get meaningful alerts.
1.
2.
3.
4.
5.
A lot of time and effort has been invested in SAP BW 7.30 to improve the performance of operations on
DataStore Objects, such as request activation. The most important improvements are:
database partitioning for DataStore Objects
mass lookups in activation of requests in DataStore Objects
dynamic flag unique data records
faster request activation in DataStore Objects on databases with massively parallel processing
architecture
lookup into DataStore Objects in transformation
The following sections describe these points in more detail.
The E fact tables of InfoCubes can be partitioned, by month or fiscal period, on databases that support
range partitioning (for more details, see here). In SAP BW 7.30, this is now also possible for the active
tables of standard DataStore Objects. More precisely, if a standard DataStore Object has at least one key
field referring to InfoObject 0CALMONTH or InfoObject 0FISCPER, then the active table of this DataStore
Object can be partitioned by one of these key fields. The advantage of this partitioning is faster access to
data due to partition pruning, provided that the partition criterion is part of the selection criterion (where
clause).
To activate partitioning, go to the menu and choose Extras DB Performance DB Partitioning (in edit
mode of a standard DataStore Object).
A popup appears where you can select a key field that refers to the InfoObject 0CALMONTH or InfoObject
0FISCPER (you cannot select anything if there is no field referring to one of these InfoObjects).
Any changes made here will become effective after activating the DataStore Object.
Note that you can only change the partitioning settings if the DataStore Object is empty. This means that
you should decide whether you want a DataStore Object to be partitioned before you load any data into it.
Confirm the dialog. The system will now run the activation without lookup.
For activation processes scheduled as part of a process chain, you also have the option to force the system
to omit the lookup for init requests. Choose Change... next to the text Uniqueness of Data: Use
DataStore Settings on the maintenance screen of your variant for an activation.
A popup appears where you can choose Init requests always return unique data records.
Confirm the dialog. The system will now omit the lookup for init requests.
(4) New Activation Method for Databases with Massively Parallel Processing
Architecture
The traditional activation of requests in a standard DataStore Object is done by splitting the data of the
activation queue into packages that are activated independently of each other (and usually
simultaneously). The system ensures that records with identical keys are placed into the same package
and stay in their original sequence. It is very important that records with identical keys are activated in the
correct sequence to guarantee correct results. Therefore a package is activated sequentially - record by
record.
Usually there are not many records with identical keys in an activation run. SAP BW 7.30 offers you a new
activation method that can be used on databases with massively parallel processing architecture (these
are currently (January 2011) IBM DB2 for Linux, UNIX, and Windows and Teradata Foundation for SAP
NetWeaver BW).
This new activation method finds all the records in the activation queue that do not have unique keys and
activates them in the traditional way. All the other records have unique keys in the activation queue and
can be activated regardless of the sequence. These records are activated simultaneously using a few SQL
statements. This means that the majority of records are no longer read from the database to the
application server, processed and written back to the database. Instead they are processed directly in the
database. This results in a considerable improvement of the activation runtime, in databases with a
massively-parallel-processing architecture.
This new activation method is used automatically by the system (i.e. no user action is needed), provided
that certain conditions are met (list not complete):
this new activation method is supported for the database used (currently only IBM DB2 for Linux,
UNIX, and Windows and Teradata Foundation for SAP NetWeaver BW, as mentioned above)
the aggregation behaviors of all (load) requests that are activated in one go are identical
This will help in reusability like avoiding the LISTCUBE tcode again and again for the same selections.
This will also helps you to reduce certain amount of your manual work.
3. Same time give the program name starting with Z or Y ( This is the program which is going to be reused) and please make a note of
it.
4. Now execute the screen , which displays your selection screen and select your needed field for the
selection, later also select your needed fields for the output using field selection for output tab
.
5. After selecting it, kindly select the Save button which is there on the top of the screen to save it as variant.
6. You will get the popup screen in which it will ask you to update with Variant name and its Description, enter the needed info and save
it again through the save button or through the file menu Variant --->Save.
7. Use can make use of options check box which is available in the screen likeProtect Variant, which protects others to changes
this variant on the same program, means it can be changed only by the created user, other is not certainly required, still if you need to
know its purpose kindly press F1 on corresponding check box for its use.
8. Now go to TCODE SE38 and give the program name as the one which you gave it in the LISTCUBE transaction and execute it.
9. Once you execute it you will get the screen as you decided in listcube, click on variant button for save variants or click on field
selection for output tab for your changes like to save another variant or to make the changes in the current variant.
10. Here if you want to view the save variants from list cube you can use the variant button and select your variants to display it which
has been stored already.
11. Select the need variant and execute the program to get your desired outputs.
Note: you can also store N number of variants from this program itself.
Now instead of going LISTCUBE again and again to make your selection you can make use of this program when you want to make
use of the same info provider to display your output for n number of times, this will reduce your time in selecting your selection option,
and your desired output screen.
Dependencies: If the structure of the Info Provider changes, a program generated once is not adapted
automatically. The program has to be manually deleted and generated again to enable selections of new
fields.
Scenario1.
Consider a R/3 system having a CMOD code for one of the data source and there you have been given a situation to
identify few data from BW system based on that , records from R/3 has to be fetched to complete your extraction. Now
in order to achieve this, a Function Module has been written in BW system to fetch the data, but that Function module
has to be called from R/3 system using Remote function call, situation seems to very simple, but while you write the
code in R/3 system there comes a complexity to identify from which BW system(either QA or Dev OR PRD) the data
has to be read, because you cant write same code in your Dev, Quality and Production System, as the client and
system name will differs accordingly your remote call system will differ
If you are having only three systems then you can go ahead and hardcode the BW system values as DEV, QA, PRD
using if condition and your R/3 system client details you can write your code, but tomorrow if one more system called (
Sandbox or Readiness) has been added, in that case again you have to go ahead and change the code again to work
in that environment which makes you complex each and every time, so in order to avoid such situation, there is run
time structure which identifies your Info package details, accordingly you can get your Target system details from your
IP and pass that value in your remote Function Module to get the BW data from R/3 system for your manual
calculations.
BW system: BWSCLNT010.
With the announcement of HANA*, some customers, analysts and others have raised the question on how HANA
relates to BW with a few of them even adding their own, home made answer in the sense that they speculate that
HANA would succeed BW. In this blog, I like to throw in some food for thought on this.
Currently, HANA's predominant value propositions are
(i) extremely good performance forany type of workload
(ii) a real-time replication mechanism between an operational system (like SAP ERP) and HANA
Let's match those for a moment with the original motivation for building up a decision-support system (DSS) or data
warehouse (DW). In the 1990s, a typical list of arguments in favour of such a system looked like this:
1.
2.
Provide data models that are more suitable and efficient for analysis and reporting.
3.
4.
Historize - store a longer history of data (e.g. for compliance reasons), thereby relieving OLTPs from that task
and the related data volumes.
5.
6.
stages depending on the type of furniture you want to produce - shelves might require less steps (i.e. "layers") than a
cupboard.
Finally, I believe that there is an excellent case for building a BW on top of HANA, i.e. to combine both - see the bottom
right scenario in the figure below. HANA can be seen as an evolution of BWA and, as such, this combination has already
proven to be extremely successful: Understanding Query Performance in NW BI and BIA have been in the market for
about 5 years,SAP BW 7.3 beta is available albeit mainly focusing on the analytic layer of BW (in contrast to the DW
layer). When you continue this line of thought and when you assume that HANA is not only BWA but also able to
comply with primary storage requirements (ACIDetc.) then the huge potential opens up to support, for example,
integrated planning (BW-IP): atomic planning operators (used in planning functions) can be implemented
natively inside HANA, thereby benefitting from the scalability and performance as seen with BWA and OLAP and also
from avoiding to transport huge volumes of data from a DB server to an application server,
data store objects (DSOs): one can think of implementing such an object natively (maybe as a special type of
table) in HANA, thereby accelerating performance critical operations such as the data activation.
This is just a flavour of what is possible. So, overall, there is 4 potential and interesting HANA-based scenarios that I
see and that are summarized in figure 1. I believe that HANA is great technology that will only come to shine if the
apps exploit it properly. SAP, as the business application company, has a huge opportunity to create those apps. BW
(as a DW app) is one example which has started quite some time ago on this path. So the question on the BW-HANA
relationship has an obvious answer.
* High Performance ANalytic Appliance
** The case remains valid even if there are a few supporting data feeds, e.g. from small complementary sources.
Figure 1: How HANA can potentially evolve in existing SAP system landscapes ... in a non-disruptive way.
to improve performance of SAP BW Query. Subset of an InfoCube data is stored in an Aggregate. As a result, the response time that
we get out of reading the data from aggregate will be much faster than reading from an InfoCube.
When a query gets executed it is split into several sub queries. Split of the query is based on the following rules:
Condition 1: Parts of the query on different aggregation levels are split.
Condition 2: Different Selections on characteristics are combined
Condition 3: Parts on different hierarchy levels or parts using different hierarchies are split.
Aggregates that we build should meet the needs of query navigation and several sub queries within each query. If we have more than
one aggregate built on top of a cube, after the query split, OLAP Processor searches for an optimal aggregate for each part. This
means, a single query using a multiple sub query can access more than one aggregate to get the desired output. Refer to the flow
chart shown below. At the end, OLAP processor consolidates all the results and gives the desired output.
If a query performance is poor, we can use the following 2 thumb rules to decide if Aggregates will actually help improve performance.
There is a tool called Work Load Monitor (ST03N) that is used to track the System Performance. It basically
helps to analyze the queries with highest run time.
DB time can also be determined by executing the Query through RSRT. The EVENT ID 9000 (Data Manager):
highlighted in the screen shot below gives the DB time.
Summarization ratio can be seen by executing the query from RSRT transaction. Select the query you want to monitor the
performance and click on execute and debug button as shown in the screen print below
Usage: Every time a query hits the aggregate, Usage counter gets incremented by 1. If your Usage counter is zero, in 99% of the
cases, you can safely assume that your aggregates are not being used.
Last Used Date: Along with the Usage Indicator, another important parameter to be kept in mind is the Last Used Date. For an
Aggregate, Usage Indicator might show some big number. But when you actually take a closer look at the last used date, It might show
a date 6 months prior to the current date. This means that this aggregate has not been used by any query in the last 6 months. The
reason might be that there were many changes made at the query level but the aggregates were not updated or modified accordingly.
Valuation:
Valuation is the systems best guess of judging how good an aggregate is. Usually, if the summarization ratio is high, the number of +
signs in the valuation will be more. There could be some aggregates where the number of plus signs in an aggregate is more, but the
aggregate as such is never used. Though the valuation will give a fair idea about the aggregates created, it need not be 100% right.
Once the data is loaded into the cube, following steps are to be carried out- for the data to be available for reporting at the aggregate
level.
Change Run (Master Data Activation): To activate the changes of the master data. All the data containing the
navigational attributes gets realigned.
Adjustment of time dependent aggregates: This is done to recalculate the aggregates with time dependent
navigational attributes.
Any aggregate that is created on a cube has to be periodically monitored for its usage and the last used date.
If there is any aggregate that is not being used, it has to be either modified or removed
Un-used aggregates add to the load performance. It is always a good practice to drop the aggregates that are
not being used.
By default blocksize is set to 100.000.000. SAP recommends changing this setting to a value between 5.000.000 and 10.000.000. In
this way, we reduce joining or sorting on disk and also reduce log space consumption. You should not set BLOCKSIZE to lower values,
because this can result in a WHERE condition that forces the optimizer to use an index other than the clustered index. Suggest to use
5.000.000 for systems with less than 10 GB of real memory. If you have more real memory SAP recommends setting BLOCKSIZE to a
value up to 10.000.000.
Ideally the index on the Time dimensionshould be used when reading data from the fact table, because the fact table is clustered
according to the time dimension. In many cases index on data request dimension is chosen.
The tool helps you to identify how much block size you can actually generate with your existing aggregates. Use button "Pre-Analysis of
the aggregate filling" in the aggregate maintenance window to see the SQL statement that is used depending on the value of parameter
BLOCKSIZE.
We can explain this statement in ST05 and check, which INDEX is used to access the FACT table and then act accordingly.
During conversion DataStore objects are not available for reporting / staging. Querying on the InfoCubes is however
possible during this time.
After migration to the SAP HANA database, normal standard InfoCubes are in the SAP HANA database's column-based
store and have a logical index (Calculation Scenario). In the analysis, they behave like BWA-indexed InfoCubes.If the
InfoCubes have data persistency in BWA, the content is deleted during the system migration to HANA and the InfoCubes
are set to inactive. If you want to continue using one of these InfoCubes as a standard InfoCube, you need to activate it
again and reload the data from the former primary persistence (DataStore object for example).
We can make DSO as an In-Memory based DSO only if it is of Standard type. Also Standard DataStore objects can be
converted into SAP HANA-optimized DataStores only if:
1) The DSO is not included in a BW 3.x data flow (for example, by using update rules).
Exception: If you use the BW 3.x data flow only on an inbound basis (but not on an outbound basis), you can still convert
the DataStore. However, in general, we recommend that you migrate the old data flow before converting the DataStore.
See: http://help.sap.com/saphelp_nw73/helpdata/en/8d/6b1b58cc1744e1b</p><p>ce7898a50e19368/frameset.htm
2) DSO is not supplied by real-time data acquisition. Since small data volumes are usually produced for each activation
step when data is supplied by RDA, RDA-supplied DataStores do not benefit in the same way as staging-supplied objects.
Continue to use classic DataStore objects for this use case instead.
3) DSO is not part of a HybridProvider or a semantically partitioned object (SPO).You can recreate SPOs based on SAP
HANA-optimized DataStore objects
Different options available for DSO migration:
1) Simple Migration: Only Active data gets converted and change log history gets deleted. Conversion is faster and
requests cannot be rolled back after migration.
2) Full Migration: Active data gets converted along with change log history. Conversion is slower but there is no restriction
on rollback of requests.
So if the source system delta extraction is to be achieved for the timely base, the same data can be pulled into BW easily
by a process chain.
However, you can change this standard delivery. For more information, see note 485958.
Solution
It can be achieved by changing the settings at back end in the source system.
As the requirement is to get the deltas for every 30 minutes (1800 Secs), it needs to be update in the table
BWOM_SETTING. We need to create an entry or change the existing entry for the field/Param_Name BWFINSAF with
value 1800 (secs).
So now the Delta pointer for the datasource will be set to for every 30 mins, So that new deltas are created in source
system.
Coming to BW side, we need to schedule the start variant in the process chain for every one hour ot two hours, depending
on the time it takes for completion of the chain. So now the latest FI GL data can be reported.
So after loading the data into the cubes, the Bex queries will be able to fetch the latest financial data in the report output.
Even the business object reports, OLAP universes based on the Bex queries will fetch the latest FI data and show the
same data for the BO reports such as WebI or crystal reports based on the BO Universe.
Related SAP Notes:
SAP Note 991429
SAP Note 485958
1) Go to DWWB (RSA1).
In the present
version of Master data deletion functionality, we find that system checks if
the desired info object has any
dependencies i.e. info object being used in Master data like Hierarchies or in
Transaction data like Info Cubes.
So the system takes huge time to delete and also may lead to
memory overflows.
The new functionality looks like as shown below:
1) Delete SIDs:
<br /></p>
<p>This option is
used to delete the SID values in the /BIC/S<Info object name>.</p>
<p> We can choose if we want to delete the SIDs
also. If we choose to delete SIDs we have to check option Delete SIDs. This
will ensure that all SIDs are deleted.</p><p> </p><p>!https://weblogs.sdn.sap.com/weblogs/images/252146487/3d.jpg|
height=250|alt=|width=600|src=https://weblogs.sdn.sap.com/weblogs/images/252146487/3d.jpg!</p><p> </p><p>System
will pop up a warning, before you delete SIDs as
shown below.</p><p> </p><p>!https://weblogs.sdn.sap.com/weblogs/images/252146487/16d.jpg|height=150|alt=|
width=600|src=https://weblogs.sdn.sap.com/weblogs/images/252146487/16d.jpg!</p><p> </p><p>SAP recommends not
deleting
SIDs as it may lead to consistencies when reloading the same data.* *</p>
<p>In Older version, before deleting we have
to ensure that the chances or reloading the same characteristics is less. Or
else when we reload the data it may lead to inconsistencies if the where-used
list check performed by the system was not complete.</p>
<p>In the newer version, we can check the usage of
the Master data using Search Mode as per our requirement. We have 4 different
search modes which I will be discussing in detail later in the section.</p><p> </p><p>2) Delete Texts:
<br />
3) Simulation mode:
When we delete
master data, we have to check if it is used in info cubes, hierarchies before
deleting. As we have to lock these dependent objects for deleting data in
them. So we need a down time to execute
deletion.
Now you can monitor the background job using SM37. The job will be created by the user name who created it or choose
Application log from the below screen for the same action.
Where-used Table:
<br />
Here you can see the message stating that master data
is deleted in the background as below:
!https://weblogs.sdn.sap.com/weblogs/images/252146487/9d.gif|height=455|alt=|width=395|
src=https://weblogs.sdn.sap.com/weblogs/images/252146487/9d.gif !
2) One usage per value per Object:
We use this to
know about the usage of master data in all InfoProviders.
The system performs where-used check and if
it finds a value being used by each InfoProvider then search continues for
other values in the InfoProvider.
!https://weblogs.sdn.sap.com/weblogs/images/252146487/12d.jpg|height=250|alt=|width=600|
src=https://weblogs.sdn.sap.com/weblogs/images/252146487/12d.jpg!
<br />
us with complete where-used list and takes the maximum run time.
Selective Deletion:
We can use report RSDMDD_DELETE_BATCH for selective deletion from
any Info Object and give the selections using FILTER button. Simulation options can be given in P_SMODE.
Data Acquisition DataSource (RDA), which could potentially provide near real-time reporting from a BWA.
A typical usage scenario could be that you want to extract your PurchaseOrders from R/3 and make available for
reporting. Using a HybridProvider, as soon as the data is loaded into a DSO they then become available for reporting with
all the benefits of an InfoCube and
BWA.
A hybrid Provider within the Administrator Workbench
2. Graphical Dataflow modeler
A graphical way of building a datamodel has been introduced in BW 7.3. The Graphical Dataflow Modeler is a new
function within the Administrator Workbench (transaction RSA1).
This provides:* Top-down modeling of a new dataflow* Organization for existing dataflows* Creation of template dataflows
using best practice* SAP delivered pre-defined dataflows
Using a drag-and-drop interface, a developer can build a dataflow by selecting either existing objects, or by directly
creating new objects from within the modeler. The entire dataflow can be modeled in this way, from the DataSource to the
MultiProvider, and even as an OpenHub destination.
Any BW system with multiple complex projects would benefit from the retrospective implementation of dataflows. This tool
reduces the effort and complexity associated with the administration and maintenance of a system. For example, a single
dataflow could be delivered for all the data used within an application, meaning the system will become easier (and
therefore cheaper) to support.
3. Accelerated data loads
Accelerated data loads are now possible!!! - SAP has rewritten underlying code for data loads for optimized performance.
In many places in BW 7.3, the technical coding has been optimized to improve the loading performance. Load times into
DSO objects has been reduced by up to 40% when compared to BI7, plus the loading of Master Data has been
accelerated.
Improvements in load performance translate into benefits for the business. Data can be made available earlier in the day,
reloads are quicker (so cost less) and larger volumes of data can be loaded each and every day.
4. Semantic partitioning Turned Automatic!!!
With BW 7.3, the BW systems can automatically Semantically Partition your InfoProviders. Using a wizard interface, you
can create multiple partitions that store your data, handle the mappings between partitions and schedule the loading
processes. The need for the manual creation of multiple transformations, DTPs has been removed J
Semantic Partitioning is where we spread similar data across several InfoCubes or DSOs, split according to region or
year. This allows increased performance and stability, though comes with at a cost of increased development and
maintenance.
A typical scenario would be to Semantically Partition an InfoCube that is used for Global Sales reporting. By keeping to
data for each region or year separate, performance is improved, and maintenance and daily loading is easier.
Semantic Partitioning development will provide the business with both quicker and more reliable reporting, at a lower cost.
Rather than manually building Semantic Partitioning where critical, you can use it in smaller applications at little extra
cost.
Accelerated data loads - SAP has rewritten underlying code for data loads for optimized performance.
Graphical Data flow modeling and best practice data modeling patterns
1.
A robust modeling language the modeling language should be robust enough in order to express most of the
requirements from an enterprise application. If it is too simple( such as configuration tools which allows you to define a
very limited set of options and then create an application out of it) then it will cover only limited set of requirements and
therefore will help only on a very specific cases. On the other hand the modeling language should not be too complex as it
will make the modeling tool tedious and not user friendly, which will not fulfill my second requirement J.
2.
A simple modeling tool modeling tool should be used by business experts and usually not by developers, so the
modeling tool should be simple enough so the business expert will be able to focus on the business logic and not on
technical/complex staff.
3.
UI extension points as the modeling language will not cover all customer requirements, a major point here is to
provide the ability to extent the UI offering of the modeling tool. By providing the UI extension points the customers can
cover all of his specific requirements.
4.
Connectivity to various backend the ability to consume services is a must, but in the enterprise world today
companies have various sources of data ( Web services, Databases , R3 ). If you can consume different subsets of data
you are able to create applications which displays and consolidate data from various data sources.
5.
LCM- one last and very important in the enterprise world is to enable customers to deliver the applications from
development to production or to enable to work on the applications with multiple users.
Visual composer do just that. We still have some points that can be improved but in general Visual composer fulfill the
above requirements, and therefore is the right solution for creating a fully fledged enterprise applications.
I will appreciate your comments and feedback, and of course, if you think on a concrete requirement that is missing in
Visual Composer, please feel free to suggest an idea in Visual Composer Idea place.
3) This action leads to the below screen, where I have to select whether to copy the upward, downward or both the data
flows of the cube.
I choose YES to continue. This will help in creating required IP's and DTP's automatically.
5) This will lead to Dataflow copy wizard. Where I will configure below steps:
A) No. Of Copies
B) Source Systems
C) Data Sources
D) Info Provider
E) Transformations
F) Directly dependent Process
G) Indirectly dependent process
Now lets discuss more about features present in the wizard.
We can see that wizard is divided into 2 parts. Left side tells you the step which is currently in progress as well as the
errors RED traffic light (if any) in the steps. If you find all these steps with GREEN traffic light, then you can precede
with copying the data flow else the wizard will not allow you to do the same.
We will now discuss in detail about what each step in this wizard is responsible for:
A) No. Of Copies:
We can determine the number of copies to be created in this step. We can create a maximum of 99999999999 (We may
not use these many copies in the real environment).We need to mention the replacement for placeholders for technical &
description names as you can see below.
In my case, I will create only 1 copy. If it is "one" copy, we need not worry about the replacement holders as shown above.
Now I press continue to proceed to the next step in the wizard i.e. Source Systems.
B) Source Systems:
We cannot create a new source system or change the existing source system for the copy flow. But we can create a data
source from the existing source system to the new source system i.e. we can create an identical data source for a flat file
system based on SAP source system (or we can use an existing flat file system).
We have an additional feature in the wizard. We can checkDisplay Technical Names to display technical names of
the objects as shown below.
If we assign a wrong object or no object, you can find RED status as shown below.
Now I will assign a flat file source system (ex: YFF_SALES) to our new flow as shown below.
Now the status will turn to GREEN and we can precede to the next step i.e. Data Sources.
C) Data Sources:
In this Step, I will use the option Create with Template. This means we create a new data source with help of new
template.
E) Transformations:
Before copying transformations, we have to check if source and target fields are present in the Target object. If we dont
have those fields present in the Target object that particular rule cannot be copied (as the field in Target is missing).
With these I have completed all the important steps and I can proceed with copying the data flow.
Now system will ask you if I want to continue with data flow copy in the dialog or background.
If we want to check the logs once again, we can use transaction RSCOPY.
6) Hence the copying is completed. Hope you understood the benefits of this new tool.
Patching the SAP JVM on an SAP NetWeaver system is only supported using the Java Support Package Manager
(JSPM) but there is issue with patching SAPJVM6 through JSPM for distributed system JSPM tries to start PAS and DB
for SAPJVM deployment, but its not able to do so because they are on different servers and it gives error.
In that case we can do manuall patching like below first modify Instance profile parameters for JVM 6
_CPARG0 = list:$(DIR_CT_SAPJVM)/sapjvm_6.lst
_CPARG1 = source:$(DIR_CT_SAPJVM)
SAPJVM_VERSION = 6.1.032
DIR_SAPJVM = $(DIR_EXECUTABLE)$(DIR_SEP)sapjvm_6
jstartup/vm/home = $(DIR_SAPJVM)
Then uncar sapjvm6 and keep it at location of jvm directory, there will be variation in directory name as per your OS. If you
are not using PI or Java then this directory structure may not present and upgrade is not needed in that case.
Start SUM
With sid<adm> on Primary Application Server
> /usr/sap/<SID>/SUM/sdt/exe/DSUGui
Please take a glimpse of this tool it will remind you of SAP version upgrade and Ehp ugrade tool !
For detail please check SAP Note 1557404 - Central Note - Software Update Manager
http://help.sap.com/saphelp_nwpi71/helpdata/en/46/c24a5fb8db0e5be10000000a1553f7/content.htm
rdisp/shutdown/disable_gui_login - This parameter will suppress GUI logons during the shutdown. This applies
irrespective of user roles or authorizations
rdisp/shutdown/gui_auto_logout - Defines the period for which dialog users can be inactive during the kernel
shutdown, before they are automatically logged off. If greater value than 0 will be used, the maximum wait time is
calculated from the minimum of the parameters rdisp/gui_auto_logout and rdisp/shutdown/gui_auto_logout.
rdisp/shutdown/idle_wp_timeout - How long the kernel shutdown waits for all work processes to be in the status
"waiting".
rdisp/shutdown/message_frequency - How frequently dialog users are requested to log off during the server
shutdown.
rdisp/shutdown/j2ee_timeout - How long the kernel shutdown waits for the AS Java (JEE Engine) to shutdown.
rdisp/shutdown/load_balance_wait_time - How long the kernel shutdown waits for the server to be deleted from
the load balance information. During this time, all requests can continue to be processed.
rdisp/shutdown/abap_trigger_timeout - Defines how long the kernel shutdown waits for the shutdown trigger to be
read by the Auto ABAP.
The Soft Shutdown option is available from SAP transaction SM51 and with help of SAP MMC
HANA
For reference please visit below links
http://www.sap.com/corporate-en/press.epx?pressid=15178
http://www.sapcloudcomputing.com/technology/architecture.html
http://www.wftcloud.com/
With these 20 points I would like to wind up this blog, but I will appriciate if readers suggest more points which they know
but we should stick to the Basis perspective
----------------Adding this point as suggested by Hemanth Kumar Jul 10, 2012---------------21. From NetWeaver7.1 sap introduced new tool in place of JCMON. This tool is JSMON
you can type help and get multiple options that can be used, like below screenshot you can type instance, process,
display and see the result.
Email This