4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

O racle

Blogs Home Products & Services Downloads Support Partners Communities About Login Oracle Blog
Oracle BI applications blog Oracle BI applications blog

Oracle BI apps 11.1.... | Main | Implementing Oracle...

Load Plan Generator An Inside Look

By Saurabh Verma on May 14, 2013 Authored By : Don Co Seng Oracle BI Applications (BIAPPS) 11.1.1.7.1 was recently released and is now available for download. One of the biggest features in this release is the introduction of Oracle Data Integrator (ODI) as the embedded ETL tool. With the introduction of ODI, a new method of orchestrating the task of loading your data warehouse is now required. This requirement brings about the introduction of Load Plan Generator (LPG). LPG is a utility for generating ODI load plans for a desired subset of fact tables to be populated into BIAPPS Data Warehouse against one or more source systems. An ODI load plan is an executable object in ODI allowing you to
https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside 1/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

organize tasks based on pre-defined order on the basis of the fact tables being loaded. LPG is invoked from Configuration Manager (CM) and makes use of metadata stored in CM and ODI repository. There is no separate repository required for LPG. This results in significantly lower metadata development and maintenance costs as LPG uses same metadata as in the ETL tool repository. Key Concepts Before we get to the inner workings of LPG, lets take a quick look at a few key concepts that drive the LPG behavior. BI Apps taxonomy At the top of the BIAPPS taxonomy are the different offerings available such as Financial Analytics, HR, CRM, etc. Under the offerings are the functional areas such as accounts payable in finance or payroll in HR. Below the functional areas are the different fact groups like AP Transactions and Balance or Payroll Balance. For each fact group, there are dimension groups associated with it. A dimension group can be specific to a particular fact group or shared across different fact group. The BI apps taxonomy drives both load plan generation as well as the setup flows in Functional Setup manager. Offering (contains) -> Functional Areas (contains) -> Fact Groups (associated to) -> Dimension Groups Phase As for BIAPPS load phases, it is similar to that of previous BIAPPS releases. There are 3 main phases: Source Data Extract (SDE), Source Independent Load (SIL), and Post Load Process (PLP). The SDE phase consists of tasks extracting data from your different source systems. This phase loads all your staging tables and requires source system downtime. The SIL phase loads data from your staging tables into their respective dimension or fact tables. The PLP phase loads data into aggregate tables or some other facts requiring additional data processing. The SIL and PLP phase requires data warehouse downtime to complete the load process. With BI apps 11.1.1.7.1, due to clear separation of SDE and SIL/PLP phases its possible to control and optimize the source system / warehouse downtime to the corresponding phases. Design Time vs. Runtime dependencies The LPG algorithm deals with the design time dependencies and run time dependencies separately. Any intra-entity specific design time dependencies are seeded in the repository. For example if a dimension has multiple steps to load the target table in
https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside 2/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

SIL phase, these steps are seeded once in the ODI repository as they are known at design time and never change. The run time dependencies i.e. association of dimension to fact, or association of a dimension or fact to corresponding staging tables on the basis of sources are calculated by the LPG engine. This approach reduces the number of iterations and the metadata required to get task ordering correct at the entire graph level as the design time dependencies are always consistently honored. LPG behavior is deterministic and does not require multiple iterations to get your entire graph behave correctly. Load Plan Generator Now, lets take a closer look at how LPG works. At the core of LPG are load plan components. The load plan components capture the design time dependencies. There is a load plan component per dimension or fact per phase in general. Load plan components are used as building blocks by LPG to generate a complete load plan. Load plan components are further classified into two categories: Development components are defined at the grain of fact groups or dimension groups as described earlier. Each component contains one or more references to actual ODI scenarios. Each development component requires all steps for loading a particular dimension or fact group. The order of scenarios is pre-defined during development. Most of the development components are for a specific dimension or fact group. However, there are also those defined for supporting tables like persistent staging or general tables. System components are defined to capture the phases and ordering of phases. The system components are mostly static, since BIAPPS load phases rarely change. For special cases, the dependencies across dimension groups and fact groups are defined here. Also, inclusion of support tables such as persistent staging table is controlled here. System components contain references to development components. Generating a Load Plan As mentioned earlier, LPG is invoked from CM. Generating a load plan is a two step process. First, a user creates a load plan definition using CM. A load plan definition is mainly a list of one or more fact groups per source system which the user desires to load. The user then invokes LPG to generate a load plan based on this load plan definition. When complete, the generated load plan is stored back in a load plan folder in ODI repository. Load Plan generation logic

https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside

3/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

So, how does LPG figure out which dimension or fact group component to include in the load plan to be generated? LPG generates a load plan utilizing the following information: 1. Fact tables belonging to selected fact groups o LPG determines the fact tables and fact group association via an ODI flex field at the fact table level in the ODI data store. 2. Dimensions dependencies to Facts o The main source of dependency information between dimensions and facts are the foreign key constraints between them. Dimension to dimension dependency (snow-flake) is also supported up to the 2nd level. 3. Staging tables related to Facts and Dimensions o Staging and warehouse tables are resolved via BIAPPS data model naming standard. For example XXX_DS is the staging table for XXX_D dimension table. 4. Related PS or TMP tables used as source in scenarios o PS and TMP dependencies are resolved based on which are used as source or lookup table in scenarios required for the dimension or fact group. 5. Keywords in load plan steps for domains and class dimensions o An additional resolution is through the use of keywords in load plan steps which are used mainly for resolving steps within class dimensions or domains. These are generic tables used across facts / dimensions and the keywords help associate as subset task for these tables to specific facts or dimensions. Table Maintenance For table maintenance, additional steps are added in the generated load plan to perform truncate, index management, and table analysis based on pre-defined settings and table types. We will cover this in more detail in a follow up post. Multisource Support For multi-source support, the user can create a load plan definition containing one or more source system for a fact particular fact group. There are 3 sequencing options in this release for multi-source support: 1. Extract-Load, Extract-Load, Extract-Load (EL, EL, EL) LPG generates separate load plans containing all phases for each source system. Each load plan is executed serially of each other. 2. Extract, Extract, Extract, Load (E,E,E,L) LPG generates multiple SDE only load plan for each source and a single SIL/PLP load plan. Each SDE only load plan will be sourcing from a particular adaptor. The SIL/PLP load
https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside 4/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

plan should be executed after all of the SDE load plans. Each load plan is executed serially of each other. 3. Extract-Extract-Extract-Load (E-E-E-L) LPG generates a single load plan containing multiple SDE phases and one SIL phase. This load plan will simultaneously extract data from different source systems. After the SDE phase completes, the SIL followed by PLP phase ensues. For more details and information on LPG, please refer to documentation available in OTN. Category: Oracle Tags: 11.1.1.7.1 apps bi generator load lpg multisource odi plan Permanent link to this entry Oracle BI apps 11.1.... | Main | Implementing Oracle... Comments: Hi! What about how to change steps in generated load plan? I found an error in one SDE step. An only way I found to change it is to export this step, change Oracle package name in xml file and load it back to load plan. Also it seems to be necessary to delete old steps and rename new one to old name. Have you any knowledge about how to make load plan steps changes correctly? Ths. p.s. This step have an mistake in package name: SDE_ORAR1213_ADAPTOR_SDE_ORA_BOMITEMFACT_EXPLOSION Posted by Roman on November 07, 2013 at 12:55 AM PST # Hi, If i want to add a new fact table to which Fact Groups do i add it to. How can i create new Fact Group, Functional Areas etc ? Thanks Krishna Posted by guest on February 17, 2014 at 03:05 AM PST # Hi,
https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside 5/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

If i want to add a new fact table to which Fact Groups do i add it to. How can i create new Fact Group, Functional Areas etc ? Thanks Krishna Posted by Krishna on February 17, 2014 at 03:06 AM PST # Hi Roman, There are two ways to correct the issue 1. If there is an issue with the scenario name for a particular step in a generated load plan, then this may be a bug with the load plan dev component. The scenario name needs to be corrected in the load plan dev component and then the LP needs to be regenerated once the correction is done. 2. To manually edit a step in a generated load plan, the user can simply go to the properties window of the step and edit the scenario name of the step. This is a workaround for this particular generated load plan. The root cause though may still be in the load plan dev component or the scenario name generated for a package. Having said that, we checked and the scenario name is correct for the BOM Explosion LP step in PS1 release. Well have to get more detail from you to pursue this alleged mismatch further. Posted by Don on February 17, 2014 at 11:54 AM PST # Hi Krishna, If you want to add steps for loading a new custom fact table, as of this release, the recommended approach is to add it to the predefined load plan development components for X_CUSTOM_FG. Thanks, Don Posted by Don on February 17, 2014 at 11:55 AM PST #
https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside 6/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

What if you want to add a new custom PLP mapping to the load plan?Where will that go? Posted by Ashwin on March 14, 2014 at 08:17 AM PDT # Hi Ashwin, To add a new custom PLP mapping, it should go to the X_CUSTOM_FG dev components. If it is dependent on a customized fact table, make sure that the PLP load plan step is dependent on the load plan step for loading the customized fact table. Thanks, Don Posted by guest on March 14, 2014 at 10:37 AM PDT # BI Apps 11.1.1.7.1 Warehouse Schema and Repository cleanup instructions I have recently installed BI Apps 11.1.1.7.1 including ODI. During my load plans to load data from Oracle EBS 12.1.1 data source, the load plans generated various erros. I had been looking for instructions and/or documentation pertaining to clean/truncate all data loaded so far on the BI Apps Warehouse Schema and corresponding entries on the BI Apps Repository so that I can generate a fresh load plan Can you please provide the detailed instructions on this Thanks Hari Posted by Hari on March 24, 2014 at 01:12 PM PDT # Hi Hari, To reset the full warehouse, you need to 'Execute Reset Data Warehouse Scenario' Refer to this link for more information http://docs.oracle.com/cd/E38317_01/doc.11117/e37987/a1_ui_ref.htm#CACIJJBD
https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside 7/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

Thanks, Don Posted by guest on March 25, 2014 at 10:49 AM PDT # Hi Don Thank you for your quick response I did refer to the 'Execute Reset Data Warehouse Scenario' but this does not seem to work for my scenario The document explains "Execute Reset Data Warehouse Scenario: This command resets the data warehouse by truncating the W_ETL_LOAD_DATES table. This ensures that the subsequent load will truncate all target tables and do a fresh full load" In my case, initially I had loaded all Financial Analytics data including GL, AP, AR, FA etc., which did not go well.I was only able to see setup/parameter choice list data on the Answers Dashboard like Organizations, Periods etc but none of the reports contained transactional or summary data. I now want to cleanup all old data and do a fresh data to include only a small portion like GL Transactions so that I can debug any mapping issue or other issues during the load. The "Execute Reset Data Warehouse Scenario" will only truncate my new load plan related tables which is limited to GL Transactions Do you know of any other means to completely wipe out the BI Apps Datawarehouse data and start fresh Thanks Hari Posted by Hari on March 26, 2014 at 05:37 PM PDT # Hi Hari, If you want to just totally empty out the warehouse. You can go ahead and truncate all your tables.
https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside 8/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

Thanks, Don Posted by Don on March 26, 2014 at 08:26 PM PDT # Post a Comment: Name: guest E-Mail: URL: Notify me by email of new comments Remember Information?

Your Comment: HTML Syntax: NOT allowed Please answer this simple math question 9 + 65 =
Preview About Post

Oracle Blogs Admin-Oracle

https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside

9/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

Search

Enter search term: Search only this blog

Recent Posts

How does TYPE2_FLG Work in ETL 3 Modes for Moving Data to the BI Applications DW from a Source Application Database Snapshot Facts in OBIA (3) Snapshot Facts in OBIA (2) Snapshot Facts in OBIA (1) BI apps 11.1.1.7.1 Cumulative Patch 1 is available now How to Implement Object Security in Project Analytics in OBIA 11.1.1.7.1 Notes for implementing Universal adapter for OBIA Project analytics How to include Fusion DFFs Into OBIA 11.1.1.7.1 In Premise Data Warehouse BIAPPS List of ODI Variables
Top Tags

11.1.1.7.1 account adapter adaptor apps bi biapps changing chartfield compare config config_manager conversion
https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside 10/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

currency data development dff dimension duplicate etl financials fusion generator group informatica load lpg manager measure model multisource number obia obiee odi of oracle patch plan project projects psft rpd security sia slowly snapshot
https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside 11/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

type2_flg unit universal

Categories

Oracle Personal
Archives

April 2014 SunMonTue WedThuFriSat 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Today

Bookmarks

Events Oracle Technology Network Oracle Technology Network Blog Recent Articles Technology Newsletters
Menu

Blogs Home Weblog

https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside 12/13

4/29/2014

Load Plan Generator An Inside Look (Oracle BI applications blog)

Login
Feeds

RSS All /Oracle /Personal Comments Atom All /Oracle /Personal Comments The views expressed on this blog are those of the author and do not necessarily reflect the views of Oracle. Terms of Use | Your Privacy Rights | Cookie Preferences

https://blogs.oracle.com/biapps/entry/load_plan_generator_an_inside

13/13