1

Cache Management
Decision support queries sometimes require a large amount of database processing. Requests for the same information require frequent trips to the back-end databases to retrieve the query results. Such trips can increase query response time and result in poor performance from the users perspective.

Cache
Cache is a physical memory in server hard disk. Oracle BI Server can save the results of a query in cache files and then reuse them later when a similar query is requested. The query cache allows Oracle BI Server to satisfy many subsequent query requests without having to access back-end databases. This reduction in communication costs can dramatically decrease query response time.

Oracle BI Query Cache Architecture

A query cache is a facility that stores the results from queries. If a query is fulfilled by the results stored in the cache, it is called a cache hit. A cache hit means that the server was able to use the cache to answer the query and did not go to the database at all.

Advantages of using Cache

Eliminates unnecessary database processing because pre-computed results are stored in a local cache Improves query performance by fulfilling a query from the cache as opposed to searching through the database. Conserves network resources by avoiding a connection to the database server.

Disadvantages of not using Cache

Significantly reduces query performance Increases network traffic Increases database processing Increases BI Server processing overhead

Cache Management

Disadvantage of an improperly maintained cache

Contain stale data (Old data) Contain irrelevant or rarely used data Contain data too specific to be useful in improving performance

Types of Cache
There were 2 types of cache BI Server Cache Presentation Server Cache If there is scheduler server is installed then there will be one more cache type Scheduler cache

BI Server Cache
BI Server cache is stored in the directory: C:\Oracle\OracleBIData\cache (C = the disk Where the OBIEE is installed). BIquerycacherequiresdedicatedhighspeed,highreliabilitydiskspace BI Server Cache can be managed by using NQSConfig file. We can disable cache for a single table. In Physical layer open a table properties, in the general tab disable the check box Cacheable. CacheautomaticallyenablesforAliastableswhenweenableforbasetable.

The RPD should be in online to handle the cache (Sever should be up to handle cache).

Managing BI Server Cache (using NQSConfig.ini file) Enable/Disable BI Server Cache

We can enable or disable cache for entire environment of BI Server by making NQSConfig file [Enable = Yes/No] under the head CACHE.
We can also change the following BI server cache settings in NQSConfig file. The location/directory path for the cache files Size of the directory The number of cache entries in the path The size of each cache entry The maximum number of rows per each cache entry

Cache Management

Presentation Server Cache

BI Server cache is stored in the directory: C:\Oracle\OracleBIData\tmp (C = the disk Where the OBIEE is installed). By default there is no cache for presentation server, to enable Presentation server cache we need to write an xml code in instanceconfig.xml file. (Tip: xml code will be available in internet) In the Presentation services Manage Sessions what we see is log files they are not actually caches. They are just information provider log files. By default, the cache files for the presentation server reside in the \tmp directory within the respective subdirectories sawcharts, sawrptcache and sawvc; while the xml cache files lying in the \tmp folder itself. Chart Cache - \OracleBIData\tmp\sawcharts\ Report Cache - \OracleBIData\tmp\sawrptcache\ State Pool Cache - \OracleBIData\tmp\sawvc\ XML Cache - \OracleBIData\tmp\ Presentation Server Cache can be managed by using instanceconfig.xml file. Enable/Disable Presentation Server Cache To enable Presentation Services Cache we need to write the following code in instanceconfig file. <ServerInstance> <Cache> <Query> <MaxEntries>100</MaxEntries> <MaxExpireMinutes>60</MaxExpireMinutes> <MinExpireMinutes>10</MinExpireMinutes> <MinUserExpireMinutes>10</MinUserExpireMinutes> </Query> </Cache> </ServerInstance> To disable Presentation services cache we need to write the following code in instanceconfig file. <ServerInstance> <ForceRefresh>TRUE</ForceRefresh> </ServerInstance>

Cache Management

Purging
Cache files always produce the same results, even after a database has been updated. Not purging outdated caches (known as stale caches) can potentially return inaccurate results over time and consume disk space. To maintain cache properly we need to Purge stale data. Purge means cleaning/deleting cache. Stale data means old or historical data. Purging can be done only in Online mode

Methods of Cache Purging

a. Manual Purging i. Purging Cache for Single entry in BI Server ii. Purging Cache for all entries in BI Server iii. Purging single request Cache in Presentation Server iv. Purging Cache for all entries in Presentation Server v. Purging all caches at a time (Bi Server & Presentation Server) b. Automatic Purging i. Persistence time on physical table ii. Event pooling table iii. Connection scripts iv. Purging cache with a batch file v. Ibots c. Bypassing Cache for a single request i. Bypassing cache for a single Presentation server request ii. Bypassing cache for a single BI Server request

Manual Purging
Manual purging means purging the cache manually i) Purging Cache for Single entry in BI Server: Select Manage > Cache to open the Cache Manager. You will observe the list of cache records; select the cache files which you want to purge. The Cache Manager can be viewed in online mode only

o o

o o

Select Edit > Purge to purge the records. This will automatically deletes cache files in the path C:\OracleBIData\cache.

Cache Management

5
o It is possible to delete cache entries for a single table, multiple tables or the entire schema. Select the Physical tab in the Cache manager and select the tables or schema and go to Edit>Purge. ii) Purging Cache for all entries in BI Server: Select all the cache entries in the cache manager and go to Edit> Purge, so all the cache entries will be purged. iii) Purging single request Cache in Presentation Server: To purge presentation server caches, go to Presentation Services. Go to Settings > Administration > Manage Sessions > -To purge a single Presentation Server cache Click on Close of that particular request.

o o

iv) Purging Cache for all entries in Presentation Server: To purge all the long running requests, click on Close All Cursors button v) Purging all caches at a time (Bi Server & Presentation Server): In Presentation Services go to Settings > Administration > Issue SQL Write the command Call SAPurgeAllcache() and click on Issue SQL. (Command is case sensitive) All the presentation server caches and Bi server caches will be purged.

o o o

Cache Management

Automatic Purging
i) Persistence time on physical table: I. By default each and every physical table will have property of Cache never expires. II. If you enable caching for a table, any query involving the table is added to the cache. III. If we know source table loading time accurately then we can purge data using Cache persistence time. IV. Set cache persistence time to indicate how long entries for this table should be kept in the cache. V. In this picture example, for the Channels physical table we selected Cache persistence time = 2 hours. That means if a report is run using Channels table as a source then that report output cache will be purged in every 2 hours.

ii) Event Polling Table The event table is a physical table that resides on a database accessible to Oracle BI Server.

The use of an Oracle BI Server event polling table (event table) is a way to notify the Oracle BI Server that one or more physical tables have been updated. o Whenever there is an update (insert or delete or update) in the physical tables, a row will be inserted in event table. o Steps to implement Event polling table 1. Creating Event pooling table a. With the installation of OBI, the EPT table syntax notepad will be stored in the location. C:\OracleBI\server\Schema\SAEPT.Oracle

Cache Management

7
b. Create S_NQ_EPT table in database 2. Creating Trigger or post SQL a. We need to create a Trigger or post SQL (SQL INSERT statements) to insert a record each time when there is an update in the physical table. b. In SQL Plus execute below script: Create or replace Trigger Event_Table_Update_Trigger after insert or update or delete on SH.Products for each row begin insert into SH.S_NQ_EPT (Update_Type, Update_TS, Table_Name) Values (1,sysdate,Products); End; / c. If we are using backend ETL tool as Informatica then we can use Session Post SQL to insert the data. 3. Importing S_NQ_EPT table into Physical layer and configuring as EPT table a. Import the created S_NQ_EPT table into Physical layer. b. To configure the table as EPT table, go to Tools > Utilities > Oracle BI Event Tables c. Select S_NQ_EPT table and give Pooling frequency time. d. Oracle BI Server observes this new data S_NQ_EPT and purges Products table corresponding request output.

iii) Connection scripts o Go to connection pool properties, open the Connection Scripts tab. o Write the prebuilt query script in any of the executing procedure according to your requirement. Ex: Call SAPurgeAllcache()

Cache Management

8
iv) Purging cache with a batch file o It is the most used method of purging the cache in real time. Steps: o Create a notepad with .sql extension with any one of the cache functions to purge cache according to your requirement. Ex: Here we are creating Purge.sql with the function Call SAPurgeAllcache() o Save the file in the folder C:\OracleBI\server\Repository where the OBIEE is installed. o Create purge.log file in the same folder, which helps to check the status of the batch file. Creating log file is not mandatory. o Open note pad and create the following purge_batch batch file and save it in a folder. nqcmd -d AnalyticsWeb -u Administrator -p Administrator -s C:\OracleBI\server\Repository \purge.sql -o C:\OracleBI\server\Repository \purge.log The file has the following format: nqcmd Oracle BI Server command utility -d Oracle BI Server data source name -u repository username -p repository password -s path to the create aggregate SQL script -o path to the log file o Ensure that the Oracle BI Server service is started. o Open command prompt and go the path where the purge_batch batch file saved. Run the batch file. Or we can just double click on the created batch file. o All the cache entries will be purged. Note: To run the batch file we need password for the repository and the Administration toll must be opened to create batch file.

Bypassing Cache for a single request

o o I) Bypassing single Presentation server request If we want to purge for a single presentation server request. In the Presentation services page, go to Advanced after building a request. Enable Bypass Oracle Presentation Services Cache check box.

o o

ii) Bypassing cache for a single BI Server request In Answers, go to Advanced Reporting tab when building report, set Prefix value: SET VARIABLE DISABLE_CACHE_HIT=1, DISABLE_CACHE_SEED=1, LOGLEVEL=7;

Cache Management

Cache Seeding
Inserting data into cache is called as Seeding. Seeding the cache is the process of pre-populating the cache with queries that are known to generate cache hits. One of the main advantages of seeding the cache is the improvement of query performance. Is performed by running prebuilt queries during off hours or immediately after purging o We can do seeding in 2 ways 1. Scheduling a report 2. Manually executing request 1. Scheduling a Report: Use Oracle BI Delivers to schedule queries automatically to run at a specified time. Create an iBot to run a request automatically at a specified time, it will create cache automatically. 2. Manually executing request: Executing a request in Oracle BI Answers manually.

Cache Functions (Scripts)

By using some prebuilt queries, we can purge the entire cache in one shot or purge the cache by query or purge the cache by table or purge the cache by database. We can use these functions in Connection scripts, Issue SQL page of presentation services. i. SAPurgeAllCache() This is the function will clear the entire cached query in one shot. Ex: Call SAPurgeAllCache(); ii. SAPurgeCacheByQuery() We can issue a command to purge a specific query from the cache; the command takes a single parameter, which should be equal to the exact query registered in the cache. An example is given. Ex: Call SAPurgeCacheByQuery(SELECT EMPD, EMPNAME from HR_EMP_D WHERE SALARY > 100000 ); iii. SAPurgeCacheByTable() We can select to purge the cache for all queries using a specific table; we must specify the database, physical catalog, schema and table as parameters. Ex: Call SAPurgeCacheByTable( DBName, PhyCatName, SchemaName, TableName ); iv. SAPurgeCacheByDatabase() We can purge the cache for all queries to a specific database. Ex: Call SAPurgeCacheByTable( DatabaseName );

Cache Management

10

Standards and Best Practice

1. Choose a cache management strategy based on how often the data in the database is updated. 2. Queries returning a large amount of data may not be ideal candidates to be cached as large data set will also cause the cache file to be large. The larger the cache, the less performance enhancement can be derived from the cache. 3. Situations in which particular data elements must be refreshed frequently or on a real time basis may also not be good candidates for caching solution. 4. Purging the cache entries using an Event polling table is not beneficial when the incremental ETL is run once a day or overnight. 5. Parameters for the event table contain table names only and cannot contain an alias. This can lead to misleading results and an alternative purging strategy must be found for the alias. 6. A single event polling table cannot be shared by multiple Oracle BI Servers. When you set up multiple Oracle BI Servers, you need to create an event polling table for each one. 7. Do not set the polling frequency to less than 10 minutes. If you want a very short polling interval, consider making some or all of the tables non-cacheable. 8. Specify multiple cache files on different local high performance disks for the parameter DATA_STORAGE_PATHS. This allows the BI Server to store and retrieve cached results by spreading the I/O across the disks used. Furthermore have a separate disk for each cache folder in the following format:
DATA_STORAGE_PATHS = /<localhighperformancedisk>/OracleBIData/cache1 1024 MB, /<localhighperformancedisk>/OracleBIData/cache2 1024 MB, /<localhighperformancedisk>/OracleBIData/cache3 1024 MB;

Cluster aware caching specifies multiple cache files on different shared high performance disks for the parameter GLOBAL_CACHE_STORAGE_PATHS.
GLOBAL_CACHE_STORAGE_PATH = /<sharedhighperformancedisk1>/cache1 1024 MB, /<sharedhighperformancedisk>/cache2 1024 MB, /<sharedhighperformancedisk>/cache3 1024 MB;

NOTE: Those folders Cache1, Cache2 and Cache3 are separate hard disks, not just three partitions on one hard disk.

Cache Seeding
1. 2. 3. 4. Use queries that require a great deal of database or server processing. For example, queries with many joins or a great deal of sorting. Use queries with no WHERE clause that will populate many sub queries. For example, a seed statement of i. SELECT User, SR from SERVICE_REQUESTS ii. Will provide cache hits for all queries of the form iii. SELECT User, SR from SERVIC_REQUESTS iv. Where User = xx 5. Note: The query that gets seeded must be under the governance limits. 6. User most commonly used queries 7. Caching a query with 23 joins that is called once a day is not as important as caching a query with no joins that is called 5000 times a day 8. Use SELECT lists with no expressions.

Cache Management

11

FAQ
Q: What is Caching? List the merits of Caching? Ans: Caching is a OBIEE feature that enables to cache the query results against the UserID for a business model. This feature is used to improve performance by caching query results of requests that are used repeatedly. Reduces workload on database and reduces network traffic. Q: OBIEE cache is enabled, but my query is not cached? Why? Ans: Physical tables in the OBIEE repository can be marked non-cacheable. If a query makes a reference to a table that has been marked as non-cacheable, then the results are not cached even if all other tables are marked as cacheable. Q: All my physical tables are marked as cacheable, but still the query is not cached? Ans: 1) If a request contains certain SQL functions, OBIEE will not cache the query. The functions are CURRENT_TIMESTAMP, CURRENT_DATE, CURRENT_TIME, RAND, POPULATE. OBIEE will also not cache queries that contain parameter markers. 2) Result set is too big: The query result set may have too many rows, or may consume too many bytes. Than specified in nqsconfig file. 3) Query execution is cancelled: If the query is cancelled from the presentation server or if a timeout has occurred, cache is not created. Q: Tell me your roles of cache management? Tell me Cache management techniques you work? Ans: Cache Management Techniques Configuring the cache parameters Setting caching and cache persistence for tables Using the Cache Manager Inspecting SQL for cache entries Modifying the Cache Manager Column display Inspecting the cache reports Purging the cache entries manually using the Cache Manager Purging the cache entries automatically Using event polling tables Seeding the cache Q: I have 6 tables for 5 tables cache is enabled and for 1 table cache is disabled. If we run a query involved all the 6 tables, then is the query cacheable or not? Ans: No, as one table cache is disabled, all the querys involved in that table cache will not generated. Q: How to purge BI Server cache when the developer has no access to repository? Ans: By using Call SAPurgeAllcache() command in Issue SQL page of presentation services page Q: what is the BI Server file name format? Ans: NQS_RRITEC_734757_78447_00000001.TBL NQS(Prefix)_VMVGGOBI(Originating Server Name)_733547(Days passed since 1-10000)40458(Seconds passed since last midnight)_00000006(Incremental number since last BI server start).TBL

Cache Management

12

Q) When a dynamic repository variable is updated, cache is automatically purged? Why? Ans: This is designed behavior. Cache will be purged whenever the initialization block that populates dynamic repository variable is refreshed. The reason that refreshing a variable purges cache is that if a variable was used in a calculation, and the variable changed, then cache would have invalid data. By purging cache when a variable changes, this problem is eliminated. Q: What are the disadvantages of using Event Pooling Table to purge cache? Ans: EPT is not applicable for Alias tables, so we need to create separate EPT for alias. For each schema we need create a separate EPT table. So in real time mostly we dont use EVT for purging the cache.

Cache Management