Professional Documents
Culture Documents
net/oracle/how-to-max-out-your-fast-refresh-materialized-views/
Community
10 Dec
At Blue Gecko, we occasionally come across a problem, while providing remote DBA services for clients, that cannot be solved by only
reading the Oracle documentation. One such problem was for a client who wanted to use a MAX aggregate function in a complex, fast
refresh materialized view. The Oracle documentation says that this is not possible. As a materialized view query goes from simple to complex,
Oracle’s fast refresh mechanism starts to fail. Is it possible to overcome these limitations with a fast refresh materialized view? And if so, how?
Can we have our MAX and FAST REFRESH it too? The answer is yes and I’ll show you how.
How does a simple materialized view query with a MAX aggregate function work?
Before we dive into how to make a complex materialized view (mview) query work, let’s look at a simple mview query. How exactly does a fast
refresh of a simple mview query work when it includes an aggregate function like MAX? Let’s work through an example.
To show how oracle performs a fast refresh of a simple materialized view with a MAX aggregate function, we will start with a fairly generic
“orders” table and create a materialized view that contains the maximum order amount for each customer.
Let’s look at the contents of the orders table and the contents of the mview based on some data I added to the orders table:
6 1020 08-DEC-10 O
9 1030 08-DEC-10 O 13.33
Let’s now update a row to change the max_order_amount value for one of the customers. We will then refresh the mview and see the
change.
1 row updated.
SQL> COMMIT;
Commit complete.
We can see that customer_id of 1020 has a new max_order_amount of 100.11, up from 99.15. That went well. Let’s update the data back to
their original values to verify that fast refresh works when the max_order_amount decreases as well.
1 row updated.
SQL> COMMIT;
Commit complete.
Pretty cool. So how does Oracle accomplish this? If we take a 10046 trace of the fast refresh command we see that Oracle issues a DELETE
against the mview for the effected rows in the materialized view log (mlog) table. In this case, it is only for the one customer_id that we
updated. It then turns around does an INSERT of those rows again, getting the data from a SELECT against the orders table and filtering
down to just the customer_id values it wants by using the mlog table. The mlog table in this case is called mlog$_orders and was created in
our original CREATE MATERIALIZED VIEW LOG command. Here are the DELETE and INSERT statements from the trace:
We can see that these DELETE and INSERT statements are fairly complicated for even our simple mview query. As our mview query goes
from simple to complex, the logic to generate the appropriate DELETE and INSERT statements becomes more and more difficult.
One change we could make that will cause Oracle to stop being able to fast refresh the materialized view is to add a WHERE clause to our
mview query. There are other things we could change to make it complex as well. For example, we could add a SUM aggregate function, or a
COUNT aggregate that includes a CASE statement. We won’t go into all the different ways an mview query could become complex here. Lets
just look at what happens when we add the WHERE clause and try to refresh the mview.
For our WHERE clause, let’s restrict our query to only those orders that are in a “closed” state. We will need to add the state column to the
list of filter columns on the materialized view log. Let’s do that and also change the name of the mview from “orders_by_cust_mv” to
“closed_orders_by_cust_mv” while we are at it.
So far so good, but we haven’t updated any data yet. Let’s see what happens when we do.
1 row updated.
SQL> COMMIT;
Commit complete.
*
ERROR at line 1:
ORA-32314: REFRESH FAST of "EXAMPLE"."CLOSED_ORDERS_BY_CUST_MV" unsupported
after deletes/updates
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 2251
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 2457
And we blow up. We get an ORA-32314 error and our mview will no longer fast refresh.
We could try to use the DBMS_ADVISOR.TUNE_MVIEW procedure to try to see how Oracle would recommend modifing this mview in order
to make it fast refresh, but we strike out there as well:
DECLARE
v_task_name varchar2(30);
BEGIN
DBMS_ADVISOR.TUNE_MVIEW(v_task_name,
'CREATE MATERIALIZED VIEW closed_orders_by_cust_mv REFRESH FAST
AS SELECT customer_id , COUNT(*) as num_orders , MAX(order_amount)
as max_order_amount FROM orders
WHERE state = ''C'' GROUP BY customer_id');
END;
/
ERROR at line 1:
ORA-13600: error encountered in Advisor
QSM-03113: Cannot tune the MATERIALIZED VIEW statement
QSM-02086: mv uses the MIN or MAX aggregate functions
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 86
ORA-06512: at "SYS.PRVT_ACCESS_ADVISOR", line 202
ORA-06512: at "SYS.PRVT_TUNE_MVIEW", line 1232
ORA-06512: at "SYS.DBMS_ADVISOR", line 753
ORA-06512: at line 4
Is possible to work around this apparent roadblock? Yes, but not solely within the functionality of mviews that Oracle provides. To make this
work we are going to need to utilize mviews in conjunction with an additional table we create and some PL/SQL. Here is how.
In the simple mview example, Oracle was deleting and re-inserting the MAX value based on the changes that it saw in the mlog table. We’ll
do the same thing, although I’m going to use the MERGE SQL statement to replace the DELETE and INSERT (more on that later). After we
have the aggregate data we need in this new table, we will create a materialized view on top of both the original orders table and this new
aggregate table. We will tie it all together with a PL/SQL procedure that updates the aggregate table and refreshes the mview at the same
time.
First let’s create the new aggregate table and populate it with all the data:
In order to get our mview to fast refresh we will need a materialized view log on the new aggregate table as well, so let’s do that first. We then
modify the original mview query to pull the max_order_amount data from this new aggregate table. We join back to the orders table on the
customer_id column to complete the new version of the materialized view.
The initial data looks good. The max_order_amount column does not get updated when we refresh the mview, and it shouldn’t at this point.
The aggregate table is a plain old table, separate from the orders table. There is nothing currently keeping the aggregate table in sync with
the orders table.
Let’s run a test to show that the aggregate table does not get updated when we update the orders table. When we then refresh the mview,
we should expect the data to be out of sync. We’ll fix this in a minute.
1 row updated.
SQL> COMMIT;
Commit complete.
We updated the orders table, but the mview does not see the change as we expected. In order to have the data in the aggregate table
updated properly, we’ll want to run another MERGE query each time we do a fast refresh. This merge query will be slightly different than the
one we created to initially populate the aggregate table. We’ll do what Oracle did and use the materialized view log table to filter down the
number of rows we want to update. Let’s take a look back at the original MERGE statement we used to populate the aggregate table:
We will want to update the aggregate data with every fast refresh, so let’s create a PL/SQL procedure that will combine this MERGE query
with the refresh of the mview. We’ll grab the customer_id values that changed from the mlog table and update all the rows in the aggregate
table that match those customer_id values. If the max_order_amount doesn’t change, we don’t need to update the aggregate table. For this
optimization, we will add a WHERE clause to our UPDATE. I’ll underline the changes to the MERGE statement in our new refresh procedure
below:
Now if we reset the order data back to the way it was and try our update again, we should see the change reflected in the mview.
1 row updated.
1 row updated.
It works just the way we wanted it to, but we’re not quite done yet.
This ghost mview is an added layer of redundancy. If you feel like you have control over how and who may issue refresh commands to your
mviews, this second mview is not strictly necessary. For safety, I recommend creating this additional mview to pin the mlog entries until we
run our MERGE command.
Let’s look at what can happen without the additional ghost mview to pin the mlog entries.
COUNT(*)
----------
0
1 row updated.
SQL> COMMIT;
Commit complete.
COUNT(*)
----------
2
COUNT(*)
----------
0
We can see that the accidental refresh of the closed_orders_by_cust_mv materialized view removes the entries from the mlog. The
subsequent execution of the PL/SQL refresh procedure we created misses the update because these mlog rows no longer exist. Our data is
out of sync and nobody is happy.
Let’s add the ghost mview to pin the mlog entries. Then we will modify the procedure to refresh the ghost mview after the merge and try this
experiment again.
VALUES
(tab.customer_id, tab.max_order_amount);
COMMIT;
DBMS_MVIEW.REFRESH('closed_orders_by_cust_pin','FAST');
DBMS_MVIEW.REFRESH('closed_orders_by_cust_mv','FAST');
END;
/
When we run our update statement now, we see that an accidental refresh of the mview does not remove the entries from the mlog table.
COUNT(*)
----------
0
1 row updated.
SQL> COMMIT;
Commit complete.
COUNT(*)
----------
2
COUNT(*)
----------
0
That looks better. The ghost mview to pin the mlog entries does the trick. The accidental refresh of the closed_orders_by_cust_mv
materialized view does not remove the mlog rows and the subsequent procedure call correctly updates the aggregate table.
9 rows updated.
SQL> commit;
Commit complete.
CUSTOMER_ID MAX_ORDER_AMOUNT
----------- ----------------
1010 123.41
1020 99.15
no rows selected
The data in the aggregate table is wrong, but the final mview is remains correct. If I wanted to expose this aggregate table to the end users,
then I would change my MERGE to the more expensive DELETE and INSERT.
And there you have it. A workaround for using aggregate functions like MAX in a complex, fast refresh materialized view.
This code was tested on both 10g and 11g versions of Oracle. We started with the following materialized view definition that would not fast
refresh after updates to the data.
And we ended with a reworked materialized view, an additional table and PL/SQL procedure to support the MAX aggregate data, and a ghost
mview to pin the mlog entries for safety.
FROM orders
WHERE state = 'C'
GROUP BY customer_id) tab
ON (tab.customer_id = agg.customer_id)
WHEN MATCHED THEN
UPDATE SET agg.max_order_amount = tab.max_order_amount
WHEN NOT MATCHED THEN
INSERT
(customer_id, max_order_amount)
VALUES
(tab.customer_id, tab.max_order_amount);
COMMIT;
AND (customer_id) IN
(SELECT customer_id FROM mlog$_orders)
GROUP BY customer_id) tab
ON (tab.customer_id = agg.customer_id)
WHEN MATCHED THEN
UPDATE SET agg.max_order_amount = tab.max_order_amount
WHERE agg.max_order_amount != tab.max_order_amount
WHEN NOT MATCHED THEN
INSERT
(customer_id, max_order_amount)
VALUES
(tab.customer_id, tab.max_order_amount);
COMMIT;
DBMS_MVIEW.REFRESH('closed_orders_by_cust_pin','FAST');
DBMS_MVIEW.REFRESH('closed_orders_by_cust_mv','FAST');
END;
/
Dallas Willett
Blue Gecko – Remote DBA Services
Comments 1 Comment
Categories Oracle
Author dwillett
Leave a Reply
Name (required)
E-mail (Required)
Website
Blog
Script Library
Whitepapers
Free Oracle Health Check
Blog Categories
Search Blog
© 2011 Blue Gecko. All Rights Reserved. Site by Portent, an Internet Marketing Company
Blog
Contact
Privacy Policy
Terms of Use