oracle-developer.

net
Home Art icle s

Search

method 4 dynamic sql in pl/sql

Dynamic SQL and PL/SQL has been supported in Oracle for many versions. Before Oracle 8i, we used the low- level DBMS_SQL 11g New Features 10g New Features 9i New Features 8i New Features Misce llane o us SELECT * FROM some_package.some_function('<any dynamic query you like>'); package and more recently we have used Native Dynamic SQL to parse, bind and execute dynamic statements. Method 4 dynamic SQL, however, has never been truly possible natively in Oracle (i.e. in PL/SQL). For example, consider the following pseudo- statements:

Utilities Links Subscribe Disclaimer

How do we deal with " any dynamic query you like" ? What is it? What does it look like? How do we fetch it? How do we display it? We will answer these questions in this article and demonstrate how to fetch data dynamically using the following SQL statement...

SELECT * FROM TABLE ( dla_pkg.query_view( p_select_string ) );

...where p_select_string is an unknown SELECT statement.

an overview of method 4 dynamic sql

A Method 4 scenario is one in which any dynamic statement can be programmatically understood, executed and fetched without knowing any of the structures involved at the time of compilation. Imagine the various IDEs and middle- tier applications we use to query data from Oracle. Whether they be sqlplus, TOAD, SQL Developer or home- grown applications in Java, PHP, C++ etc, one thing that these tools and languages have in common is the ability to take a cursor or ref cursor and fully display its associated resultset, regardless of the structure of the SQL statement we execute. These tools and languages are able to und e rst and the record structures that are being returned. This is possible due PDFmyURL.com

to the various protocols used to access Oracle (OCI, OCCI, JDBC, ODBC etc). Now consider how we might handle Method 4 dynamic SQL in PL/SQL. We can easily prepare and parse a dynamic statement. We can also programmatically handle bind variables without knowing how many we are going to bind (if we use DBMS_SQL). We can execute this statement without needing to know its form and structure. But what do we fetch the returning data into? In PL/SQL we regularly fetch into variables, records and collections of records, but regardless of which variable type we use, we ne e d t o kno w it s st ruct ure at co mp ile t ime . For this reason, true Method 4 dynamic SQL is not possible in native, static PL/SQL. There are ways of achieving this however, but they are complicated and involve DBMS_SQL describe APIs and PL/SQL to build and execute a dynamic anonymous PL/SQL block. For an example of this, see t his o racle - d e ve lo p e r.ne t ut ilit y. With the Oracle Data Cartridge framework, we have an alternative method of achieving Method 4 scenarios for dynamic statements that return datasets.

an overview of the data cartridge framework

One of the lesser- known features of Oracle is the Data Cartridge framework which, according to the documentation, provides "the mechanism for extending the capabilities of the Oracle server" . What this actually means is that we can create a set of processing rules inside one or more object types and " plug them into" the Oracle server to use in various scenarios. One of these scenarios comprises pipelined functions that return instances of ANYDATASET (a generic type introduced in Oracle 9i). For this, Oracle provides the development framework for us to create a pipelined function, implemented as a Data Cartridge, that will return any re co rd st ruct ure we re q ue st o f it . In other words, we can exploit the Data Cartridge framework and combine it with pipelined functions to create a true Method 4 " engine" inside the database, as we will see in this article. For readers who wish to familiarise themselves with the Oracle Data Cartridge framework, t his o racle - d e ve lo p e r.ne t art icle describes its most common and simple use: user- defined aggregate functions. Many readers will be familiar with Tom Kyte's " STRAGG" function which is built on the same principles and framework. Note that the techniques we will use below will be more complicated than aggregate function implementations, so a solid grounding in the principles of building Data Cartridges is advised.

introducing the dictionary long application

We are going to use an existing oracle- developer.net application of Data Cartridge and pipelined ANYDATASET functions to demonstrate native Method 4 capabilities. This application is named (not particularly creatively) the " Dictionary Long Application" . It is built as a Data Cartridge with a single purpose: to query any dictionary view with LONG column(s) and return CLOB(s) instead. Examples are DBA_VIEWS and DBA_TAB_PARTITIONS, where we often wish to search inside the LONG columns but cannot due to their inflexibility. As CLOBs, these columns are as SQL- friendly as a regular VARCHAR2 column (since Oracle 9i of course). The Dictionary Long Application (we will call it DLA from now on) will accept any dynamic SQL statement and return the relevant PDFmyURL.com

dataset, as though we were querying statically in sqlplus. As an aside, the DLA should technically work from all versions of 10g upwards. However, due to a bug in Oracle's CLOB- handling, the version of the DLA we will see in this article will only work from 10g Release 2 (10.2) onwards. For 9i and 10g Release 1 (10.1), there is an alternative version of the DLA (available in the download at the end of this article).

elements of a data cartridge

A native Data Cartridge of the type we require is built of two components: an object type that implements the rules of the application using well- defined APIs (contained in Oracle's Data Cartridge development framework); and a PL/SQL function that declares itself as implemented by the object type. The well- defined APIs referenced above are built- in static and member methods, prefixed with ODCI*. Some of these methods are mandatory and some are optional based on functionality we may or may not wish to implement. There are several types of Data Cartridge and each varies in the built- in methods they need to include, but we will be using the ODCITable* static and member functions. In addition to an object type based on ODCITable* functions, we will also create a pipelined function. Unlike a " regular" pipelined function, this function will be fully implemented by the object type and not created in PL/SQL. We will see the details later in this article.

building the method 4 dictionary long application

With reference to the DLA, we will demonstrate how to build a Data Cartridge for a Method 4 dynamic SQL application. The implementing type for the DLA is quite complicated, but remember that in any Method 4 application we are trying to develop a program that can parse, describe, bind and fetch data for any SQL statement. The added twist with the DLA is that it converts LONGs to CLOBs, but this doesn't detract from its Method 4 capability.

dla t ype spe cif icat io n

We will build the implementing type for the DLA in small stages, breaking to describe certain elements of the syntax and logic. We will begin with the type specification, which provides a good overview of the methods we will need to implement. The implementing type is named DLA_OT and is defined as follows.

SQL> CREATE TYPE dla_ot AS OBJECT PDFmyURL.com

SQL> CREATE TYPE dla_ot AS OBJECT 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 ); / , MEMBER FUNCTION ODCITableClose ( SELF IN dla_ot ) RETURN NUMBER , MEMBER FUNCTION ODCITableFetch ( SELF rws IN OUT dla_ot, NUMBER , ANYDATASET OUT nrows IN , STATIC FUNCTION ODCITableStart ( sctx IN OUT dla_ot, stmt IN VARCHAR2 ) RETURN NUMBER , STATIC FUNCTION ODCITablePrepare ( sctx stmt OUT dla_ot, sys.ODCITabFuncInfo , VARCHAR2 IN tf_info IN , STATIC FUNCTION ODCITableDescribe ( rtype OUT ANYTYPE , stmt IN VARCHAR2 ) RETURN NUMBER ( atype ANYTYPE --<-- transient record type

) RETURN NUMBER

) RETURN NUMBER

Type created. The names of the static and member functions provide a good summary of Method 4 requirements. The most interesting method is the ODCITableFetch function because this is the area where PL/SQL traditionally breaks down in Method 4 scenarios. Note how this function is passing out an instance of ANYDATASET (i.e. any record or data structure). The individual ANYTYPE attribute will describe the structure of the records that the pipelined function will stream.

PDFmyURL.com

dla package spe cif icat io n: pipe line d f unct io n, t ype s and st at e variable
Before we build our implementing type body, we will create a package specification to wrap our pipelined function. This is unnecessary of course (the function can be standalone), but we will also make use of packaged types and state variables in our type body to avoid repetition and unnecessary work in the object type. The DLA_PKG package specification is as follows.

SQL> CREATE PACKAGE dla_pkg AS 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 TYPE rt_anytype_metadata IS RECORD ( precision PLS_INTEGER , scale , length , csid , csfrm , schema , type , name , version , attr_cnt PLS_INTEGER PLS_INTEGER PLS_INTEGER PLS_INTEGER VARCHAR2 (30) ANYTYPE VARCHAR2 (30) VARCHAR2 (30) PLS_INTEGER /* || Record types for use across multiple DLA_OT methods. */ TYPE rt_dynamic_sql IS RECORD ( cursor , column_cnt , execute ); INTEGER PLS_INTEGER INTEGER /* || Pipelined function interface. */ FUNCTION query_view( p_stmt IN VARCHAR2 ) RETURN ANYDATASET PIPELINED USING dla_ot;

, description DBMS_SQL.DESC_TAB2

, attr_type ANYTYPE , attr_name VARCHAR2 (128) PDFmyURL.com

33 34 35 36 37 38 39 40 41 42

, typecode ); /*

PLS_INTEGER

|| State variable for use across multiple DLA_OT methods. */ r_sql rt_dynamic_sql; END dla_pkg; /

Package created. Note how we define our QUERY_VIEW pipelined function. Firstly, we are returning an instance of ANYDATASET as our data type. This will contain arrays of whatever record structures we need to return (depending on the statement passed into the function). This gives it a Method 4 capability. We also declare this function to be implemented by our DLA_OT type with the USING clause. This is known as an interface method pipelined function.

dla t ype bo dy: de scribe phase

We will now build the implementing type body for the DLA_OT type. We will break it down into smaller chunks, based on each of the methods we need to implement, beginning with the OCITableDescribe static function, as follows.

SQL> CREATE TYPE BODY dla_ot AS 2 3 4 5 6 7 8 9 10 11 12 13 14 /* || Parse the SQL and describe its format and structure. PDFmyURL.com BEGIN r_sql dla_pkg.rt_dynamic_sql; v_rtype ANYTYPE ; STATIC FUNCTION ODCITableDescribe ( rtype OUT ANYTYPE , stmt IN VARCHAR2 ) RETURN NUMBER IS

15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53

*/ r_sql. cursor := DBMS_SQL.OPEN_CURSOR ; DBMS_SQL.PARSE ( r_sql. cursor , stmt, DBMS_SQL . NATIVE ); DBMS_SQL.DESCRIBE_COLUMNS2 ( r_sql. cursor , r_sql.column_cnt, r_sql.description ); DBMS_SQL.CLOSE_CURSOR ( r_sql. cursor ); /* || Create the ANYTYPE record structure from this SQL structure. || Replace LONG columns with CLOB... */ ANYTYPE.BeginCreate ( DBMS_TYPES.TYPECODE_OBJECT , v_rtype ); FOR i IN 1 .. r_sql.column_cnt LOOP v_rtype.AddAttr( r_sql.description(i).col_name, CASE --<>-WHEN r_sql.description(i).col_type IN (1,96,11,208) THEN DBMS_TYPES.TYPECODE_VARCHAR2 --<>-WHEN r_sql.description(i).col_type = 2 THEN DBMS_TYPES.TYPECODE_NUMBER ---WHEN r_sql.description(i).col_type IN (8,112) THEN DBMS_TYPES.TYPECODE_CLOB --<>-WHEN r_sql.description(i).col_type = 12 THEN DBMS_TYPES.TYPECODE_DATE --<>-WHEN r_sql.description(i).col_type = 23 THEN DBMS_TYPES.TYPECODE_RAW --<>-WHEN r_sql.description(i).col_type = 180 THEN DBMS_TYPES.TYPECODE_TIMESTAMP --<>-WHEN r_sql.description(i).col_type = 181 THEN DBMS_TYPES.TYPECODE_TIMESTAMP_TZ --<>-WHEN r_sql.description(i).col_type = 182 PDFmyURL.com

54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 END ; /* v_rtype .EndCreate ; END LOOP ;

THEN DBMS_TYPES.TYPECODE_INTERVAL_YM --<>-WHEN r_sql.description(i).col_type = 183 THEN DBMS_TYPES.TYPECODE_INTERVAL_DS --<>-WHEN r_sql.description(i).col_type = 231 THEN DBMS_TYPES.TYPECODE_TIMESTAMP_LTZ --<>-END , r_sql.description(i).col_precision, r_sql.description(i).col_scale, r_sql.description(i).col_max_len, r_sql.description(i).col_charsetid, r_sql.description(i).col_charsetform );

|| Now we can use this transient record structure to create a table type || of the same. This will create a set of types on the database for use || by the pipelined function... */ ANYTYPE.BeginCreate ( DBMS_TYPES.TYPECODE_TABLE , rtype ); rtype .SetInfo ( NULL , NULL , NULL , NULL , NULL , v_rtype, DBMS_TYPES.TYPECODE_OBJECT , 0 ); rtype .EndCreate (); RETURN ODCIConst.Success ;

We do a lot of setup work in the ODCITableDescribe static function. For efficiency, this method is only executed when a dynamic query is hard- parsed for the first time. At this stage, Oracle will create two types in our schema (one object type and one collection type), based on the structures being described (they are created by the ANYTYPE.BeginCreate static method calls). The ODCITableDescribe function is a new feature of 10g that enables us to develop Method 4 applications. The main elements of this function are as follows. PDFmyURL.com

Line s 16 - 19: using DBMS_SQL, we first describe the dynamic SQL cursor that we will ultimately be trying to execute. This gives us an array of information on the columns in the cursor's resultset; Line s 25- 70: using the cursor description, we create a transient instance of ANYTYPE. The structure of this type instance matches the described SQL cursor, with one exception described in the next bullet- point. The ANYTYPE instance will be the defining record structure of the ANYDATASET pipelined function for a given dynamic SQL statement; Line s 38- 39: the purpose of the DLA is to convert LONGs to CLOBs to make dictionary views easier to use. A LONG is of typecode 8, so when we are dealing with a cursor attribute of this type, we set the ANYTYPE attribute to CLOB instead. This is the only point at which the incoming SQL cursor and the ANYTYPE instance records differ; and Line s 77- 80: we create a transient collection type based on the transient object type created above. As with any pipelined function, we must always create an object type to define a record, followed by a collection type of this object. The fact that ODCITableDescribe is only called once per unique query means we can repeat the dynamic SQL call in and across database sessions and never have this method called again. For this reason, we have not made use of the DLA_PKG state variable for the DBMS_SQL elements of this function, as it will not be available to the other methods in the DLA_OT type. It has been stated already that the return record type is where PL/SQL's Method 4 capabilities fall down. The ANYTYPE built- in type used in the ODCITableDescribe static function above overcomes this by enabling us to create transient data structures that match the incoming dynamic SQL cursor (based on the information we can retrieve with DBMS_SQL).

dla t ype bo dy: pre pare phase

The ODCITablePrepare static function is where we initialise an instance of DLA_OT for use in other methods (and ultimately in the generation of a resultset). For example, we store the instance of ANYTYPE that was passed out of the ODCITableDescribe function in our DLA_OT instance. The well- defined interface of the ODCITablePrepare function requires that the ANYTYPE instance is passed in as a parameter, as below.

86 87 88 89 90 91 92 93 94 95

STATIC FUNCTION ODCITablePrepare ( sctx stmt OUT dla_ot, sys.ODCITabFuncInfo , VARCHAR2 IN tf_info IN

) RETURN NUMBER IS r_meta dla_pkg.rt_anytype_metadata; BEGIN

PDFmyURL.com

96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113

/* || We prepare the dataset that our pipelined function will return by || describing the ANYTYPE that contains the transient record structure... */ r_meta.typecode := tf_info.rettype .GetAttrElemInfo ( 1, r_meta.precision, r_meta.scale, r_meta. length , r_meta.csid, r_meta.csfrm, r_meta. type , r_meta.name ); /* || Using this, we initialise the scan context for use in this and || subsequent executions of the same dynamic SQL cursor... */ sctx := dla_ot(r_meta. type ); RETURN ODCIConst.Success ; END ;

This static function is quite simple and performs the following actions. Line s 117- 124 : the ANYTYPE.GetAttrElemInfo method provides us with a range of information about our transient type, including the type instance itself, which is the data we require; and Line 129: we initialise an instance of our DLA_OT, setting the transient ANYTYPE attribute for the current dynamic cursor. Like the ODCITableDescribe method, the ODCITablePrepare function is executed only at query compilation (hard- parse) time. This means that the scan context we created above (the instance of DLA_OT containing the ANYTYPE definition) is available across repeated calls of the same dynamic SQL statement. This reduces the time we spend executing a query as some of the time- intensive setup work is already done for us. Without a prepare phase, the scan context initialisation would be needed on every execution of a given SQL statement.

dla t ype bo dy: st art phase

The ODCITableStart static function is where we return to DBMS_SQL to define and execute our dynamic SQL cursor. There are some important points to note about the techniques used in the following function, which will be described after the code listing.

115

STATIC FUNCTION ODCITableStart ( PDFmyURL.com

116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 --<>-/* /* BEGIN

sctx IN OUT dla_ot, stmt IN VARCHAR2 ) RETURN NUMBER IS r_meta dla_pkg.rt_anytype_metadata;

|| We now describe the cursor again and use this and the described || ANYTYPE structure to define and execute the SQL statement... */ dla_pkg.r_sql. cursor := DBMS_SQL.OPEN_CURSOR ; DBMS_SQL.PARSE ( dla_pkg.r_sql. cursor , stmt, DBMS_SQL . NATIVE ); DBMS_SQL.DESCRIBE_COLUMNS2 ( dla_pkg.r_sql. cursor , dla_pkg.r_sql.column_cnt, dla_pkg.r_sql.description ); FOR i IN 1 .. dla_pkg.r_sql.column_cnt LOOP

|| Get the ANYTYPE attribute at this position... */ r_meta.typecode := sctx.atype .GetAttrElemInfo ( i, r_meta.precision, r_meta.scale, r_meta. length , r_meta.csid, r_meta.csfrm, r_meta. type , r_meta.name ); CASE r_meta.typecode --<>-WHEN DBMS_TYPES.TYPECODE_VARCHAR2 THEN DBMS_SQL.DEFINE_COLUMN ( dla_pkg.r_sql. cursor , i, '', 32767 ); WHEN DBMS_TYPES.TYPECODE_NUMBER THEN DBMS_SQL.DEFINE_COLUMN ( PDFmyURL.com

155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 --<>---<>---<>---<>---<>---<>---<>--

dla_pkg.r_sql. cursor , i, CAST ( NULL AS NUMBER ) ); WHEN DBMS_TYPES.TYPECODE_DATE THEN DBMS_SQL.DEFINE_COLUMN ( dla_pkg.r_sql. cursor , i, CAST ( NULL AS DATE ) ); WHEN DBMS_TYPES.TYPECODE_RAW THEN DBMS_SQL.DEFINE_COLUMN_RAW ( dla_pkg.r_sql. cursor , i, CAST ( NULL AS RAW ), r_meta. length ); WHEN DBMS_TYPES.TYPECODE_TIMESTAMP THEN DBMS_SQL.DEFINE_COLUMN ( dla_pkg.r_sql. cursor , i, CAST ( NULL AS TIMESTAMP ) ); WHEN DBMS_TYPES.TYPECODE_TIMESTAMP_TZ THEN DBMS_SQL.DEFINE_COLUMN ( dla_pkg.r_sql. cursor , i, CAST ( NULL AS TIMESTAMP WITH TIME ZONE ) ); WHEN DBMS_TYPES.TYPECODE_TIMESTAMP_LTZ THEN DBMS_SQL.DEFINE_COLUMN ( dla_pkg.r_sql. cursor , i, CAST ( NULL AS TIMESTAMP WITH LOCAL TIME ZONE ) ); WHEN DBMS_TYPES.TYPECODE_INTERVAL_YM THEN DBMS_SQL.DEFINE_COLUMN ( dla_pkg.r_sql. cursor , i, CAST ( NULL AS INTERVAL YEAR TO MONTH ) ); PDFmyURL.com

194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 END ; /*

WHEN DBMS_TYPES.TYPECODE_INTERVAL_DS THEN DBMS_SQL.DEFINE_COLUMN ( dla_pkg.r_sql. cursor , i, CAST ( NULL AS INTERVAL DAY TO SECOND ) ); --<>-WHEN DBMS_TYPES.TYPECODE_CLOB THEN --<>-CASE dla_pkg.r_sql.description(i).col_type WHEN 8 THEN DBMS_SQL.DEFINE_COLUMN_LONG ( dla_pkg.r_sql. cursor , i ); ELSE DBMS_SQL.DEFINE_COLUMN ( dla_pkg.r_sql. cursor , i, CAST ( NULL AS CLOB ) ); END CASE ; --<>-END CASE ; END LOOP ;

|| The cursor is prepared according to the structure of the type we wish || to fetch it into. We can now execute it and we are done for this method... */ dla_pkg.r_sql .execute := DBMS_SQL.EXECUTE ( dla_pkg.r_sql. cursor ); RETURN ODCIConst.Success ;

At first glance, we can see that we are defining and executing a dynamic SQL cursor, which will be familiar to many developers. There are some interesting points to note about the implementation of this, however, as follows. Line s 128- 132: we open, parse and describe the dynamic SQL statement again. We need to describe the cursor again PDFmyURL.com

because we cannot guarantee that the preceding ODCITableDescribe function will be invoked (remember the describe step is only invoked the first time a query is parsed). Describing the cursor again seems wasteful but it is necessary. The ODCITableStart function will be invoked on every query execution, so at this point we can make use of package state by using the record variable in DLA_PKG; Line s 134 - 216 : we loop through the cursor description (i.e. the array of projected columns). For each element in the array, we extract the corresponding ANYTYPE attribute (remember that we set these in the ODCITableDescribe function) and use their typecodes to define each column for fetch. We use the ANYTYPE typecodes rather than the DBMS_SQL versions because these correspond with named constants in DBMS_TYPES, which makes it easier to understand. By decoding the typecodes, we can call the correct DBMS_SQL.DEFINE_XXX API to setup the output placeholders for fetching into later; Line s 200- 212: for any LONG columns in the incoming dynamic SQL cursor, the ODCITableDescribe method sets the corresponding ANYTYPE attribute to CLOB. When extracting the attribute metadata from the ANYTYPE instance in the ODCITableStart method above, we need to know whether the attribute was always a CLOB or was originally a LONG. This is because we need to use the specific DBMS_SQL.DEFINE_COLUMN_LONG procedure to setup the LONG column for fetch. This is where having both the original cursor description and the ANYTYPE metadata becomes essential; and Line 222: we execute the dynamic SQL cursor and are ready to fetch data. We can see that much of the subtlety of the DLA conversion logic exists in this function, as described above. At this stage, however, we have reached the end of PL/SQL's native Method 4 capabilities, as we now need to fetch data. As stated earlier, to achieve Method 4 dynamic SQL in " straight PL/SQL" requires us to write dynamic PL/SQL. The DBMS_SQL APIs we have seen so far enable us to do this by defining dynamic variable names, types, fetch structures etc at runtime (i.e. using PL/SQL to write PL/SQL). Oracle Data Cartridge, together with ANYDATASET and particularly ANYTYPE, enables us to avoid dynamic PL/SQL by fetching into a transient type, as we will see below.

dla t ype bo dy: f e t ch phase

In the preceding type functions, we have parsed, described, defined and executed our dynamic SQL cursor. Based on the description of this cursor, we have also instantiated two transient types using ANYTYPE (an object type and a collection type: the pre- requisites for pipelined functions). We are now ready to implement Method 4 by fetching the data from the cursor, which we do using the ODCITableFetch member function below.

228 229 230 231 232

MEMBER FUNCTION ODCITableFetch ( SELF nrows rws IN OUT dla_ot, IN OUT NUMBER , ANYDATASET

) RETURN NUMBER IS PDFmyURL.com

233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 /* || We can now begin to piece together our returning dataset. We create an || instance of ANYDATASET and then fetch the attributes off the DBMS_SQL || cursor using the metadata from the ANYTYPE. LONGs are converted to CLOBs... PDFmyURL.com /* || First we describe our current ANYTYPE instance (SELF.A) to determine || the number and types of the attributes... */ r_meta.typecode := SELF .atype .GetInfo ( r_meta.precision, r_meta.scale, r_meta. length , r_meta.csid, r_meta.csfrm, r_meta.schema, r_meta.name, r_meta.version, r_meta.attr_cnt ); IF DBMS_SQL.FETCH_ROWS ( dla_pkg.r_sql. cursor ) > 0 THEN BEGIN TYPE rt_fetch_attributes IS RECORD ( v2_column , num_column , date_column , clob_column , raw_column , raw_error , raw_length , ids_column , iym_column , ts_column , tstz_column , cvl_offset , cvl_length ); r_fetch rt_fetch_attributes; r_meta dla_pkg.rt_anytype_metadata; VARCHAR2 (32767) NUMBER DATE CLOB RAW (32767) NUMBER INTEGER INTERVAL DAY TO SECOND INTERVAL YEAR TO MONTH TIMESTAMP TIMESTAMP WITH TIME ZONE INTEGER := 0 INTEGER

, tsltz_column TIMESTAMP WITH LOCAL TIME ZONE

272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310

*/ ANYDATASET.BeginCreate ( DBMS_TYPES.TYPECODE_OBJECT , SELF .atype, rws ); rws .AddInstance (); rws .PieceWise (); FOR i IN 1 .. dla_pkg.r_sql.column_cnt LOOP r_meta.typecode := SELF .atype .GetAttrElemInfo ( i, r_meta.precision, r_meta.scale, r_meta. length , r_meta.csid, r_meta.csfrm, r_meta.attr_type, r_meta.attr_name ); CASE r_meta.typecode --<>-WHEN DBMS_TYPES.TYPECODE_VARCHAR2 THEN DBMS_SQL.COLUMN_VALUE ( dla_pkg.r_sql. cursor , i, r_fetch.v2_column ); rws .SetVarchar2 ( r_fetch.v2_column ); --<>-WHEN DBMS_TYPES.TYPECODE_NUMBER THEN DBMS_SQL.COLUMN_VALUE ( dla_pkg.r_sql. cursor , i, r_fetch.num_column ); rws .SetNumber ( r_fetch.num_column ); --<>-WHEN DBMS_TYPES.TYPECODE_DATE THEN DBMS_SQL.COLUMN_VALUE ( dla_pkg.r_sql. cursor , i, r_fetch.date_column ); rws .SetDate ( r_fetch.date_column ); --<>-WHEN DBMS_TYPES.TYPECODE_RAW THEN DBMS_SQL.COLUMN_VALUE_RAW ( PDFmyURL.com

311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 --<>---<>---<>---<>---<>--

dla_pkg.r_sql. cursor , i, r_fetch.raw_column, r_fetch.raw_error, r_fetch.raw_length ); rws .SetRaw ( r_fetch.raw_column ); WHEN DBMS_TYPES.TYPECODE_INTERVAL_DS THEN DBMS_SQL.COLUMN_VALUE ( dla_pkg.r_sql. cursor , i, r_fetch.ids_column ); rws .SetIntervalDS ( r_fetch.ids_column ); WHEN DBMS_TYPES.TYPECODE_INTERVAL_YM THEN DBMS_SQL.COLUMN_VALUE ( dla_pkg.r_sql. cursor , i, r_fetch.iym_column ); rws .SetIntervalYM ( r_fetch.iym_column ); WHEN DBMS_TYPES.TYPECODE_TIMESTAMP THEN DBMS_SQL.COLUMN_VALUE ( dla_pkg.r_sql. cursor , i, r_fetch.ts_column ); rws .SetTimestamp ( r_fetch.ts_column ); WHEN DBMS_TYPES.TYPECODE_TIMESTAMP_TZ THEN DBMS_SQL.COLUMN_VALUE ( dla_pkg.r_sql. cursor , i, r_fetch.tstz_column ); rws .SetTimestampTZ ( r_fetch.tstz_column ); WHEN DBMS_TYPES.TYPECODE_TIMESTAMP_LTZ THEN DBMS_SQL.COLUMN_VALUE ( dla_pkg.r_sql. cursor , i, r_fetch.tsltz_column ); rws .SetTimestamplTZ ( r_fetch.tsltz_column ); PDFmyURL.com

350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 END ; END IF ; /*

--<>-WHEN DBMS_TYPES.TYPECODE_CLOB THEN --<>-CASE dla_pkg.r_sql.description(i).col_type WHEN 8 THEN LOOP DBMS_SQL.COLUMN_VALUE_LONG ( dla_pkg.r_sql. cursor , i, 32767, r_fetch.cvl_offset, r_fetch.v2_column, r_fetch.cvl_length ); r_fetch.clob_column := r_fetch.clob_column || r_fetch.v2_column; r_fetch.cvl_offset := r_fetch.cvl_offset + 32767; EXIT WHEN r_fetch.cvl_length < 32767; END LOOP ; ELSE DBMS_SQL.COLUMN_VALUE ( dla_pkg.r_sql. cursor , i, r_fetch.clob_column ); END CASE ; rws .SetClob ( r_fetch.clob_column ); --<>-END CASE ; END LOOP ;

|| Our ANYDATASET instance is complete. We end our create session... */ rws .EndCreate ();

RETURN ODCIConst.Success ;

PDFmyURL.com

We can see a pattern in how we handle the cursor and ANYTYPE attributes. As in the describe and start phases, the underlying data type of each attribute dictates the DBMS_SQL API and ANYTYPE method that we need to use. The ODCITableFetch member function above is no different, except this time we are setting the final data structure for piping to the end- user. Note in particular the following: Line s 234 - 24 9: we define a record structure for our data fetches. This record type includes an attribute for each data type we might need to fetch from the SQL cursor; Line s 26 2- 26 6 : we retrieve the metadata relating to our ANYTYPE instance as this will be used to drive the fetching and the use of the correct DBMS_SQL.COLUMN_VALUE(_XXX) API. It will also be used to add data into the ANYDATASET instance that our pipelined function will return, as we will describe below; Line s 273- 275: we instantiate an ANYDATASET, based on the record structure in our ANYTYPE instance. We call the Piecewise member function to enable us to add data elements to our ANYDATASET instance one at a time, as we fetch them off the SQL cursor; Line s 277- 375: we loop through the attributes in our ANYTYPE instance and fetch data off the SQL cursor using the appropriate DBMS_SQL procedures. In addition, we add each fetched column data into our ANYDATASET instance use the relevant type- specific method; Line s 354 - 372: when we fetch a CLOB, we need to know whether it was originally a LONG. Remember that we saved the corresponding cursor description using a state variable in DLA_PKG. If the SQL cursor attribute is a LONG, we fetch it piecewise using the DBMS_SQL.COLUMN_VALUE_LONG procedure, adding it to our ANYDATASET instance as a CLOB on completion; and Line 380: we complete the fetch into our ANYDATASET instance, by which stage the pipelined function that is implemented via the DLA_OT type will have piped most of this data.

dla t ype bo dy: clo se phase

The remaining method we need to code is the ODCITableClose member function. We can see below that all we need to do in this function is close the dynamic SQL cursor and reset our package state.

388 389 390 391 392 393 394

MEMBER FUNCTION ODCITableClose ( SELF IN dla_ot ) RETURN NUMBER IS BEGIN DBMS_SQL.CLOSE_CURSOR ( dla_pkg.r_sql. cursor ); dla_pkg.r_sql := NULL ; RETURN ODCIConst.Success ; PDFmyURL.com

395 396 397 398

END ; END ; /

Type body created. This completes our type implementation and we are now able to test our pipelined function. Before we do this, however, we can summarise the type's processing phases as follows: d e scrib e : we describe the incoming cursor and create object and collection instances of ANYTYPE based on this information. This is executed once- only for a new SQL statement, at which point Oracle creates two types to support the pipelined function implementation; p re p are : we initialise a scan context once for a unique cursor. This method is invoked once at query compile (parse) time and this prevents Oracle from initialising an instance of DLA_OT every time a particular query is restarted (i.e. executed again); st art : we describe the dynamic cursor again, using the metadata to define the API calls to the DBMS_SQL package. The cursor is executed and is ready for fetch. We store the cursor information in a package variable for sharing across methods; f e t ch: using the cursor description and ANYTYPE metadata, we fetch data off the dynamic SQL cursor into a relevant type variable. We create an instance of ANYDATASET and assign the fetched data piecewise; and clo se : we cleanup our operation by closing the cursor and resetting package state.

testing the function

We can now test our QUERY_VIEW pipelined function. As the DLA is designed as a Method 4 application to convert LONGs to CLOBs, we will execute a dynamic query against DBA_VIEWS. We will fetch a single row for simplicity. Remember that the DLA (or any Method 4 application that uses Data Cartridge and ANYDATASET in this way) will run any query against it within the domain of datatypes we support.

SQL> SELECT * 2 3 4 5 FROM TABLE ( dla_pkg.query_view( 'SELECT * FROM dba_views' PDFmyURL.com

6 7 WHERE

) ) ROWNUM = 1;

OWNER TEXT TYPE_TEXT

VIEW_NAME

TEXT_LENGTH TYPE_TEXT_LENGTH

------------------------------ ------------------------------ ---------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------OID_TEXT_LENGTH --------------OID_TEXT ----------------------------------------------------------------------------------------------VIEW_TYPE_OWNER SYS VIEW_TYPE V_$MAP_LIBRARY SUPERVIEW_NAME 160 ------------------------------ ------------------------------ -----------------------------select "LIB_IDX","LIB_NAME","VENDOR_NAME","PROTOCOL_NUM","VERSION_NUM","PATH_N AME","MAP_FILE","FILE_CFGID","MAP_ELEM","ELEM_CFGID","MAP_SYNC" from v$map_lib rary 1 row selected. We can see that the DLA has a true Method 4 capability. It has described and understood the incoming dynamic SQL statement and fetched it into a structure of its own creation. If we examine the data dictionary, we can see that Oracle has created two physical types to support this particular cursor.

SQL> SELECT type_name, typecode 2 3 FROM WHERE user_types type_name LIKE 'SYS%';

TYPE_NAME SYSTPHvttROLoRF+3LWYLkOFoww== SYSTPrH9i9v1OR0KwHDbeEiPKUA== 2 rows selected.

TYPECODE COLLECTION OBJECT

------------------------------ ------------------------------

PDFmyURL.com

We can also query the structure of the object type created to support this particular query, as follows.

SQL> SELECT attr_no 2 3 4 5 6 7 8 9 ORDER BY attr_no; , , FROM WHERE attr_name attr_type_name user_type_attrs type_name IN ( SELECT type_name FROM WHERE user_types type_name LIKE 'SYS%' )

ATTR_NO ATTR_NAME 1 OWNER 2 VIEW_NAME 3 TEXT_LENGTH 4 TEXT 5 TYPE_TEXT_LENGTH 6 TYPE_TEXT 7 OID_TEXT_LENGTH 8 OID_TEXT 9 VIEW_TYPE_OWNER 10 VIEW_TYPE 11 SUPERVIEW_NAME 11 rows selected.

ATTR_TYPE_NAME VARCHAR2 VARCHAR2 NUMBER CLOB NUMBER VARCHAR2 NUMBER VARCHAR2 VARCHAR2 VARCHAR2 VARCHAR2

---------- ------------------------------ --------------------------

As expected, this " record structure" tallies with the DBA_VIEW column description, with the exception that the TEXT column is a CLOB, rather than a LONG. We will test the DLA's Method 4 capability with another simple query, as follows.

SQL> SELECT * 2 3 4 5 FROM TABLE ( dla_pkg.query_view( 'SELECT trigger_name, trigger_body FROM dba_triggers' PDFmyURL.com

6 7 WHERE

) ) ROWNUM = 1;

TRIGGER_NAME DEF$_PROPAGATOR_TRIG

TRIGGER_BODY DECLARE prop_count BEGIN SELECT count(*) into prop_count FROM system.def$_propagator; IF (prop_count > 0) THEN -- Raise duplicate propagator error sys.dbms_sys_error.raise_system_error(-23394); END IF; END; NUMBER;

------------------------------ --------------------------------------------------------

1 row selected. Despite the complex initial setup, we can see that interface- method pipelined functions (using Oracle Data Cartridge in 10.2 and ANYTYPE/ANYDATASET generic types) provide a good means to produce Method 4 SQL applications.

performance considerations
Having such flexibility comes at a cost, as we will see below. In the following example, we will compare a query against DBA_VIEWS with a synonymous query using the DLA. Oracle's object implementation, combined with the fact that the DLA converts the LONG column to a CLOB, increases the time and resources that Oracle must spend to satisfy this Method 4 implementation. We will use autotrace to reduce the output and also a variation of Tom Kyte's RUNSTATS utility to compare the two queries.

SQL> set autotrace traceonly statistics SQL> exec runstats_pkg.rs_start;

PL/SQL procedure successfully completed.

PDFmyURL.com

SQL> SELECT * 2 FROM dba_views;

3691 rows selected.

Statistics ---------------------------------------------------------301 0 8546 0 0 3175717 41118 3693 10 0 3691 recursive calls db block gets consistent gets physical reads redo size bytes sent via SQL*Net to client bytes received via SQL*Net from client SQL*Net roundtrips to/from client sorts (memory) sorts (disk) rows processed

SQL> exec runstats_pkg.rs_middle;

PL/SQL procedure successfully completed.

SQL> SELECT * 2 3 4 5 FROM TABLE ( dla_pkg.query_view( 'SELECT * FROM dba_views' ));

3691 rows selected.

Statistics ---------------------------------------------------------7457 recursive calls PDFmyURL.com

89270 28634 0 0 4042814 2381976 19819 1 0 3691

db block gets consistent gets physical reads redo size bytes sent via SQL*Net to client bytes received via SQL*Net from client SQL*Net roundtrips to/from client sorts (memory) sorts (disk) rows processed

SQL> exec runstats_pkg.rs_stop(1000); SQL> exec runstats_pkg.rs_stop(1000); Run1 ran in 268 hsecs Run2 ran in 447 hsecs Run1 ran in 59.96% of the time

Name STAT..no work - consistent rea STAT..table fetch by rowid STAT..recursive calls STAT..lob writes STAT..lob writes unaligned STAT..free buffer requested STAT..buffer is not pinned cou STAT..lob reads LATCH.cache buffers lru chain LATCH.object queue header oper LATCH.simulator hash latch LATCH.simulator lru latch STAT..SQL*Net roundtrips to/fr STAT..user calls STAT..consistent gets STAT..consistent gets from cac STAT..consistent changes STAT..db block changes

Run1 8,498 3,709 1,035 0 0 0 7,934 0 36 34 638 638 3,700 3,704 8,549 8,549 0 0

Run2 12,204 7,477 8,191 7,384 7,384 7,443 15,499 12,435 14,886 14,887 15,751 15,751 19,826 19,830 28,640 28,640 29,765 29,765

Diff 3,706 3,768 7,156 7,384 7,384 7,443 7,565 12,435 14,850 14,853 15,113 15,113 16,126 16,126 20,091 20,091 29,765 29,765 PDFmyURL.com

LATCH.session idle bit STAT..calls to get snapshot sc STAT..session pga memory max LATCH.library cache pin STAT..db block gets STAT..db block gets from cache STAT..session logical reads LATCH.library cache STAT..session uga memory max LATCH.cache buffers chains STAT..bytes sent via SQL*Net t STAT..session pga memory STAT..bytes received via SQL*N

7,425 27 0 95 0 0 8,549 182 0 17,117 3,176,380 327,680 42,105

39,677 38,372 65,536 74,019 89,270 89,270 117,910 111,131 130,928 295,027 4,043,477 -786,432 2,382,967

32,252 38,345 65,536 73,924 89,270 89,270 109,361 110,949 130,928 277,910 867,097 -1,114,112 2,340,862

Run1 latches total versus run2 -- difference and pct Run1 27,376 Run2 582,312 Diff 554,936 Pct 4.70%

PL/SQL procedure successfully completed. From the autotrace output alone we can see that the Data Cartridge application incurs a large amount of I/O when compared with the static query. This is due to the CLOB implementation which is specific to the DLA (and will not necessarily be present in a more general- purpose application of the Data Cartridge framework). In addition, the DLA also generates a high volume of recursive SQL, required to support such a metadata- driven application. The RUNSTATS output provides more information on the resource usage of Oracle's Data Cartridge framework, in particular the latching. Oracle's object type implementation seems to use proportionately high numbers of latches (and this can be seen in most applications that make use of object types). The DLA is no different and uses far more latches than the static query (the static query uses just 5% of the latches required by the DLA). The initial setup work involved in a Data Cartridge application, such as the creation of types, appears to have little impact on the runtimes or latches used (i.e. the resource usage is similar for subsequent executions of the same cursor). To reduce the cost of executing Method 4 dynamic queries, the full DLA application has some additional features over those described in this article. Full details are available in the download file (available below) and include a set of pre- defined views (e.g. V_DBA_VIEWS, V_DBA_TAB_PARTITIONS and so on) and, more critically, the ability to limit the volume of data being generated and returned, using application context.

PDFmyURL.com

further reading
For more information on Oracle Data Cartridge, read the D at a C art rid g e D e ve lo p e r' s G uid e . In particular, read t his se ct io n on interface pipelined functions.

downloads
The Dictionary Long Application can be downloaded from he re . The oracle- developer.net variation on RUNSTATS is available he re .

acknowledgements
Many thanks to Jonathan Heller for pointing out a typecode bug for CHAR in the original DLA_OT implementation. Jonathan also noted that pseudo- columns such as ROWNUM, USER etc require an alias to work with this application. I've added these instructions to the usage notes in the DLA_PKG specification (available in the download file).

Ad rian B illing t o n, Aug ust 2007 B ack t o To p

o rac le -d e ve lo p e r.ne t 20 0 2-20 12 c o p yrig ht Ad rian Billing to n all rig hts re s e rve d | o rig inal te mp late b y SmallPark | las t up d ate d 0 2 Ap ril 20 12

PDFmyURL.com