Professional Documents
Culture Documents
Information Management
V2012-07-06
Table of Contents
Table of Contents
1 Introduction ..........................................................................................................................................................................................3
2 About this Lab......................................................................................................................................................................................3
3 Suggested Reading .............................................................................................................................................................................3
4 Environment Requirements and Setup .............................................................................................................................................3
4.1 Unpacking the image ..........................................................................................................................................................................................3
4.2 Preparation Step .................................................................................................................................................................................................4
4.3 Start the Virtual Machine .....................................................................................................................................................................................4
4.4 Login to the Virtual Machine and Accept the License Agreement.......................................................................................................................6
4.5 Open the Terminal Window.................................................................................................................................................................................9
7 Summary.............................................................................................................................................................................................22
8 Cleanup...............................................................................................................................................................................................22
1 Introduction
DB2 10 is an enterprise database engine with a rich set of features and tools providing unmatched capabilities to achieve best
performance. It contains innovative features for delivering information on demand and scaling databases to new levels.
In this part of the lab, the attendee will work with simply preparing the DB2 environment for proceeding through the labs.
3 Suggested Reading
Best practices for DB2 for Linux, UNIX, and Windows
Practical guidance for the most common DB2 10 product configurations and use this knowledge to improve the value of your
DB2 data servers.
http://www.ibm.com/developerworks/data/bestpractices/
Start the VMware image by clicking the button in VMware Workstation if it is not already on.
Note: Please wait for the first boot setup process to complete.
The system will power up like any other Linux system and will come to the state as shown in the next picture below.
After the virtual machine has finished booting up, you can now work inside the virtual machine environment. To bring focus into
the virtual machine environment, click inside the virtual machine screen with your mouse or click on the “Full Screen” button
After clicking on the screen, you may not see your mouse pointer anymore, this is normal as you are now operating in a
command line mode inside the virtual machine.
You can bring focus to the host operating system at any point by pressing “Alt + Ctrl” at the same time.
4.4 Login to the Virtual Machine and Accept the License Agreement
You will see a pop-up message asking you to read and accept the License Agreement.
Use the “Page Down” key to scroll down and read the full license agreement. Once you have read the agreement,
select “Yes, I Agree to the License Agreement” and click “Next” to go to the next screen.
This will take you to the VMware Tools License Agreement.
Use the “Page Down” key to scroll down and read the full license agreement. Once you have read the agreement,
select “Yes, I Agree to the License Agreement” to go to the next screen. This will take you the IBM DB2 10
License Agreement:
Use the “Page Down” key to scroll down and read the full license agreement. Once you have read the agreement,
select “Yes, I Agree to the License Agreement” to go to the next screen. This will take you the IBM
Installation Manager License Agreement:
This screen shows all of the Software Licenses, and Non-IBM license agreements that were displayed in all of the
previous screens. In order to use this image, you must accept all of the listed agreements that were displayed in the
previous screens. Select “Next” to finish. If you do not agree with the license agreements, select “Abort” and the
virtual machine will be shutdown automatically.
1. Open a new terminal window by right-clicking on the Desktop and choosing the “Open Terminal” item:
Enter the command “db2start” to start the database instance. Make sure the instance is started successfully.
(Note: If you notice a message that says “The database manager is already active”, that’s OK.
Please continue to the next section).
In order to enable the aliases defined for this lab and all labs that follow, copy the /home/db2inst1/labbashrc to
/home/db2inst1/.bashrc and source that file.
cp /home/db2inst1/labbashrc .bashrc
. .bashrc
• Configuration Advisor running against the database with somewhat conservative options
o Best Practice: Run the Configuration Advisor, after database has been created and input information, such as
the concurrent number of users and types of transactions. Doing this assumes that DB2 tuning experience is at
a beginner level.
o Best Practice: To ensure that the optimizer has a good set of working statistics leave this setting as is. Certain
tables may require special designations, like “Volatile.” Additionally, a statistics profile can be registered for
tables requiring more than the default collection method. This profile will be respected by the automatic
statistics gathering process. See the DB2 Information Center for more information and restrictions on this
feature.
o Best Practice: Leave this feature on when the system is a dedicated DB2 server. However, with respect to the
nature and criticality of memory allocation, it is best employed after thorough system testing. Monitor system
memory and the db2diag.log for actual heap allocations. For a DPF environment, see DB2 Information Center
for further considerations including selecting a “tuning node”.
• Unicode database
o This designation is required if you intend to use the XML native data type or if multi byte code set is required.
If required, be mindful to the collating sequence you choose, Unicode culturally correct collating sequences like
UCA500R1_xxx can cause additional CPU overhead. There are alternative collations like SYSTEM_819_BE
that take advantage of Unicode databases that contain data in only one language. They use the same lookup
table-based collation algorithm as single-byte collations such as SYSTEM_819, making them very efficient.
• Automatic Storage using a single location for containers (defined by the database manager configuration parameter
DFTDBPATH)
o Though additional storage paths can be added to a database with the ALTER DATABASE command, it is
advisable to establish multiple storage paths on dedicated devices before the database is populated. When
you create tablespaces, by default they will be designated to use automatic storage, be mindful to specify
attributes like INITIALSIZE, MAXSIZE, and INCREASESIZE.
• Database control and log files being stored in same location as tablespace containers (also defined by the database
manager configuration parameter DFTDBPATH)
o For optimal performance move the transaction log files to their own dedicated file system after database
creation.
• Default page size of 4K for default buffer pool and the SYSCATSPACE, TEMPSPACE1, and USERSPACE1
tablespaces
• Isolation level being set to Cursor Stability with currently committed semantics
• Detailed deadlocks event monitor (DB2DETAILDEADLOCK) created and enabled (This monitor is deprecated, but used
for transitional purposes
Some of the best practices cited above cannot actually be addressed until after the database has been created, while other
factors can be addressed with the CREATE DATABASE command itself. The autonomic and automatic default options should
be kept.
You will definitely want to address the placement of log files and tablespace containers. By default, db2 database metadata,
transaction log files and tablespace containers are all placed in the same default location. While this works, in a real production
system this can result in severe performance degradation, especially for large, transaction laden databases.
2. The home directory is acceptable for database metadata but leaving the log data and control files in the same location
is not the best for transactional performance. Issue the following command to verify location of log files:
Expected output:
You can see that the log file location is the same as the one for database metadata.
home/db2inst1/db2inst1/NODE0000/SQL00001
Database directory
Contains files needed for:
3. Change the path to a completely separate location. Issue this command from terminal:
Expected output:
Attention: Note that “Path to log files” still shows the old location but the effective log path is the location indicated by
NEWLOGPATH parameter.
We had also specified the RESTRICTIVE key word in the create database command. (This can be checked by displaying the
database configuration file with the “get db cfg” command under Restrict Access). Without it, db2 catalog information is readable
by PUBLIC, i.e. anyone connected to the database. This will revoke the authorization to PUBLIC and make the database more
secure. See DB2 Information center for other restrictions.
• Database configuration
• Registry variables
So far, we have looked at database and database manager configurations. In this section we look at some significant
performance related registry variables. Note that the Configuration Advisor does not enable any registry variables. Remember
that for registry variables to take effect a complete db2 recycle using db2stop and db2start is required.
5.2.1 DB2_MEM_TUNING_RANGE
This registry variable is related to self tuning memory manager (STMM). If STMM is being used, it will take more memory, as
needed, on the server as long as there is free memory available. This may cause problems on systems where you need to
leave some memory for another application or for the system. The syntax is as follows:
MINFREE and MAXFREE are percentage numbers. So if you want to dedicate no more than 60% of system memory to db2,
you would set MINFREE to 40% which means STMM should take no more than 60% of system memory so there is always at
least 40% free memory available. STMM will release memory when it is not needed but you may want to control how much
memory STMM gives up. For example, if the requirement is that STMM should retain al least 20% of system memory, then
MAXFREE would be set to 80%.
1. In order to make these settings, execute the following from command terminal:
db2set DB2_MEM_TUNING_RANGE=40,80
5.2.2 DB2MEMDISCLAIM
DB2 agents may have some associated paging space. This paging space may remain allocated even after the associated
memory has been freed depending on the platform and virtual memory tunings. DB2MEMDISCLAIM, when set, indicates that
explicit requests should be made by DB2 to free associated paging space as well. This leads to smaller paging space
requirements and possibly less disk activity from paging. On systems where memory is tight and paging space is being used,
this may provide a performance improvement.
db2set DB2MEMDISCLAIM=YES
db2stop force
db2start
5.2.3 DB2_SMS_TRUNC_TMPTABLE_THRESH
By default, when a temporary table is not needed it is truncated to zero pages. If there is a lot of temp table activity, this can
degrade performance. Using this registry variable, one can specify the number of extents that should be preserved upon
truncation so that when the table is reused there are pre allocated extents available for use.
1. Run the following command to indicate that 10 extents should be preserved when truncating any temp table:
db2set DB2_SMS_TRUNC_TMPTABLE_THRESH=10
db2stop force
db2start
USERSPACE1 is the default tablespace created and it has a page size of 4k. DB2 supports four page sizes: 4k, 8k, 16k and
32k. You may have tables with large rows that do not fit on a 4k page, so tablespaces with larger page sizes should also be
created. However, larger page size tablespaces need buffer pools with matching page size.
2. Check to see what kind of buffer pools have been created by using the following query:
There is only one bufferpool for all type of tables and activities in the entire database. Also note that it is enabled for Automatic
memory tuning. NPAGES value of -2 indicates that IBMDEFAULTBP buffer pool has been enabled for automatic tuning.
Expected output:
3. Now that buffer pools have been created, we can create the corresponding tablespaces. Execute the following from
command terminal:
4. Verify that all tablespaces are associated with the intended buffer pools. Run the following query:
Expected output:
BPNAME TBSPACE
------------------------- -------------------------
IBMDEFAULTBP SYSCATSPACE
IBMDEFAULTBP TEMPSPACE1
IBMDEFAULTBP USERSPACE1
IBMDEFAULTBP SYSTOOLSPACE
BP8K TBSP8K
BP16K TBSP16K
BP32K TBSP32K
All looks fine except that TEMPSPACE1 is still associated with IBMDEFAULTBP. Recall that we had created a dedicated
bufferpool for TEMPSPACE1 but we never altered the tablespace to point to it. So TEMPBP buffer pool exists but is not being
used by TEMPSPACE1.
1. Execute the following command to create dedicated tablespace for LOB data:
db2set DB2_ENABLE_AUTOCONFIG_DEFAULT=NO
Configuration advisor was also run for INTRO database with default values but we want to run it again with user supplied input.
2. You will use the AUTCONFIGURE command to provide database usage characteristics and get recommended values
to improve the initial configuration of the database with respect to the input. Here we are only retrieving
recommendations (apply none) but you can also optionally apply recommendations as well (apply db only,
apply db and dbm).
Lab
Keyword Values Default Explanation
Setting
Workload_type Simple, Mixed Simple workloads tend to be I/O intensive mostly Default
transactions whereas complex ones tend to be CPU
Mixed,
intensive and mostly queries
Complex
Admin_priority Performance, Both Optimize for better performance or better recovery time. Perf.
Recovery,
Both
Isolation RR, RS, CS. RR Maximum Isolation level applications connecting to this CS
database
UR
AUTOCONFIGURE OUTPUT:
In this section, we will look at some of the tools that are typically used to collect this information. We will also explore how to
collect DB2 configuration information. Keeping track of such system and database level information is important in both
understanding how the system changes over time and to help gauge what effects of any tuning has on the system.
Additionally, it nicely transcends the different operating systems and can be used anywhere where DB2 is installed. The content
can differ across operating systems, but more or less provides the same information,
As you can tell, this is all very useful information to know prior to database tuning.
Similar information can also be obtained using SQL if you have already created a database and are connected to it. This is
useful if you want to programmatically collect such information.
1. To see the system information for INTRO database, issue the following commands:
Such details usually include configuration information about the database manager and database configuration files, buffer
pools, tablespaces, and even the DDL of your database objects.
The db2support problem analysis and environment collection tool is essentially a one-stop-shop for all environment
information. With one command you can collect all of the environment and database information such as database manager
configuration, database configuration, registry variables, tablespace layouts, buffer pool sizes and a lot more. Though it is
primarily intended for DB2 support purposes, it also works nicely to collect information about a system with which you may not
be familiar.
cd /home/db2inst1/
mkdir envdata
cd ~/envdata
db2support . –d INTRO –c
Attention: For the purpose of this lab, you should take a look at the generated files, which can be found in the
/home/db2inst1/envdata directory, please continue in the next instruction.
2. Once the tool finishes, it will have all the information collected and available in db2support.zip file. Unzip the file to see
the db2support.html file by issuing the following command.
unzip db2support.zip
3. Start up Firefox from My computer menu in the lower left hand corner. Once Firefox browser is up, go to File Open
File, specify location for db2support.html file to bring it up in the browser.
(Note: If you notice a message that says “Server not found”, just ignore it)
You should now be able to see all the environment configuration information in the browser broken up into different sections. A
sample of the output produced is listed below:
Browse and review the configuration information dumped by db2support. Based on the output produced, answer the following
questions:
Q2: Can you identify the tablespace containers for all the tablespaces in the INTRO database?
So far we have looked at tools to check the configuration information. There are times when you need to dump the database
schema containing DDL for all the tables, indexes, triggers, stored procedures and functions that were created by the user. This
can be done with the db2look command. This command offers a host of options to tailor the results to your needs. With
respect to performance related information the following parameters can be used:
• -l : bufferpools and tablespaces, created
• -f : database parameters that impact the query optimizer
• -x or -xd : authorizations specified with the RESTRICTIVE parameter of the CREATE DATABASE command
• -m ; statistics.
Issue the following command from command terminal:
View the output files specified. In the queryParameters.out file, you will see the database manager and database parameters
along with any registry settings that affect performance. In the BPS_TBS.out file, you will see the bufferpools and tablespaces
and their DDL.
7 Summary
You can see by now that there are three key areas that are important to understand when trying to avoid performance
degradations of your system: configuration, monitoring, and performance troubleshooting. One of the best practices around
performance tuning and monitoring is to collect baseline information. We explored some of the tools provided by DB2 to gather
the initial data that will help you to understand system performance under both operational and troubleshooting conditions.
8 Cleanup
At this point, we will must clean up both the database and environment to prepare for the next section. Please issue the
following commands:
IBM Canada
8200 Warden Avenue
Markham, ON
L6G 1C7
Canada
IBM, the IBM logo, ibm.com and Tivoli are trademarks or registered
trademarks of International Business Machines Corporation in the
United States, other countries, or both. If these and other
IBM trademarked terms are marked on their first occurrence in this
information with a trademark symbol (® or ™), these symbols indicate
U.S. registered or common law trademarks owned by IBM at the time
this information was published. Such trademarks may also be
registered or common law trademarks in other countries. A current list
of IBM trademarks is available on the Web at “Copyright and
trademark information” at ibm.com/legal/copytrade.shtml
Product data has been reviewed for accuracy as of the date of initial
publication. Product data is subject to change without notice. Any
statements regarding IBM’s future direction and intent are subject to
change or withdrawal without notice, and represent goals and
objectives only.
Table of Contents
1 Introduction..........................................................................................................................................................................................4
2 Suggested Reading .............................................................................................................................................................................4
3 Lab Preparation ...................................................................................................................................................................................5
3.1 Start Virtual Machine and DB2............................................................................................................................................................................ 5
3.2 Initialize Environment.......................................................................................................................................................................................... 7
10 Tools ...................................................................................................................................................................................................48
10.1 DB2TOP............................................................................................................................................................................................................ 48
12 Cleanup.............................................................................................................................................................................................101
1 Introduction
DB2’s performance monitoring framework has been improved in DB2 10. Historically DB2’s monitoring utilities and interfaces
have been based on the monitoring elements presented by the Snapshot Monitoring stream. DB2’s Health monitor, snapshot
commands, administrative views and table functions all utilize the Snapshot Monitoring stream of monitoring elements.
DB2 V10 brings along with it a new and full complement of monitoring table functions and administrative views that enable you
to take advantage of the monitoring framework. The monitoring framework is robust, light weight and features many identical or
similar monitoring elements as the Snapshot monitoring.
This lab is intended to teach you the essentials to monitoring a DB2 database system. The scope of a “DB2 database system”
extends capabilities through the Enterprise Edition, and up to and including version 10. This module is intended for someone
who has some experience with DB2 and wishes to broaden their knowledge in the area of performance-related monitoring.
These labs are performed on a VMware image of a 32bit SUSE Linux machine with minimal resources declared. There are
some commands used in the lab that are typically found on Linux/Unix systems, however basic Unix command-line tools can be
downloaded and easily employed on Windows-based machines at no cost. Additionally, comparable commands are available in
the Windows environment, like find in place of grep.
There are two main tips that will make the lab progress better.
1. Use the up arrow key to recall previous commands. These commands can be edited and re-executed as needed.
2. Follow all the instructions, don’t skip steps. These labs have been created so that they can be rerun but to avoid that
please follow all instructions.
2 Suggested Reading
Information Management Technical Library at developerWorks
Contains articles and tutorials and best practices
http://www.ibm.com/developerworks/views/db2/library.jsp
http://www.ibm.com/developerworks/data/bestpractices
3 Lab Preparation
Before getting started with the lab exercises you will set up you’re environment then work through a series of exercises focused
on DB2 performance monitoring techniques.
1. Choose the DB2_Enterprise_10….vmx file when first opening the lab image in the /DB2v10Performance... folder and
run it.
2. If this is the first time this virtual machine has been started, a series of licensing agreements will prompt you. To
proceed, acknowledge by responding by selecting the “Yes, I Agree to the License Agreement” selection.
3. Login in at the console prompt as db2inst1. You will be prompted for the password which is password. You may be
prompted for the root password as well, if so, it is also password.
From the new terminal session start the Database Manager as follows:
• Enter the command “db2start” to start the database instance. Make sure the instance is started successfully. (Note:
If you notice a message that says “The database manager is already active”, that’s “OK. Please continue to the
next step.)
• In order to enable the aliases defined for this lab, copy the /home/db2inst1/labbashrc to /home/db2inst1/.bashrc and
source that file.
cd ~
cp labbashrc .bashrc
. .bashrc
1. If you no longer have the terminal opened from previous exercise: By right clicking on the Gnome desktop (as in 3.1)
open a new terminal window.
2. While logged in as user “db2inst1”, change your working directory to the folder containing the script mentioned.
edb2scripts
The command will take you to directory /home/db2inst1/Documents/LabScripts/db2pt/essential/db2scripts.
3. Before executing, take a look at the script, and note how the database is created. It will first try to drop the db2pt
database, in case one exists from previous labs, and then create a new version of the database.
cat createDB.db2
Note: You can use the arrow keys to scroll up and down in the script and type the letter “q” to exit anytime.
Note: Make sure that the create database SQL statement has finished successfully. The following message should be
returned after the successful completion of this SQL statement:
With the new monitor elements and infrastructure, you can use SQL statements to efficiently collect monitor data from memory
to determine whether specific aspects of the system are working correctly and to help you diagnose performance problems.
With the new access methods, you can get the data you need without using the snapshot interfaces. The increased monitoring
granularity gives you more control over the data collection process; collect the data you want from the source you want. The
monitoring framework also reduces performance monitoring overhead to a minimum. Though the monitoring framework was
designed to work with the Performance Optimization feature pack, it can still be used with the default workloads that
are predefined for each database created.
Monitoring information is collected about the work performed by your applications and reported through table functions and
administrative views at the following three levels:
System level
These monitoring elements provide details about all work being performed on the system. Monitor-element access
points include service subclass, workload definition, unit of work, and connection.
Activity level
These monitor elements provide details about activities being performed on the system (a specific subset of the work
being performed on the system). You can use these elements to understand the behaviour and performance of
activities. Monitor-element access points include individual activities, and entries in the database package cache.
For database-wide control over the collection of monitoring data at the System, Activity and Data Object Levels, and the
generation of events in the new event monitors, ten database configuration parameters have been added. With the workload
management feature enabled, these controls can be enforced at both the database and at activity levels defined in workloads.
In this short exercise you will learn how to look at a database configuration file, examine the configurations for the monitoring
framework, and then actually make a few updates to the lock monitoring configurations.
1. If you no longer have the terminal opened from the previous exercise, right click on the Gnome desktop (as in 3.1) and
open a new terminal window.
2. Enter the command to get and display the database configuration file. The new configurations will appear at the bottom
of the file’s listing under Monitor Collect Settings.
db2start
db2 get db cfg for db2pt | grep -i MON
The request (REQ), activity (ACT) and Object Metrics (OBJ) are all set on (BASE) by default, which means all metrics are
collected for all requests executed on the data server.
3. Next you are going to turn on maximum collection for lock wait and deadlock monitoring and set the threshold for
lockwaits to 3 seconds.
4. Show the new database configurations You may want to “get db cfg show detail” and explain:
This short exercise will teach you how to set and reset these controls.
1. By right clicking on the Gnome desktop (as in 3.1) open a new terminal window or you may use the one from 4.1 if
still open.
2. Connect to the database and list the system monitor switches from the database manager configuration file. You will
find most switches are set to OFF.
4. Now you will turn OFF the switch for bufferpools at the session level. It should currently be set to ON (session level),
adopted from the DFT_MON_BUFPOOL value in the db manager configuration file. First activate the database and take
a snapshot of the database, then turn the switch ON and look again to see that bufferpool metrics are being collected
for your session.
There are two ways to monitor operations in your database. You can view information that shows the state of various aspects of
the database at a specific point in time. Or, you can set up event monitors to capture historical information as specific types of
database events take place.
You can monitor your database operations in real-time using monitoring table functions. For example, you can use a monitoring
table function to examine the total amount of space used in a table space. These table functions let you examine monitor
elements and metrics that report on virtually all aspects of database operations using SQL.
1. Create the event monitor. To create an event monitor, use the appropriate version of the CREATE EVENT
MONITOR statement. When you create an event monitor, you must choose how to record the data that the event
monitor collects. All event monitors can write their output to relational tables; however, depending on your specific
purposes, there are different options that might be more appropriate.
db2stop force
db2start
db2 connect to db2pt
db2 "SELECT SUBSTR(appl_name,1,20)
appl_name,application_handle,event_timestamp, event_type,
substr(system_authid,1,20) system_authid FROM chg_summary_history"
edb2scripts
db2 -tvf create_objects_evm.sql
1. Identify a table for which you want to view object usage statistics. You can use the MON_GET_TABLE table function to
view monitor metrics for one or more tables.
2. Create an usage list for the table by using the CREATE USAGE LIST statement
db2 "CREATE USAGE LIST DOCUMENTSUL FOR TABLE DOCUMENTS"
3. Activate the collection of object usage statistics by using the SET USAGE LIST STATE statement
db2 "SET USAGE LIST DOCUMENTSUL STATE = ACTIVE"
4. During the collection of object statistics, ensure that the usage list is active and that sufficient memory is allocated for
the usage list by using the MON_GET_USAGE_LIST_STATUS table function. To check the status of the
DOCUMENTSUL usage list, issue the following command:
db2 "SELECT MEMBER, STATE, LIST_SIZE, USED_ENTRIES, WRAPPED
FROM TABLE(MON_GET_USAGE_LIST_STATUS('DOCUMENTS', 'DOCUMENTSUL', -2))"
6. When the time period for which you want to collect object usage statistics is elapsed, deactivate the collection of usage
list data by using the SET USAGE LIST STATE statement.
db2 "SET USAGE LIST DOCUMENTSUL STATE = INACTIVE"
7. View the information that you collected by using the MON_GET_TABLE_USAGE_LIST function. You can view statistics
for a subset or for all of the statements that affected the table during the time period for which you collected statistics
db2 "SELECT MEMBER, EXECUTABLE_ID, NUM_REFERENCES, NUM_REF_WITH_METRICS,
ROWS_READ, ROWS_INSERTED, ROWS_UPDATED, ROWS_DELETED
FROM TABLE(MON_GET_TABLE_USAGE_LIST('DB2INST1', 'DOCUMENTSUL', -2))
ORDER BY ROWS_READ DESC
FETCH FIRST 10 ROWS ONLY"
8. If you want to view the text of a statement that affected the table, use the value of the executable_id element in the
MON_GET_TABLE_USAGE_LIST output as input for the MON_GET_PKG_CACHE_STMT table function
The various options for table event monitors are set in the CREATE EVENT MONITOR statement. For further assistance in
generating CREATE EVENT MONITOR SQL statements for write-to-table event monitors, you can use the db2evtbl command.
Simply provide the name of the event monitor and the required event type (or types), and the CREATE EVENT MONITOR
statement is generated, complete with listings of all the target tables. You can then copy the generated statement, make
modifications, and then execute the statement from the command line processor.
Formulate a CREATE EVENT MONITOR statement using the WRITE TO TABLE clause to indicate that event monitor data is to
be collected in a table (or set of tables).
• ACTIVITIES
• BUFFERPOOLS
• CHANGE HISTORY
• CONNECTIONS
• DATABASE
• DEADLOCKS
• LOCKING
• PACKAGE CACHE
• STATEMENTS
• STATISTICS
• TABLE
• TABLESPACE
• THRESHOLD VIOLATIONS
• TRANSACTIONS
• UNIT OF WORK
1. To create a unit of work event monitor called myevmon, use a statement like the one that follows
db2 "CREATE EVENT MONITOR myevmon FOR UNIT OF WORK WRITE TO TABLE"
The preceding statement creates a unit of work event monitor that uses defaults for the logical groups of monitor elements
collected, the corresponding output table names, and the target table spaces for the table.
• connheader_evm_test
• stmt_ test
• subsection_ test
• control_ evm_test
2. Optional: Specify the logical groups for which you want data collected. By default, event data is collected for all logical
data groups for the event monitor type. (See Target tables, control tables, and event monitor table management for
details.)
If you want only data for selected logical groups collected, you can specify the names of the logical groups to include in
the CREATE EVENT MONITOR statement. For example, with a locking event monitor, you might want to collect only
the information associated with the LOCK and PARTICIPANT logical groups. To include only these logical groups, you
could use a statement like the one that follows:
You can also use a catalog view to see which event monitors exist for monitoring a specific type of event. The
SYSCAT.EVENTS view returns a list of event monitors and the type of events for which they record data.
As event monitors are enhanced in the DB2 product, the tables they produce might change. For example, new columns might be
added to a table for reporting new monitor elements. Before Version 10, if you had existing event monitors that wrote to tables
that contained data that you wanted to retain, and you wanted to collect the data in the newly-added columns, you were required
to manually alter them after upgrading to the new release. This alteration involved adding any of the new columns that you might
want to use. If you did not add the new columns, the event monitor would work as it had in the previous release, capturing only
the data supported by that the event monitor in that release.
Unformatted event tables that had changed could not be upgraded at all; you were required to drop them and then re-create
them.
The EVMON_UPGRADE_TABLES stored procedure upgrades the definitions of existing event monitor tables to match those
produced by the current level of the DB2 product. This feature lets you keep any existing tables that you might have, along with
all the data they contain, eliminating the need to manually alter, or to drop and re-create tables.
Note: Starting in Version 10, you can also use the ALTER EVENT MONITOR statement to add new logical groups to an event
monitor. You can use this approach as an alternative to EVMON_UPGRADE_TABLES to add logical data groups added in a
new release. However, you cannot use ALTER EVENT MONITOR to modify logical groups that are already associated with the
event monitor; if a logical data group already associated with the event monitor has changed, the only way to modify the event
monitor is using the EVMON_UPGRADE_TABLES procedure.
The EVMON_UPGRADE_TABLES procedure works with both regular and UE tables. For regular tables, the procedure adds
any new columns needed, drops old columns that are no longer required, and alters any columns as needed. For UE tables, the
procedure adds new columns and modifies existing columns as needed to allow the UE table to be processed by the
db2evmonfmt tool, or the EVMON_FORMAT_UE_TO_TABLES or EVMON_FORMAT_UE_TO_XML routines.
db2 "create event monitor lock for locking write to unformatted event table"
db2 "create event monitor act for activities write to table control"
db2 "create event monitor stat for statistics write to table"
db2 "create event monitor conn for connections write to table"
After upgrading the database to the current release they upgrade all the event monitor tables using the following command:
If instead they only wanted to upgrade act, they could use this command:
Alternatively they could choose to upgrade only the statistic event monitors by using this command:
Even when you have set up monitoring and diligently followed all the steps to keeping your system performing well, inevitably
you will encounter a performance issue, typically identified when an alert is issued or when application response times are
noticeably slower. You may need to quickly figure out what activity is causing a problem and moreover, figure out why it is
problematic and recommend a solution.
This is when you go into triage mode. You need to investigate, quickly identify, and prioritize any potential activities that may be
contributing to a slowdown. You may have alerts set up or monitoring data that you have collected, but they may prove
inconsequential to the situation at hand, often taking significant time to format, sequence, and analyze. There are several key
areas to check on when trying to determine what is going on with your slow or unresponsive database system, start by asking
and answering a few simple questions.
• Are there more users on the system?
• Are there locking problems occurring?
• What is the overall state of the database?
• Are statistics current?
• What is status and usage of tablespaces and their containers?
• Are bufferpools being used efficiently?
• Is there excessive sorting or sort overflows happening?
• Is the database accepting new connections? remotely? locally?
• Have changes been made to the system?
In these exercises you will practice several methods to acquiring insight into “what is happening?” on your database system. By
doing this, you will gain a better understanding as to what activities can adversely affect performance and some of the options
available for gathering basic performance information.
6.1 Applications
APPLICATIONS: Here you will start a workload to generate some activity on the system and learn how to gain insight into what
applications are running and what they are doing.
1. Open up two terminal sessions one will be referred to as the “DB2 terminal” the other the “Workload Expert terminal”.
2. From the Workload Expert terminal: enter the alias that will put you in the correct directory and start the scenario.
cd /home/db2inst1
. .bashrc
edb2scripts
./we_run_live_random.sh
This will start the Workload Expert in graphical mode, which is required by this scenario. Wait for the scenario to give the
indication that it has started and is actually running. The appearance of the Live Connection Control is when you know it has
started successfully.
3. For the rest of the exercise enter the commands from the DB2 Terminal:
db2 list applications for database db2pt
Notice the various connection attributes being displayed especially the Appl. Handle and the Application Id. These are
often used as identifiers when looking at actual application activity or looking at locks conflicts.
This is a simple way to count how many applications are connected. Here you have qualified the lines to count with the grep of
DB2INST1 and used the wc (word count) utility to count the lines, words, and characters of the piped output. The first figure
reported is the number of applications connected.
Lab tip: you can select, copy, and then paste Application IDs from the terminal
Using the spacebar to page through the results, see if you can find how many locks are being held, or perhaps how many sorts
have been performed.
Here you will use an administrative view to retrieve information on connected applications. Any information that you get from
using the “get snapshot” command can be obtained via SQL with corresponding administrative views.
**********************************DB2PD****************************************
The db2pd command can yield deep insight into application activity as well as many other critical monitoring objects like
bufferpools, tables, tablespaces, memory sets, and transactions. With the db2pd command you can get information on individual
threads or produce a stack trace for all engine dispatchable units (consisting mostly of threads). The db2pd command’s results
takes some practice to get used to, but can supply very valuable information and insight as to what is currently happening on a
system. It has been expanded to include workload management objects like service classes, work action sets, and work class
assets.
Here you see some familiar data like AppHandl, Appid, and Status with several other IDs used as keys to other db2pd inquiries.
This is merely a very simple sample of what this command can do.
Now you will use db2pd to determine which edu, engine dispatchable unit, primarily threads, are consuming the most CPU
cycles. This can be a valuable diagnostic tool when you start with just the knowledge that a CPU bottleneck has arisen.
Use the space bar on your keyboard to go through the result pages
You can leave the Workload Expert scenario running and the DB2 Terminal open to continue with lab exercises.
The metrics returned by the MON_GET_CONNECTION table function represent the accumulation of all metrics for requests that
were submitted by a connection. Metrics are rolled up at unit of work boundaries, and periodically during the execution of
requests. Therefore, the values reported by this table function reflect the current state of the system at the time of the most
recent rollup. Metrics are strictly increasing in value. To determine the value of a given metric for an interval of time, use the
MON_GET_CONNECTION table function to query the metric at the start and end of the interval, and compute the difference.
Tip: As a connection can be mapped to more than one service superclass during its lifetime, the metrics reported through the
MON_GET_CONNECTION table function might represent a subset of the metrics for all requests submitted over the connection.
This might occur if metrics collection is disabled for some of the superclasses that are mapped by the connection.
The MON_GET_CONNECTION table function returns one row of data per connection and per member. No aggregation across
members (for a service class or more), is performed. However, aggregation can be achieved through SQL queries.
Display connections that return the highest volume of data to clients, ordered by rows returned.
Metrics collected by this function are controlled at the database level using the mon_obj_metrics configuration parameter. By
default, metrics collection is enabled.
Figure 35 – List one row of data per database table and per database member
There is an optional input argument (either "D" or "S") of type CHAR(1) that specifies information type for the returned statement.
If the argument is NULL or an empty string, information is returned for all SQL statements. Not case sensitive: "D" stands for
dynamic; "S" for static.
List all the dynamic SQL statements from the database package cache ordered by the average CPU time.
6.6 Statistics
The DB2 optimizer evaluates the data in your databases to develop access plans as efficiently as possible (using minimal
resources) to return a queries result set. If statistics have not been run or are inaccurate, your system will run a less-than-
optimal level. Statistics can make a huge impact on performance. One quick and easy way to get a picture on how recently
statistics have been run is to query the syscat.tables view. In this very short exercise you will practice querying the system
catalog views. The system catalog views represent both the statistics on tables and indexes along with the metadata of all
database objects that DB2 uses. You will then update the statistics and query again to see the results.
1. By right clicking on the Gnome desktop (as in 3.1) open a new terminal window.
2. You will run a script to create a schema (RSTATS), and a table with a single index (primary key). From the open
terminal:
edb2scripts
conndb
db2 -tvf createRSTATS_T1.db2
Now you will run a script that will query both the SYSCAT.TABLES and SYSCAT.INDEXES views to check to see if statistics
have been run. At this point the STATS_TIME should be null “-“, indicating statistics have not been run. Then you will update
the statistics for the table and see that the STATS_TIME has been updated to reflect the latest statistics update. From the same
terminal:
db2 terminate
7 System Bottlenecks
Note: Your results may be a little different.
Monitoring your database system at the system level and identifying bottlenecks can help provide clues as to where to look for
tuning opportunities.
In this exercise, you will proceed as if you have a range partitioned table that is created in a single tablespace. You will run a
scenario that will create such a table and create I/O against it, after which, you will collect information like how many physical
reads are occurring on the tablespace and how much read time is accumulating against the underlying container. Then you will
run the same scenario, except the table is spread across two tablespaces. You will see how I/O gets spread and hopefully
realize the benefits gained by spreading data across different resources. All the while, you will be running the system
monitoring tools iostat and vmstat. You will learn what to look for to identify excessive I/O with these tools.
A note on spreading data across resources: There are a number of ways to “spread” data out to maximize I/O potential.
Range partitioning is only one technique you can use to spread your data out across disk resources. It should be noted that
range partitioning is primarily employed for a number of reasons, outside its potential to spread data. For instance, it also allows
for the seamless rolling-in and out of data since each range is contained physically in its own file. It also improves performance
through a feature called partition elimination. This occurs when a query includes the partitioning key as part of the searchable
predicate; the DB2 optimizer is made aware of the partitioning and can eliminate ranges that are not part of the query. Of course,
spreading data with range partitioning will not improve performance unless the tablespace containers have dedicated disk
resources assigned; its use in this lab is more for demonstrative purposes than to advocate a preferred technique of spreading
data.
Best practices dictate that spreading data starts with striping at the storage system level. You can create stripe sets across
multiple disks and build your file systems or logical volumes on top of these. Another approach is to present additional disk
resources to the database system. Then create new file systems or logical volumes on the new disk resources. It is more
effective to stripe at the lowest level possible. This exercise emphasizes using system monitoring tools to identify bottlenecks as
part of your approach to investigating and monitoring DB2 systems for performance issues.
1. Open up three terminal sessions. One will be referred to as the “vmstat terminal”, the second as the “iostat terminal”,
and the third as the “Workload Expert terminal”.
2. From the vmstat terminal: Start the vmstat tool with a 2 second interval.
vmstat 2
Take note of the cpu/wa column, this represents %cpu time in wait
3. From the Iostat terminal: Start the “iostat tool” with a 2 second interval
iostat 2
4. From the Workload Expert terminal: Start the RangeNoSpread.scn workload scenario. This will take about a minute
to prepare and run for about a minute there after. Once running switch your focus to the vmstat and iostat terminals
output, where you should see high CPU wait times, well in excess of 25%. This is indicative of a disk bottleneck.
cd ~
cp labbashrc .bashrc
. .bashrc
edb2scripts
db2stop force
db2start
db2 activate database db2pt
./we_run_RangeNoSpread.sh
Watch the vmstat and iostat terminals to see the high CPU wait time being registered
Note: When you see this output in the WE console the scenario is done setting up and starting the actual workload.
VMSTAT
IOSTAT
5. Once the scenario has finished, from the Workload Expert terminal: Connect to the database and take a snapshot of
the tablespaces. Note how many physical reads are being done against the TS_RANGE1 tablespace. Then run the
script that will get the read times on a per container basis. You can compare these results with the results from the
second scenario.
conndb
db2 get snapshot for tablespaces on db2pt | more
6. Clean up the data objects created in the scenario. This will drop the objects created in the “No Spread” scenario.
Then you will run the same scenario except that the ranges will be dedicated to separate tablespaces and underlying
containers. This will result in the I/O being spread out across containers. You can compare these results with the
original results. The total number of I/O’s should be very close but spread out onto different containers.
Note: Since actual disk resources for both containers are the same, you will not see a big difference in CPU Wait time
(maybe a little bit, attributed to multiple prefetchers working). The emphasis here is that you will see that the I/O is indeed
being spread between the containers with a snapshot of the tablespaces and the MON_GET_CONTAINERS table function.
• Switch your attention to the Monitoring terminals to see the high wait time being registered in the
vmstat output
vmstat
conndb
db2 get snapshot for tablespaces on db2pt | more
7. Finally, clean up the objects created in the scenario and close all terminals.
db2 -tvf cleanRange.db2
• Close all terminals and processes. Use Ctrl+c to break out of vmstat and iostat.
One of the most important things to monitor on a regular basis, if not the most important, is I/O activity. Excessive I/O activity
results from many different database operations; table and index scans, mismanaged or lack of memory to relieve I/O sort
overflows, batch activity, excessive logging, temporary tables, lack of sufficient resources, and poor database designs, are some
typical causes.
In the following exercises, you will use both the monitoring table functions and snapshot monitoring tools to help identify
potential and actual “I/O hot spots”.
It is a good idea to transition from snapshots to the monitoring framework wherever and whenever practical. It represents the
future of DB2 monitoring, however it should be noted that if you collect metrics for both you will incur overhead for both
monitoring streams.
Before you start, familiarize yourself with some of the important DB2 monitoring elements that report on I/O. The following table
represents some of these monitoring elements.
1. Open up two terminals sessions. One will be referred to as the “DB2 terminal” and the other as the “Workload Expert
terminal”.
2. From the DB2 terminal: stop and start the instance. This will reset all counters for both monitoring streams. Connect to
the database.
db2stop force
db2start
3. From the Workload Expert terminal enter the alias that will put you in the correct directory and start the scenario.
cd ~
cp labbashrc .bashrc
. .bashrc
edb2scripts
./we_run_live_random.sh
This will start the Workload Expert in graphical mode, which is required by this scenario. Wait for the scenario to give the
indication that it has started and is actually running. The appearance of the Live Connection Control is when you know it
has started successfully.
First take note of which is the later timestamp, the first database connect timestamp or the Last reset timestamp, the later
one represents when counters started for these data points. This is a simple and quick way to check on read activity on
your database. You can quickly compare the number of physical to logical reads, evaluate the number of direct reads, and
gauge how much Insert, Update, Select, and Delete activity has been going on.
You can use corresponding, snapshot based, administrative views and table functions to collect this data into relational
tables for further comparisons and analysis (SYSIBMADM.SNAPDB and SNAP_GET_DB).
In the subsequent exercises you will enhance some of the basic monitoring elements by calculating ratios from the base
monitoring counters.
5. Cancel the workload by clicking on the Cancel button under the Scheduler Manager window of the Workload Expert
graphical interface. Once the scheduler has been terminated, clic “X” out of the window closing the Workload Expert.
Respond YES to any prompts alerting you to the fact that the application has stopped processing (due to the cancel).
6. Close the open terminal.
The first metric you will capture is Reads per Transaction. You will run the SQL in the reads_per_trx.db2 file to get results.
The administrative view SYSIBMADM.SNAPDB is used here to calculate the average number of rows read per each transaction
from applications connected to the database.
This is a very high level monitoring view. It is best used when initially assessing a system or for trending I/O usage. These same
monitoring elements can use at more granular levels when taking application level snapshots.
The next ratio is the number of Rows Read to the Number of Rows Returned it is somewhat similar to reads per transaction in
that it is measuring read efficiency, looking to see how many rows need to be read for each row that is actually returned to the
application. You will run the SQL in the rread_per_rrtrn.db2 file to obtain results. The monitoring table function
MON_GET_WORKLOAD() is used as the basis to the query.
Lastly, you will look at the Highest Read Time per Container. This will help you determine where I/O time is being spent. It will,
in part, verify if your physical layout works well with respect to the activities on your database. You will run the SQL in the
HighestReadTimePerContainer.db2 file. The monitoring table function MON_GET_CONTAINER() is used as the basis to the
query.
{By looking at the number of rows read per rows returned ratio you might assess that additional indexes are required. The read
time per container can tell you physically where the most time is being spent on reading and may assist you in determining if and
where more containers are needed for your tablespaces and which tablespaces, including temporary tablespaces are the
busiest.}
reads_per_trx.db2: in this screen shot you see a result of 110 reads/trx, a fairly good rate for an OLTP system. Under 10 would
be very good, over 50 worth investigating.
2. Now get a picture of the rows read for every row returned to the applications.
db2 –tvf rread_per_rrtrnd.db2
rread_per_rrtrnd.db2: Since we are using the monitoring framework, statistics are aggregated at the workload level, in this
case even without the workload management feature being used the default system workloads are available to query.
3. Finally, find the highest read time per container. This will indicate which containers and tablespaces are being heavily
utilized.
db2 –tvf HighestReadTimePerContainer.db2
HighestReadTimePerContainer.db2: Here it is quite easy to see which tablespace’s underlying containers have the
highest read time. Keep in mind that this is done on a per container basis so this would be a great way to evaluate how well
data is spread with respect to the requests being made, with the basis of evaluation being actual usage, as opposed to
counting rows or looking at data statistics and assuming equal usage.
Figure 52 – Status, platform, location, and connect time as an aggregate view of all members
Bufferpools are real memory heaps used to cache pages of data and indexes read in from disk resources. When comparing
access times between memory and disk we are talking about nanoseconds vs. milliseconds, since memory is a much faster
resource. You can monitor and tune your bufferpools when necessary, or let STMM do it automatically for you.
4. From the DB2 terminal: Connect to the database and run the file that will calculate BP hit ratios for regular data with
both the snapshot monitor administrative view (SNAPBP) and one of the monitoring table functions
(MON_GET_BUFFERPOOL). SQL statements for both are in the same file; the first listing is the SNAPDB based query,
while the second uses the MON_GET_BUFFERPOOL table function.
edb2scripts
conndb
db2 -tvf get_bphits_all.db2
You can use the up key on your keyboard and re-enter the “db2 –tvf get_bphits_all.db2” command a few times to
inspect the output at different points in time.
5. Examine the output from each set of metrics. Note that the bufferpool hitratio is low for the BPMONTUNE bufferpool,
and make a note of what that value is and the number of logical reads that have been performed.
6. Still from the DB2 terminal: Alter the size of the bufferpool to 5000 and examine bufferpool metrics again. The
bufferpool will dynamically change size. You should see some improvement but you would probably like to see better.
(The longer the scenario has been running, the longer it will take to see improvement). Then alter the BPMONTUNE
bufferpool to 10000 pages. Soon after this change you should see hit ratios approaching/exceeding 90 %. Depending
how long the Workload Expert Scenario has been running, it may take a few minutes before the hit ratios reach such a
level.
Lab Tip: Use the up arrow on your keyboard to recall commands.
7. Finally alter buffepool to automatic. When you do so it will be dynamically tuned automatically by DB2’s Self Tuning
Memory Manager and its sophisticated algorithms.
db2 alter bufferpool bpmontune size automatic
db2 –tvf get_bphits_all.db2
8. From the Workload Expert terminal you will stop the running scenario by typing quit and the enter key.
quit
9. Close both terminals
Note: If cleanup fails for any reason and you wish to run the lab again, run the manual cleanup file: db2 –tvf
manual_cleanup_bpmontune.db2
10 Tools
Note: your results may be a little different.
You have learned about some of the essential DB2 monitoring elements, why they are important, and some of the DB2
interfaces available for monitoring performance related activities. It is possible to create your own scripts, applications, and
monitoring repositories with these interfaces but this can be a daunting task especially in larger environments. This is when you
start thinking about finding tools, tools that already make good use of DB2 monitoring interfaces. Tools offer many advantages
like logical groupings of related metrics, interactive design, and the ability to quickly save performance data for further analysis.
In this lab exercise you will learn about the db2top tool, which comes with DB2 on Unix/Linux installations.
10.1 DB2TOP
The db2top tool runs on and comes with Unix/Linux installations of DB2. You can look at both cumulative and delta
perspectives for a wide variety of metrics and activities including: Databases, Tablespaces, Bufferpools, Dynamic SQL, Locks,
Memory Pools and more.
In this exercise, you will start a workload with the Workload expert utility and examine what kinds of performance related activity
is occurring from a number of different views. Then we will look for expensive SQL, display and save the actual statement into a
file where it can be benchmarked and tuned with other db2 tools such as db2batch.
1. Open up two terminal sessions. One will be referred to as the “DB2 terminal”, the other as the “Workload Expert
terminal”.
2. From the DB2 terminal: stop and start the instance. This will reset all counters for both monitoring streams. Connect to
the database.
db2stop force
db2start
3. From the Workload Expert terminal enter the alias that will put you in the correct directory and start the scenario.
cd ~
. .bashrc
edb2scripts
./we_run_live_random.sh
This command will start the Workload Expert in graphical mode and run a mixture of random SQL against the db2pt
database (Live SQL Mixer). It will take a minute or two before it is fully running. The Live Connection Control will appear
as shown below when fully running:
4. Once you’ve seen that the scenario is running use the graphical controls, increase connections to around 5-10 for each
control this will make the workload a bit more substantial.
5. From the DB2 terminal: Start the db2top facility and invoke the online help utility.
db2top -d db2pt
h
Note : command “h” is for help.
Here you can see the various keystroke based options available. There is a wide range of objects, activities and
perspectives to choose from. There are also options relating to the display and behavior of the tool. Most notably, O is
used to look at current settings for db2top, k is used to toggle between actual and delta values and C enables the
capturing of the session into a file that can be replayed and thus reanalyzed.
6. Now you will explore some of the options first hand. First you will look at Tables (T), Bufferpools (b), and finally Dynamic
SQL (D).
Here you can see the most active tables and what kind of I/O is occurring.
Now you are at the terminal command prompt. You should be able to look at the mySQL.out file you created. It is ready
for further analysis and tuning. You will create a simple benchmark with the db2batch tool.
The mySQL.out file should be located in the directory you called db2top from, so you shouldn’t have to navigate to work
on the file.
Lab Note: you may need to enter clear to reset the terminal display (get rid of black).
Now you can see how long it takes for DB2 to process this query in CPU time. You could use other tools like the explain
utility or db2advis to further analyze and tune. You will use these tools in greater detail in the SQL Tuning Lab.
db2sampl
Note: This can take some minutes, you need to wait until the message “SERVER: ACTIVE” appears in the console as shown in the
next image:
Note: Before to continue you need to be sure the instance is started, so in a new terminal window enter the command “db2start” to
start the database instance. Make sure the instance is started successfully. If you notice a message that says “The database manager
is already active”, that’s OK. Please close the terminal window and continue to the next step.
5. Type-in User name db2inst2 and password password in the login window and click Log In.
4. Click OK to confirm.
3. Fill in the details for the Sample database. Click OK to add the connection.
5. Select the Use predefined template radio button and click the dropdown to select OLTP Production with all details. Click
Next.
6. On the next screen, click the pencil button next to Retention times and sampling intervals.
7. Notice that you can modify the aggregation intervals in this screen to influence the retention period of the detailed data,
but this is not neccessary in this case. Click Cancel.
8. Check the Workload Manager option in Monitoring profiles. Click on the pencil button next to Workload Manager.
11. Click Next two times and click Finish. Fill in the password password and click OK.
15. Now we need to create the table called OPM_SAMPLE_TABLE for use by the subsequent labs.
16. Open a new terminal window by right-clicking on the Desktop and choosing the “Open Terminal” item.
3. Specify password as password (just in case ask for it), click Log In.
4. Through this Alert configuration, you will be notified if the database is not reachable or quiesced. [Note: We will not
change anything here.] Click Cancel.
Note: You can click some other Alert Type shown and click to select it and then click Edit to review. Please do not modify and click
Cancel.
Note: Just FYI: The Alert Configuration can also be done through the Health Summary dashboard as shown.
2. Click Connect. [Note: If you see YES instead of the Connect button, skip to the next step.]
7. Click Apply.
Figure 95 – Apply
Figure 96 – By Database
Note: After changing one or more alerts for the current database, the same settings can be applied to another database using the
Copy Settings button. [Note: We will not change anything here].
The OPM allows you to manage performance alerts easily. It gives you the ability to:
Note: Just FYI: You could have also clicked Open Services to open Email and SNMP services.
4. In Configure the Email Service, the SMTP server information can be specified to enable email notifications. [We will not
change anything here.] Click Cancel.
5. Click SNMP to select it. Click Configure.
10. Please notice that you can configure notifications for each alert separately for warning, critical or both using either email
or SNMP or both. This allows OPM to send different alerts to different people. [Note: We will not modify anything here.]
Note: The script CORE01.SH will run a work load through a Java program with no retry for the deadlocks.
When you run this script, you will notice java exceptions being thrown for SQL error -911 indicating deadlocks between competing
transactions. This is by design and normal.
4. Run the CORE02.SH script from the second terminal. This script is the same as CORE01.SH except that it connects as
user DB2POWER.
./CORE02.SH
Note: If the workload stops before you complete this lab exercise, please restart it again.
5. In the Health Summary dashboard, you have the convenience of looking at the overall health of the configured
databases through a single screen as shown below. Based upon the alerts and warnings, you have the ability to drill
down further to explore any problem areas.
6. Please wait for a minute or two for the Critical Alerts number to go more than 50 if this is possible.
7. Click on the number shown under the Critical Alerts.
8. Notice that the majority of the alerts are lock wait, which is expected due to -911 errors. Click Analyze.
9. Click the Participants tab. You can see Owner and Requester application causing the lock wait.
10. Click the Statements tab. You can see the SQL statements from each application causing lock wait.
5. Click the Current Blocking Connections tab. Select any lock wait.
6. Click Analyze to open details on the locking issue for any selected lock wait.
DB2 Performance Tuning and Monitoring Clinic Workbook page 100 of 251.
IBM Software
Information Management
7. Click Close.
DB2 Performance Tuning and Monitoring Clinic Workbook page 101 of 251.
IBM Software
Information Management
3. View the Sort and Sort Overflow diagrams. Click the Actions tab.
5. In the Workload dashboard, check number of sorts per minute, sorts per transactions, total sort times, sort time, piped
sort and sort memory charts.
DB2 Performance Tuning and Monitoring Clinic Workbook page 102 of 251.
IBM Software
Information Management
6. While you are on the Workload dashboard, review the transaction, row and statement throughput at the database level.
DB2 Performance Tuning and Monitoring Clinic Workbook page 103 of 251.
IBM Software
Information Management
3. Review the alerts shown for Package and Catalog Cache Hit Ratio.
DB2 Performance Tuning and Monitoring Clinic Workbook page 104 of 251.
IBM Software
Information Management
5. The memory dashboard shows memory heaps related to the Database Global Memory.
6. Check the memory sizes for each and the related database parameter.
DB2 Performance Tuning and Monitoring Clinic Workbook page 105 of 251.
IBM Software
Information Management
Note: Remember that we drilled down from overall health summary for all systems to a very specific problem notified through an
alert. This feature helps to pin point hot spots quickly.
DB2 Performance Tuning and Monitoring Clinic Workbook page 106 of 251.
IBM Software
Information Management
2. On the Overview dashboard, you can focus on areas that are shown in red.
3. When you hover the mouse over a line, the Open Details icon becomes visible, which can be clicked to open the related
dashboard.
DB2 Performance Tuning and Monitoring Clinic Workbook page 107 of 251.
IBM Software
Information Management
2. You can also drag the blue 1 Hour button backward on the timeline to see the past data.
3. While viewing past data, the aggregation level indicates if you are viewing detailed or aggregated data.
Note: The aggregation levels were defined at time of adding the monitored database. The aggregation levels can be changed
through Open Database dashboard.
DB2 Performance Tuning and Monitoring Clinic Workbook page 108 of 251.
IBM Software
Information Management
2. You can either choose for screen to refresh automatically through a chosen period (default 1 minute) or refresh it at any
time by clicking Refresh Now.
DB2 Performance Tuning and Monitoring Clinic Workbook page 109 of 251.
IBM Software
Information Management
2. Click to uncheck Refresh Every. Make sure you are in the Real-Time Data view.
DB2 Performance Tuning and Monitoring Clinic Workbook page 110 of 251.
IBM Software
Information Management
4. Review your connections and notice CPU and different wait times.
DB2 Performance Tuning and Monitoring Clinic Workbook page 111 of 251.
IBM Software
Information Management
10. Review the Rows and Transactions, Locking and Communication and Utilities tabs.
Note: Through the connection details dashboard, you can see aggregated information at the connection level.
The next section shows the individual SQL details in a connection.
DB2 Performance Tuning and Monitoring Clinic Workbook page 112 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 113 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 114 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 115 of 251.
IBM Software
Information Management
Note: Since our work load is OLTP like with short running queries, you will not be able to see actual SQL.
If you are running data warehouse type long running queries, you will be able to see them in this dashboard.
The next section shows the individual SQL details in a connection.
6. Click Close.
7. Close the Current Application Connections dashboard.
DB2 Performance Tuning and Monitoring Clinic Workbook page 116 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 117 of 251.
IBM Software
Information Management
6. Select OPM_SAMPLE_BP_1 buffer pool. Click the button Show Contained Objects and the screen will
automatically switch to the Table Spaces tab with a filter on only table spaces for this buffer pool.
DB2 Performance Tuning and Monitoring Clinic Workbook page 118 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 119 of 251.
IBM Software
Information Management
11. The Tables tab shows tables contained in the table space OPM_SAMPLE_TS_1.
DB2 Performance Tuning and Monitoring Clinic Workbook page 120 of 251.
IBM Software
Information Management
14. The SQL dashboard shows only those SQL statements containing the OPM_SAMPLE_TABLE.
15. Close the SQL Statements and Buffer Pool and I/O dashboards.
11.7.1.4 Reports
1. Click Run performance reports for a database. [Please wait for reports to load.]
DB2 Performance Tuning and Monitoring Clinic Workbook page 121 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 122 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 123 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 124 of 251.
IBM Software
Information Management
• Close up all process and terminals. This is the end of the lab exercises.
LAB FILES:
DB2 Performance Tuning and Monitoring Clinic Workbook page 125 of 251.
IBM Software
Information Management
12 Cleanup
If you want to cleanup your work you can use these commands.
DB2 Performance Tuning and Monitoring Clinic Workbook page 126 of 251.
© Copyright IBM Corporation 2012
All Rights Reserved.
IBM Canada
8200 Warden Avenue
Markham, ON
L6G 1C7
Canada
IBM, the IBM logo, ibm.com and Tivoli are trademarks or registered
trademarks of International Business Machines Corporation in the
United States, other countries, or both. If these and other
IBM trademarked terms are marked on their first occurrence in this
information with a trademark symbol (® or ™), these symbols indicate
U.S. registered or common law trademarks owned by IBM at the time
this information was published. Such trademarks may also be
registered or common law trademarks in other countries. A current list
of IBM trademarks is available on the Web at “Copyright and
trademark information” at ibm.com/legal/copytrade.shtml
Product data has been reviewed for accuracy as of the date of initial
publication. Product data is subject to change without notice. Any
statements regarding IBM’s future direction and intent are subject to
change or withdrawal without notice, and represent goals and
objectives only.
DB2 Performance Tuning and Monitoring Clinic Workbook page 127 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 128 of 251.
IBM Software
Information Management
Table of Contents
1 Objectives.............................................................................................................................................................................................3
2 Suggested reading ..............................................................................................................................................................................3
3 DB2 Design Advisor Lab .....................................................................................................................................................................3
3.1 Start DB2.............................................................................................................................................................................................................4
3.2 Create Database and Tables ..............................................................................................................................................................................5
3.3 Examine and Execute Workload .........................................................................................................................................................................6
3.4 DB2 Design Advisor ..........................................................................................................................................................................................17
3.5 Examine and Execute Workload after recommendations .................................................................................................................................19
DB2 Performance Tuning and Monitoring Clinic Workbook page 129 of 251.
IBM Software
Information Management
1 Objectives
DB2 10 contains numerous SQL performance enhancements that continue to make the DB2 data server an industry - leading
data server solution that is suitable for any size of organization.
A number of performance improvements have been included in DB2 10 to improve the speed of many queries. These improve-
ments are automatic; there are no configuration settings or changes to the SQL statements required.
DB2 10 has up to 3x faster query performance than its predecessor, with new functionality and enhanced features like:
– Queries over star schemas
– Queries with joins and sorts
– Queries with aggregation
– Hash joins
In this lab, you will learn:
How to identify problematic SQL statements in your application
How to monitor SQL statement resource usage
How to view SQL query execution plans
Which database features help to improve the SQL statement performance
How to use the DB2 Design Advisor to gather SQL performance tuning recommendations
How to use the Optim™ Query Tuner graphic tool to obtained recommendation to improve performance based on a
workload
How to influence access plan selections to optimize query execution
2 Suggested reading
Information Management Technical Library at developerWorks
Best Practices: Tuning and Monitoring Database System Performance
http://www.ibm.com/developerworks/data/bestpractices/systemperformance/
DB2 Performance Tuning and Monitoring Clinic Workbook page 130 of 251.
IBM Software
Information Management
User: db2inst1
Password: password
2. Open a terminal window as by right-clicking on the Desktop area and choose the “Open Terminal” item.
DB2 Performance Tuning and Monitoring Clinic Workbook page 131 of 251.
IBM Software
Information Management
4. In order to enable the aliases defined for this lab and all labs that follow, copy the /home/db2inst1/labbashrc to
/home/db2inst1/.bashrc and source that file.
cd ~
cp labbashrc .bashrc
. .bashrc
day_day int
sales_fact day_of_week int
day_date date day_month int
prod_code char(6) day_month_name char(20)
store_num char(5) day_quarter char(2)
quantity integer day_quarter_name char(20)
day_year int
store_dim
store_num char(5)
state char(2)
country varchar(255)
1. While logged in as user “db2inst1”, change your working directory to the folder containing the work scripts.
qtdir
Or
cd ~/Documents/LabScripts/db2pt/querytuning
2. Before executing, take a look at the script, and note how the database, and tables are created.
cat crDbAndTabs.sql
3. Execute the script as follows:
DB2 Performance Tuning and Monitoring Clinic Workbook page 132 of 251.
IBM Software
Information Management
Note: Make sure that each SQL statement, except the first (database drop may fail because ad_whse had not been created yet), has
finished successfully. The following message should be returned after the successful completion of the SQL statement:
4. With the database created run the following two scripts to populate the database tables and update the system catalogs
with the data statistics.
Note: In DB2 10, a number of improvements have been made to the RUNSTATS command to make statistics gathering faster. One of
these improvements is that you no longer have to fully qualify object names by specifying a schema name. If you do not specify a schema
name, the default schema is used. You can view this displaying the file run_stats.sql
5. In this exercise the DB2 Design Advisor and the Visual Explain utility are used. These require the system Explain ta-
bles. Create the Explain tables by running the following commands:
midir
db2 –tvf EXPLAIN.DDL
1. Display the file batch_query.sql to view the two SQL statements. You may need to resize your terminal window to see
the first query.
qtdir
cat batch_query.sql
Leave this window open.
2. Open IBM Data Studio by double-clicking on the IBM Data Studio icon on the Desktop.
3. If this is the first time Data Studio is launched, it may take a moment. Click OK to accept the default workspace and
check the option “Use this as the default and do not ask again”.
DB2 Performance Tuning and Monitoring Clinic Workbook page 133 of 251.
IBM Software
Information Management
4. By default, the Database Administration perspective is opened. Your workspace should look like the following:
5. We want to change to the IBM Query Tuning perspective. To do this, click on the “Open Perspective” icon and click
Other, as shown below.
DB2 Performance Tuning and Monitoring Clinic Workbook page 134 of 251.
IBM Software
Information Management
Note: Perspective defines the initial set and layout of the components in your screen. Each perspective provides a set of functional-
ity aimed at accomplishing a specific type of tasks or works with specific types of resources.
7. In Data Source Explorer, expand the folder Database Connections until you see AD_WHSE.
DB2 Performance Tuning and Monitoring Clinic Workbook page 135 of 251.
IBM Software
Information Management
8. Still in Data Source Explorer, right click AD_WHSE and select Connect.
9. In the pop-up dialog, fill in the connection details as follows and Click OK:
Database: AD_WHSE
Host: localhost
Port number: 50001
User name: db2inst1
Password: password
10. After a connection has been successfully established, you should see this icon beside the AD_WHSE database in
the Data Source Explorer, which indicates a connected database. Disconnected databases are listed with . Expand-
ing the AD_WHSE database folder will list supported database object types, as seen in the following screen capture:
DB2 Performance Tuning and Monitoring Clinic Workbook page 136 of 251.
IBM Software
Information Management
11. At the top menu bar in Data Studio, select File -> Open File, then open
/home/db2inst1/Documents/LabScripts/db2pt/querytuning/batch_query.sql, which is the file you just examined in Step
1 of this section.
12. You may need to specify the database connection against which the script is going to be executed. To do that, click on
“No Connection”, select AD_WHSE, and click Finish.
DB2 Performance Tuning and Monitoring Clinic Workbook page 137 of 251.
IBM Software
Information Management
13. Right click anywhere on the query and choose ‘Open Visual Explain’, click ‘Next’ to explore the options, then choose
“Finish” to generate the explain plan and diagram. You may want to double-click the tab
to maximize the view.
Note: In DB2 10, the Control Center and related components have been discontinued. These have been replaced by a new set of GUI
tools: IBM Data Studio and IBM InfoSphere Optim tools.
You’ll see the current query access plan that includes table scan operations. Drill down into the objects for additional de-
tails. The access plan is provided below for your reference. Notice the total cost is approximately 5,000 timerons.
Cost, in the context of access plans, is the estimated total resource usage necessary to execute the access plan for a
statement (or the elements of a statement).
Cost is derived from a combination of CPU cost (in number of instructions), I/O (in number of seeks and page transfers),
and communications cost.
The unit of cost is the timeron. A timeron does not directly equate to any actual elapsed time, but gives a rough relative es-
timate of the resources (cost) required by the database manager to execute different plans for the same query.
The cost shown in each operator node of an access plan graph is the cumulative cost, from the start of access plan execu-
tion up to and including the execution of that particular operator. It does not reflect other factors, such as the workload on
the system or the cost of returning rows of data to the user.
DB2 Performance Tuning and Monitoring Clinic Workbook page 138 of 251.
IBM Software
Information Management
Digging deeper into Statement 2, the join is on four tables, SALES_FACT, STORE_DIM, PRODUCT_DIM and
DATE_DIM. Table scans (TBSCAN) are being performed on each table. Three of these are dimension tables and are
small in size.
Right click on the ‘DB2INST1.STORE_DIM’ node and select ‘Show Description’. The Statistics time indicates when
the last time statistics were updated in the system catalog for the table you are viewing. Make sure the time is current.
Scroll through the statistics look for information like the Number of columns, the Number of rows, etc. The STORE_DIM
table has just 49 rows, making indexes unhelpful. You can do the same analysis on the other dimension tables too.
DB2 Performance Tuning and Monitoring Clinic Workbook page 139 of 251.
IBM Software
Information Management
Right click the ‘DB2INST1.SALES_FACT’ node. Here you can look for ‘Number of rows’ and see that there are 707520
rows. You may think indexes will help, but look again. This warehouse statement has to read all the data in the table;
an index won’t help in this case either.
DB2 Performance Tuning and Monitoring Clinic Workbook page 140 of 251.
IBM Software
Information Management
In this scenario, DB2 optimizer first performs a Hash Join between SALES_FACT and STORE_DIM tables on column
STORE_NUM.
Right click on the HSJOIN(8) node joining the SALES_FACT and DATE_DIM tables and select ‘Show description’.
The ‘Cumulative IO cost’ information is the estimated cumulative I/O cost (in data page I/Os) of executing the chosen
access plan up to and including this operator. The ‘Cumulative CPU Cost’ is the estimated cumulative CPU cost (in
instructions) of executing the chosen access plan up to and including this operator. The ‘Cumulative Total Cost’ is the
estimated cumulative cost (in timerons) of fetching the first row for the access plan up to and including this operator and
includes any initial overhead required.
The Cumulative properties section provides information on the tables in the join, the selected columns and the join
columns.
DB2 Performance Tuning and Monitoring Clinic Workbook page 141 of 251.
IBM Software
Information Management
Hash join executes by requiring one or more predicates in the form table1.columnX = table2.columnY. In DB2 10, the
column types may be different.
First, the designated inner table, DATE_DIM is scanned and rows are copied into memory buffers that are drawn from
the sort heap. The memory buffers are divided into sections, based on a hash value that is computed on the columns of
the join predicates. If the size of the inner table exceeds the available sort heap space, buffers from selected sections
are written to temporary tables.
When the inner table has been processed, the second (or outer) table, SALES_FACT is scanned and its rows are
matched with rows from the inner table by first comparing the hash value that was computed for the columns of the join
predicates. If the hash value for the outer row column matches the hash value for the inner row column, the actual join
predicate column values are compared.
Outer table rows that correspond to portions of the table that are not written to a temporary table are matched
immediately with inner table rows in memory. If the corresponding portion of the inner table was written to a temporary
table, the outer row is also written to a temporary table. Finally, matching pairs of table portions from temporary tables
are read, the hash values of their rows are matched, and the join predicates are checked.
Note: Default value is subject to change by the DB2 Configuration Advisor after initial database creation.
For the full performance benefit of hash joins, the value of the sortheap database configuration parameter and the
sheapthres database manager configuration parameter may have to be changed. Alternatively, you can configure
sort heaps to be controlled by STMM by setting the sheapthres to 0 and the sortheap to automatic.
DB2 Performance Tuning and Monitoring Clinic Workbook page 142 of 251.
IBM Software
Information Management
Note: sheapthres default value is subject to change by Configuration Advisor after initial database creation.
Hash join performance is best if you can avoid hash loops and overflow to disk. To tune hash join performance,
estimate the maximum amount of memory that is available for sheapthres and then tune the sortheap parameter.
Increase its setting until you avoid as many hash loops and disk overflows as possible, but do not reach the limit that is
specified by the sheapthres parameter.
DB2 introduces new columns to SNAPAPPL Administrative view like DBPARTITIONNUM, TOTAL_OLAP_FUNCS,
OLAP_FUNC_OVERFLOWS and MEMBER. Consult SNAPAPPL administrative view and SNAP_GET_APPL table
function section from DB 10 Information Center for details.
Use the db2batch utility to find out how long it takes to execute this workload. The db2batch tool can read SQL
statements from either a flat file or from standard input, dynamically prepare and execute the statements, and return a
result set. It also enables you to control the number of rows that are returned to db2batch and the number of rows that
are displayed. You can specify the level of performance-related information that is returned, including elapsed time,
processor time, buffer pool usage, locking, and other statistics collected from the database monitor. If you are timing a
set of SQL statements, db2batch also summarizes the performance results and provides both arithmetic and
geometric means.
DB2 10 introduces Command parameter WFO which specifies that the db2batch operation should wait for the
outcome of an operation. For Cursor Stability and higher scans, db2batch will wait for the commit or rollback when
encountering data in the process of being updated or deleted. Rows in the process of being inserted are not skipped.
14. Run the following command in the terminal window:
This command will run the two workload statements 25 times and capture the results in the files result.out and timings
in the file summary.out.
15. View the contents of the .out files by running the commands:
cat summary.out
cat result.out
DB2 Performance Tuning and Monitoring Clinic Workbook page 143 of 251.
IBM Software
Information Management
The contents of the summary.out file are provided below. Consider this result can be a little different.
* Summary Table:
Type Number Repetitions Total Time (s) Min Time (s) Max Time (s) Arithmetic Mean
Geometric Mean Row(s) Fetched Row(s) Output
--------- ----------- ----------- -------------- -------------- -------------- ---------------
-------------- -------------- -------------
Statement 1 25 19.777257 0.382162 1.551512 0.791090
0.756770 4 4
Statement 2 25 49.666674 1.087114 3.072242 1.986667
1.905023 132 132
* Total Entries: 2
* Total Time: 69.443931 seconds
* Minimum Time: 19.777257 seconds
* Maximum Time: 49.666674 seconds
* Arithmetic Mean Time: 34.721965 seconds
* Geometric Mean Time: 31.341196 seconds
In this scenario it took about 49 seconds to execute statement 2 and about 69 seconds to execute the workload. The perform-
ance of this workload can be improved. In the next section, we’ll show you how by be using the DB2 Design Advisor. You can
leave all windows and applications open, as we’ll use them in the next section.
Given a set of SQL statements in a workload, the Design Advisor will generate recommendations for:
• New indexes
• New clustering indexes
• New Materialized Query Tables (MQTs)
• Conversion to multidimensional clustering (MDC) tables
• The redistribution of tables in a partitioned database
Note: The wizard interface for the design advisor is discontinued in DB2 10, but the design advisor is still available using the
db2advis command.
1. To gather the recommendations of the DB2 Design Advisor, run the following command:
DB2 Performance Tuning and Monitoring Clinic Workbook page 144 of 251.
IBM Software
Information Management
In this workload the first statement will be run 250 times and the second statement will run 200 times. The statements
are available in the file batch_query.sql.
The recommendations of the DB2 Design Advisor are captured in the file advise.out.
Note: Since the Query Patroller is no longer supported in Version 10, the –qp option parameter for the db2advis command has been
discontinued.
cat advise.out
For our test workload, the DB2 Design Advisor provided the following recommendations. Typically you would want to
edit this file and change the system generated names to something more meaningful.
DB2 Performance Tuning and Monitoring Clinic Workbook page 145 of 251.
IBM Software
Information Management
1. You should still have batch_query.sql open in the editor inside your Data Studio workspace. Right click anywhere
within the SQL editor, select Open Visual Explain, keep all default values, click Finish to view the updated access plan
graph. Maximize the Access Plan Diagram view if you want.
2. The access plan for the second SQL statement is provided below for your reference. In the example, the query execu-
tion cost has changed and now the execution of the query has been improved.
This will run the two workload statements 25 times and capture the results in a result_new.out and timings in sum-
mary_new.out file.
4. To compare the old and new timings, display the .out files:
cat summary.out
cat summary_new.out
Sample contents of the summary_new.out file are provided below, actual numbers may vary.
DB2 Performance Tuning and Monitoring Clinic Workbook page 146 of 251.
IBM Software
Information Management
* Summary Table:
Type Number Repetitions Total Time (s) Min Time (s) Max Time (s) Arithmetic Mean
Geometric Mean Row(s) Fetched Row(s) Output
--------- ----------- ----------- -------------- -------------- -------------- ---------------
-------------- -------------- -------------
Statement 1 25 14.132961 0.389929 0.804882 0.565318
0.553693 4 4
Stat 2 25 35.373943 0.885524 2.556759 1.414958
1.362658 132 132
* Total Entries: 2
* Total Time: 49.506904 seconds
* Minimum Time: 14.132961 seconds
* Maximum Time: 35.373943 seconds
* Arithmetic Mean Time: 24.753452 seconds
* Geometric Mean Time: 22.359306 seconds
As you can see the first statement did not benefit from the MQT or the indexes. But the second statement execution time has
substantially reduced due to the MQT. The net result is that the total workload time has been reduced due to the DB2 Design
Advisor recommendations.
You can close the IBM Data Studio but leave the command window open for the next set of exercises.
In this lab, we’ll use the MON_GET_PKG_CACHE_STMT table function interface to identify the problematic SQL statements.
MON_GET_PKG_CACHE_STMT reports information for both static and dynamic SQL statements, unlike the dynamic SQL
snapshot which only reports information for dynamic statements. We’ll use the DB2 Design Advisor to suggest indexes to
increase the performance of these SQL statements. These will be applied followed by an execution time comparison.
First you will execute a DB2 script file to create the new event monitor for locking. Open up a terminal window and change your
current directory to '/home/db2inst1/Documents/LabScripts/db2pt/querytuning':
DB2 Performance Tuning and Monitoring Clinic Workbook page 147 of 251.
IBM Software
Information Management
1. Restart the database and check if the database configuration parameter mon_act_metrics which provides metrics
about activities being performed on the database is turned on. It is on by default, with a setting of BASE. To confirm
execute:
db2stop force
db2start
qtdir
db2 connect to ad_whse
db2 get db cfg | grep –i mon
Note: Make sure that MON_ACT_METRICS=BASE. On databases created prior to V9.7 and then upgraded to V9.7 or higher, the
mon_act_metrics parameter is set to NONE by default. In version 10 the BASE setting of MON_ACT_METRICS includes metrics re-
ported through MON_GET_PKG_CACHE_STMT and MON_GET_PKG_CACHE_STMT_DETAILS .
cat badquery.sql
cat beforesummary.out
2. Let’s investigate by collecting the top ten dynamic & static SQL statements ordered by the average CPU time by using
the MON_GET_PKG_CACHE_STMT table function. It returns a point-in-time view of SQL statements in the database
package cache. This allows us to examine the aggregated metrics for a particular SQL statement, allowing us to quickly
determine the reasons for poor query performance.
Note: In DB2 10, MON_GET_PKG_CACHE_STMT returns additional columns that report metrics about I/O server efficiency, processing
time for authentication, statistics generation, statement execution, high water mark input values, and extended latch waits.
It also allows us to compare the behavior of an individual cached section, relative to the other statements, to assist in
identifying the most expensive section or statements.
DB2 Performance Tuning and Monitoring Clinic Workbook page 148 of 251.
IBM Software
Information Management
3. Let’s look at the query plan of this bad query to confirm that it maybe missing an index. You will use DB2 Explain to do
so. Run this command to get the query plan:
Operator
(ID)
RETURN
( 1)
|
HSJOIN
( 2)
/----/ \-----\
TBSCAN FETCH
( 3) ( 4)
| /--/ \
Table: IXSCAN Table:
DB2INST1 ( 5) DB2INST1
SALES_FACT | PRODUCT_DIM
Index:
DB2INST1
IDX1203262340360
We can see that the query is doing a table scan on the table SALES_FACT which has 707520 rows. An index on this
table should improve performance by reducing the I/O.
4. Now let us use the DB2 Design Advisor to get recommendations to improve this query.
DB2 Performance Tuning and Monitoring Clinic Workbook page 149 of 251.
IBM Software
Information Management
("QUANTITY" ASC, "PROD_CODE" DESC) ALLOW REVERSE SCANS COLLECT SAMPLED DETAILED STATISTICS;
COMMIT WORK ;
5. Let us create the recommended index for this query and run the statistics.
We can see the plan has improved by doing an Index scan instead of a table scan and the estimated cost of the query
is lower.
Operator
(ID)
RETURN
( 1)
|
HSJOIN
( 2)
/------/ \------\
IXSCAN FETCH
( 3) ( 4)
| /--/ \
Index: IXSCAN Table:
DB2INST1 ( 5) DB2INST1
IDX1203270014270 | PRODUCT_DIM
Index:
DB2INST1
IDX1203262340360
1. We need to create a test table for this exercise. This command will create a table with more than a million rows.
cat badquery2.sql
db2batch –d ad_whse –f badquery2.sql –r oldresults.out,oldsummary.out
This will run the bad query2 and capture the results in oldresult.out and timings in oldsummary.out.
DB2 Performance Tuning and Monitoring Clinic Workbook page 150 of 251.
IBM Software
Information Management
cat oldsummary.out
4. Let us again get the top ten dynamic & static SQL statements ordered by the average CPU time using the
MON_GET_PKG_CACHE_STMT table function. It returns a point-in-time view of SQL statements in the database
package cache. This allows us to examine the aggregated metrics for a particular SQL statement, allowing us to quickly
determine the reasons for poor query performance.
5. Let us now see the query plan of this bad query2 to confirm that it may be missing an index. Run this command to get
the query plan.
6. Now let us use the design advisor to provide recommendations to improve this query.
7. Let us create the recommended index for this query and run the statistics.
DB2 Performance Tuning and Monitoring Clinic Workbook page 151 of 251.
IBM Software
Information Management
9. Execute the bad query2 statement again using the db2batch utility.
10. Compare the OLD and NEW execution times and observe the improvement in query performance.
cat oldsummary.out
cat newsummary.out
qtdir
mon_switch_on
2. The workload for this exercise consists of five SQL statements that are continuously run against the database server.
To start the workload, in the terminal windows type the following commands:
start_batch
3. To examine the five SQL statements type the following command:
less all_query.sql
Note: You can use the arrow keys to scroll up and down in the script and type the letter “q” to exit anytime.
DB2 Performance Tuning and Monitoring Clinic Workbook page 152 of 251.
IBM Software
Information Management
use the SYSIBMADM.SNAPDYN_SQL administrative view and SNAP_GET_DYN_SQL table function, which are available
in all supported versions of DB2, to return the top 5 dynamic SQL statements with regards to total CPU consumption.
3. To display a list of statements with information about the time required to prepare the statement, we use the QUE-
RY_PREP_COST administrative view. When selecting from the view, an order by clause can be used to identify queries
with the highest prep cost. We can examine this view to see how frequently a query is run as well as the average exe-
cution time for each of these queries. If the time it takes to compile and optimize a query is almost as long as it takes for
the query to execute, you might want to change the optimization class that you are using. Lowering the optimization
class might make the query complete optimization more rapidly, returning a result sooner. However, if a query takes a
significant amount of time to prepare but is executed thousands of times (without being prepared again) then the opti-
mization class might not be an issue.
7. First stop the workload by shutting down the database server. This will throw some error messages that can be safely
ignored. Run.
db2stop force
8. Restart the database server for the next step.
db2start
DB2 Performance Tuning and Monitoring Clinic Workbook page 153 of 251.
IBM Software
Information Management
Note: in Version 10, some system catalog views and administrative SQL routines changed to include the database member infor-
mation.
cat summary_OLD.out
2. The recommendations of the DB2 Design Advisor are captured in the file all_advise.out. View the recommendations us-
ing:
cat all_advise.out
3. As part of the recommendations new MQTs have been recommended and the earlier one will be reused. Implement the
recommendations by running:
cat summary_OLD.out
cat summary_NEW.out
Compare the old and new execution times and observe the improvement in query performance.
DB2 Performance Tuning and Monitoring Clinic Workbook page 154 of 251.
IBM Software
Information Management
For SQL statements that need to be executed many times, where the only difference in the statement are the variables used as
predicates, it is often beneficial to prepare the SQL statement once and reuse the query plan by using parameter markers to
substitute the predicate values during runtime.
In this exercise you will first run two workloads, both will create 10 connections, each inserting a row into a single predefined
table. The process will repeat 10,000 times inserting a total 100,000 rows into the table. The difference between the workloads
is that one utilizes parameter markers and the other does not. Then, you will again run the workload that does not use
parameter markers, but this time with the new statement concentrator variable turned on. At the end of each run, a snapshot of
the database will be taken to compare relative Host Execution Elapsed Time and other counters.
qtdir
./mon_switch_on
2. Enter the alias to take you to the workload expert working directory, and then run the noPmarkers.scn file to have the
application insert 100,000 rows into the predefined table.
DB2 Performance Tuning and Monitoring Clinic Workbook page 155 of 251.
IBM Software
Information Management
db2 terminate
db2stop force
db2start
db2 activate database ad_whse
db2 get snapshot for database on ad_whse | grep Host
DB2 Performance Tuning and Monitoring Clinic Workbook page 156 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 157 of 251.
IBM Software
Information Management
2. Use the terminal window directory to prepare the database with script oqwt01.sh located in /home/db2inst1/Documents
/LabScripts/db2pt/querytuning/oqwt. Run the following command:
oqwtdir
./oqwt01.sh
After some time, the following should display, indicating that everything has been processed successfully:
DB2 Performance Tuning and Monitoring Clinic Workbook page 158 of 251.
IBM Software
Information Management
-----------------------------------------------------------------------------
Run EXPLAIN.DDL in the SAMPLE database
-----------------------------------------------------------------------------
CREATE FUNCTION DB2OE.QT_LIC() RETURNS VARCHAR(255) LANGUAGE SQL CONTAINS SQL NO EXTERNAL ACTION
DETERMINISTIC RETURN VARCHAR('wVLwF/a4OBBLTyNX4bsUKw==')
DB20000I The SQL command completed successfully.
DB2 Performance Tuning and Monitoring Clinic Workbook page 159 of 251.
IBM Software
Information Management
1. Open IBM Data Studio by double-clicking on the IBM Data Studio icon on the Desktop.
2. If this is the first time Data Studio is launched, it may take a moment. Click OK to accept the default workspace and
check the option “Use this as the default and do not ask again”.
DB2 Performance Tuning and Monitoring Clinic Workbook page 160 of 251.
IBM Software
Information Management
4. We want to change to the IBM Query Tuning perspective. To do this, click on the “Open Perspective” icon and click
Other, as shown below.
DB2 Performance Tuning and Monitoring Clinic Workbook page 161 of 251.
IBM Software
Information Management
1. Move your mouse pointer to the Project Explorer section, right-click and select New, and then Project…
DB2 Performance Tuning and Monitoring Clinic Workbook page 162 of 251.
IBM Software
Information Management
2. Select Query Tuner Project from the wizard list and click Next.
3. Leave the Project Name as suggested in the wizard and click Next.
DB2 Performance Tuning and Monitoring Clinic Workbook page 163 of 251.
IBM Software
Information Management
4. Select the SAMPLE database for the connection and click Next. If necessary create connection following steps 6 to 8
in Lab 3.3 of this workbook. Remember use SAMPLE as database name. Then click Finish.
5. In this lab, we will use a workload file, with multiple SQL statements. First select File and click Browse.
DB2 Performance Tuning and Monitoring Clinic Workbook page 164 of 251.
IBM Software
Information Management
7. Use “;” as statement delimiter, click Capture and, click Capture. Once the statement appears on screen, click Save
All to Workload…
DB2 Performance Tuning and Monitoring Clinic Workbook page 165 of 251.
IBM Software
Information Management
2. Check Re-collect EXPLAIN information before running workload advisors. Click Select What To Run…. Click
Select All, and click OK.
DB2 Performance Tuning and Monitoring Clinic Workbook page 166 of 251.
IBM Software
Information Management
4. There may be Unexplained Statements such as “set current schema= ‘TPCDS’”, Click Yes to continue.
DB2 Performance Tuning and Monitoring Clinic Workbook page 167 of 251.
IBM Software
Information Management
Figure 35 – Status
5. Workload advisors will start working on the workload. Please wait for this to finish.
7. After this, the recommendations are presented. We’ll review them in the next few steps.
DB2 Performance Tuning and Monitoring Clinic Workbook page 168 of 251.
IBM Software
Information Management
10. Click Finish to execute the RUNSTATS COMMANDS on the database. Click OK in the confirmation window.
DB2 Performance Tuning and Monitoring Clinic Workbook page 169 of 251.
IBM Software
Information Management
Note: In DB2 10, a number of improvements have been made to the RUNSTATS command to make statistics gathering faster. The
command parameters have also been simplified.
DB2 Performance Tuning and Monitoring Clinic Workbook page 170 of 251.
IBM Software
Information Management
1. Open a new terminal window by right-clicking on the Desktop and choosing “Open in Terminal.”:
DB2 Performance Tuning and Monitoring Clinic Workbook page 171 of 251.
IBM Software
Information Management
oqwtdir
./oqwt03.sh
3. Back in Data Studio, right-click on the QTProject1 and select Create Query Group. [We saved this project in the
previous section.]
DB2 Performance Tuning and Monitoring Clinic Workbook page 172 of 251.
IBM Software
Information Management
5. Right click on Query Group X, where X is the generated group number, and select the option Tune Query.
DB2 Performance Tuning and Monitoring Clinic Workbook page 173 of 251.
IBM Software
Information Management
6. As in the previous section we need a file with a SQL statement to optimize Click File, and then click Browse.
DB2 Performance Tuning and Monitoring Clinic Workbook page 174 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 175 of 251.
IBM Software
Information Management
8. Select workload name you just saved in the previous step, double click to open.
DB2 Performance Tuning and Monitoring Clinic Workbook page 176 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 177 of 251.
IBM Software
Information Management
4. The single query tune advisor will start to run. Please wait for this to finish.
DB2 Performance Tuning and Monitoring Clinic Workbook page 178 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 179 of 251.
IBM Software
Information Management
7. Notice in the access plan graph, there is a HASH JOIN between a full table scan on the table OQWT_T2 and
IXSCAN on the table OQWT_T1. We will take this as an example and show you how to create an optimization
profile to influence the query access path.
To set up the optimization profile, we will use the access plan graph of the query we saw in the previous section.
We will create an optimization profile using the following requirements.
1. OQWT_T1 table to be accessed with IXSCAN and it should use index OQWT_T1_IX1
2. OQWT_T2 must be a leading table in the join sequence and the join method for OQWT_T1 must be nested-loop join
and not the hash-join shown in the access plan graph
DB2 Performance Tuning and Monitoring Clinic Workbook page 180 of 251.
IBM Software
Information Management
2. Please notice that the Plan-Hint 1 is created. You will see two panes on the right-hand side. One of them is the Join
Diagram, based on existing access plan. The other is the Editable Join Sequence Diagram. We will work from the
second graph window.
DB2 Performance Tuning and Monitoring Clinic Workbook page 181 of 251.
IBM Software
Information Management
2. Click the Join Type dropdown, select NLJOIN and click OK.
3. An NLJOIN box is created in the diagram window. Click to select it. Right-click and select Add table reference as a
child node.
DB2 Performance Tuning and Monitoring Clinic Workbook page 182 of 251.
IBM Software
Information Management
5. Right-click on the NLJOIN box. Select Add table reference as a child node.
DB2 Performance Tuning and Monitoring Clinic Workbook page 183 of 251.
IBM Software
Information Management
2. Click the Value in Guideline dropdown. Select IXSCAN and click Select Index. Check DB2INST1.OQWT_T1_IX1
and click OK.
DB2 Performance Tuning and Monitoring Clinic Workbook page 184 of 251.
IBM Software
Information Management
3. You will notice the following join graph between two tables.
DB2 Performance Tuning and Monitoring Clinic Workbook page 185 of 251.
IBM Software
Information Management
3. A new window will open with two access path graphs. Compare them and notice that the new plan is using the
nested loop join. Close the window.
DB2 Performance Tuning and Monitoring Clinic Workbook page 186 of 251.
IBM Software
Information Management
3. A new window will open. Scroll to the bottom, browsing through the script generated. Click OK to deploy this
optimization profile to the database.
DB2 Performance Tuning and Monitoring Clinic Workbook page 187 of 251.
IBM Software
Information Management
Figure 75 – Deploy
qtdir
. cleanup.sh
DB2 Performance Tuning and Monitoring Clinic Workbook page 188 of 251.
IBM Software
Information Management
IBM Canada
8200 Warden Avenue
Markham, ON
L6G 1C7
Canada
IBM, the IBM logo, ibm.com and Tivoli are trademarks or registered
trademarks of International Business Machines Corporation in the
United States, other countries, or both. If these and other
IBM trademarked terms are marked on their first occurrence in this
information with a trademark symbol (® or ™), these symbols indicate
U.S. registered or common law trademarks owned by IBM at the time
this information was published. Such trademarks may also be
registered or common law trademarks in other countries. A current list
of IBM trademarks is available on the Web at “Copyright and
trademark information” at ibm.com/legal/copytrade.shtml
Product data has been reviewed for accuracy as of the date of initial
publication. Product data is subject to change without notice. Any
statements regarding IBM’s future direction and intent are subject to
change or withdrawal without notice, and represent goals and
objectives only.
DB2 Performance Tuning and Monitoring Clinic Workbook page 189 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 190 of 251.
IBM Software
Information Management
Table of Contents
1 Objectives.............................................................................................................................................................................................3
2 Lab Setup..............................................................................................................................................................................................3
2.1 Login to the Virtual Machine................................................................................................................................................................................3
2.2 Create the SAMPLE Database ...........................................................................................................................................................................4
2.3 Modifying Lab Environment.................................................................................................................................................................................4
4 Event Monitor for Locking – Deadlock detection and formatting to relational tables..................................................................9
4.1 Create Event Monitor ........................................................................................................................................................................................10
4.2 Simulating deadlocks ........................................................................................................................................................................................10
6 Summary.............................................................................................................................................................................................17
7 Scripts provided for this lab .............................................................................................................................................................18
7.1 snapdblocks.sql.................................................................................................................................................................................................18
7.2 lockwaits.sql ......................................................................................................................................................................................................18
7.3 createEventMonitorLocking.db2........................................................................................................................................................................18
7.4 call_EF.sh .........................................................................................................................................................................................................18
DB2 Performance Tuning and Monitoring Clinic Workbook page 191 of 251.
IBM Software
Information Management
1 Objectives
After completion of this lab, the student should be able to:
Use snapshots, event monitors, administrative views and db2pd to identify locking issues through the reporting
provided by key monitoring elements.
Be able to identify the type of lock event causing a performance slowdown, identify the SQL statement(s) involved and
the locks being requested and encountered by applications.
2 Lab Setup
2.1 Login to the Virtual Machine
User: db2inst1
Password: password
2. Open a terminal window as by right-clicking on the Desktop area and choose the “Open Terminal” item.
DB2 Performance Tuning and Monitoring Clinic Workbook page 192 of 251.
IBM Software
Information Management
db2sampl –force
The database used in this lab will need to have CUR_COMMIT turned off. CUR_COMMIT provides significant relief from locking
issues in a variety of scenarios.
Newly created databases will have the feature turned on by default. Any database updated from V9.5 or earlier will not have
CUR_COMMIT enabled.
db2stop force
db2start
DB2 Performance Tuning and Monitoring Clinic Workbook page 193 of 251.
IBM Software
Information Management
Wait for the scenario to give the indication that it has started and is actually running. Do not close this window. You can
minimize it since it will not be used until the end of the exercise.
After the scenario has been running for about 20 seconds, you should have collected enough data in the lock event monitor that
you started in section 3.1. To avoid having a large amount of data to browse through in the output file, you can turn the event
monitor off by setting it to 0. (Do NOT stop the Workload Expert scenario running in the first terminal window. You will need it
running for the next several exercises). Open a second terminal window, connect to the database, and enter:
A database snapshot is a good place to start when investigating locking issues. The snapshot administrative view
SYSIBMADM.SNAPDB allows database level monitor elements to be returned via SQL SELECT statements. The use of queries
to retrieve snapshot monitor data allows filtering to focus on specific monitor data.
In the second terminal window, connect to the database and execute the following:
This query will return database level monitoring elements for each of the locking scenarios; lock waits, deadlocks, lock
escalations and lock timeouts. It is easy to see that LOCK WAITS are occurring. Why?
This can be explained by reviewing the database configuration.
DB2 Performance Tuning and Monitoring Clinic Workbook page 194 of 251.
IBM Software
Information Management
The DB CFG parameter LOCKTIMEOUT has not been changed from the default value of -1.
When LOCKTIMEOUT is set to -1 all LOCK WAIT conditions will wait indefinitely for the encountered lock to be released. Thus all
queries will become suspended in a lock-wait state until the lock encountered is released. If you plan on setting this parameter
to a specific time period, make sure that the application is prepared accordingly. DB2 will return an error code to the application
(SQL0911). From the command line interface it looks like this:
DB21034E The command was processed as an SQL statement because it was not a valid Command
Line Processor command. During SQL processing it returned:
SQL0911N The current transaction has been rolled back because of a deadlock or timeout.
Reason code "68". SQLSTATE=40001
Sample Output:
db2inst1@db2v10:~/Documents/LabScripts/db2pt/Locking> db2 "select locks_held, lock_waits from
sysibmadm.snapdb"
LOCKS_HELD LOCK_WAITS
-------------------- --------------------
13 14
If the number reported under LOCK_WAITS is greater than zero, then that is the number of times that applications or
connections waited for locks.
To narrow the scope, and filter database locks to only the applications in a lock wait status, use the Snapshot administrative
view SYSIBMADM.MON_LOCKWAITS:
DB2 Performance Tuning and Monitoring Clinic Workbook page 195 of 251.
IBM Software
Information Management
Some administrative views like SYSIBMADM.LOCKWAITS have been deprecated in DB2 10, so it is necessary to use new
administrative views like SYSIBMADM.MON_LOCKWAITS. You can review the following link:
http://publib.boulder.ibm.com/infocenter/db2luw/v10r1/topic/com.ibm.db2.luw.sql.rtn.doc/doc/r0023171.html
There are two applications waiting. Both have encountered an exclusive lock (X) placed by a third application. One is
attempting to acquire a X-lock, which would be expected when issuing an INSERT, UPDATE or DELETE; the other is
requesting a share (NS) which reflects a SELECT. MOVE THIS ENTIRE PARAGRAPH TO BELOW THE GREEN NOTE
ABOVE.
To obtain the locking event report, use the db2evmonfmt tool. This tool is included with DB2 but is not compiled. You will find the
source in the Instance’s home directory under samples/java/jdbc.
For more details refer to the “db2evmonfmt tool for reading event monitor data” topic in Information Center v10.
For this lab, the utility has already been compiled and is in the Locking directory. Redirect the output to a file name to ease
review using a text editor.
DB2 Performance Tuning and Monitoring Clinic Workbook page 196 of 251.
IBM Software
Information Management
Note: You can find the db2evmonfmt.out output file in ~/sqllib/java/jdk32/jre/bin/java directory.
Switch to the terminal running Workload Expert and enter quit to stop the process:
quit
As good practice, so that the lab can be rerun, turn off and drop the event monitor lockevmon, and drop the lock event table
lockwaits:
In order to continue with the next exercise is necessary to update the DB CFG parameter MON_LOCKWAIT with its previous
value:
DB2 Performance Tuning and Monitoring Clinic Workbook page 197 of 251.
IBM Software
Information Management
Now if you want to monitor deadlock events in DB2 10, using the CREATE EVENT MONITOR FOR LOCKING statement is the
recommended method. This event monitor not only monitors for deadlocks but also lock timeouts and excessive lock wait
occurrences.
In the previous exercise, you used this same approach (event monitor for locking) when monitoring lockwaits, now we will
focus on deadlocks. Previously we used the db2evmonfmt tool to generate formatted output and details about the
lockwaits, in this exercise you will use a system based stored procedure to store this information in a set of relational tables
then query these tables.
1. Create the event monitor for locking. Output for this monitor is generated in the unformatted event (UE) table structure.
The create statement can specify placement of the UE table in a separate tablespace, a preferable technique to use
when you want to reduce I/O contention.
2. Generate transactions that will create deadlocks. The deadlock events will be written out to the UE tables
3. Format UE table data into relational table structure using a stored procedure developed expressly for this purpose.
evmon_format_ue_to_tables
('LOCKING', NULL, NULL,
Deadlocked Event Monitor NULL, NULL, NULL, call_EF.sh
Applications “Mylocks” 'RECREATE_FORCE', -1,
for Locking 'SELECT * FROM MYLOCKS
ORDER BY EVENT_TIMESTAMP')"
SQL Analysis
MYLOCKS
(Unformatted
Event Table) “LOCK”
Event relational
tables
DB2 Performance Tuning and Monitoring Clinic Workbook page 198 of 251.
IBM Software
Information Management
First you will execute a DB2 script file to create the event monitor for locking. Open up a terminal window and change your
current directory to:
Before executing, take a look at the script; first the old event monitor is disabled, then an event monitor for locking is created
with specific designations for the table name (mylocks) and table space location (userspace1) before the new monitor is
turned on.
less createEventMonitorLocking.db2
Note: You can use the arrow keys to scroll up and down in the script and type the letter “q” to exit anytime.
Note: Ignore the SQL0204N errors; they are a result of DROP statements which precede the CREATE and SET statements in the
script so you can repeat execution.
In a second terminal window, launch a workload simulation that will accumulate deadlocks. Locking information will be collected
in the unformatted event tables (UE).
DB2 Performance Tuning and Monitoring Clinic Workbook page 199 of 251.
IBM Software
Information Management
Let this scenario run for 15-20 seconds before entering “quit” to return to the operating system prompt:
quit
Now check that deadlocks were generated and captured into the mylocks table (defined in create event monitor statement).
You should see a count of some rows in the mylocks table. (If you don’t, try re-executing the deadlock scenario).
Since the mylocks table is unformatted, you will execute a stored procedure to format the table into relational tables.
Use the call_EF.sh script to invoke EVMON_FORMAT_UE_TO_TABLES stored procedure. Three tables will be created to store
the unformatted data in a relational format: LOCK_EVENT, LOCK_PARTICIPANTS and LOCK_PARTICIPANT_ACTIVITIES.
./call_EF.sh
code in call_EF.sh :
DB2 Performance Tuning and Monitoring Clinic Workbook page 200 of 251.
IBM Software
Information Management
The last step is to query the newly created (and populated) relational tables. You can use a provided script that will pick out the
first ID (xmlid) that is returned from the LOCK_EVENT table and query the other tables based on that ID.
The query’s result displays the participants in a deadlock event, the application(s) that were involved, the SQL code, and
isolation levels used. This data provides a “good start” to resolving a deadlock issue.
Remember to quit the simulation workload by entering “quit” to return to the operating system prompt:
quit
DB2 Performance Tuning and Monitoring Clinic Workbook page 201 of 251.
IBM Software
Information Management
Under the Currently Committed isolation level, only committed data is returned, as was the case previously, but now readers do
not wait for updaters to release row locks. Instead, readers return data that is based on the currently committed version; that is,
data prior to the start of the write operation.
In DB2 10, Currently Committed is turned on by default for new databases. This allows any application to take advantage of the
new behavior without any modification. The database configuration parameter cur_commit can be used to override this
behavior. This might be useful, for example, in the case of applications that require the pre-9.7 behavior of blocking on writers
to synchronize internal logic. Upgraded databases have cur_commit disabled by default.
Currently Committed semantics apply only to read-only scans that do not involve catalog tables or the internal scans that are
used to evaluate constraints.
Note that, because Currently Committed is decided at the scan level, a writer's access plan might include Currently Committed
scans. For example, the scan for a read-only sub-query can involve currently committed semantics
Log data is required for retrieving the currently committed image of the row. Depending on the workload, this can have an
insignificant or substantial impact on the total log space and log buffer space required. The requirement for additional log space
does not apply when cur_commit is disabled.
Consider the following scenario, in which deadlocks are avoided under the Currently Committed isolation level. In this scenario,
two applications update two separate tables, but do not commit. Each application then attempts to read (with a read-only cursor)
from the table that the other application has updated.
Without Currently Committed, transactions running under the Cursor Stability isolation level might create a deadlock, causing
one of the transactions to fail. This happens when each transaction needs to read data that is being updated by the other.
Using Currently Committed, if the query (of either application) happens to require the data currently being updated by the other
transaction, that transaction does not wait for the lock to be released, avoiding a deadlock since the previously committed
version of the data is located and used.
DB2 Performance Tuning and Monitoring Clinic Workbook page 202 of 251.
IBM Software
Information Management
In this lab, we’ll demonstrate the effect of the Currently Committed feature. To do so, we will create a scenario where a potential
read / write block can happen when 2 queries are running concurrently. We will then compare the difference in results and
execution time as we toggle the parameter CUR_COMMIT.
We will use command line processor (CLP) to simulate the applications accessing the database at the same time.
First, let’s examine the existing setting for currently committed. From within a Terminal window, type in the following:
The CUR_COMMIT parameter is located near the end of the list. It should display as DISABLED, since we turned it off earlier in
the exercise. If it is ON, disable it by issuing the following command:
After changing the value, we need to disconnect all connections, thus deactivating the database and allowing for the new value
to take effect upon the next activation or connect.
In order to mimic the behavior of a long running transaction, we first need to disable CLP’s auto-commit feature, which is ON by
default. When auto-commit is active, CLP automatically issues a COMMIT after every executed SQL statement. Therefore, we
need to disable it so we are able to specify when the transaction will be committed. In terminal A, enter the CLP prompt by
typing the command below. The “+c” option will disable the auto-commit feature for this session.
export DB2OPTIONS="+c"
You can check that the auto-commit feature is off by executing the command below:
DB2 Performance Tuning and Monitoring Clinic Workbook page 203 of 251.
IBM Software
Information Management
Since auto-commit is OFF, from now on all SQL statements that you execute will be part of the same transaction until you issue
a “commit” or “rollback”. Now, connect to the database:
We will then execute an update query which will put a lock on the rows for as long as the transaction is not committed.
Next, we will launch a query that will read the data locked by Terminal A.
The time command will allow us to quantify the wait time. We can see that the query waits and does not return any result. In
fact, it is waiting on the locks of the first terminal window’s UPDATE statement to be released.
DB2 Performance Tuning and Monitoring Clinic Workbook page 204 of 251.
IBM Software
Information Management
With the two terminal windows open beside each other, we will observe the effect of committing the UPDATE of the first terminal
window. In Terminal A, commit the transaction by executing the following command:
db2 commit
You can see the second terminal windows query is instantly returned with the updated values. The locks placed by the first
terminal window’s update have been released with the issuance of the commit statement.
After changing the value, we need to disconnect all connections thus deactivating the database allowing for the new value to
take effect upon the next activation.
In both terminal windows issue a connect reset: In the first terminal window:
DB2 Performance Tuning and Monitoring Clinic Workbook page 205 of 251.
IBM Software
Information Management
In the second terminal window, reconnect to the database and try to retrieve the values from the table:
Notice the amount of time the query took to return this time. The query returned instantly because there was no access block to
the data. Also, notice the values returned were not from the most recent update since the update has not yet been committed,
but from the last committed update.
db2 commit
Switch the focus back to Terminal B. We want to execute the selection query again by pressing the up arrow button once to
retrieve the last executed command, and then press Enter. If you cannot find the last command, type in
Notice the values returned this time reflects our last update since the transaction in terminal A has ended and the updates
committed to the database.
To finalize the lab, enable auto-commit so that CLP will automatically issues a COMMIT after every executed SQL statement. Do
that by typing the command below. The “-c” option will enable the auto-commit feature for this session.
export DB2OPTIONS="-c"
You can check that the auto-commit feature is off by executing the command below:
6 Summary
You can now identify and diagnose locking issues that can affect the performance of your databases and applications.
Now you have some tips which you can apply in your environment in order to design better databases to avoid locking issues.
Now you can use DB2 tools such as event monitors and administrative views to monitor your databases.
DB2 Performance Tuning and Monitoring Clinic Workbook page 206 of 251.
IBM Software
Information Management
7.1 snapdblocks.sql
#------------------------------------------------------------------------------------------
SELECT LOCK_WAITS, DEADLOCKS, LOCK_ESCALS, LOCK_TIMEOUTS FROM SYSIBMADM.SNAPDB;
7.2 lockwaits.sql
SELECT SUBSTR(TABSCHEMA,1,8) AS TABSCHEMA,
SUBSTR(TABNAME,1,15) AS TABNAME,
REQ_AGENT_TID,
LOCK_OBJECT_TYPE, LOCK_MODE,
LOCK_MODE_REQUESTED,
HLD_APPLICATION_HANDLE
FROM SYSIBMADM.MON_LOCKWAITS;
7.3 createEventMonitorLocking.db2
-- This will shutdown the old default deadlock monitor
set event monitor db2detaildeadlock state 0;
-- This will shutdown mylocks so that lab can be rerun
set event monitor mylocks state 0;
-- Clean up just in case ok if this statement fails
drop event monitor mylocks;
drop table mylocks;
-- Create new event monitor for locking
create event monitor mylocks for locking write to unformatted event
table(table mylocks in userspace1);
set event monitor mylocks state 1 ;;
7.4 call_EF.sh
#------------------------------------------------------------------------------------------
db2 "connect to db2pt97"
db2 "call evmon_format_ue_to_tables('LOCKING', NULL, NULL, NULL, NULL, NULL,
'RECREATE_FORCE', -1, 'SELECT * FROM MYLOCKS ORDER BY EVENT_TIMESTAMP')"
exit
DB2 Performance Tuning and Monitoring Clinic Workbook page 207 of 251.
IBM Software
Information Management
IBM Canada
8200 Warden Avenue
Markham, ON
L6G 1C7
Canada
IBM, the IBM logo, ibm.com and Tivoli are trademarks or registered
trademarks of International Business Machines Corporation in the
United States, other countries, or both. If these and other
IBM trademarked terms are marked on their first occurrence in this
information with a trademark symbol (® or ™), these symbols indicate
U.S. registered or common law trademarks owned by IBM at the time
this information was published. Such trademarks may also be
registered or common law trademarks in other countries. A current list
of IBM trademarks is available on the Web at “Copyright and
trademark information” at ibm.com/legal/copytrade.shtml
Product data has been reviewed for accuracy as of the date of initial
publication. Product data is subject to change without notice. Any
statements regarding IBM’s future direction and intent are subject to
change or withdrawal without notice, and represent goals and
objectives only.
DB2 Performance Tuning and Monitoring Clinic Workbook page 208 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 209 of 251.
IBM Software
Information Management
Table of Contents
6 Summary ............................................................................................................................................................................................12
7 Cleanup...............................................................................................................................................................................................12
DB2 Performance Tuning and Monitoring Clinic Workbook page 210 of 251.
IBM Software
Information Management
This lab will utilize the SAMPLE database. To create (or re-create) the database, enter the following command in a terminal
window:
DB2 Performance Tuning and Monitoring Clinic Workbook page 211 of 251.
IBM Software
Information Management
Forward Recovery must be enabled for the database. Forward Recovery is placed into effect by setting either of the two DB
CFG parameters LOGARCHMETH1 or LOGARCHMETH2 and then performing an offline backup of the database.
Before continuing, verify that Archive Logging is enabled by taking a second database backup, but this time includes the
ONLINE option. Only when Archive Logging is enabled can an online backup be performed.
DB2 Performance Tuning and Monitoring Clinic Workbook page 212 of 251.
IBM Software
Information Management
With Archive Logging enabled, the Online Backup should have truncated and archived the current transaction log. In the
terminal session, issue the command Verify the path has been changed. Issue this command:
----------------------------------------------------------------------------
Comment:
Start Time: 20120402194209
End Time: 20120402194643
Status: A
----------------------------------------------------------------------------
EID: 3 Location:
/home/db2inst1/archive/db2inst1/SAMPLE/NODE0000/LOGSTREAM0000/C0000000/S0000000.LOG
DB2 Performance Tuning and Monitoring Clinic Workbook page 213 of 251.
IBM Software
Information Management
LOGARCHCOMPR1: This parameter specifies whether the log files written to the primary archive destination for logs are
compressed.
iostat –t 3
DB2 Performance Tuning and Monitoring Clinic Workbook page 214 of 251.
IBM Software
Information Management
The iostat will repeatedly display every 3 seconds until you enter Control-C to terminate. You will see that most of the I/O
activity is occurring on the /sda and /sdc file systems.
Enter Control-C to terminate iostat then issue the following command to see the mount point of /sda:
Let’s query the database to see the location of the table spaces. You will notice they share the same device as the transaction
logs. In the terminal session, issue the following commands:
DB2 Performance Tuning and Monitoring Clinic Workbook page 215 of 251.
IBM Software
Information Management
Note the location of the transaction logs. By default, they are created on the same file system (and physical device) identified by
the database path (DBPATH). In this case, that is the same path as the database’s table space containers.
quit
db2 terminate
This will truncate and archive the current log file as well as remove any secondary log files. Once reactivated the database will
create log files in the new location.
DB2 Performance Tuning and Monitoring Clinic Workbook page 216 of 251.
IBM Software
Information Management
After database activation, the log files now reside on the /logs file system located on /dev/sdf1. Restart the workload simulation
and observe the change using the iostat utility. Now, database disk activity will be observed on both sda, sda2 and sdb, sdb1
devices.
iostat –t 3
Notice how significant I/O activity is on the transaction log files. Changing the log path to a separate local high-speed disk, which
is not in use for anything other than logging, can significantly improve performance. Enter Control-c to stop the IO stat process
and quit to end the Workload Expert scenario.
DB2 Performance Tuning and Monitoring Clinic Workbook page 217 of 251.
IBM Software
Information Management
Within a short period of time, SQL process will be aborted due to a log-full condition:
DB2 Performance Tuning and Monitoring Clinic Workbook page 218 of 251.
IBM Software
Information Management
The available log space, i.e. (LOGPRIMARY + LOGSECOND) * LOGFILSIZ, is not enough to keep up with the number of
transactions being processed.
Let’s establish a more reasonable value to LOGFILSZ parameter, and add more primary log and secondary log files. In the
terminal session, issue the command:
The increased capacity for transaction log records will now permit the SIMPLESQL scenario to execute until the script is
terminated.
DB2 Performance Tuning and Monitoring Clinic Workbook page 219 of 251.
IBM Software
Information Management
Restart the scenario, this time it will be necessary for you to enter “quit” to abort the script’s execution.
6 Summary
You can now identify and diagnose performance issues relating to the database’s transaction logs.
Now you be able to remedy the common issues relating to the database’s transaction logs which contribute performance
degradation
7 Cleanup
If you want to cleanup your work you can use these commands.
DB2 Performance Tuning and Monitoring Clinic Workbook page 220 of 251.
© Copyright IBM Corporation 2012
All Rights Reserved.
IBM Canada
8200 Warden Avenue
Markham, ON
L6G 1C7
Canada
IBM, the IBM logo, ibm.com and Tivoli are trademarks or registered
trademarks of International Business Machines Corporation in the
United States, other countries, or both. If these and other
IBM trademarked terms are marked on their first occurrence in this
information with a trademark symbol (® or ™), these symbols indicate
U.S. registered or common law trademarks owned by IBM at the time
this information was published. Such trademarks may also be
registered or common law trademarks in other countries. A current list
of IBM trademarks is available on the Web at “Copyright and
trademark information” at ibm.com/legal/copytrade.shtml
Product data has been reviewed for accuracy as of the date of initial
publication. Product data is subject to change without notice. Any
statements regarding IBM’s future direction and intent are subject to
change or withdrawal without notice, and represent goals and
objectives only.
DB2 Performance Tuning and Monitoring Clinic Workbook page 221 of 251.
IBM Software
Information Management
DB2 Performance Tuning and Monitoring Clinic Workbook page 222 of 251.
IBM Software
Information Management
Table of Contents
1 Introduction ..........................................................................................................................................................................................3
2 Suggested Reading .............................................................................................................................................................................3
3 Lab Setup..............................................................................................................................................................................................3
4 Create Database...................................................................................................................................................................................4
4.1 Create Database .................................................................................................................................................................................................4
4.1.1 Activate the Database.........................................................................................................................................................................5
4.2 REORGCHK, REORG and RUNSTATS.............................................................................................................................................................6
4.2.1 REORGCHK .......................................................................................................................................................................................6
4.2.2 REORG ...............................................................................................................................................................................................8
4.2.3 RUNSTATS.........................................................................................................................................................................................8
DB2 Performance Tuning and Monitoring Clinic Workbook page 223 of 251.
IBM Software
Information Management
1 Introduction
DB2 10 contains innovative features for delivering information on demand and scaling databases to new levels.
• Automatic tuning DB2 performs for you when you use utilities
• Manual tuning considerations and options for utilities
• Monitoring the utilities DB2 provided for maintaining the integrity of your data
• Features and facilities available for configuring and monitoring the performance of DB2 and your applications
Performance and security are often left out of initial design considerations for new or migrated databases. We will address these
as we proceed through the workshop.
Some of the information in this document has been taken directly from the DB2 Version 10 Information Center. The information
may or may not have been modified for the purposes of this document.
2 Suggested Reading
Information Management Technical Library at developerWorks
Contains articles and tutorials
http://www.ibm.com/developerworks/views/data/libraryview.jsp
3 Lab Setup
Power on the virtual machine. Login as db2inst1. The password is password. Many of the commands you use to configure
performance for DB2 can be quite long and complicated. We've put these commands into text files so you don't have to do as
much typing in this lab. Please review each shell script or SQL file before you use it. To make it easier to review the code you'll
be executing throughout the lab, open of the files with SQL extension in an editor. Write permission for these files has been
removed so you won't damage them while they're open in the editor. Alternatively, refer to the end of this document for the
source code.
DB2 Performance Clinic Lab
© Copyright IBM Corp. 2012. All rights reserved Page 3 of 30
DB2 Performance Tuning and Monitoring Clinic Workbook page 224 of 251.
IBM Software
Information Management
/home/db2inst1/Documents/LabScripts/db2pt/utilities
• Select all the files with SQL extension in the directory and click the Open pushbutton.
4 Create Database
Lab Tip: Use the up arrow on your keyboard to recall commands.
In the previous exercise we placed our database logs in directory /logs. This directory will be reused in this exercise. Therefore,
remove any old log files in this directory:
1. Open a terminal window as by right-clicking on the Desktop area and choose the “Open Terminal” item.
DB2 Performance Tuning and Monitoring Clinic Workbook page 225 of 251.
IBM Software
Information Management
rm –rf /logs/*
We provide a script that contains the commands to create the database by restoring a backup of the xmldb database. Open a
terminal session and enter the following commands to restore the database and the database objects needed for the lab
exercises.
cd ~/Documents/LabScripts/db2pt/utilities
db2start
db2 -tvf restoreXMLdb.sql
Deployment consideration: Database buffer pools, agents and other objects are not allocated in memory until the database is
activated or an application connects to the database.
If a database has not been activated, the first application to connect to the database will incur a performance penalty as it waits
for the database objects to be allocated and instantiated. In addition, when the last application disconnects from the database,
the database objects are de-allocated. This means the next application to connect to the database will have to wait until the
database objects are allocated. Another reason to activate the database is for monitoring purposes; performance metrics are
typically accumulated upon either activation or first connection to the database.
You can remove this performance inhibitor by activating the database when the system starts. Database objects will remain
allocated until the deactivate database command is issued. This will also give DB2's autonomic computing features the
best opportunity to provide their benefits. Refer to the activate database command for more information on this subject
db2stop
db2start
db2 activate db xmldb
db2 connect to xmldb
DB2 Performance Tuning and Monitoring Clinic Workbook page 226 of 251.
IBM Software
Information Management
4.2.1 REORGCHK
The REORGCHK command calculates statistics on the database to determine if tables, indexes or both need to be reorganized
or cleaned up.
We will run the reorgchk command twice. The first time we will use the current statistics for the database, and then we will
supply an option that will update the statistics and then check if a reorg is needed. Enter the following command on a single
line:
DB2 Performance Tuning and Monitoring Clinic Workbook page 227 of 251.
IBM Software
Information Management
You can review the reorgchk command's results by opening the reorgchk_1.txt by displaying its contents with the cat, more or
less commands. Results during lab development showed that table did not require a reorg (F1 F2 F3 REORG results were ---),
but that an index reorg was recommended (F4 F5 F6 F7 F8 REORG results were *----).
cat reorgchk_1.txt
The UPDATE STATISTICS parameter calls the RUNSTATS routine to update table and index statistics, and then uses the
updated statistics to determine if table or index reorganization is required. Run the command to update the statistics and review
the results. Enter the following command on a single line:
Results during lab development showed similar results for both commands, meaning that the statistics for this table were
already up-to-date. This is a small table with relatively simple statistics. Results for production databases may show that you
should use the UPDATE STATISTICS option for more accurate results, provided that you have a large enough window.
(RUNSTATS may be costly in terms of lapsed time and catalog locking.) Alternatively, you can enable automatic RUNSTATS, so
that DB2 will gather statistics in the background as needed.
DB2 Performance Tuning and Monitoring Clinic Workbook page 228 of 251.
IBM Software
Information Management
4.2.2 REORG
The reorg command is used to reorganize (defragment) an index or a table. Running a reorg for indexes rebuilds the index
data into unfragmented, physically contiguous pages. Running a reorg for a table reconstructs, or rebuilds the rows, compacting
the table by eliminating fragmented data. In addition, you can rebuild the data compression dictionary and recompress the data
accordingly.
On a partitioned table, you can reorganize a single partition. You can reorganize a specific nonpartitioned index, or you can
reorganize all the partitioned indexes on a specific data partition.
If you specify the CLEANUP option of the index clause, cleanup of logically deleted indexes is performed without rebuilding the
indexes. This command cannot be used against indexes on declared temporary tables or created temporary tables (SQLSTATE
42995).
Enter the following commands to reorg the indexes and table data and create a new compression dictionary.
db2 reorg indexes all for table db2inst1.xmlfiles
db2 reorg table db2inst1.xmlfiles resetdictionary
4.2.3 RUNSTATS
There are two essential times you should execute the runstats utility against a table: after the table has had a lot of updates
or after you reorganize the table. You should execute the runstats utility against statistical views that have been enabled for
optimization when modifications to their underlying tables affect the rows returned by the views.
You do not need any explicit privilege to use this command on any declared temporary table that exists within its connection.
We want to update the statistics for the database because we have just run a reorg operation. We can base the statistics on a
sample, or percentage of the data in the table. DB2's runstats command offers two types of sampling: Bernoulli and system.
Bernoulli sampling evaluates a sample set of rows. System sampling evaluates a sample set of data pages. System sampling
introduces less overhead than Bernoulli sampling but may not produce as accurate results when your data is highly clustered.
In this example we specify the util_impact_priority parameter with its lowest value so it will run at a very low priority. We
specify “allow read access” so other applications will be able to access the data while the command runs. We specify 100 for
both the sampling method and the repeatable parameter so that every row (100 percent of the rows) will be analyzed. You will
probably want to use a much lower value for larger tables. You may need to compare results with no sampling to find the lowest
percentage that will yield “good” statistics. This can become critical to performance when hundreds of millions of rows exist.
Enter the following command to capture the new statistics for our database.
DB2 Performance Tuning and Monitoring Clinic Workbook page 229 of 251.
IBM Software
Information Management
The first reorg command we ran in the last section was for the indexes. Review that reorg command's results by running the
reogrchk command again, as shown below. Enter the following command on a single line:
Compare files reorgchk_2.txt and reorgchk_3.txt. There should no longer be any indexes needing a reorg
The second reorg command we ran in the last section reorganized the data, creating a new compression dictionary. Enter the
following command to display the current compression statistics. Record the results in to table below.
DB2 Performance Tuning and Monitoring Clinic Workbook page 230 of 251.
IBM Software
Information Management
The throttling system ensures that the throttled utilities are run as frequently as possible when necessary without violating the
impact policy. You can throttle statistics collection, backup operations, rebalancing operations, and asynchronous index
cleanups.
First, you define the impact policy by setting the UTIL_IMPACT_LIM configuration parameter.
Check the current setting for the impact policy with the following command.
Figure 6 – UTIL_IMPACT_LIM
DB2 Performance Tuning and Monitoring Clinic Workbook page 231 of 251.
IBM Software
Information Management
The value of the util_impact_lim parameter determines what impact all running utilities can have on the total database
workload. For example, if the value of this parameter is 10, then all running utilities together can have no more than a 10%
average impact upon the total workload for the database. If the value is set to 100, then no throttling will occur.
Refining this a bit further, you can manage the priority of individual utilities within the average impact allotment. For example,
you could give more resources to RUNSTATS and fewer to BACKUP, or vice versa. You use the UTIL_IMPACT_PRIORITY
clause of the RUNSTATS or BACKUP command to set the relative priority of an individual utility within the UTIL_IMPACT_LIM
percentage. If you omit the UTIL_IMPACT_PRIORITY clause, the utility runs unthrottled, regardless of the setting of
UTIL_IMPACT_LIM.
You may also use the SET UTIL_IMPACT_PRIORITY command or the db2UtilityControl API to enable throttling, or to set or
change the relative priority of an individual utility. By default, within the impact allotment, each utility has a utility impact priority
of 50 (acceptable values are between 1 and 100, with 0 indicating no throttling).
5.2 Backup
DB2 allows you to backup an entire database or selected tablespaces. You can perform a hot (online) or cold (offline) backup
depending on how the transaction logging is configured.
Data, backups, active transaction logs and archived transaction logs should be stored on separate physical devices for two
reasons:
• First, if you lose the active database you don't lose the backup and transaction logs, too. If you did there would be no
way to recover or restore the database.
• Second, disk I/O will not be a bottleneck as you read from the database and write to the backup and/or transaction logs.
We have configured separate drives for active transaction logs (/logs), archived transaction logs (/archive), and database
backups (/backups).
Let’s suppose that the Service Level Agreement (SLA) states that the database should remain online at all times except during
the quarterly maintenance periods. To reduce the amount of time it would take to recover a database after a disaster you should
perform a full database offline backup before putting the database online. Then you could take another full offline backup
during each maintenance period.
Before performing a backup operation we need to change transaction logging from circular mode to archival mode. This will
allow you to take an online backup of the database. In order to do this we need to set the logarchmeth1 parameter.
db2 connect to xmldb
db2 update db cfg using logarchmeth1 disk:/archive
We have a disk drive set aside for transaction logs. Specify the new transaction log setting with the following command.
DB2 Performance Tuning and Monitoring Clinic Workbook page 232 of 251.
IBM Software
Information Management
The messages issued when you executed the previous commands indicate that the changes have not yet taken effect. We need
to terminate all connections to the database to activate the changes. To be certain, and since we aren't in production mode,
we'll also restart DB2. Enter the following commands to activate the logging changes.
Note: We cannot activate the database, nor can applications connect to the database, which is now in a BACKUP PENDING state.
A message to this effect is returned by the activate command and is also placed in the DB2 diagnostics log. By default, this log can be
found at ~/sqllib/db2dump/db2diag.log
Start a new terminal session or terminal session tab. We'll use it to monitor utilities as they run. Start a WHILE loop in the new
session. Press Enter after each of the following lines. This loop will return message SQL1611W when no utilities are running.
while true
do
db2 list utilities
sleep 5
done
DB2 Performance Tuning and Monitoring Clinic Workbook page 233 of 251.
IBM Software
Information Management
DB2 automatically calculates the numbers of buffers it needs for a backup and the parallelism to be used to read tablespaces
concurrently. DB2 records the actions it takes to optimize backup performance using its autonomic computing features in the
db2diag.log. Enter the following command to backup the database, and then switch to the other terminal session to monitor the
backup process.
In the diagnostics log (/home/db2inst1/sqllib/db2dump/db2diag.log) you can see STMM modifying memory consumers
parameters. We observed, during the creation of the lab materials, that DB2 set the parallelism to 2 and used 4 backup buffers
of size 4096.
DB2 Performance Tuning and Monitoring Clinic Workbook page 234 of 251.
IBM Software
Information Management
Figure 9 – db2diag.log
The output of the LIST UTILITIES command shows that the backup was unthrottled. It also gave us an estimated
percentage complete.
DB2 Performance Tuning and Monitoring Clinic Workbook page 235 of 251.
IBM Software
Information Management
Switch to the terminal session tab that isn't monitoring activities. In this session you will activate the database (it's ready for
production use now), add more files to the database, and then perform an online backup which you will throttle. Add more data
with the following command.
DB2 Performance Tuning and Monitoring Clinic Workbook page 236 of 251.
IBM Software
Information Management
After the script completes, enter the following command on a single line to backup the database, and then switch to the other
terminal session to monitor the backup process. Providing the util_impact_priority clause and value will throttle the
backup operation.
You should notice in db2diag.log that an online backup was started and the transaction log was archived. The WHILE loop
should show you that the backup was started in throttled mode using the percentage you supplied, and that specifying the
UTIL_IMPACT_PRIORITY on the BACKUP command changed to the priority to 80.
DB2 Performance Tuning and Monitoring Clinic Workbook page 237 of 251.
IBM Software
Information Management
UTIL_IMPACT_PRIORITY is set using the SET UTIL_IMPACT_PRIORITY command. It sets the priority that a particular utility has over the
resources available to throttled utilities as defined by UTIL_IMPACT_LIM. (0 = unthrottled)
In addition to the storage savings you can achieve through row compression in your active database, you can also use backup
compression to reduce the size of your database backups.
Whereas row compression works on a table-by-table basis, when you use compression for your backups, all of the data in the
backup image is compressed, including catalog tables, index objects, LOB objects, auxiliary database files and database meta-
data.
You can use backup compression with tables that use row compression. Keep in mind, however, that backup compression
requires additional CPU resources and extra time. It may be sufficient to use table compression alone to achieve a reduction in
your backup storage requirements. If you are using row compression, consider using backup compression only if storage
optimization is of higher priority than the extra time it takes to perform the backup.
Tip: Consider using backup compression only on table spaces that do not contain compressed data if the following conditions
apply:
• Data and index objects are separate from LOB and long field data, and
DB2 Performance Tuning and Monitoring Clinic Workbook page 238 of 251.
IBM Software
Information Management
• You use row and index compression on the majority of your data tables and indexes, respectively
To use compression for your backups, use the COMPRESS option on the BACKUP DATABASE command.
The EXPORT command has no performance optimization parameters. There are a few ways to improve the export utility's
performance. As the export utility is an embedded SQL application and does SQL fetches internally, optimizations that apply to
SQL operations apply to the export utility as well. Consider taking advantage of large buffer pools, plus indexing and sort heap if
a WHERE clause is used. In addition, try to minimize device contention on the output files by placing them away from the
containers and log devices. Large objects and XML files can be exported in round robin fashion by specifying multiple
directories.
EXPORT does not move database objects associated with the tables such as aliases, views, or triggers, or the objects tables
may depend on, such as user-defined types or user-defined functions. You will need to manually move these objects. They can
be created with the DB2LOOK command.
Create a directory to store the extracted data in and list the number of records in the table with the following commands.
mkdir /tmp/db2move
cd /tmp/db2move
db2 connect to xmldb
db2 "select count(*) from reldata"
db2 connect reset
DB2 Performance Tuning and Monitoring Clinic Workbook page 239 of 251.
IBM Software
Information Management
Choose one of the following commands to export the data. Both versions of this command should create identical output
and results. Notice that the DB2MOVE command syntax is simpler. The EXPORT command should be entered on a single line.
Either:
db2 connect to xmldb
db2move xmldb export -tn reldata
Or:
db2 connect to xmldb
db2 "export to tab1.ixf of ixf messages tab1.msg select * from reldata"
You can review the files created by the EXPORT command with the following commands. The EXPORT.out file is available if you
used the DB2MOVE command.
ls
cat EXPORT.out
cat tab1.msg
Delete the data in the table and verify the records have been deleted with the following commands.
db2 connect to xmldb
db2 truncate table reldata reuse storage immediate
db2 "select count(*) from reldata"
db2 connect reset
DB2 Performance Tuning and Monitoring Clinic Workbook page 240 of 251.
IBM Software
Information Management
You can improve the performance of the IMPORT utility if the data is on separate drives from the log files and tablespace
containers. You can reduce the impact to other applications by importing individual tables or doing hierarchical imports when
system utilitization is lower. The following two parameters impact the performance of this utility.
ALLOW NO ACCESS is the default for IMPORT operations. DB2 will acquire an exclusive (X) lock on the table before any rows
are inserted. In this case, the table is considered to be offline to other applications and they will not be able to access the table
or its contents. This allows for the fastest IMPORT operation.
Specifying WRITE causes the IMPORT to run in online mode, allowing other applications to access the table. There are several
IMPORT options which are not compatible with online mode. These are REPLACE, CREATE, or REPLACE_CREATE.
The COMMITCOUNT parameter specifies how frequently the data is to be written to disk. IMPORT performs a COMMIT after every
n records are imported if a number is provided. When AUTOMATIC is specified, IMPORT determines when a commit needs to be
performed. The IMPORT utility will automatically commit for either one of two reasons:
DB2 Performance Tuning and Monitoring Clinic Workbook page 241 of 251.
IBM Software
Information Management
If the ALLOW WRITE ACCESS option is specified and the COMMITCOUNT option is not specified, the IMPORT utility will perform
commits as if COMMITCOUNT AUTOMATIC had been specified.
In order to meet our SLA we need to keep the database online. Import the data back into the reldata table using the following
commands and performance options. The IMPORT command should be entered on a single line.
There should be a message for each COMMIT the IMPORT utility made after each 50 rows, and when the operation ended.
less import.msg
For reference, notice that the DB2MOVE command syntax is much simpler than the IMPORT command. DB2MOVE will use the
information in file db2move.lst to determine which files and tables to access.
DB2 Performance Tuning and Monitoring Clinic Workbook page 242 of 251.
IBM Software
Information Management
The ingest utility ingests pre-processed data directly or from files output by ETL tools or other means. It can run continually and
thus it can process a continuous data stream through pipes. The data is ingested at speeds that are high enough to populate
even large databases in partitioned database environments.
An INGEST command updates the target table with low latency in a single step. The ingest utility uses row locking, so it has
minimal interference with other user activities on the same table.
With this utility, you can perform DML operations on a table using an SQL-like interface without locking the target table. These
ingest operations support the following SQL statements: INSERT, UPDATE, MERGE, REPLACE, and DELETE. The ingest
utility also supports the use of SQL expressions to build individual column values from more than one data field.
1. Transport
2. Format
3. Flush
During this exercise we will just replace data into a table using import, ingest and load, each one several times to see and
compare execution times.
Depending on hardware infrastructure, you may reach to the conclusion that the fastest one is LOAD. However, INGEST not
only is as fast as IMPORT, but also has the option to automatically restart from the point of cancellation if it occurs. If we disable
this functionality in INGEST, it will run twice as fast as IMPORT. Furthermore, INGEST can input data from several files at the
same time, or from pipes, etc.
DB2 Performance Tuning and Monitoring Clinic Workbook page 243 of 251.
IBM Software
Information Management
We first need to create the table that INGEST tool uses for restart information, and the table that we will use for our example, to
input data:
cd ~/Documents/LabScripts/db2pt/utilities
./compare_times.sh
You will see output similar to the results below (result times may vary depending on the physical machine where VMware
resides). Notice the execution time for each operation:
DB2 Performance Tuning and Monitoring Clinic Workbook page 244 of 251.
IBM Software
Information Management
STARTING LOAD
"/home/db2inst1/Documents/LabScripts/db2pt/utilities/ingest_data_200k".
13:57:07.219370".
SQL3110N The utility has completed processing. "200000" rows were read from
SQL3515W The utility has finished the "LOAD" phase at time "03/23/2012
13:57:11.797938".
real 0m2.632s
user 0m0.004s
sys 0m0.008s
SQL2914I The ingest utility has started the following ingest job:
DB2 Performance Tuning and Monitoring Clinic Workbook page 245 of 251.
IBM Software
Information Management
"DB21001:20120323.135714.571600:00003:00006".
13:57:28.934178"
real 0m14.512s
user 0m0.004s
sys 0m0.008s
SQL2914I The ingest utility has started the following ingest job:
"DB21001:20120323.135729.156538:00003:00006".
13:57:40.456536"
real 0m11.531s
user 0m0.008s
sys 0m0.008s
SQL3418W The NOCHARDEL file type modifier should not be specified if the data
was exported using DB2. It is provided to support vendor data files that do
DB2 Performance Tuning and Monitoring Clinic Workbook page 246 of 251.
IBM Software
Information Management
SQL3110N The utility has completed processing. "200000" rows were read from
SQL3149N "200000" rows were processed from the input file. "200000" rows
were successfully inserted into the table. "0" rows were rejected.
As you can see, the execution time of IMPORT and INGEST with restart capability is almost the same, but INGEST provides the
functionality of managing jobs. Also, notice that INGEST execution time is about HALF of IMPORT execution time, without the
limitations of the LOAD command.
The LOAD, like the EXPORT and IMPORT utilities, is cross-platform compatible. This utility can be used as an alternative to the
IMPORT utility. Use the LOAD utility if you want to move large quantities of data into empty tables or tables that already contain
data.
DB2 Performance Tuning and Monitoring Clinic Workbook page 247 of 251.
IBM Software
Information Management
SAVECOUNT n
CPU_PARALLELISM n
DISK_PARALLELISM
FETCH_PARALLELISM
INDEXING MODE
REBUILD
INCREMENTAL
AUTOSELECT
DEFERRED
If a value is not specified for DATA BUFFER, CPU_PARALLELISM, or DISK_PARALLELISM, an intelligent default is calculated
by the utility at run time.
As shown in Figure 18, the record order in the source data can be preserved when CPU_PARALLELISM is set to allow multiple
processes or threads.
During the development of this lab we were not able to capture much information with the LOAD QUERY command because our
table is small and the LOAD completes very quickly. You can try if you like with the following loop, or you can skip this set of
commands. If you want to try, open a new terminal session and enter these commands to monitor the progress of the LOAD
operation. Press Enter after each line.
DB2 Performance Tuning and Monitoring Clinic Workbook page 248 of 251.
IBM Software
Information Management
Switch to an unused terminal session. Delete the data in the table and verify the records have been deleted with the following
commands.
Start the LOAD and verify the records were loaded with the following commands. The LOAD command should be entered on a
single line.
cd /tmp/db2move
db2 load from tab1.ixf of ixf savecount 50 messages load.msg insert into
reldata cpu_parallelism 2
db2 "select count(*) from reldata"
db2 connect reset
If you tried to monitor the LOAD progress with the loop, switch back to that terminal session and press Ctrl+C to cancel the loop.
You can review the output of the command in file loadQuery.txt. Note: you may have to repeat the load a couple of times for the
query to show results since it is such a small load.
If you require 24x7 availability then you will probably implement a form of high availability for your database. It is possible to
eliminate some maintenance activity against a primary production server by applying maintenance to the standby, or backup
server.
Many companies and organizations have established Service Level Agreements or SLAs. These formalized requirements will
have a significant bearing on how you will configure your DB2 system and commands, often requiring you to trade availability for
performance.
DB2 Performance Tuning and Monitoring Clinic Workbook page 249 of 251.
IBM Software
Information Management
7 Cleanup
If you want to cleanup your work you can use these commands.
DB2 Performance Tuning and Monitoring Clinic Workbook page 250 of 251.
IBM Software
Information Management
IBM Canada
8200 Warden Avenue
Markham, ON
L6G 1C7
Canada
IBM, the IBM logo, ibm.com and Tivoli are trademarks or registered
trademarks of International Business Machines Corporation in the
United States, other countries, or both. If these and other
IBM trademarked terms are marked on their first occurrence in this
information with a trademark symbol (® or ™), these symbols indicate
U.S. registered or common law trademarks owned by IBM at the time
this information was published. Such trademarks may also be
registered or common law trademarks in other countries. A current list
of IBM trademarks is available on the Web at “Copyright and
trademark information” at ibm.com/legal/copytrade.shtml
Product data has been reviewed for accuracy as of the date of initial
publication. Product data is subject to change without notice. Any
statements regarding IBM’s future direction and intent are subject to
change or withdrawal without notice, and represent goals and
objectives only.
DB2 Performance Tuning and Monitoring Clinic Workbook page 251 of 251.