Professional Documents
Culture Documents
com
* The Ebook starts from the next page : Enjoy !
Front cover
Chris Rayns George Bogner Simon Calthrop Rob Jones Ken Ray Joe Sewell
ibm.com/redbooks
International Technical Support Organization IBM Tools: CICS Performance Monitor V1.1 April 2003
SG24-6922-00
Note: Before using this information and the product it supports, read the information in Notices on page ix.
First Edition (April 2003) This edition applies to Version 1, Release 1, of IBM CICS Performance Monitor for OS/390, Version 1 Release 3 of CICS Transaction Server, and Version 2 Release 2 of CICS Transaction Server.
Copyright International Business Machines Corporation 2003. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Contents
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .x Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...... ...... ...... ...... ....... ....... ....... ....... . xi . xi . xii . xii
Part 1. Product overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 1. CICS online performance overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 How to approach CICS performance monitoring and tuning. . . . . . . . . . . . . . . . . . . . . . 1.2 CICS tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 CICS Performance Analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 CICS Performance Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.3 CICS Performance Analyzer and CICS Performance Monitor benefits . . . . . . . . . 1.2.4 CICS Online Transmission Time Optimizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.5 CICS Interdependency Analyzer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.6 Other relevant information sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 2. CICS PM overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 What is CICS PM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 CICS PM architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Minimum CICSPlex SM concepts for CICS PM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Basic CICSPlex SM components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 CICSPlex SM for CICS PM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 CICS PM GUI application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Event monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.2 Threshold definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 Installed Threshold Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 CICS PM use of Web user interface view sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 The CICS PM server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 System requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.1 Server software prereqs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.2 Workstation prereqs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Post SMP/E install and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 3. CICSPlex SM installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Overview of the installation of CICSPlex SM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Planning your installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 MVS environment setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Build the system components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 CICSPlex SM data sets and initialization job CMZDEFDS . . . . . . . . . . . . . . . . . . 3.3.2 Create the CICSPlex SM CSD definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Coordinating address space (CAS) creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 Editing sample CAS procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Starting the CAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 CICSPlex SM address space (CMAS) creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Create CMAS region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4 4 5 6 7 8 8 9
11 12 12 13 13 14 15 16 18 20 21 22 22 22 23 24 25 27 27 29 29 29 31 33 33 34 35 35
iii
3.5.2 Editing the sample CMAS procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.3 Creating the CMAS parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.4 CMAS SIT SYSIN parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.5 Start the CMAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Create the CICSplex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.1 Editing CMZBATCH JCL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.2 Definition statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 CICSPlex SM Web User Interface (WUI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.1 Create Web user interface region data sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.2 Editing the sample Web user interface procedure . . . . . . . . . . . . . . . . . . . . . . . . 3.7.3 Code page conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.4 Language support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.5 CICS initialization parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.6 CICSPlex SM initialization parameters for the Web user interface . . . . . . . . . . . . 3.7.7 Server initialization parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.8 Starting the Web user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Creating a Managed Application System (MAS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8.1 Adding libraries to your CICS regions JCL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8.2 CSD definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8.3 System initialization parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8.4 LMAS parameters and JCL changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8.5 Restart the LMAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.9 Security implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.9.1 Securing CICSPlex SM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.9.2 PassTicket support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 4. Customizing an existing CPSM installation to use CICS PM . . . . 4.1 Upgrade your Web user interface region CSD with the CICS PM resources . 4.2 Import the CICS PM threshold definitions into your CMAS repository . . . . . . 4.3 Add CICS PM support to your Web user interface server. . . . . . . . . . . . . . . . 4.4 Add PassTicket support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....... ....... ....... ....... .......
36 38 38 40 41 41 43 46 46 46 48 50 50 51 52 53 55 55 56 56 56 57 58 58 59 61 62 63 64 65 67 68 68 68 69 69 71
Chapter 5. CICS PM client installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Reviewing prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Download the CICS PM workstation client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 FTP download . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Terminal emulation download . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Unload and Install the CICS PM workstation client . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Connect and start working with CICS PM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 6. GUI interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 6.1 Workstation client startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.2 Workstation client components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 6.3 CICS PM window navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 6.3.1 Monitor window status area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 6.4 Monitoring component. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 6.4.1 Event Monitor window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 6.5 Threshold definition component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 6.5.1 Threshold definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 6.5.2 Threshold definition groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 6.5.3 Threshold Definitions window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.5.4 Installed threshold definition window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.6 CMZBase threshold definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
iv
Chapter 7. The CICSPlex SM Web User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Using the CICSPlex SM Web user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 The CICS PM Web user interface main menu . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.2 CICS regions views. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.3 View menus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 The CICSPlex SM Web user interface view editor . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Creating your own menus and view sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 CICSPlex SM Web user interface administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 The CICSPlex SM Web user interface transaction COVC . . . . . . . . . . . . . . . . . 7.3.2 Security considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
121 122 122 126 142 147 147 155 155 158
Part 2. CICS PM in action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Chapter 8. System setup and scenario overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 CICS region setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.1 Configuration for traditional workloads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.2 Configuration for Enterprise JavaBean workloads . . . . . . . . . . . . . . . . . . . . . . . 8.1.3 CICS Web Support and 3270 Bridge setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.4 CICS Transaction Gateway setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.1 CICS VSAM problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.2 Tuning the CICS-DB2 attachment facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.3 CICS ISC problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.4 CICS storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.5 Java applications in CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 9. VSAM system problems scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 VSAM LSR high volume scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1.1 Setting up the thresholds for the VSAM LSR scenario . . . . . . . . . . . . . . . . . . . . 9.1.2 Running the VSAM LSR scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 VSAM Enqueues problem scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3 VSAM RLS problem scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.1 Setting up the thresholds for the VSAM RLS scenario . . . . . . . . . . . . . . . . . . . . 9.3.2 Running the VSAM RLS scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 10. DB2 system problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1 CICS and DB2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.1 Overview of the CICS DB2 attachment facility . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 CICS DB2 attachment tuning overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.1 CICS PM monitoring of the CICS DB2 attachment facility . . . . . . . . . . . . . . . . 10.3 CICS PM DB2 thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.1 DB2 connection thresholds and alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.2 DB2 Entries thresholds and alerts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.3 DB2 Trans thresholds and alerts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.4 DB2 Threads thresholds and alerts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.5 DB2 subsys thresholds and alerts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.6 XTASK thresholds and alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4 CICS PM DB2 threshold monitoring scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.1 Scenario: DB2 connection drop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.2 Scenario: DB2 transaction workload using dedicated entry threads. . . . . . . . . 10.4.3 Scenario: DB2 transaction workload using pool threads. . . . . . . . . . . . . . . . . . 10.4.4 Scenario: DB2 transaction workload lockout. . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.5 Scenario: DB2 transaction workload entry vs. pool threads . . . . . . . . . . . . . . .
Contents
163 164 164 164 165 165 167 168 168 168 168 168 171 172 172 175 187 193 194 196 207 209 210 210 213 215 215 216 218 220 221 223 225 226 227 236 241 250 262 v
10.4.6 CICS PA threshold monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 10.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 Chapter 11. Storage scenario in CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 The CICS dynamic storage areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.1 CICS subpools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.2 Monitoring CICS dynamic storage areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Scenario one . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Scenario two description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 12. Java scenario in CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 CICS and Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.1 Java language programs in CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.2 Stateless CORBA objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.3 Enterprise JavaBeans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Monitoring JVM performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.1 JVM-related Threshold Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.2 Threshold values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.3 Threshold groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.4 Threshold group JVM_reset_NO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.5 Threshold group JVM_reset_YES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.6 Installing the threshold definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3 JCICS workload scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.1 Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.2 Monitoring AOR with Xresettable=NO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.3 Monitoring AOR with Xresettable=YES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4 EJB workload scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.1 Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.2 Monitoring the AORs for EJBs with Xresettable=YES . . . . . . . . . . . . . . . . . . . 12.4.3 Resettable events and JVM pool phaseout . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5 Monitoring an IIOP Listener TCPIPSERVICE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.1 TCP/IP-related thresholds. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.2 IIOP Listener scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.3 TCPIPSERVICE status scenario. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.4 MAXSOCKETS condition scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 13. ISC scenario in CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1 Overview of the scenario. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.1 Creating a threshold group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.2 Add thresholds to group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.3 The Attribute field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.4 Operator, Eval Data fields and event severity. . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.5 Save and install. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.6 Setting up other thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.1 Dealing with the first problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.2 Our next problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 274 275 275 279 290 298 301 302 302 302 302 303 303 303 304 304 306 308 308 308 310 312 315 316 320 324 326 326 327 327 329 332 333 334 334 335 338 339 339 340 341 341 343 348
How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 IBM Redbooks collections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
Contents
vii
viii
Notices
This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces.
ix
Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:
CICS Connection CICS CICSPlex ^ IBM.COM IBM IMS MVS MVS/ESA OS/2 OS/390 Parallel Sysplex RACF Redbooks(logo) Redbooks RMF S/390 System/390 Tivoli VTAM WebSphere z/OS
The following terms are trademarks of other companies: ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. C-bus is a trademark of Corollary, Inc. in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC. Other company, product, and service names may be trademarks or service marks of others.
Preface
This IBM Redbook is primarily intended for CICS Transaction Server V1.3 and V2.2 customers who plan to implement IBM CICS Performance Monitor. CICS PM is an online monitoring tool that provides real-time performance management, monitoring, and troubleshooting solutions for CICS TS. It enables you to detect performance problems early, identify causes, and change system and resource parameters to avoid problems. The book starts with an overview of CICS-provided tools and utilities that help you gather and analyze performance data. We then introduce the CICS Performance Monitor, provide installation details, and look at the various interfaces: the GUI, followed by the WUI. Also detailed here is the installation procedure for CICSPlex SM. The second part of the book reviews a number of scenarios in which you would use this product to help diagnose a system problem. The scenarios cover the major CICS components and interfaces: CICS-VSAM CICS-DB2 CICS-Java CICS Storage CICS ISC
xi
Joe Sewell is a Advisory IT Specialist working with the IGS Americas Migration Team in the Unites States. He has 35 years of experience in the mainframe data processing field. His areas of expertise include MVS, CICS, and Tivoli Decision Support for OS/390 . Thanks to the following people for their contributions to this project: Eugene Deborin International Technical Support Organization, Poughkeepsie Center Robert Haimowitz International Technical Support Organization, Poughkeepsie Center Richard M Conway International Technical Support Organization, Poughkeepsie Center Grant Shayler IBM Hursley Peter B Siddell IBM Hursley Norbert Verbestel IBM Belgium Dave Rushalll IBM Hursley Peter Klein IBM Germany Peter Siddell IBM Hursley
Comments welcome
Your comments are important to us! We want our Redbooks to be as helpful as possible. Send us your comments about this or other Redbooks in one of the following ways: Use the online Contact us review redbook form found at: xii
IBM Tools: CICS Performance Monitor V1.1
ibm.com/redbooks
Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYJ Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400
Preface
xiii
xiv
Part 1
Part
Product overview
In this part we offer an overview of CICS online performance. We introduce CICS Performance Monitor, a component of the IBM CICS tools family. We cover CICSPlex SM installation, and how to customize an existing CPSM installation to use CICS PM. There is a chapter on CICS PM Client installation, and we then examine the GUI CICS PM interface and the CPSM Web User Interface.
Chapter 1.
Let us, first, have a look at the first two products that have to do with CICS performance monitoring and analysis.
PQ63527
UQ68731
APAR PQ63143
PTF
UQ68396,
UQ68398,
UQ68400
CICS TS V1.3
PQ63141
Global and task related user exits The view sets provide access to all resources defined to the CICS systems, including: Transactions Transaction classes Programs Files, data tables, and local shared resource (LSR) pools Connections Logstreams Journals Temporary storage queues Transient data queues Terminals DB2 entries System and transaction dumps Enterprise Java resources
Other view sets are provided, giving access to the performance information about all the active tasks in the CICS systems.
The two products, CICS PA and CICS PM, complement each other. Table 1-2 provides a functional comparison.
Table 1-2 CICS PA and CICS PM comparison
Function Invocation Mode Input origin/Output destination Output format Information source History Deals with Additional usage CICS PA Offline Passive 3270/Reports Tabular/Graphical SMF As far as you keep SMF data Exceptions Capacity planning CICS PM Online Active Windows workstation GUI CICSPlex SM Recent Alerts Status monitoring
When the transaction resource class of monitoring is switched off When CICS itself quiesces When monitoring itself is deactivated or when there is an immediate shutdown of CICS, the transaction resource records are not written to SMF and the data is lost. EXEC CICS SET MONITOR STATUS(ON) PERFCLASS(PERF) This information could be used periodically to calculate the charges applicable to different tasks. If you want to set up algorithms for charging users for resources used by them, you can use this class of data collection to update the charging information in your organizations accounting programs. (For older versions of CICS, it was not recommended to charge primarily on exact resource usage, because of the overheads involved in getting these figures.)
GTF is generally used to monitor short periods of system activity. You should run it accordingly. No data reduction programs are provided with GTF. To extract and summarize the data into a meaningful and manageable form, you can either write a data reduction program yourself or use one of the available program offerings.
10
Chapter 2.
CICS PM overview
This chapter introduces the IBM CICS Performance Monitor at a high level. The following topics are covered: What is CICS PM? CICS PM architecture Minimum CICSPlex SM concepts for CICSPM CICS PM GUI application CICS PM Web user interface view set The CICS PM Server System requirements Post SMP/E install and configuration
11
CICS PM User
ESSS
z/OS
Figure 2-1 CICS Performance Monitor architecture
In order to implement CICS PM, regions must be grouped into one or more CICSplexes. A CICSplex is any group of CICS regions that are defined to and managed by CICSPlex SM (CPSM). CICSPlex SM is included as part of CICS Transaction Sever for OS/390 1.3, or CICS Transaction Sever for z/OS 2.2.
12
CICSPlex
CICS TS CPSM WUI Region
CICSPM SERVER CODE
CICS PM GUI app and Web Browser
CPSM CAS
CPSM Agent
ESSS
CPSM Agent
CPSM Agent
CPSM Agent
CPSM Agent
WUI Repository
CMAS Repository
z/OS
Figure 2-2 Basic CPSM components required by CICS PM
13
Note: We recommend that in a real production environment the Web user interface server be placed in a special Web user interface server CICSplex. An installation that currently utilizes CICSPlex SM will already have at least the CAS and CMAS regions running. In this case, the existing CMAS needs to be configured for CICS PM, and a Web user interface region added and configured.
What a CMAS is
The CMAS is a CICS region dedicated solely to the CICSPlex SM function. It is responsible for managing and reporting of all CICS regions and resources within the defined CICSplex(es). The CMAS interacts with CICSPlex SM agent code running on each managed CICS region (MAS) to define events or conditions of interest, and collect information gathered as a result of such definitions. The CICSplex topology, events, and conditional definitions together with collected data are all stored in the CMAS repository data set, EYUDREP. In more complex setups, multiple CMAS regions can communicate between MVS regions to build up CICSplexes that may consist of CICS regions across a sysplex or located at physically disparate sites. An MAS belongs to a CICSplex and can only be a member of one CICSplex. A maintenance point CMAS is the owner of a CICSplex. When more than one CMAS is involved in managing a CICSplex, then the CMAS which was used to create the CICSplex is the maintenance point. This is basically the master repository for the CICSplex. When a definition is changed, then the maintenance point CMAS must be available. For simplicity, we only consider a single CMAS for the Web user interface server and all MAS regions here. Note: A CMAS region is not part of a managed CICSplex.
What a MAS is
Every CICS region to be monitored using CICS PM must be under the management of CICSPlex SM. To become an MAS, a CICS region must have the CICSPlex SM agent load libraries added to the DFHRPL, STEPLIB concatenations of their startup batch job or procedure. The CICSPlex SM agent code is executed during CICS initialization to register the MAS with its respective managing CMAS. More specifically, a z/OS-based MAS region is known as a Local MAS (or LMAS). An OS/2 or Microsoft Windows-based CICS is known as a Remote MAS (or RMAS). We only deal with LMAS regions here, and refer to them simply as MAS regions. Agent code running on the MAS communicates pertinent statistical and monitoring data back to the CMAS it connected to.
14
What a CAS is
The Coordinating Address Space (CAS) is a standalone CICSPlex SM address space (it is not a CICS region), running as a dynamically created subsystem. The CAS automatically defines itself to MVS as a subsystem, but unlike ESSS the installation must create the JCL for the CAS job/proc and start it. The CAS is to support the EUI. A CAS can have its own topology, which is independent of the CICSPlex SM/CICSPlex topology. The CMAS establishes a connection with the CAS during initialization. For the purposes of CICS PM, configuration work required for the CAS region is minimal. There is only one CAS per MVS region.
The ESSS
There is one further address space that is automatically created upon startup of the CMAS, called the Environment Services System Services (ESSS). It is a limited function system address space that provides MVS system services to the CICSPlex SM components. Note: There is only one ESSS per MVS region, and no special setup work is required.
Further reading
More complex environments may require further background reading, such as CICSPlex SM for z/OS Concepts and Planning, SC34-6015.
15
CICSplex topology information is downloaded and the GUI presents the Event Monitor window, shown in Figure 2-3.
Figure 2-3 Screen shot of the CICS PM GUI application following successful signon and topology refresh
The CICS PM GUI offers the user three main categories to work with: Event monitor Threshold definitions Installed threshold definitions Through these three windows, threshold definitions can be managed (created, modified, saved, deleted, installed, discarded) and the resultant events monitored. We will now look at each of the main windows individually.
Topology
The Topology display is a graphical tree representation of the CICSplex(es), with a CICSplex name being the top level node. Subnodes of the CICSplex consist of CICSplex SM groups. CICSPlex SM groups may contain either individual CICS regions, or further CICSPlex SM groups. The organization of CICS regions into CICSPlex SM groups needs to be carefully considered during the planning stage for CICSPlex SM installation/configuration. 16
IBM Tools: CICS Performance Monitor V1.1
Using the topology tree, the user can select the current scope to be used by the other two panes for system status and event history. Each tree node has a current status that reflects the highest severity of any outstanding event on any subnodes. Therefore, a high severity event in any CICS system will be visible to the CICS PM user even when the GUI is set to a CICSplex-wide scope.
Region Histories
The Region Histories subwindow shows the status of the currently selected node in the Topology pane, together with the status of any subnodes immediately below the selected node (if there are any). The indicated status is subject to the currently selected Resource Filter. Each node in the Topology tree has an icon next to its name indicating current status, and then a chronological indication of status of that node over the previous five sample periods. This can be seen in Figure 2-4.
Time
Figure 2-4 An entry on the Region Histories pane of the Event Monitor window
Resource Filter
The current setting of the Resource Filter is shown directly below the subwindow tabs. The Resource Filter and the current scope form a mask for the Region Histories and the Event Log displays. Figure 2-5 shows the currently selected scope and resource filter.
Figure 2-5 Display of current scope and resource filter on the Event Monitor window
Resource Summary
Each entry on the Resource Summary tab has three buttons, which have functions as illustrated in Figure 2-6 on page 18.
17
Set Filter to labelled CICS Resource and switch to Region History tab
Figure 2-6 Actions for buttons on the Event Monitor window Resource Summary pane
The first button in the set, labelled with a CICS Resource, is a Web browser hyperlink to a Web user interface view for that resource type. For further information on Web user interface view hyperlinks, see 2.5, CICS PM use of Web user interface view sets on page 21. The second and third button both cause the Resource Filter to be set. However, the second button causes the GUI to switch the pane back to the Region Histories tab, after setting the Resource Filter.
Event log
The lower right pane contains the Event Monitor log, which is also subject to the scope currently selected on the topology pane. These events appear as they happen (within the selected scope and filter), but only remain present while the event condition holds true. Events are automatically removed from this pane at the next refresh, when the condition no longer holds true. From any line in the event monitor log, the user can link to any one of Event (within the GUI), Region, Resource Type or Resource Identifier (all WUI views). The events can be sorted by the data in any of the available columns. Figure 2-7 shows an example of the event log pane with several outstanding events.
Figure 2-7 The event log pane of the Event Monitor window
18
Threshold Groups
CICS PM supplies a sample set of thresholds named CMZBase. This is very much a sample set, and should be regarded as read-only. The intent is that post-install, a copy is made of the CMZBase threshold group, which can then be customized to suit the needs of the site as required. In Figure 2-8, the CMZBase Threshold Group has been selected, which causes the CICS PM GUI to download the definitions from the CICS PM Server region. Alternatively, new Threshold Groups can be created to organize commonly themed sets of Threshold Definitions. Threshold Groups can be installed in any available scope.
CICS Resources
Threshold Definitions are categorized under the CICS Resource to which they pertain. The middle pane, CICS Resources, shows which CICS Resources have had Threshold Definitions made against them, indicated by the presence of a golden bell icon. In Figure 2-8, the Region CICS Resource under the Region category has been selected. The currently selected CICS Resource item acts as a filter for the contents of the right-hand pane.
19
Threshold Definitions
A Threshold Definition describes a condition which when satisfied produces an event. The condition describes a state in relation to a CICS resource or statistic. An example may be SOS equal to SOS (or SOSABOVE or SOSBELOW). Note: Defined Thresholds are just that; they are not necessarily installed or active. They are comparable to CICS definitions stored on the CSD. Individual Threshold Definitions cannot be installed in isolation. In order to install a Threshold Definition, it must belong to a Threshold Group and that group must be installed as a whole.
The CICS PM GUI is discussed further in Chapter 6, GUI interface on page 75.
20
Figure 2-10 Screen shot of a WUI view for a specific CICS region
The supplied Web user interface view sets can also be used independently of the GUI. Further information on the Web user interface can be found in Chapter 7, The CICSPlex SM Web User Interface on page 121.
21
Required maintenance
RACF
Note that OS/390 2.10 only requires APAR OW44393 PTF UW72529.
CICS TS 1.3
CICS Transaction Server for OS/390 1.3 Element CICS APAR numbers PQ63141 PQ65166 PQ66101 PQ58762 PQ59865 PQ66469 Corresponding PTF numbers UQ70905, UQ70908, UQ70913 UQ71532 UQ71488 UQ71427, UQ71439(WUI) UQ71511, UQ71512(WUI) UQ71510
CICSPlex SM
22
CICS TS 2.2
CICS Transaction Server for z/OS 2.2 Element CICS APAR numbers PQ63143 PQ65168 PQ66102 PQ58660,PQ62212, PQ62937 PQ59827 PQ66470 Corresponding PTF numbers UQ68396, UQ68398, UQ68400 UQ71534 UQ71490 UQ71440 UQ71515 UQ71509
CICSPlex SM
Recommended maintenance
CICS TS 1.3
CICS Transaction Server for z/OS 1.3 Element CICSPlex SM APAR numbers PQ65959 PQ67901 PQ67903 PQ67650 Corresponding PTF numbers Open at time of writing Open at time of writing UQ71926 Open at time of writing
CICS TS 2.2
CICS Transaction Server for z/OS 2.2 Element CICSPlex SM APAR numbers PQ65961 PQ67902 PQ67904 PQ67652 Corresponding PTF numbers Open at time of writing Open at time of writing UQ71929 Open at time of writing
CICS PM 1.1
CICS Transaction Server for z/OS 2.2 Element CICS PM APAR numbers PQ69345 Corresponding PTF numbers Open at time of writing
Operating system
CICS PM is only supported for the following operating systems:
23
Microsoft Windows NT 4 Service pack 6 or higher Microsoft Windows 2000 Professional Service pack 2 Microsoft Windows XP Professional
Web browser
The CICS PM Web user interface View sets require a browser to have the following capabilities: HTML 3.2 or later, with frames and tables HTTP 1.0 or 1.1 HTTP cookie support Further functionality is also available to browsers that provide support for: Java 1.0.2 classes JavaScript 1.1 scripts Multiple browser windows.
24
Chapter 3.
CICSPlex SM installation
This chapter is for those who have no CICSPlex SM currently installed and have little or no knowledge of it. For those who already have a CICSPlex SM running to be used with CICS/PM, refer to Chapter 4, Customizing an existing CPSM installation to use CICS PM on page 61.This chapter does not include the client download and install, which is covered in Chapter 5, CICS PM client installation on page 67. In this chapter we describe our experiences in installing CICSPlex SM and tailoring it for use with CICS/PM. We also provide a sample worksheet and useful command summary as we found this to be the best way of keeping track of Dataset High Level Qualifiers, Sysids and Applids, etc. On completion of this chapter we will have a running CICSPlex SM system with the necessary components for a basic CICS/PM setup. We build this section by section until we reach the setup in Figure 3-1 on page 26. We added CICS regions B, C, and D just to show potential application groupings.
25
CM AS SCSCCM A1 SC 53PLEX
26
Occurrence
CICSPlex SM run time data sets HLQ CICSPlex SM system data sets HLQ CICS/PM system data sets HLQ LMAS CSD grouplist Web user interface CSD grouplist TCP/IP Port for CICS/PM CAS subsystem id (ssid) CMAS sysid CMAS applid CICS system data set HLQ CMAS CSD grouplist CICSPlex SM release number CMAS name CICSplex name CMAS CICS version 1st group name Regions of 1st group 2nd group name
27
Occurrence
Value
Regions of 2nd group 3rd group name Contents of 3rd group WUI sysid Web user interface applid TCP/IP host name TCP/IP port for the client
CICSPlex SM run-time data sets HLQ CICSPlex SM system data sets HLQ CICS/PM system data sets HLQ LMAS CSD grouplist Web user interface CSD grouplist TCP/IP Port for CICS/PM CAS subsystem id (ssid) CMAS sysid CMAS applid CICS system data set HLQ CMAS CSD grouplist CICSPlex SM release number CMAS name CICSplex name CMAS CICS version 1st group name Regions of 1st group 2nd group name Regions of 2nd group 3rd group name Contents of 3rd group
28
Occurrence
Web user interface sysid Web user interface applid TCP/IP host name TCP/IP port for the client
29
@userid@ - Job userid We set this to our TSO userid for the JCL jobcard. An alternative is to create a separate pds member containing your jobcard to be copied into any jobs you run. In Step DREPINIT you will need to specify your time zone in the parms for program EYU9XDUT. Table 3-4 shows the code for your specific time zone.
Table 3-3 Dataset summary
DD Name EYUDREP Data set type VSAM Description Contains CPSM repository, holding the definitions of your CICSplex. Contains CPSM WUI repository holding the view sets and menus. This is the parameter repository, which is only used for communications definitions if there is more than 1 CAS. Holds the screen configuration definitions when used via CPSM in TSO. The parameters used by CMAS, WUI and MAS at initialization. Usage Used by CMAS
EYUWREP EYUIPRM
VSAM PDS
EYUSDEF
PDS
The CPSM TSO interface Any Region used within the CPSM structure
EYUPARM
PDS
30
Code
GMT Offset
Code
GMT Offset
* Note Time zone A can only be specified in a period definition (Refer to the CICSPlex System Manager Administration Guide)
Once tailoring has been completed, submit the job and expect a zero return code. The CMZDEFDS job can be rerun at this point, as the data sets are deleted prior to being defined. Note: The CMZDEFDS job assumes you are using SMS to manage your data set placement. If this is not the case, add the required VOL=SER= for the PDS defines and a suitable VOLUME( ) to the VSAM defines. After completing this step, our picture is as shown in Figure 3-2.
CM A S R e p o s ito r y (E Y U D R E P )
CAS PARM S ( E Y U IP R M )
P A R M L ib (E Y U P A R M ) W UI R e p o s ito r y (E Y U W R E P )
31
If you are using a shared CSD across your CMAS, Web user interface and LMAS regions, then this job only needs to be run once, otherwise see Table 3-5 for CICS/TS 1.3 or Table 3-6 for CICS/TS 2.2, to decide what programs need to run against the region CSDs.
Table 3-5 CICS/TS 1.3 CSD upgrade programs
Program EYU953G0 Region type CMAS Description Creates CSD group EYU140G0. Creates EYU140L0 grouplist, which contains all the groups from DFHLIST and group EY140G0. Creates CSD group EYU140G1, which you then need to add to an appropriate Web user interface and LMAS grouplist. Creates CSD group EYU140GW, which you then need to add to an appropriate Web user interface grouplist. Creates CSD groups CMZPMSRV and CMZIPSRV, which you then need to add to an appropriate Web user interface grouplist.
WUI and all LMAS regions Web user interface Web user interface
EYU962G1
Web user interface and all LMAS regions Web user interface
CMZ62CSD
Creates CSD groups CMZPMSRV and CMZIPSRV, which you then need to add to an appropriate Web user interface grouplist.
Select the grouplist you wish the CSD definitions to be part of and supply the information in the job for the ADD commands. The relevant CSD groups need to be added to grouplists and included in your SIT parameters. The last action of this job is to ALTER the TCPIPSERVICE to your selected port. This is a different port from that used for the Web user interface server, specified in the TCPIPPORT Web user interface initialization parameter. Submit the tailored job and expect a return code of zero. Example 3-1 on page 33 shows our tailored JCL and parameters for a CICS/TS 2.2 region. Note: CMZ13CSD and CMZ22CSD are only different in the TCPIPSERVICE, as the definitions have changed between CICS/TS 1.3 and CICS/TS 2.2.
32
33
changing @sysid@ to a unique MVS subsystem ID. This should not be predefined in the MVS IEFSSNxx subsystem PARMLIB member. In our setup, we chose to use the sysid of CICSPlex SM. If you are unsure about this, refer to your MVS systems programmer for advice. The command /D SSI is useful to display current SSIDs. The remaining data set high-level qualifiers need to be changed so that @cicshlq@ is set to pick up the CICSPlex SM base code and system files and @userhlq@ is set to reference the EYUIPRM start-up parameters (which, in this case, is an empty data set). Example 3-2 shows the JCL tailored for the CAS. The CAS is started using default options for simplicity. The parameters SPCF=Y and CONVXCF=Y are required to allow CAS-to-CAS communication using MVS cross-system coupling facility (XCF) instead of VTAM. We recommend that you do not alter these or any other parameters regardless of coupling facility availability. Note: We did not enable the security options on our system, so take the necessary steps to enable your CAS to run, that is, add a suitable STARTED general resource class and permissions.
Example 3-2 CAS start-up procedure
//*---------------------------------------------------------------//* Use this procedure to start a CICSPLEX SM CAS //*---------------------------------------------------------------//CMZCAS PROC SSID='CICSPlex SM', // CICSHLQ='CICSTS22', // USERHLQ='CICSSYSF.CICSPM.SC53' //***************************************************************** //* SSID - MVS/ESA subsystem ID //* CICSHLQ - High-level qualifier of CICS TS system libraries //* USERHLQ - High-level qualifier of the user EYUIPRM data set //***************************************************************** //CPSMCAS EXEC PGM=BBM9ZA00, // PARM=('SSID=&SSID,XDM=N', // 'SPCF=Y,CONVXCF=Y', // 'COLD=N,DUMP=ALL'), // TIME=1440, // REGION=4096K //* //STEPLIB DD DISP=SHR, // DSN=&CICSHLQ..CPSM.SEYUAUTH //BBACTDEF DD DISP=SHR, // DSN=&CICSHLQ..CPSM.SEYUADEF //BBSECURE DD DISP=SHR, // DSN=&CICSHLQ..CPSM.SEYUPARM //BBVDEF DD DISP=SHR, // DSN=&CICSHLQ..CPSM.SEYUVDEF //BBIPARM DD DISP=SHR, // DSN=&USERHLQ..EYUIPRM //*
To shut down the CAS at a later stage, issue the MVS STOP command. You would not normally need to do this outside a shutdown for IPL. Example 3-3 shows a normal and successful start-up.
Example 3-3 Successful CAS start using default options
IEF403I SC53CAS - STARTED - TIME=09.52.03 - ASID=00BE - SC53 BBMYAB62I Default system values used for CAS definition BBMZS100I SPCF is active, GROUP=EYUGROUP,MEMBER=SC53 BBMXCL41I Default system values used for target definition BBMXBI30I Security Parameter Modification member BBMTSS00 in BBSECURE used BBMXBI29I Security Class Modification member BBMTSP00 not found in parmlib BBSECURE BBMXBI26I Default security resource properties used BBMXCL36I Default security resource definition used for COMMON resources BBMSS001I Security - ESMTYPE(RACF) SUBSYS(CPSM) REQSTOR(<asis>) APPL(CPSM) BBMZA001I CAS(SC53) SSID(CPSM) INITIALIZATION COMPLETE - R3.3.8 (CPSMvrm) BBMXCL36I Default security resource definition used for PLEXMGR resources
P A R M L ib (E Y U P A R M ) W UI R e p o s ito r y (E Y U W R E P )
35
JCL, or by using the IBM-supplied DFHDEFDS job found in the CICS base library SDFHINST. A change is required to the DFHAUXT data set definition to specify a minimum of 25 cylinders of DASD. Once the data sets are defined correctly, copy procedure CMZCMAST from the SCMZPROC library to your system procedure library, renaming it to your site standard. The copied procedure needs to be edited to point at your load libraries and other data sets. The main differences between the CMAS and a normal CICS region are: The CMAS executes program EYU9XECS instead of DFHSIP. The STEPLIB should only include the APF-authorized libraries SEYUAUTH and SDFHAUTH. The DFHRPL should only contain libraries SEYULOAD and SDFHLOAD. Two DD statements are added; BBACTDEF for the SEYUADEF library, and BBVDEF for the SEYUVDEF library. A DD statement EYULOG for CMAS and managed CICS region logs via a transient data queue is added. The DD statement for EYUPARM, which will contain the CMAS initialization parameters, is added. The CMAS repository data set EYUDREP is included. No DFHSIT initialization table is specifiedthis must be allowed to default. SYSIN is used to specify the CMAS specific values.
36
//* OUTC - PRINT OUTPUT CLASS //* SIP - SUFFIX OF SYSTEM INITIALIZATION PARAMETERS. //* USE 13 FOR CICS TS 1.3 OR 22 for CICS TS 2.2 //* //******************************************************************** //************************** RUN CMAS *************************** //******************************************************************** //CMAS EXEC PGM=EYU9XECS,REGION=®,TIME=1440, // PARM=('START=&START,APPLID=&APPLID,SYSIDNT=&SYSIDNT,SYSIN') //* //SYSIN DD DSN=CICSSYSF.&CMZHLQ..SCMZSAMP(CMZCSI&SIP),DISP=SHR //* The CICS TS STEPLIB //STEPLIB DD DSN=&CICSHLQ..CICS.SDFHAUTH,DISP=SHR // DD DSN=&CICSHLQ..CPSM.SEYUAUTH,DISP=SHR //* The CICS TS LIBRARY (DFHRPL) //DFHRPL DD DSN=&CICSHLQ..CICS.SDFHLOAD,DISP=SHR // DD DSN=&CICSHLQ..CPSM.SEYULOAD,DISP=SHR //* -----------------------------------------//* The CICS SYSTEM DEFINITION data set //DFHCSD DD DISP=SHR,DSN=&CSDHLQ..DFHCSD //* ----------------------------------------------//* The AUXILIARY TEMPORARY STORAGE data set //DFHTEMP DD DISP=SHR, // DSN=&USERHLQ..DFHTEMP //* ---------------------------------//* The INTRAPARTITION data set //DFHINTRA DD DISP=SHR, // DSN=&USERHLQ..DFHINTRA //* -----------------------------------//* The AUXILIARY TRACE data sets //DFHAUXT DD DISP=SHR,DCB=BUFNO=5, // DSN=&USERHLQ..DFHAUXT //DFHBUXT DD DISP=SHR,DCB=BUFNO=5, // DSN=&USERHLQ..DFHBUXT //* --------------------------------//* The CICS CATALOG data sets //DFHLCD DD DISP=SHR, // DSN=&USERHLQ..DFHLCD //DFHGCD DD DISP=SHR, // DSN=&USERHLQ..DFHGCD //* -----------------------------------//* CICS EXTRAPARTITION data sets //DFHCXRF DD SYSOUT=&OUTC //MSGUSR DD SYSOUT=&OUTC,DCB=(DSORG=PS,RECFM=V,BLKSIZE=136) //* CICS TRANSACTION DUMP data sets A & B //DFHDMPA DD DISP=SHR, // DSN=&USERHLQ..DFHDMPA //DFHDMPB DD DISP=SHR, // DSN=&USERHLQ..DFHDMPB //* ---------------------------------------//* EXTRAPARTITION data sets for CPSM //* CPSM Parameters: COPR TDQ //EYUPARM DD DISP=SHR, // DSN=&CPSMHLQ..EYUPARM(CMZCMASP) //* CPSM: COLG TDQ for CPSM LOG messages //EYULOG DD SYSOUT=* //* The CMAS REPOSITORY data set //EYUDREP DD DISP=SHR, // DSN=&CPSMHLQ..EYUDREP //*
37
38
If you do not use the default CICS SVC number, change @nnn@ to reflect the SVC number you do use. If you do use the default, comment out or remove this parameter. If you are using an external security package, change @userid@ to specify your site default userid, otherwise this can be commented out or removed. Also, if you are not using an external security package, set SEC=NO. You need to add IBM-supplied CSD group DFHTERMC to a CMAS grouplist and change @userlist@ in the GRPLIST parameter. The DFHTERMC group is required for the autoinstall console model. An example of the SYSIN parameters can be seen in Example 3-6.
Example 3-6 CMAS SIT SYSIN parameters
* ********************************************************************* * CICS TS 2.2 CMAS system initialization overrides for the default SIT* * Replace the following : * * @nnn@ - CICS SVC number * * @userid@ - CICS default userid * * @userlist@ - Group list name for the CMAS * * ********************************************************************* * APPLID=is provided in the JCL PARM parameter AICONS=AUTO Autoinstall for consoles AUXTR=ON Auxiliary trace on AUXTRSW=NEXT Switch auxiliary trace data sets once *ICSSVC=@nnn@ The CICS SVC number CPSMCONN=CMAS Initialize region as a CPSM CMAS *FLTUSER=@userid@ CICS default userid DUMPDS=A DUMPSW=NEXT EDSALIM=100M Initial value (to be reviewed over time) GRPLIST=(EYU220L0,CMA1LIST) GMTEXT='CICSPLEX System Manager (CMAS) address space' ICV=100 Region exit time interval. ICVR=20000 Runaway task interval ICVTSD=1 Terminal scan delay interval IRCSTRT=YES Start interregion communication ISC=YES Include the intersystem communication progrm MAXSOCKETS=1 Sockets support not required: 1 is minimum valu MXT=300 Set maximum tasks to 300 for CMAS RENTPGM=PROTECT Allocate ERDSA from read-only storage SEC=NO Initialize CICS security domain SPOOL=YES Start JES spool support SSLTCBS=0 No SSL support SUBTSKS=1 Subtasking yes * SYSIDNT is provided in the JCL PARM parameter SYSTR=OFF Master trace switch off TRTABSZ=2048 Size of internal trace table TS=COLD Cold start temporary storage WRKAREA=1024 1KB CWA required by CPSM CMAS XCMD=NO ! XDB2=NO No XDCT=NO security XEJB=NO on XFCT=NO any XJCT=NO of XPCT=NO these XPPT=NO resources XPSB=NO although XTST=NO security XTRAN=NO is
39
XUSER=NO .END
ON !
We have found that when changes are made to any of the CICSPlex SM CMAS SYSIN parameters, a COLD start is always required. Otherwise, other than the INITIAL start, an AUTO start is preferred. Failure to do so may produce unpredictable results. After this step, our setup looks as in Figure 3-4 on page 41.
40
CM AS SCSCCM A1
41
//********************************************************************* //* * //* @BANNER_START 01 * //* Licensed Materials - Property of IBM * //* * //* 5655-I46 * //* * //* (C) Copyright IBM Corp. 2002 * //* * //* * //* CICS Performance Monitor for z/OS, Version 1 Release 1 * //* * //* @BANNER_END * //* * //* STATUS = 1.1.0 * //* * //* CHANGE ACTIVITY : * //* * //* $MOD(CMZBTCH) COMP(INSTALL) PROD(CICSPM ): * //* * //* PN= REASON REL YYMMDD HDXXIII : REMARKS * //* $P0= D05139 110 021002 HD4HAPF : Installation enhancements * //* $D0= I07279 110 020808 HD4HAPF : Creation * //* $P1= D05105 110 020926 HD3SCWG : Add REXX Alternate Library * //* $P2= D05177 110 021018 HDGGDEW : More starter Set changes * //* $P3= D05621 110 021024 HDGOGJS : Add SYSMDUMP * //* * //********************************************************************* //* //*-------------------------------------------------------------------//* Replace the following : //* @cicshlq@ - High-level qualifier of CICS TS system libraries //* @cmzhlq@ - High-level qualifier of CICS PM system libraries //* @actinfo@ - Accounting information //* @userid@ - Job userid //* @parm@ - CPSM release number //* @cmasname@ - CMAS name //* @plexname@ - CICSPlex name //* @ctsver@ - two digit CICS TS version: e.g 13 for CICS TS 1.3 //* 22 for CICS TS 2.2 etc //* @regnam1@ - Region name of a CICS system within the CICSPlex //* @applid1@ - Applid of a CICS system within the CICSPlex //* @sys1@ - Sysid of a CICS system within the CICSPlex //* @group1@ - Group name //*-------------------------------------------------------------------//*------------------------------------* //* Perform CPSM Object Definitions * //*------------------------------------* //* Specify the CPSM release number in the PARM string. For example //* PARM='0140' for CICS TS 1.3 & PARM='0220' for CICS TS 2.2 //DEFINE1 EXEC PGM=CMZBATCH,PARM='0220' //* //STEPLIB DD DISP=SHR,DSN=CMZ.SCMZLOAD // DD DISP=SHR,DSN=CICSTS22.CPSM.SEYUAUTH //* DD DISP=SHR,DSN=CMZ.SEAGALT //* Un-comment the above line to use the REXX Alternate Library only //* if you do not have the REXX/370 runtime environment installed //* @P1A //* //SYSTSPRT DD SYSOUT=*
42
//EYUTRCF DD SYSOUT=* //* //*SYSMDUMP DD DISP=(,CATLG),DSN=@userid@.CMZBTCH.SYSMDUMP, //* UNIT=SYSDA,SPACE=(CYL,(50,25)), //* DCB=(DSORG=PS,RECFM=FBS,LRECL=4160,BLKSIZE=24960) //* Un-comment the above lines if you need to add a SYSMDUMP DD. //* You may need to modify this statement to meet your //* installations requirements @P3A //* //CPLEXDEF DD DISP=SHR,DSN=CMZ.SCMZPARM(CMZCPDEF) //CSYSDEF DD DISP=SHR,DSN=CMZ.SCMZPARM(CMZCSDEF) //CSYSGRP DD DISP=SHR,DSN=CMZ.SCMZPARM(CMZCSGRP) //CMZSTDEF DD DISP=SHR,DSN=CMZ.SCMZPARM(CMZIMP22) //* //CMZDEFS DD *
If you are using your own copied jobcard, @actinfo@ and @userid@ can be ignored. @parm@ should reflect your CICSPlex SM release, that is, 0140 = CICS/TS 1.3 and 0220 = CICS/TS 2.2. If you do not have the REXX/370 run-time environment installed, uncomment the SEAGALT library in the STEPLIB. This is because the CMZBATCH program is compiled REXX.
43
12.List the CICSplex structure. Example 3-9 shows our tailored definition statements.
Example 3-9 Sample tailored definition statements
* * First set the Context to the CMAS: CONTEXT SCSCCMA1 * * and then define a new CICSPlex to that CMAS: DEFINE CICSPLEX SC53PLEX * * Now reset the context to the new CICSPlex: CONTEXT SC53PLEX * * and then define the Name, Applid, Sysid and connecting CMAS * for each region to be managed under the CICSPlex: DEFINE REGION SCSCMMA1 APPLID SCSCMMA1 SYSID MMA1 CMASID SCSCCMA1 * *DEFINE REGION @regnam2@ * APPLID @applid2@ * SYSID @sys2@ * CMASID SCSCCMA1 * *DEFINE REGION @regnam3@ * APPLID @applid3@ * SYSID @sys3@ * CMASID SCSCCMA1 * * Now define a CICS Group to associate these Regions collectively: DEFINE CICSGRP SC53GRP1 * * and then ADD each Region to the new group: ADD TOGROUP SC53GRP1 REGION SCSCMMA1 * *DD TOGROUP @group1@ * REGION @regnam2@ * *DD TOGROUP @group1@ * REGION @regnam3@ * * Now repeat these definitions for another Region/Group set: DEFINE REGION SCSCWMA1 APPLID SCSCWMA1 SYSID WMA1 CMASID SCSCCMA1 * *DEFINE REGION @regnam5@ * APPLID @applid5@ * SYSID @sys5@ * CMASID SCSCCMA1 * *DEFINE REGION @regnam6@ * APPLID @applid6@ * SYSID @sys6@ * CMASID SCSCCMA1
44
* DEFINE CICSGRP SC53GRP2 * * and then ADD each of these Regions to the second group: ADD TOGROUP SC53GRP2 REGION SCSCWMA1 * *DD TOGROUP @group2@ * REGION @regnam5@ * *DD TOGROUP @group2@ * REGION @regnam6@ * * Now define another group to scope all regions in the CICSPlex: DEFINE CICSGRP SC53GALL * * and add the previously defined CICS Groups to it. ADD TOGROUP SC53GALL CICSGRP SC53GRP1 * ADD TOGROUP SC53GALL CICSGRP SC53GRP2 * * Finally, print all definitions associated with * the current Context ( CICSPlex @plexname@). PRINT * * /* //
Once the CMZBATCH job has been tailored, submit it and expect a zero return code. More information about CMZBATCH can be found in the Performance Monitor Users Guide. After completing the CMZBATCH job, our setup looks as shown in Figure 3-5.
45
CM AS SCSCCM A1 SC 53PLEX
P A R M L ib (E Y U P A R M ) W UI R e p o s ito r y (E Y U W R E P ) C M ZC M A SP
C M Z W U IP M C M Z W U IIN
46
variables carefully (remember this is a sample job). Example 3-10 shows our edited start-up procedure.
Example 3-10 WUI procedure
//CMZWUIT PROC START=AUTO, // USERHLQ='CICSSYSF.CICS620.WMA1', // CPSMHLQ='CICSSYSF.CICSPM.SC53', // CSDHLQ='CICSSYSF.CICSTS22.XMA1', // CICSHLQ='CICSTS22', // CMZHLQ='CMZ', // SYSIDNT='WMA1', // APPLID='SCSCWMA1', // REG='150M', // OUTC='*', // SIP=22 //******************************************************************** //* USERHLQ - HIGH-LEVEL QUALIFIER OF USER DEFINED CICS RUN TIME //* DATA SETS //* CICSHLQ - HIGH-LEVEL QUALIFIER OF CICS TS SYSTEM LIBRARIES //* CMZHLQ - HIGH-LEVEL QUALIFIER OF CICS PM SYSTEM LIBRARIES //* SYSDINT - SYSID OF CICS REGION AND 2ND HALF OF APPLID //* REG - STORAGE REQUIRED //* START - TYPE OF CICS START-UP REQUIRED //* OUTC - PRINT OUTPUT CLASS //* SIP - SUFFIX OF SYSTEM INITIALIZATION PARAMETERS. //* USE 13 FOR CICS TS 1.3 OR 22 for CICS TS 2.2 //* //******************************************************************** //************** Run CICSPlex SM Web User Interface server *********** //******************************************************************** //CWUI EXEC PGM=DFHSIP,REGION=®,TIME=1440, // PARM=('START=&START,APPLID=&APPLID,SYSIDNT=&SYSIDNT,SYSIN') //* //SYSIN DD DSN=CICSSYSF.&CMZHLQ..SCMZSAMP(CMZWSI&SIP),DISP=SHR //* The CICS STEPLIB //STEPLIB DD DSN=&CICSHLQ..CICS.SDFHAUTH,DISP=SHR // DD DSN=&CICSHLQ..CPSM.SEYUAUTH,DISP=SHR //* The CICS LIBRARY (DFHRPL) //* Include your private load library //DFHRPL DD DSN=CICSSYSF.CICSPM.SC53.LOAD,DISP=SHR // DD DSN=&CICSHLQ..CICS.SDFHLOAD,DISP=SHR // DD DSN=&CICSHLQ..CPSM.SEYULOAD,DISP=SHR // DD DSN=&CMZHLQ..SCMZLOAD,DISP=SHR //* -----------------------------------------//* The CICS system definition data set //DFHCSD DD DISP=SHR,DSN=&CSDHLQ..DFHCSD //* ----------------------------------------------//* The AUXILIARY TEMPORARY STORAGE data set //DFHTEMP DD DISP=SHR, // DSN=&USERHLQ..DFHTEMP //* ---------------------------------//* The INTRAPARTITION data set //DFHINTRA DD DISP=SHR, // DSN=&USERHLQ..DFHINTRA //* -----------------------------------//* The AUXILIARY TRACE data sets //DFHAUXT DD DISP=SHR,DCB=BUFNO=5, // DSN=&USERHLQ..DFHAUXT //DFHBUXT DD DISP=SHR,DCB=BUFNO=5, // DSN=&USERHLQ..DFHBUXT
47
//* --------------------------------//* The CICS CATALOG data sets //DFHLCD DD DISP=SHR, // DSN=&USERHLQ..DFHLCD //DFHGCD DD DISP=SHR, // DSN=&USERHLQ..DFHGCD //* ---------------------------------//* The CICS EXTRAPARTITION data sets //DFHCXRF DD SYSOUT=&OUTC //MSGUSR DD SYSOUT=&OUTC,DCB=(DSORG=PS,RECFM=V,BLKSIZE=136) //* -----------------------------------//* CICS TRANSACTION DUMP data sets A & B //* CICS TRANSACTION DUMP data sets A & B //DFHDMPA DD DISP=SHR, // DSN=&USERHLQ..DFHDMPA //DFHDMPB DD DISP=SHR, // DSN=&USERHLQ..DFHDMPB //* The Web User Interface REPOSITORY data set //EYUWREP DD DISP=SHR, // DSN=&CPSMHLQ..EYUWREP //* EXTRAPARTITION data sets for CPSM //* CPSM Parameters: COPR TDQ //EYUPARM DD DISP=SHR, // DSN=&CPSMHLQ..EYUPARM(CMZWUIPM) //* WEB User Interface: COVP TDQ //EYUWUI DD DISP=SHR, // DSN=&CPSMHLQ..EYUPARM(CMZWUIIN) //* WEB User Interface: COVI TDQ for IMPORT //EYUCOVI DD DISP=SHR, // DSN=&CMZHLQ..SCMZVIEW(CMZEVIEW) //* CPSM: COLG TDQ for CPSM LOG messages //EYULOG DD SYSOUT=* //* END OF CICS START PROCEDURE
48
* CICS Performance Monitor for z/OS, Version 1 Release 1 * * * * @BANNER_END * * * * STATUS = 1.1.0 * * * * FUNCTION = * * This module is a sample DFHCNV for CICS PM. * * * *-------------------------------------------------------------------- * * * * ENTRY POINT = CMZCVW$ * * * *-------------------------------------------------------------------- * * * * CHANGE ACTIVITY : * * * * $MOD(CMZCVW$),COMP(GEMSTONE),PROD(CICSPM ): * * * * PN= REASON REL YYMMDD HDXXIII : REMARKS * * $D0= I07279 110 020806 HD4HAPF : Gemstone Samples * * $P1= D05051 110 020923 HDGOGJS : Remove CSECT line from CMZCVW$ * * $P2= D05102 110 020926 HD2GJST : Remove NACT02 and NACT05 entries* * * *********************************************************************** DFHCNV TYPE=INITIAL * DFHCNV TYPE=ENTRY,RTYPE=PC,RNAME=DFHWBHH,USREXIT=NO, SRVERCP=037,CLINTCP=8859-1 DFHCNV TYPE=SELECT,OPTION=DEFAULT DFHCNV TYPE=FIELD,OFFSET=0,DATATYP=CHARACTER,DATALEN=32767, * LAST=YES * DFHCNV TYPE=ENTRY,RTYPE=PC,RNAME=DFHWBUD,USREXIT=NO, * CLINTCP=8859-1,SRVERCP=037 DFHCNV TYPE=SELECT,OPTION=DEFAULT DFHCNV TYPE=FIELD,OFFSET=0,DATATYP=CHARACTER,DATALEN=32767, * LAST=YES * COPY EYU$CNV1 * DFHCNV TYPE=FINAL END
Example 3-12 shows our assemble job. The DFHAUPLE procedure is supplied as part of the CICS/TS 2.2 base with modifications for data set HLQs.
Example 3-12 Assembler JCL
//DFHCNVCW JOB (999,POK),'L06R',CLASS=A,REGION=4096K, // MSGCLASS=T,TIME=10,MSGLEVEL=(1,1),NOTIFY=&SYSUID //* //* //* //* //* /*JOBPARM SYSAFF=SC53 //PLEASE EXEC DFHAUPLE,INDEX='CICSTS22.CICS' //ASSEM.SYSUT1 DD DSN=CICSSYSF.CMZ.SCMZSAMP(CMZCVW$),DISP=SHR //ASM.SYSLIB DD DSN=CICSTS22.CICS.SDFHMAC,DISP=SHR
49
@BA29831
Further information on page codes can be found in the CICS Transaction Server for z/OS Communicating from CICS on System/390.
50
CPSMCONN=WUI Initialize the regions as a CPSM Web server CWAKEY=CICS *FLTUSER=@userid@ CICS default userid DSALIM=7M EDSALIM=128M GRPLIST=(DFHLIST,WMA1LIST) Initialize with CICS & CPSM groups GMTEXT='CICSPlex SM Web User Interface Server' INITPARM=(EYU9VKEC='ENU',EYU9VWAN='ENU1') ISC=YES MAXSOCKETS=255 MXT=300 *EC=YES Run with security SEC=NO Run without security for SC53 * SYSIDNT is provided in the JCL PARM parameter of CWUITASK SYSTR=OFF TCPIP=YES USERTR=ON WRKAREA=1024 XCMD=NO ! XDB2=NO No XDCT=NO security XEJB=NO on XFCT=NO any XJCT=NO of XPCT=NO these XPPT=NO resources XPSB=NO although XTST=NO security XTRAN=NO is XUSER=NO ON ! .END
51
* * Note: Add a NAME(@regionname@) parameter if the REGION name is NOT * the same as the APPLID
Following are details about the parameters in Example 3-15: DEFAULTMENU(IBM_CMZMENU) This parameter specifies the default menu presented to the Web user interface for CICS/PM. This can be changed to (IBM_CMZMAINMENU) to give both CPSM and CICS/PM view sets if AUTOIMPORTTDQ is set to import them. DEFAULTNAVIGATE(IBM_CMZNAVIGATE) This is the navigate frame name in the Web user interface for CICS/PM. DEFAULTCMASCTXT(SCSCCMA1) This must be the name of the CMAS that the Web user interface server connects to. If it is not, then CICS PM will not work. We used SCSCCMA1. DEFAULTCONTEXT(SC53PLEX) Specify the CICSplex name you wish to be set for CONTEXT, which can be changed at any time once CICS/PM is active.
52
DEFAULTSCOPE(SC53PLEX) Specify the default scope for CICS/PM. This can be a plex, a group, or an individual CICS region. The scope can be changed (depending on what you need to view) once CICS/PM is active. TCPIPHOSTNAME(WTSC53.ITSO.IBM.COM) This is the address of your System/390 host that is used by your Web browser to contact the Web user interface. TCPIPPORT(9001) This is the port number that is to be used when contacting your System/390 host. AUTOIMPORTTDQ(COVI) This is the name of the CICS TDQ that the CICS/PM view sets are to be imported from to populate the Web user interface repository. The default is (COVI), which, through the CSD TDQ definitions, reads from the EYUCOVI DD statement of the Web user interface procedure. We recommend that you only use this parameter for the following reasons: Initial start of your Web user interface to import view sets. After applying maintenance to the view sets to refresh them, that is, after applying PTFs. Importing the CICSPlex SM view starter sets from the CICSPlex SM base library SEYUVIEW where there are three members: EYUEVX01 for US English, EYUKVX01 for Japanese, and EYUSVX01 for simplified Chinese. It is best to avoid leaving this parameter in, because the view sets will be loaded each time the Web user interface is started, resulting in delayed initialization. Another method of importing the view sets is using CICS transaction COVC.
53
EYUVS0109I EYUVS0109I EYUVS0109I EYUVS0109I EYUVS0109I EYUVS0109I EYUVS0109I EYUVS0108I EYUVS0101I EYUVS0200I EYUVS0204I EYUVS0206I EYUVS1016I (COVI). EYUVS1018I EYUVS0002I EYUVS0010I EYUVS0020I EYUVS0020I
SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1 SCSCWMA1
DEFAULTNAVIGATE(IBM_CMZNAVIGATE) DEFAULTCMASCTXT(SCSCCMA1) DEFAULTCONTEXT(SC53PLEX) DEFAULTSCOPE(SC53PLEX) TCPIPHOSTNAME(WTSC53.ITSO.IBM.COM) TCPIPPORT(9001) AUTOIMPORTTDQ(COVI) STARTUP PARAMETERS READ. Parameter service initialization complete. Starting CICS Web Interface. TCP/IP service (EYUWUI) installed successfully. CICS Web Interface enabled on TCP/IP port number 9001. Import 'ALL (OVERWRITE)' initiated for user (CICSUSER) from TDQ Import completed successfully. 95 objects read from TDQ (COVI). CICSPlex SM Web User Interface initialization complete. Server connected to CMAS, SYSID(CMA1). Task initialization complete for User (CICSRS1). Task initialization complete for User (CICSRS2).
We found that when any changes are made to the CICSPlex SM Web user interface sysin parameters, a COLD start is always required; otherwise, other than the INITIAL start, an AUTO start is preferred. Failure to do so may produce unpredictable results. Figure 3-6 on page 55 shows our setup after the WUI start.
54
CM AS SCSCCM A1 SC 53PLEX
P A R M L ib (E Y U P A R M ) W UI R e p o s ito r y (E Y U W R E P ) C M ZC M A SP
C M Z W U IP M C M Z W U IIN
55
56
This step needs to be repeated for each CICS region you wish to monitor with CICS/PM. Our setup now looks like Figure 3-7 on page 58. We added CICS regions B, C and D just to show potential application groupings.
57
CM AS SCSCCM A1 SC 53PLEX
59
60
Chapter 4.
61
4.1 Upgrade your Web user interface region CSD with the CICS PM resources
The CSD used by your CICSPlex Web user interface region must be updated to include the new CICS PM resources. CICS PM provides you with a sample job called CMZCSDUG, located in the SCMZINST data set, which will define the resources needed to build and set up a new CICSplex. This job can be used to add the CICS PM components to your existing CICSPlex Web user interface region s CSD. At this point you need to decide what TCP/IP Service port number CICS PM will use. A good idea is to choose a number similar to the port that is currently in use by your CICSPlex Web user interface server. Once you have a port number, edit and run job CMZCSDUG. The job contains editing instructions in the JCL comments that you should follow, but remember you also have to remove the extra CICSPlex EYU* and DFH* statements, keeping only the CICS PM CMZ* resources, and set your CICS PM Port number. The job will import the required CICS PM resources and update your existing Web user interface regions startup list with the new CICS PM resources. Example 4-1 shows you the SYSIN input you need.
Example 4-1 Input statements required to run CMZCSDUG for CICS TS 2.2
//SYSIN DD * * Use these statement for CICS TS 2.2 UPGRADE USING (CMZ22CSD) ADD GROUP(CMZPMSRV) LIST(PJA2LIST) ADD GROUP(CMZIPSRV) LIST(PJA2LIST) * Alter the port number for the CICS PM server: ALTER TCPIPSERVICE(CMZTCPIP) GROUP(CMZIPSRV) PORTNUMBER(9011) /*
After running CMZCSDUG and installing the new groups, you end up with two ports, one for CICSPlex under EYUWUI and one for CICS PM under CMZTCPIP.
Example 4-2 WUI region display of the required TCPIPService definitions
I TCPIPS STATUS: RESULTS - OVERTYPE TO MODIFY Tcpips(CMZSRVIP) Bac( 00010 ) Con(0000) Por(09011) Http Nos Ope Tra(CWXN) Urm( CMZANAL ) Ipa(9.12.6.29 ) Wai Tcpips(EYUWUI ) Bac( 00100 ) Con(0000) Por(09001) Http Nos Ope Tra(CWXN) Urm( EYU9VWAN ) Ipa(9.12.6.29 ) Tim
Note: When your Web user interface region starts up it will automatically define and install the EYUWUI TCP/IP Service definitions. You should also see the following resources in your Web user interface region with the two groups now in your Web user interface region s startup group list.
Example 4-3 CICS PM Resources installed by the CMZCSDUG job
E G(CMZ*) ENTER COMMANDS NAME TYPE GROUP CMZSRVIP TCPIPSERVICE CMZIPSRV
62
CMZANAL CMZCEMT CMZEVAL CMZEVLD CMZINIT CMZLVAL CMZNGEN CMZRACT CMZRRPR CMZRTDP CMZSRVR CMZSTRM CMZTREY CMZ1 CMZL
PROGRAM PROGRAM PROGRAM PROGRAM PROGRAM PROGRAM PROGRAM PROGRAM PROGRAM PROGRAM PROGRAM PROGRAM PROGRAM TRANSACTION TDQUEUE
CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV CMZPMSRV
Example 4-4 CICS PM Groups installed in the WUI region group list
E L(PJA2LIST) ENTER COMMANDS NAME TYPE ITSOEJB GROUP PJA2IIOP GROUP C52CMAC GROUP DFHEJVS GROUP PJA2GRP GROUP PJAAI62 GROUP EYU220GW GROUP TRADERBL GROUP CMZPMSRV GROUP CMZIPSRV GROUP
LIST PJA2LIST PJA2LIST PJA2LIST PJA2LIST PJA2LIST PJA2LIST PJA2LIST PJA2LIST PJA2LIST PJA2LIST
4.2 Import the CICS PM threshold definitions into your CMAS repository
CICS PM provides you with a sample job called CMZBTCH to create the sample CMZBASE definitions that you can use/copy in the GUI. You need to import the CMZBASE definitions for each CICSplex that you want to use them in. What is needed is to edit that job, removing all the extra statements so that you only add the CICS PM components to your existing CICSPlex repository. Edit and run job CMZBTCH from your SCMZINST data set. Edit the job according to the instructions in the JCL comments, making sure to set the CONTEXT parm to your CICSplex. Example 4-5 shows a list of the input statements you need.
Example 4-5 Input statements required to run CMZBTCH
//CMZDEFS DD * * * First set the Context to the CMAS: CONTEXT SC66PLEX * OPTION DUPREC UPDATE IMPORT CMZSTDEF * * *
63
* Finally, print all definitions associated with * the current Context (i.e. CICSPlex plexname). CONTEXT SC66PLEX PRINT RTADEF * PRINT EVALDEF * PRINT ACTNDEF * /*
c. Restart your Web user interface region. d. Use the COVC transaction option 5 to import all menus and views using the COVI TDQ. This step copies the CICS PM views and menus from the data set pointed to by the COVI TDQ, which should be SCMZVIEW, into your CICSPlex Web user interface repository data set, which should be EYUWREP.
64
COVC
EYUVCTI
Name of extrapartition TDQ for import Menu | Viewset | All Skip | Overwrite | DELETE
Note: The import operation could take several minutes. Also, trying to browse the SCMZVIEW data set will lock up your session for 5 or 10 minutes.
65
66
Chapter 5.
67
68
C:\>
Menu List Mode Functions Utilities Help ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss ISPF Command Shell Enter TSO or Workstation commands below: ===>
Place cursor on choice and press enter to Retrieve command => IND$FILE GET 'CMZ.SCMZDWLD(CMZEMW32)' => => => => => =>
Figure 5-1 Your host screen session must be in Command mode via ISPF option 6
C:\>receive cmzsetup.exe a:'cmz.scmzdwld(cmzemw32)' The file transfer request is being processed. Number of bytes transferred : 19167520 File transfer complete. C:\>
69
Figure 5-3 Initial install screen from running CMZSETUP.EXE; select Next
70
71
Open your preferred Web browser and select your existing CICSPlex Web user interface URL, as long as you tailored your CICSPlex EYUWUI parameters correctly to point to CICS PM. See 4.3, Add CICS PM support to your Web user interface server on page 64 if you are still connecting to your CICSPlex Web user interface instead of CICS PM.
Figure 5-7 Start off connecting to your existing CICSPlex WUI URL
Note: If the Web user interface is running with SEC=YES in the SIT, then the signon screen shown in Figure 5-8 will be different. Also, when you are running your Web user interface with SEC=NO in the SIT, any non-blank value may be specified as the value for your userid. It does not need to be defined to an external security manager (ESM), or be the same as your TSO userid.
72
Figure 5-9 The CICSPlex WUI screens take you to the CICS PM main screen
Chapter 7, The CICSPlex SM Web User Interface on page 121 discusses using the CICS PM Web user interface in detail. 2. Connecting to CICS PM via your workstation client software Using your Start menu, locate the client software. It should be under Start -> Programs -> IBM CICS Performance Monitor -> IBM CICS Performance Monitor.
73
Once the product starts, it asks you for your CICS Performance Monitor Server address and port. The server address is the address that is entered in the TCPIPHOSTNAME field pointed to by the EYUWUI card in your Web user interface region JCL. The port is the port number you chose for CICS PM (see Figure 4-1, COVC transaction option 5 on page 65). Now you will be asked for your Userid and Password, then select OK. You should get a Topology Refreshing window and after that completes, you should see Connected in the bottom left corner of the CICS PM. The IP address and port number you enter are the values that you specified on the CMZIPSRV tcpipservice. Note: The CICS PM product only asks for the CICS PM server address, but under the covers it locates the EYUWUI port to allow communication with the CICSPlex Web user interface. See Chapter 6, GUI interface on page 75 for detailed information about using the CICS PM GUI.
74
Chapter 6.
GUI interface
This chapter provides a description of the CICS PM workstation client. The workstation client is a Java application that provides a graphical user interface on a Windows workstation. The workstation client connects to a CICSPlex SM Web User Interface (WUI) server, and displays information about all the CICS regions in the CICSplexes to which the server has access.
75
Connect dialog
CICS PM requires a user ID to connect to, and maintain a single session with, a CICSPlex SM Web User Interface (WUI) server. CICS PM polls the server regularly to check for new events, and the frequency of this polling activity is normally enough to keep the session open. If the Web User Interface server is running with security enabled, the password associated with the user ID is also needed to complete the connection. The dialog used to connect the CICS PM client with the Web User Interface server is illustrated in Figure 6-2, CICS PM Connect dialog box on page 77.
76
The information needed to complete the Connect dialog is as follows: Host Name The host name of the MVS system on which the CICS PM server is running. Since the CICS PM server must run in a Web User Interface server CICS region, the name is the same as the TCPIPHOSTNAME initialization parameter in the Web user interface region. Port The port number specified in the TCPIPSERVICE definition for the CICS PM host server application. This is the port number specified in the TCPIPSERVICE definition in the CMZIPSRV CSD group. User ID A user ID as defined to the external security manager on the host. Fore example, it could be a TSO user ID. Password The password associated with the specified user ID. It is only required if the Web user interface server is running with SEC=YES. Change Password Click to change the password associated with the user ID in the external security manager. Reconnect Reconnect to the host if the specified user ID is already logged on to the host.
77
78
79
Status information is as follows: Connection and topology state The value shown in the first field is one of the following: No connection The workstation client is not connected to the CICSPlex SM Web User Interface server, and the event monitor window has no content. Building topology The workstation client is connected to the CICSPlex SM Web User Interface server and is obtaining topology information. The messages following the Building topology statement indicate the progress of the task. This transient state is in effect while the topology tree is being built. This status is displayed when the Refresh item on the Topology menu is clicked. Connected The workstation client is connected to the CICSPlex SM Web User Interface server. This is the normal operating status for the CICS PM workstation client. PM host and port number The host name of the CICSPlex SM Web User Interface server, and in parentheses the TCP/IP designation followed by the port number for the CICS PM connection, and the port number for the CICSPlex SM Web User Interface connection.
80
81
The monitor window consists of three panes. The proportions of each of these panes can be altered by dragging the pane boundaries as required. When a pane is too small to show the full content, the appropriate horizontal and vertical scroll bars appear.
82
The topology tree structure objects are navigated with standard navigation actions to expand and collapse nodes on the tree. There is a Topology menu item (see Figure 6-10) that has options that control the appearance of the topology tree, an option that refreshes the topology tree, and an option that launches the Web user interface. The underlined letter of the menu selection represents the shortcut to that item.
The eight menu options are: Refresh Refreshes the topology tree with the current set of active regions. This need only be done when a change has been made to one of the CICSplex configurations. When CICS PM polls CICSPlex SM, it does not include topology information. Collapse All Collapse all nodes of the topology tree and show CICSplexes only.
83
Expand All Expands all the nodes of the topology tree down to the lowest level, showing all CICS system groups and all members of the groups. Expand to Events Expands the topology tree, but only the nodes leading to regions against which events are currently outstanding. Individual regions are shown or not shown according to the current selection of the Show Regions option. Default Selection Resets the topology scope to the default scope, which is the first CICSplex in the tree. Show OK Icon When selected, a checkmark appears next to this selection. Also, the normal status icon shows next to regions that are in normal status. The region s appearance in the topology pane is subject to the status of the Show Regions selection. Show Regions When selected, a checkmark appears next to this selection. Also, all CICS regions in the CICSplexes are included in the topology pane. When not selected, only CICSplexes and CICS system groups show in the topology pane. Launch WUI When selected, CICS PM launches a browser window holding the CICS PM main menu for interfacing with the CICSPlex SM Web User Interface server. See Figure 6-11.
When a node is selected, that node delineates the scope that controls the source of information on the other panes in the Event Monitor window. Each node in the tree has a pair
84
of associated icons that tell the type and the status of the node. The topology type icons are explained in Table 6-1; the status icons are explained in Table 6-2.
Table 6-1 The types of nodes in a topology tree and their icons
Topology icon Name of object defined to CICSPlex SM The name of the CICSplex Description A CICSplex. If there are any current events within the CICSplex, the status icon reflects the highest level of severity in the CICSplex. A CICS system group. The status icon reflects the highest level of severity in the group.
A group for CICS regions that are not defined to a specific system group.
A CICS region. The status icon shows the highest level of severity in the region.
Warning. There is at least one current event that has been triggered by a threshold defined as warning or HW.
Severe. There is at least one current event that has been triggered by a threshold defined as severe or HS.
Very Severe. There is at least one current event that has been triggered by a threshold defined as very severe or VHS.
85
The Resources summary view contains sets of push buttons organized in arrays of three for each type of resource. The sets are in logical groups of resource type, such as activity, files and DB2, and so on. Each array relates to one of the resource types against which you can define an event threshold. The three buttons in an array are identified as follows: The left-most button is the subject button, shown in Figure 6-13.
Resource name
Figure 6-13 Subject button
This button has the name of the resource monitored and is a hypertext link to the CICSPlex SM Web user interface. Click this button to open the Web browser resource table page for the resource (Figure 6-14 on page 87). The browser shows instances of the resource for the current scope only. When there can be only one instance of the named resource (such as a DB2 connection), the details page is shown.
86
This function is equivalent to choosing the Event List -> Link To -> Resource Type menu option, or to clicking the Resource field in a row of events. However, clicking the subject button gives a direct link to the relevant browser page even when there is no current event against the resource. The center button is the region histories button, shown in Figure 6-15.
This button is a hypertext link to another CICS PM pane. Click this button to open the CICS region histories view. See Figure 6-16 on page 88. When this button is clicked, the status pane changes to show a region history view for the resource located on the node highlighted in the topology pane. If the highlighted node is a CICSplex or a CICS system group, a region history view is shown for all of the CICS system groups or CICS regions assigned. Also, when this button is clicked, the event list pane shows all events triggered for that resource within the scope highlighted on the topology pane. Event severities shown in the region summary correspond to the highest severity event currently outstanding against the resource.
87
Figure 6-16 Region histories view as a result of clicking the region histories button
This button holds an icon that indicates the status of the resource. The status is as found in Table 6-2 on page 85. This event button also acts as a filter for the event list. For example, if the event status line shows the message: Status for scope: CICSplex Filter: Tasks and the event button for Local Files is clicked, the status line message changes to: Status for scope: CICSplex Filter: Local Files and only local files events are displayed in the event list. Regions histories view This is a graphical view of the status history of the regions within the current scope. See Figure 6-18 on page 89. The status history is maintained for the past 30 minutes, displayed in six five-minute periods using the status icons as found in Table 6-2 on page 85. If no events occur in a five-minute period, the normal status icon is shown for that period. If events do occur within a five-minute period, the icon shows the highest level event triggered.
88
Note: A normal status history does not necessarily mean that no events occurred during the history period, only that CICS PM did not collect any reported events for that period. CICS PM can only report events that are active when CICS PM polls CICSPlex SM to collect details of all currently active events. Therefore, it is possible for an event detected and recorded by CICSPlex SM to be cleared before CICS PM calls CICSPlex SM at one of its regular polling intervals. In this case, CICS PM does not report the event because the event was no longer active at the time CICS PM polled CICSPlex SM.
The region histories pane has two sections. The top section contains the history of the item selected. When the selection is a CICSplex or CICS system group, an overall history view of the selected item is shown in the top section and the bottom section shows an overall history of subordinate nodes. When the selection is a CICSplex, the subordinate nodes are the CICS system groups assigned to the CICSplex and a node named Other systems that covers all CICS regions in the CICSplex not assigned to a CICS system group. When the selection is a CICS system group, the subordinate nodes are the CICS regions assigned to the group. When the selection is a CICS region, the bottom section is blank. The scope of information shown in the region histories view is shown in the event list status area below the region histories pane. The name next to Status for scope: is the name in the top section of the region histories pane. The name next to Filter: is the resource the region history is reporting on. Each node in the history view is represented by a triplet of adjoining buttons and six boxes showing event status history, with groups and CICS regions ordered in alphabetical order by name. The buttons are as follows: Left-most or node button, shown in Figure 6-19.
This button is a hypertext link to the CICSPlex SM Web User Interface. The button label is the name of the node, such as a CICSplex name, a CICS system group name, or a CICS region name.
89
Click this button to open the appropriate Web browser window for the node. This button provides direct access to the Web browser CICS region views even when there is not a current event for the region in the event list. If the node that is selected is a CICSplex or CICS system group, the browser shows the CICS region tabular view (Figure 6-20) listing all regions in the associated node.
If the node that is selected is a CICS region, the browser shows the CICS region detailed view (Figure 6-21 on page 91) for the selected region.
90
Reserved button
This button is not active and is reserved for future use. Right-most or event button, shown in Figure 6-23.
This button is labelled with the current status of the resource. The status icon explanation is shown in Table 6-2 on page 85. Note: The severity indicated is that specified in the installed threshold definitions and is not a system-determined value.
91
Clicking the current event status button resets the scope in the topology tree and the top section of the region histories pane to the node associated with the event. Also, the event list pane resets to show only events of this type triggered for thresholds defined to that node. Region history icons, shown in Figure 6-24.
The region history portion of the view is a group of six icons. Each icon represents the highest level of severity for events that occurred during that five-minute period. The history refreshes from left to right with the right-most icon being the oldest five-minute period. Together the icons show a 30-minute history of the CICSplex, CICS system group, or CICS region.
The event list pane has two components, the event list status area and the event list area. Event list status area This area is at the top of the event list pane. It indicates the scope selected for the events in the event list and whether there is a filter in force. Scope is applied in several ways. It can be applied using the topology pane to the left. It can also be applied by clicking Event on a node in the region histories pane. The filter is applied when either Region histories or Event on one of the resources summary items is clicked. The filter can also be applied using the Event List menu item at the top of the screen. The event list status area also has a Clear Filter button which removes the filter, if one is applied, and causes all events within the current scope to appear on the event list area. Event list area This area holds a table of triggered events limited by the scope and filter shown in the event status area. Table entry fields that are underlined are fields with hypertext links to
92
other windows within CICS PM, or to Web browser pages that are part of CICSPlex SM Web user interface. When appropriate, a scroll bar appears to facilitate viewing all of the information for all of the events. Note: Clicking on a field name in the event list table heading area (Figure 6-26) will cause the event list to be sorted in descending sequence, using the selected field as the sort key. Subsequent clicking of the field will cause the event list to cycle between ascending and descending sequence, using the selected field.
Click on one of the heading field names to sort the event list, using the field name as a sort key.
Figure 6-26 Event list table heading area
Each event in the table is described by data in the following columns: Event This field is a hypertext link that opens the Installed definitions window (Figure 6-27 on page 95). The field holds the name of the threshold definition that triggered the event. This name is generated when the threshold is created and stored in the CMAS repository as part of the definition. Generated names have the first three characters of CMZ. There are several events that are predefined and installed as part of the CICSPlex SM system availability monitoring function that may appear in the event list. Their names are: !!SAMOPS - A CICS system is not available. !!SAMSOS - A CICS dynamic storage area (DSA) is short on storage. !!SAMSDM - A CICS system dump is in progress. !!SAMTDM - A CICS transaction dump is in progress. !!SAMMAX - A CICS system has reached maximum tasks. !!SAMSTL - One or more tasks in a CICS system are stalled because of contention for resources such as DBCTL, DL/I, transient data, files, journals, and temporary storage.
Region This field is a hypertext link that opens the Web browser to the CICS region details screen (Figure 6-21 on page 91). The field holds the region name of the event. Resource This field is a hypertext link that opens the Web browser to the page for the named resource type (Figure 6-14 on page 87). The field holds the name of the resource type associated with the event. Identifier This field is a hypertext link that opens the Web browser to the details page of the named resource (Figure 6-28 on page 95). The field holds the name of the resource that triggered the event.
93
Attribute This field holds the attribute whose state is being tested within the resource type, that is, the operand of the threshold expression. Operator The test that is being made on the attribute. That is, the operator component of the threshold expression. EQ - Equal to. NE - Not equal to. GE - Greater than or equal to. LE - Less than or equal to. Reference Value The value the attribute and operator are to be applied to. When the expression evaluates to true, an event is triggered for this resource. For example if the Attribute is OPENSTATUS, the operator is EQ, and this value is CLOSED, the event is triggered whenever the OPENSTATUS of this resource becomes CLOSED. Current Value The value of the threshold attribute at the time the event was generated. Severity The severity assigned to the event in the threshold definition. CICS PM uses three severity icons that correspond to warning, severe, and very severe. Priority The priority of the event. (this is a CICSPlex SM attribute, which is defaulted to 1 in CICS PM and included to support coexistence of CICSPlex SM and CICS PM events.) Time The date and time the event was triggered. Description The event description entered when the threshold was created. The text can include key phases that can provide additional information not available from the generated event name. Maximum length of this field is 30 characters. Note: When a row in the event list is selected, the background color of the selected field becomes red with white lettering. The hypertext link fields become blue with white lettering. The remaining fields become black with white lettering.
94
95
The Event List menu (Figure 6-29), provides the following options:
Filter by Resource When the current view is the Event Summary pane, the filter parameter is the resource type. If this option is selected, CICS PM opens a dialog through which the resource type to be used as a filter can be specified. See Figure 6-30.
Filter by Region When the current view is the Region Summary pane, the filter parameter is a CICS region. If this option is selected, CICS PM opens a dialog through which the region to be used as a filter can be specified. See Figure 6-31.
96
Show All Selecting this option removes any filters in operation against the Event list. All events in the currently selected scope are shown, sorted to the last-used sort sequence in the default sort column. Sort Select this menu item to open a dialog to apply a sort key to the event list. Figure 6-32 shows the sort key selection dialog box. Clicking on a field name and OK in the Sort dialog box has the same effect as clicking a field name on the Event list table heading.
Link to Select this option to show a sub-menu listing hypertext links in the Event list (Figure 6-33). The submenu is only available when the current cursor focus is on a hypertext link in the Event list. Clicking on any of the submenu options has the same effect as clicking on the corresponding hypertext link on an event on the Event list pane. The submenu options are:
Event Opens the Installed Threshold Definitions window. See Figure 6-27 on page 95. The scope currently in force for the Event list and the name of the event are passed to the Installed Threshold Definitions window, which opens to show all threshold definitions for that resource type.
97
Region Opens a browser window to display CICS region details for the CICS region named in the event. See Figure 6-21 on page 91. Resource Type Opens a browser window at the page for the named resource type. See Figure 6-14 on page 87. The content is limited to instances for the current scope only. Resource Identifier Opens a browser window to display the resource details page for the named resource in the event record. See Figure 6-28 on page 95.
98
Note: When a new threshold definition group is named, CICS PM generates an 8-byte internal name for the group as its key when saving the group in the CMAS repository. The threshold group name you enter at its creation always appears in the threshold groups pane.
99
The threshold groups pane has two sections: System-supplied default group This section has only one entry, the CICS PM-supplied default group named CMZBase. This group is intended as a starter set for threshold definitions. It is recommended that the CMZBase group not be modified but copied, and modifications made to the new group. The contents of the group are supplied in the CMZIMPnn member (where nn is the version and release number of CICS TS) of the SCMZPARM data set. The member contents are imported into the CMAS repository by the CMZBATCH utility program. User-defined threshold group list This is a list of threshold groups that have been created by CICS PM workstation clients and stored in the CMAS repository. The list reflects groups created for the currently selected CICSplex. The CICSplex selection is determined by which CICSplex was selected in the window from which the threshold definition window was navigated. For example, if the threshold definition window was switched to from the event monitor window, then the CICSplex being monitored in the event monitor window would be the CICSplex for which the user groups had been defined. The current CICSplex is shown in the Current CICSplex drop-down selection list found at the bottom of the threshold pane (Figure 6-36). This selection list can be used to change the target CICSplex by clicking the down arrow and selecting the desired CICSplex from the list.
The threshold group list is displayed in group name order. Only one group name at a time can be selected. When a group is selected, it sets the context for the resource list in the CICS resources pane (Figure 6-41 on page 103). As a result of the context setting, the icon displayed in the resource list (Figure 6-42 on page 104) indicates whether any thresholds have been defined for that resource set and its associated resources. Also when a group is selected, defined thresholds that are associated with the resource highlighted in the resource pane will be displayed in the Thresholds pane (Figure 6-46 on page 109). When creating threshold groups, consider the scope of the thresholds it will contain. Within a CICSplex, a single threshold might require values that are different for each region or CICS system group. For example, dynamic storage area threshold values could be different for a terminal owning region as compared to an application owning region. Keeping thresholds together that pertain to a single region or a group of like regions helps in the installation and ongoing maintenance of the groups.
100
The threshold groups pane has a threshold groups menu selection located above the pane (Figure 6-37).
The threshold groups menu selection has the following objects: New... This selection opens a dialog (Figure 6-38) that allows the creation of a new, empty group. The name entered can be up to 30 alphanumeric characters in length.
Cut Copies the selected definition group to a temporary save area and deletes the selected definition group and its contents from the CMAS repository. Be aware that deleting a group in this way does not affect any installed thresholds associated with the group until a region with the thresholds installed is restarted. Copy Copies the selected group to a temporary save area. Paste Pastes previously cut or copied threshold definitions into the selected threshold group, with the threshold name temporarily shown as NewThreshold. A new system-generated name is assigned when you save the changes. This can be used as a method of creating new threshold definitions. Copied threshold definitions can be pasted into the same threshold group they were copied from, to another group, or to a new group. Cut threshold definitions can be pasted to another group or to a new group. Delete... CICS PM immediately removes the selected threshold definitions group from the group list. However, the group is not deleted from the CMAS repository until the delete action is confirmed in the Save changes dialog box (Figure 6-39). If you click No, the group name is restored in the threshold groups pane and nothing is deleted from the CMAS repository. If
Chapter 6. GUI interface
101
you click Yes, the group is then deleted from the CMAS repository. Installed definitions are not affected until a CICS region to which they have been associated is restarted.
Save Saves the group containing all the outstanding new and changed threshold definitions in the current context group to the CMAS repository. Install Opens a dialog (Figure 6-40) to enable the installation of all thresholds defined in the current threshold definitions group to a selected scope. The install action presents the CICSplex topology tree from which a scope can be chosen for the install.
Discard Changes Discards all unsaved threshold definition changes to the selected threshold group.
Resources pane
The Resources pane contains a list of all the resources in CICS PM against which thresholds can be defined. Using this list, a resource type is selected for which a threshold is then defined. The resources list is a tree representation of CICS resources organized in three levels. The first level is the root node. This level represents all resource types for an entire threshold 102
IBM Tools: CICS Performance Monitor V1.1
group. The second level is the resource group level. This level is a superset of the resource categories shown on the Web user interface navigation frame. See Figure 6-41. The third level is the individual resource types for which thresholds can be defined. Note: Resource types for which CICS PM thresholds cannot be defined are not included in the resource tree structure.
Figure 6-41 CICS Resources pane & Web user interface navigation frame
The icon associated with a resource entry in the tree indicates whether the currently selected threshold definition group holds one or more thresholds defined against that resource type. See Figure 6-42 on page 104 for an explanation of the resource tree icons. When a different threshold group is selected, the resource pane is refreshed and the first category in the list is shown as selected. The threshold pane is then updated accordingly.
103
This icon indicates this resource group has thresholds defined. This icon indicates this resource group has no thresholds defined.
The CICS Resources pane supports standard tree navigation actions to expand a node or collapse a node. When a resource is selected, the threshold pane is updated to show all of the threshold definitions associated with the selected resource within the selected threshold definition group. If no thresholds are defined for the selected resource, the Thresholds pane is empty. If the root node, All resources, of the resource tree is selected, the threshold pane displays all threshold definitions for all resources associated with the selected threshold group. The CICS Resources pane has a resources menu selection (Figure 6-43) located above the pane.
The Resources menu selection has the following objects: Expand all Expands every node in the resource types tree to show each resource type for which thresholds can be defined. Collapse all Collapses all the resource types tree nodes so that only the root node, All resources, is shown. Expand To Definitions Expands only those branches of the resource types tree for which thresholds have been defined in the current threshold group. Select All 104
IBM Tools: CICS Performance Monitor V1.1
Sets the current selection to the root node of the resource type tree and causes the Thresholds pane to show all threshold definitions associated with the selected threshold group.
Thresholds pane
The Thresholds pane (Figure 6-44) shows the currently selected threshold definitions list and the threshold status area. The threshold list is determined by the currently selected threshold group and the currently selected resource type for that group. Note: The Threshold Definitions pane shows only those resource definitions that have names beginning with the letters CMZ. This means they were created using the CICS PM threshold definition service.
The threshold pane has a thresholds menu selection (Figure 6-45 on page 106) above the pane.
105
The thresholds menu selection has the following items: Show All Clicking this selection sets the resource pane selection to the root node and shows all definitions for the current group. Select All Selects all the threshold definitions currently shown in the threshold pane. Deselect All Deselects all the threshold definitions in the current threshold pane. New... Adds a new threshold definition form to the threshold pane. The default values in the new form will be as described in Table 6-3 on page 107. Cut Copies the selected threshold definitions to a temporary work area and deletes the selected definitions. The delete action is completed only if a subsequent paste action is performed. If paste is not performed, the definitions remain in the list. Copy Copies the selected definitions to a temporary work area. Paste Copies the thresholds from the temporary save area to the threshold pane for the currently selected threshold group. The threshold group can be the same group the definitions were copied from or a different group. The copied thresholds are created in the target group with new generated names. Delete Deletes the currently selected threshold definitions. Sort by > Sorts the thresholds by one of the fields in the following list: Resource type The menu option is active only when the Show All menu option has been selected. When active, the thresholds in the threshold pane are sorted by the Resource column on the threshold form.
106
Resource ID Sorts definitions in the threshold pane by the Resource Identifier column of the threshold form. All characters in the column are used in the sort key according to their face value in the character set. Threshold name Sorts definitions in the threshold pane by their system-generated names in the Name column. Each threshold definition in the pane is presented as a form containing the attributes of the threshold definition. Descriptions of the Attribute field can be found in CICSPlex System Manager Resource Tables Reference, SC34-6027. The details of the form are described in Table 6-3.
Table 6-3 Threshold definition form field descriptions
Threshold definition field Resource Name Resource identifier Description Resource type, which is determined by the selection from the resource pane. A non-selectable, read-only field. A text entry field containing the specific or generic ID of a resource. This corresponds to the KEY attribute in the CICSPlex SM resource table, and also the name of the resource as defined to CICS, such as the file name or the connection name. A text entry field that can hold a description of up to 30 characters. A scrollable list of attributes, from which the attribute to be used to monitor the state of the resource is selected. The operator used in the threshold definition condition statement. It can be EQ or NE for a state attribute: or GE or LE for an integer attribute. First item in the attribute list for the specified resource type. This is determined by the type of attribute that appears first in the attribute list for the resource type. If it is a state attribute, the default is EQ. For an integer attribute the default is LE. First attribute value for state variable; blank for integer variable. Value shown in a new form First resource type in path from selected node. This is a system-generated value.
Description Attribute
Operator
The value to be tested in the condition statement. This can be a text entry from a scrollable list or a numeric value. Depending on the type of attribute (state or integer). A user-defined level of severity. There are three severity levels to choose from: HW, HS or VHS. The sampling interval between evaluations of the condition. This is an integer data entry field. The interval is a numbers of seconds in the range of 1-86400 seconds.
Event Severity
HW
Sample
107
The sampling interval: Threshold definitions are stored in the CMAS repository as several RTA analysis definitions. The sampling interval is stored in two of those analysis definitions. Using CICSPlex SM services, you can define different sampling intervals in the two analysis definitions. One interval specifies the interval for collecting data about the state of resources, and the other when the collected data is to be analyzed. CICS PM threshold definitions allow you to specify only a single value for sampling interval. CICS PM sets the collection interval to the value you specify in the threshold definition form, but sets the analysis interval to only half that value. Thus, for the default sampling interval of 60 seconds, real-time analysis data is collected every 60 seconds and analyzed every 30 seconds. The sampling interval determines when the resources state data is collected by the CICS regions and passed to the CMAS for subsequent analysis. The analysis happens at a time also determined by the sampling interval. However, data collection and data analysis are not synchronized events. They run as asynchronous activities. The effect of this is that an event can be triggered later than the sampling interval would suggest. For example, if you specify the threshold definition for an ISC or MRO connection as not equal to acquired, with a sampling interval of 60 seconds, it is possible that a connection could be in the not acquired state for almost 90 seconds before an event is triggered. Table 6-4 illustrates this situation.
Table 6-4 Connection states
Step Time in mm:ss 00:00 Actual state of connection resource Acquired Collected State RTA actions
1. Expiry of collection interval 2. Connection fails 3. Expiry of analysis interval 4. Expiry of analysis interval 5. Expiry of collection interval 6. Expiry of analysis interval
Acquired
The connection state is collected and the state is recorded as acquired. One second after the state was collected and saved, the connection fails and its actual state is now released The collected data (from step 1) is analyzed (30 seconds after the previous analysis) and found to be acquired. Result: No event is triggered. The collected data (from step 1) is analyzed (30 seconds after the previous analysis) and found to be acquired. Result: No event is triggered. The connection state is collected and the state is recorded as released. The collected state (from step 4) is analyzed again (60 seconds after step 3). Result: An event is triggered. The elapsed time since the connection first became released is 90 seconds.
00:01
Released
Acquired
00:29
Released
Acquired
00:59
Released
Acquired
01:00
Released
Released
01:29
Released
Released
Choose the sampling interval carefully to avoid unnecessary overhead in the CICS regions being monitored. The evaluation of resources in the monitored CICS regions is performed by the CICSPlex SM COIR transaction, which runs whenever it is invoked by the CMAS to
108
perform the evaluation. If the number of installed resource definitions being evaluated is particularly large (for example, programs), and you specify a short sampling interval, the time taken to evaluate their state could be excessive. Set the sampling interval to take account of all the relevant factors. Figure 6-46 illustrates the result of selecting a threshold group.
group selected
resource selected
All of the fields in a threshold definition can be modified except the name. To modify a definition, click the item to be modified and make the desired changes. Selected definitions are denoted by a highlighted rectangle (Figure 6-47 on page 110). When all modifications have been made, the group containing the changed definitions can be saved to the host by clicking Save or the Threshold Groups -> Save menu selection. The definitions can be installed by selecting Install, or the Threshold Groups -> Install menu selection. As implied by their inclusion in the Threshold Groups menu selection, the save and install actions apply to the entire selected threshold group and not just to the modified threshold definitions. Once a definition has been modified, any attempt to navigate to another CICS PM window without first saving the modified definition group will result in the save changes dialog box (Figure 6-39 on page 102) appearing.
109
To create a new threshold definition, first select the group and resource type the threshold is to be associated with. Then click Thresholds -> New. The result will be a blank form appropriate for the resource type (Figure 6-48 on page 111). Complete filling in the form before leaving it. A different choice of resource cannot be made or another function selected without completing the form. If the form is not to be completed, click Thresholds -> Delete to remove the definition from the Thresholds pane. Then navigation away from the definition can be accomplished. Once the definition is complete it can be saved and installed using the appropriate buttons at the bottom right of the window. As with modified threshold definitions, the save and install actions apply to the entire, selected threshold group and not just to the new definitions.
110
Threshold status area: The threshold definitions status area is located immediately below the Thresholds pane. It can contain two items of information: Changes pending The presence of this message indicates that changes have been made to definitions in the selected threshold group, and they have yet to be saved to the CMAS repository. Any attempt to select another threshold group when changes are outstanding will result in the appearance of the save changes dialog box for disposition of the changes. Current CICSplex The currently selected CICSplex is shown in a combo box. The threshold groups shown in the threshold groups pane are the groups defined to the selected CICSplex. The drop-down list can be used to select another CICSplex. When a new selection is made, the threshold groups associated with the new CICSplex will be shown in the threshold groups pane. The threshold status area also has two shortcut buttons located on the right side. The Save and Install buttons are shortcuts for the Save and Install options on the Threshold Groups menu selection. Figure 6-49 on page 112 is an example of the threshold status area.
111
Topology pane
This pane is similar to the topology pane in the Event monitor window with the following exceptions: All nodes have the icon that denotes the node type. Included are inactive CICS regions. The status icon is not shown. When the Installed Definitions widow is opened, the selected scope is the same as that in the Event monitor window. If the scope in the Installed Definitions window is changed, the same scope change is applied to the Event monitor window. The Topology menu in the Installed Definitions window (Figure 6-50) contains a subset of the menu items in the Topology menu in the Event monitor window.
For details of these menu items, see The topology pane on page 82.
Resources pane
The Resources pane in the Installed Definitions window is the same as the resources pane in the Threshold Definitions window. The list of resource groups corresponds directly to the resource group lists on the resources view of the status pane of the CICS PM Event monitor window.
112
Installed Definitions will only be shown when the All resources node, the resource group, or the resource type with a definition installed is selected. The resources menu on the Installed Definitions window is the same as the resources menu on the Threshold Definitions window. For details about the resources menu options and the icons associated with the resource tree, see Resources pane on page 102.
113
In the initial Installed Definitions window it should be noted that: The topology pane is at the approximate level of detail as in the Event monitor window. No resource type is selected. However, the icon for the Files & DB2 resource group indicates the presence of installed definitions for that group. The Thresholds pane is empty. In the initial screen the pane is empty because no resource selection has been made.
114
Figure 6-53 shows CMZBase Files & DB2 resource group forms.
Figure 6-54 on page 116 shows CMZBase Queues resource group forms.
115
Figure 6-55 shows CMZBase ENQ Pools resource type of the Regions resource group forms.
Figure 6-56 shows CMZBase CICS Loader resource type of the Regions resource group forms.
Figure 6-57 on page 117 shows CMZBase CICS monitor resource type of the Regions resource group forms.
116
Figure 6-58 on page 118 shows CMZBase Region resource type of the Regions resource group forms.
117
Figure 6-59 on page 119 through Figure 6-61 on page 120 show CMZBase CICS storage resource types forms.
118
119
120
Chapter 7.
121
Note: We recommend that you secure the CICS system amendment features of the CICSPlex SM Web user interface to authorized personnel.
122
Existing users of the CICSPlex SM Web user interface may wish to also access their previous CICSPlex SM-defined view sets, and user-defined views. An alternative panel is provided to allow access to both the CICS PM-provided view set menu, and the CICSPlex SM-provided view set menu, which is available if IBM_CMZMAINMENU is specified in the DEFAULTMENU parameter.
Selection criteria
There are three options for selection criteria: CMAS Context, Context, and Scope, refreshable with the SET button to the right of the panel. CMAS Context is the CMAS name, and is only used when viewing CMAS level data such as CMAS status, and CICSplexes connected to a CMAS. Context is the CICSplex name and is necessary for almost all views. Scope can be used to limit the available commands and views to a particular level, and can contain a CICSPLEX name, a CICS group, or single MAS in a group. A simple default is to fill in the name of the CMAS region, and prefill the other two fields with the name of the CICSPLEX. Hypertext link from CICS PM sets the context according to the view being linked to.
123
Navigation frame
All the CICS PM-supplied view sets are available in the Navigation frame (Figure 7-2 on page 123). The topmost selection is Home, which will return you to the menu specifed in DEFAULTMENU. There is also a function to return you to the last menu you were in, Repeat last menu. Below that are the CICS PM view sets where various views are organized into groups. The contents of each view set can be shown by selecting the expansion icon next to the view set (plus sign), and collapsed by selecting the collapse icon (minus sign) next to each expanded view set. The navigation frame view sets and a brief description of their contents are shown in Table 7-1.
Table 7-1 Navigation frame view sets
Navigation frame view sets Alerts Regions Activity Connectivity Files & DB2 Logging Queues Transactions Programs Enterprise Java Models Miscellaneous Description of contents Active events - CICS PM or CPSM - defined thresholds that have been triggered, creating event records General information about CICS regions in the current scope, storage views, job views etc. Task views, UOW views, Interval control requests, enqueues etc. Terminals, ISC, MRO, LU62, TCP/IP, FEPI views DB2-related views, VSAM-related views, data tables Journal and logstream views Intra and Extra - partition views, temporary storage views Various transaction-related views Program and DFHRPL views Corba servers, jar files, Enterprise beans, JVMs Enqueue, journal, temporary storage, terminal model views Process types and document template views
At the bottom of the Navigation frame are three further options under the heading Special: View editor - The functions of the view editor are described in section 7.1.3 beginning on page 139. Note: If the Web user interface is running with SEC=YES in the SIT, then you are only shown the View editor option if you are allowed to use this option. New window - opens the existing view in a new window. Sign off - Signoff from the Web user interface.
124
Main panel
The Main panel provides you with a subset of the views available in the Navigation frame, as shown in Table 7-2.
Table 7-2 General views
General View CICS regions Active tasks Units of work (UOW) ISC and MRO connections Description Information about CICS regions in the current scope. Information about all active tasks on all CICS regions in the current scope. Information about all active units of work on all CICS regions in the current scope. Information about all installed ISC and MRO connections on all CICS regions in the current scope, including those out of service. Information about installed TCP/IP service resource definitions on all CICS regions in the current scope. Information about installed transaction class definitions on all CICS regions in the current scope. Information about all installed local and dynamic transaction definitions in all CICS regions in the current scope. Note filters for transaction displays are case-sensitive. Information about remote transaction definitions on all CICS regions in the current scope. Information about installed local file resources on all CICS regions in the current scope. Note filters for file displays are case-sensitive. Information about installed DB2 connections on all CICS regions in the current scope. Information about Main and Auxiliary temporary storage for all CICS regions in the current scope. Information about installed remote file resources on all CICS regions in the current scope. Information about installed terminal on all CICS regions in the current scope. Note that filters for terminal displays are case-sensitive. Information about any events (e.g., monitored exceptions) that are outstanding in the current scope.
Active events
These are broadly documented in chapter 9 of CICS Performance Monitor Users Guide, GC34-6194. For the purposes of this chapter, we will follow one General view, the CICS Regions view, which when selected will provide a list of the CICS regions according to the current scope, which in this example is CICSplex-wide.
125
Note: Be careful using a CICSplex-wide scope, especially if you try to obtain details on something which has lots of resources, for example programs or terminals. If you have lots of MASes and thousands of programs in them, you can generate very large queries which can consume considerable resources, in both the CICS regions themselves, as well as in auxilliary storage used by the ESSS data spaces and CMAS.
The small triangle and the inverted triangle beneath the title of each column can be used to sort ascending or descending, or provide a summary display of the data in the column in question. Summaries of character data, such as the CICS region name, will display only the characters that are common to all the listed records, and display asterisks for the rest. Summaries of numeric or hexadecimal fields are handled according to the summary column for the appropriate value in the CICSPlex SM Resource Tables Reference. Actions can be performed against all items in a summary display by selecting the check box in the left hand
126
column. Next to the check box is a number indicating the number of records grouped into that summary. The options below the listed regions, or the summary of them, are a useful feature, allowing you to perform context-wide actions against the listed regions, which you could select either by ticking the appropriate boxes, or by amending the context. From here you could request a shutdown of all, or of a subset of your CICS systems. Similarly, you could request statistics processing, rebuild security profiles, and so on. Note: The Refresh button in the Selection criteria section of the display can be activated by selecting the button, or activated to automatic refresh by selecting the check box and specifying the refresh interval in seconds. Automatic refresh is only available to JavaScript-enabled browsers. Selecting a new panel will reset the automatic refresh function. Notice the small green question mark above the Refresh button in the Selection criteria section, shown in Figure 7-4 on page 128.
127
The icon marked d above is a help function message specific to the panel display message, in this case EYUVC12801, explaining what the display shows. The icon marked c represents the resource that you are looking at. The icon marked a can be selected to go back. The icon marked b is a particularly useful feature, the print preview feature. This has particular application when a displayed screen is one of a number of pages that display a tabular list of data, such as the list of transactions in Figure 7-4, which shows page 1 of 70 pages. By selecting the print preview icon, this can then be printed to a single screen that can then be scrolled through, as shown in Figure 7-5 on page 129.
128
On the top left of the screen is the scroll bar, which has fitted all 70 screens into one page. Figure 7-6 on page 130 Shows the data displayed when the scope of the regions display has been reduced to include the AORs defined to group CSGAOR.
129
From here an action can be taken against all or some of the CICS regions in the scope. In this case we need to shut down these two regions. Selecting Shutdown either with Select record All selected, or with the check boxes for each region selected, generates a view with the following shutdown options, shown in Figure 7-7 on page 131.
130
A number of shutdown options are presented, together with buttons to confirm the action, which is presented for the first CICS region in the scope. Selecting Yes to all will perform the requested action for all CICS regions originally selected, without needing independent confirmation for each region. Selecting Yes will perform the action for that CICS region and present the same screen for the next CICS region in the scope. We select Yes to all, refresh the screen, and shortly afterwards, both disappear from the display. Figure 7-8 on page 132 shows part of the screen generated when a single CICS region is selected from the CICS regions panel shown in Figure 7-3 on page 126.
131
As well as displaying various features of this particular CICS region, such as VTAM applid, release level, and startup type, there are a number of options to allow configuration and amendment of CICS system parameters. These are mainly the system options available through CICSPlex SM or CEMT SET functions. Changes are made as required to the various parameters, then actioned by Apply changes at the bottom of the display. If the changes are successful, you are notified accordingly: EYUVC1230I 'Apply changes' (SET) request completed successfully If the change you have tried to make is incorrect or not applicable, you will similarly be notified with appropriate response and EIB function indicators: EYUVC1231E 'Apply changes' (SET) request failed. (Tableerror, Dataerror) EYUVC1270I Action (SET) failed for 'SCSCPAA1' in 'SCSCPAA1'. Invalid request. (CICS information: RESP(16) RESP2(5) EIBFN(00005404)) Figure 7-9 on page 133 shows the bottom of the panel shown in Figure 7-8.
132
From here you also have the option to perform system-wide commands such as Shutdown, and the button to action system configuration amendments, Apply changes. There are a number of further links available here which apply to the single CICS region selected, and which can themselves link to further views. Some of these provide information displays, and some can be used to modify certain CICS settings. For example, to look at the storage settings for the selected CICS region, we select the CICS Storage link, which generates the view shown in Figure 7-10 on page 134, part of the panel generated by selecting that link on the previous display.
133
Current storage settings are displayed together with status indicators such as Short on storage status. Underlined or highlighted fields2 provide further links to other storage displays. The settings in boxes, Current DSA limit and Current EDSA limit can be overtyped and replaced with new settings and then actioned with Apply changes at the bottom of the display. In this example, we select the link for Total storage currently allocated to DSAs, which generates the view shown in Figure 7-11 on page 135, showing part of the panel generated by this selection.
134
We now have a breakdown of the DSA into its various elements, each of which can be selected, and we are presented with various other fields showing numbers of Getmains, Freemains, subpool dispositions, and so on. We also have the option again to increase or decrease the current DSALIM setting, actionable with Apply changes. In this example, we wish to see more detail for the subpool dispositions in CICS key DSA, so we select the Current Subpools (domain and task) field under the CDSA column, which has the value 33 in it in this example. This presents us with the view shown in Figure 7-12 on page 136.
135
This shows the results of selecting the CDSA subpools field, and presents yet more detailed information for the CDSA storage allocations, listed by subpool name, and if further information is required, we could view the attributes for each named subpool by selecting the names in the Domain subpool name column. Note: At this level of the display, columns can again be sorted by ascending or descending order, and may also be summarized. Also, in the Selection criteria area of the display, we can again amend the context and the scope, and limit the display by relational operators to specific values. Figure 7-13 on page 137 shows the CICS regions display, shown here again for ease of reference.
136
Apart from the CICS system name link, there is also a link available from the Current number of tasks column, which can be followed by selecting the number in the relevant row. This generates the display shown in Figure 7-14 on page 138, showing the currently active tasks on the selected CICS region.
137
The Selection criteria section of the display now contains relational operators to allow the tasks displayed to be filtered by task ID, Transaction id, user ID and Transaction class. These are case-sensitive. At the bottom of the screen are three action buttons, Set attributes, Purge and Force purge. Set attributes allows you to change the task priority when the required task is selected. Purge and Force purge perform the same function as their CEMT equivalent against any selected task. Each column may be sorted or summarized as in other tabular displays. The fields Task ID and Principle facility provide links to further information about the resource in question. Following the link for Principle facility on terminal TC2B, we get the display shown in Figure 7-15 on page 139.
138
This display shows the specific information for terminal TC2B. Various device attributes and terminal statistics are displayed, as well as modifiable fields that can be actioned by Apply changes. From here, we can also link to a more specific terminal display, to the task display, and to program and transaction information. We can also acquire and release the terminal, set it in or out of service, release it, or discard the resource definition. Similar functions and displays are available for MRO links, LU62, FEPI and TCP/IP connections, as shown in Figure 7-16 on page 140, a display for MRO connections for this CICS region.
139
This display shows the MRO connections for this CICS, and the range of CEMT commands we can perform against them. Note: With the other connectivity options in the Navigation frame on the left, we can perform similar functions as appropriate to the connection type. Figure 7-17 on page 141 shows the detailed display reached when selecting one of the active tasks from the screen shown in Figure 7-14 on page 138.
140
A wealth of further information specific to the task can be reached by following the links, some of which are visible on the screen displayed, and again the buttons Apply changes, Purge and Forcepurge are at the bottom. We can also see from this display whether or not the task is purgeable, which in this case it is. Assuming in this case a programmer has left CEDF transaction active, suspending a transaction that is holding resources, we decide to Purge it. This then passes us to a confirmation screen which we action by selecting Yes. We are then returned to the Active Task display shown in Figure 7-14 on page 138, with an indicator of whether or not the task purge has been successful, as follows: EYUVC1230I 'PURGE' (PURGE) request completed successfully.
141
The bottom of the CICS PM Web user interface main menu provides three view menus: Event views - active thresholds and events System and resource views - essentially the same options that are available in the navigation frame CICSPlex SM configuration views - CICSPlex configurations and MASs Figure 7-19 on page 143 shows the Event views panel.
142
The Event views are split into three links: Active thresholds - CICS PM-generated defaults or user-defined thresholds, or inherited CICSPlex SM-defined RTADEFs Active events - Incidences where the defined thresholds have been triggered on one or more CICS regions Active event details - Details on the threshold: event specifications and triggers Selecting the Active thresholds link generates the display shown in Figure 7-20 on page 144.
143
The Active thresholds display shows the thresholds that are active on this CICSPlex, sorted in descending order, then summarized to remove duplicate definitions (the thresholds would otherwise show up as one incidence of each definition per CICS region). The top two in Figure 7-20 are unique to CICSPLA1 in this CICSPlex. There are three differently generated threshold definitions shown here. The top one is an RTA (real time analysis) threshold that was generated under CICSPlex SM and then imported into the CICS PM view set. The others were created in a CICS PM session. Although you can view the thresholds with the Web user interface, you cannot create them. Selecting the highlighted or underlined Definition name takes you to a brief description of the Active threshold. Several of the columns in the above display are relevant to CICSPlex SM RTA definitions only: the Active definition type column, the PERIODEF column, and the Associated action name column. CICSPlex SM RTADEFs are covered in CICSPlex SM Administration, SC34-6019. At the bottom of the panel are two buttons, a Discard button, which provides the function of removing a threshold from the currently selected scope (but does not permanently remove it
144
from the stored repository), and a Deactivate button, which applies to CICSPlex SM RTADEFs with PERIODEFs and/or associated action names active. Figure 7-21 shows a page of the currently triggered thresholds on this CICSPlex, reached by selecting the Active events link shown in Figure 7-19 on page 143.
Various information about the scope and context of the event, its priority and type, is available here. All the triggered events shown here are the CICS PM-supplied defaults, and are indicative of the need to tailor the supplied thresholds to avoid displaying event occurrences that are inappropriate for your installation. Selecting the Event name provides more information about the event specification, following the Detailed information link, and then the associated EVALDEF takes you to the display shown in Figure 7-22 on page 146.
145
This shows the event details for the CMZBase threshold definition CMZPMD15, which raises a VHS severity event if the percentage of free storage in ECDSA falls below 30%, which you may or may not consider to be a meaningful threshold to have active on your CICS systems, (it probably would have been more appropriate to have a VLS set here rather than VHS, as this would point to a low value being a problem). The CICS Performance Monitor Users Guide advises you to take a copy of the CMZBase definitions and modify them as appropriate for your needs, and to deactivate the supplied defaults. This is an important step in the configuration of CICS PM (that shouldnt be avoided) to prevent your active event displays from becoming flooded with less than crucial information about triggered thresholds. The Active event details link from the selections shown in Figure 7-19 on page 143 will present you with the same display as following the Detailed information link described on the previous page, presented for all events in tabular form.
146
Note: CICS PM view sets are locked and cannot be modified or deleted using the view editor. Figure 7-23 shows the initial CICSPlex SM Web user interface editor menu you are presented with if you select View Editor from the primary options menu.
The CICSPlex SM Web user interface gives you the ability to create your own menus, views, and view sets. This is an important feature of the Web user interface. As you customize the displays to suit your requirements, or for any target group of operators or administrators. Within the views you have access to a wealth of CICS control block variables and state data. The DEFAULTMENU parameter can be modified to use a menu you create yourself as the default menu presented to users.
147
Selecting OK returns you to the Menu editor operations panel where you now select Edit, and enter the name of the new menu created, in our case NEWHOME. The Edit screen then prompts you to select the menu components you wish to amend. You can alter the title that appears on the menu and its description, and also amend the scope and context. In our case, we simply wish to amend the menu contents, so we select Menu contents. This presents us with the existing contents of IBM_CMZMAINMENU, which we have copied, and various edit options as shown in Figure 7-25 on page 149.
148
In this example we are going to add another menu option to the two default ones above from IBM_CMZMAINMENU, under the existing group title Menu. Select Append -> Menu Choice -> OK. Now select Title and annotation to give a name and description to the menu choice. You can put anything you like here; we called the new option Troubleshooting for problem tasks, and gave it an appropriate description. Select OK . Now select Destination to choose a menu choice type, and select Menu link, to link our menu option to another menu, which we also create. We also have the option here to link to another view, or to an external URL. From the Menu link components we select Target Menu, from the Target Menu screen we select Target Menu name, and type in the name of the new target menu, which doesnt exist yet, that we will call TROUBLESHOOT. Select OK -> Finish -> Finish -> OK -> Save, to go back to the Web user interface editor screen shown in Figure 7-23 on page 147. Now select View sets -> Create to create a new view set that we will name REGIONISSUES. View sets require an object that determines the area their available views will focus on. In this case we choose CICSRGN. This brings us to a view set contents screen, where we will add a view to REGIONISSUES, which we will call REGIONPROBS, as shown in Figure 7-26 on page 150.
149
We have the option to create a tabular view, showing a few relevant details for a range of objects, or a detail view, showing a number of details for a single object, or a Confirmation panel to confirm actions against objects. In this case we want a display showing key problem details on all our CICS regions, so we select Tabular view, and on the Pre-fill option below we select Key attributes to pre-fill our tabular view with key fields, which in object CICSRGN is actually only the CICS region name field EYU_CICSNAME. Select OK, which adds REGIONPROBS to the view set REGIONISSUES, and presents us with the Tabular View Components screen shown in Figure 7-27 on page 151.
150
There are various options available here. We amend the Title and annotation, giving the view a title of CICS Region-wide issues, and a brief annotation of Check for outstanding problems on CICS. All the other options will be allowed to default, except Table contents, which when selected gives us a table prefilled only with EYU_CICSNAME. We wish to display certain key variables for our CICS regions, so from the Table Contents display we select Append, which presents us with all the attributes available for the previously specified object CICSRGN, as shown in Figure 7-28 on page 152.
151
The first field we want to display after the CICS region name that has been included for us in the panel by the pre-fill option, is Current number of tasks represented in the attributes table as CURRTASKS. So we find that field in the table and select it, which presents us with three options, all of which we are going to allow to default for now. For reference they are: to Edit the column title and sort and summary options, to Add view links from this column, and to Edit the presentation options. Select Finish, which returns you to the Table Contents display. Now do the same for MAXTRCNT, TDMPTOTL, SOSSTATUS, SDMPTOTL, and CPUTIME, Appending each in turn. This then gives us the panel shown in Figure 7-29 on page 153.
152
From here, select OK -> Save -> Finish, which saves the view and view set you have created. We now want to create another view set, which we call PROBLEMS, and include a number of transaction views in it. The views will all be task-based, so we choose the Copy option on the initial view set selection, select IBM_CMZXTASK from the view set list, and PROBLEMS as the new view set name. We then choose Edit and select the new view set PROBLEMS. This gives us a list of the views in the view set copied from IBM_CMZXTASK, not all of which we need, as we are going to create four new views and link some of them to some of the detail views in this view set. Choose Add and create a view called TSKSTORAGE. Make it tabular, and pre-fill it with key attributes. In Title and annotation, we simply call it Task Storage attributes. We then Edit the table contents and append TRANID, DFHSTOR087, DFHSTOR108, DFHSTOR139, and PURGEABILITY. Now select the CICS system name column and Edit. Select View links -> Append -> View set -> OK. This gives you a link condition screen whereby you can make the link conditional on an attribute, or make it always valid, which is what well do. Select OK -> Target. This gives us a selection of the available view sets from which we can select a view to link to. Select IBM_CMZCICSSTOR -> OK. This gives you an object
153
confirmation screen for the view set CICSSTOR, then click OK. We are now presented with a list of target views. Select DETAILED -> OK. Now to make sure we have the appropriate scope for the detailed view, select Context and scope settings, scroll down to Scope settings, select Set to value of attribute, and select the attribute EYU_CICSNAME, click OK. Now Finish -> Finish -> OK -> Finish. For TRANID, we will also have a link, in this case a Local view. This will always be valid. We will link it to view Detail6 in this view set, which is Active task - storage usage. For this example we also created views in view set PROBLEMS: For Enqueues - ENQUEUES For High CPU usage (potential loops) - HIGHCPU For DB2 usage - DB2PROBLEMS In Enqueues we had a column for total file requests, on which we used a conditional link, depending on the value being higher than zero. For other fields we had links as appropriate to file displays, task activity displays, and transaction DB2 activity. Now return to the initial editor panel and edit view set REGIONISSUES, view REGIONPROBS, that we created previously. Amend the table contents, editing CICS System name to link to view set IBM_CMZCICSRGN. Then view Detailed -> CICS Region -> Overview. Similarly, amend the Current number of tasks column to link to view set IBM_CMZCICSRGN, view DETAIL2, the Short-on-storage status column to link to view set PROBLEMS, view TSKSTORAGE, and the Total CPU time used column to link to view set PROBLEMS, view HIGHCPU, amending the scope to EYU_CICSNAME in each case. Now create a new menu called TROUBLESHOOT, giving it a title text of Trouble shooting. For Menu contents, we select Append -> Group title, which we call Region Events. We now append a Menu choice called Region problems, annotating it as appropriate, and for the Destination field we select View link and REGIONISSUES, selecting REGIONPROBS as the target view. Similarly, we append another Menu choice called Active Events. For Destination we select View link and IBM_CMZEVENT, Object EVENT, Target view TABULAR. We then append a third Menu choice, calling it Task display menus and giving it a Destination of Menu this time, which we call TASKISSUES. Lastly, we create the menu TASKISSUES similarly to the menu TROUBLESHOOT. We give it a title text of Task problem displays. The Group title we make Select required display. The menu choices beneath this we point at the four views we created in view set PROBLEMS, giving them titles as appropriate. Finally we amend the DEFAULTMENU setting in the EYUWUI data set, changing it to NEWHOME, and reload our Web user interface region. This provides access to the new menus and view sets. The above process seems long-winded, but with practice is actually quite simple, and is invaluable in making full use of the CICSPlex SM Web user interface, should you decide to have access and control from that interface as well as from the CICS PM Client. Its probably
154
worth, therefore, to set up some of your own views and menus as appropriate to your installation. Figure 7-30 on page 155 shows view REGIONPROBLEMS in view set REGIONISSUES, which we created in the previous view-editing example.
155
Dynamically set trace flags Figure 7-31 shows the screen generated by entering the COVC transaction without parameters on the CICSPlex SM Web user interface region.
COVC
CICSPlex SM Web User Interface Control 1. 2. 3. 4. Start Server Stop Server Status Details User Sessions
EYUVCTR
5. Import 6. Export 7. Trace Flags CICSPlex SM, an element of CICS Transaction Server for z/OS, Ver 2 Rel 2 Licensed materials - Property of IBM. 5697-E93 (C) Copyright IBM Corp. 1994, 2001. All rights reserved Current Status : Ready Applid : SCSCPJA2 Time : 17:11:32 Date : 12/09/2002
PF
1 Help
3 Exit
12 Return
You can also enter the commands COVC START and COVC STOP to start and stop the interface. Entering COVC STOP or choosing option 2 from the COVC Web user interface control panel will also shut down the Web user interface CICS region. Choosing option 3 for status details provides some useful information about the Web user interface configuration as shown in Figure 7-32 on page 157.
156
COVC
CICSPlex SM Web User Interface Control Status Details CMAS Sysid Server Sysid CICSPlex SM Release Secure Sockets Port Hostname : CMAS : PJA2 : 0220 : No : 9001 : WTSC66OE.ITSO.IBM.COM
EYUVCTS
TCP/IP Service Name : EYUWUI TCP/IP Service Status : Open TCP/IP Address : 9.12.6.29 Current Status : Ready Applid : SCSCPJA2 Time : 17:19:35 Date : 12/09/2002
PF
1 Help
3 Exit
12 Return
Option 4 to look at User sessions might be useful in the event of user problems. It also shows the number of screens the user has open, and their originating IP address (this will be the HTTP proxy IP address, if the client is using a Web proxy), as shown in Figure 7-33 on page 158.
157
COVC
CICSPlex SM Web User Interface Control User Sessions Page Inactive Timeout :
EYUVCTU 1 of 1
30 (mins)
Number of Windows 2 15 D
PF
1 Help
3 Exit
7 Up
8 Down
12 Return
Note: Notice that in Figure 7-33, user EDD has a D in the Number of Windows column, which means that user EDD sometime during the session used the Web user interfaces Data interface. This is the interface that CICS PM uses to poll for events, which means that user EDD is probably using CICS PM. You can terminate a user session by entering P for purge, or F for force purge against the userid. Using the Import function allows you to import view sets, or menus, or both from an extrapartition TDQ where they are held. Using the Export function, similarly, allows you to export view sets or menus to an extrapartition TDQ, for example COVE. Export gives you an option to Lock exported definitions, which prevents the definitions from being modified or deleted following a subsequent Import operation. Using the Trace flags option, you can set a number of trace flags to either Y or N to control the amount of trace information being produced. This would normally be done only at the request of IBM support center personnel.
allow your workstation client and any Web browser windows launched by it, to share the same user session. This is documented in chapter 4 of CICS Performance Monitor Users Guide, GC34-6194. Access to issue Web user interface system amendment commands is controlled via CICSPlex SM Simulated Command and Resource Security. If this is activated for a given CICSplex and MAS, then CICSPlex SM simulates the command and resource security checks that would have occurred in the MAS, if the user had issued the equivalent request using the CICS SPI from a transaction defined with RESEC=YES and CMDSEC=YES. You might also consider controlling access to the view editor, which you would do by creating a RACF profile in the FACILITY class, called EYUWUI.wui_server_applid.EDITOR (where wui_server_applid is the CICS APPLID of the server), and give authorized users of the View editor UPDATE authority. Further details on this can be found in CICSPlex SM Web User Interface Guide, SC34-6018.
159
160
Part 2
Part
CICS PM in action
In this part we go through a sequence of senarios that show how CICS PM can be used to identify and resolve CICS system problems. The scenarios cover traditional application workloads (COBOL, VSAM, DB2) as well as new Web-initiated or Java workloads.
161
162
Chapter 8.
163
164
Each of the listener regions had MRO connectivity to each of the Java AORs. There is also MRO connectivity between PJA6 and PJA7 because one of our EJB applications required access to a partner AOR. The listener regions used TCP/IP port sharing, so the IIOP inbound requests could be balanced between the listener regions. The CICSPlex SM Workload Manager (WLM) facility, part of this setup, requires the use of EYU9XLOP as your routing program, as well as other CPSM definitions. These were used to balance method requests across AORs PJA6 and PJA7. The Java AORs are connected to a DB2 subsystem, D7Q2, and they also share a set of VSAM data sets opened in RLS mode used by our EJB application. There are two versions of an application that use VSAM RLS files and two versions using DB2 data (JDBC and SQLJ). For a complete description of CICS logical EJB server setup, refer to Java Applications in CICS, SC34-6000, and to Chapter 5 of Enterprise JavaBeans for z/OS and OS/390 CICS Transaction Server V2.2, SG24-6284. For a description of the ITSO EJB Trader application, refer to Chapters 10, 11, and 12 of Enterprise JavaBeans for z/OS and OS/390 CICS Transaction Server V2.2, SG24-6284.
165
description of the ECIRequest version of the ITSO Trader application, refer to Chapter 10 of Enterprise JavaBeans for z/OS and OS/390 CICS Transaction Server V2.2, SG24-6284. Figure 8-1 shows our connectivity setup. MRO connections from all the CICS regions to the CMAS, as well as IP connections between Java AORs (PJA6 and PJA7) and the LDAP server, are not shown on the chart.
SC47 TPNS
SC66
PTA1
3270 VSAM/DB2 workload
PTA2
HTTP
CMAS PAA1
LDAP
PAA6
PAA4
Browser
CI EX
z/OS CTG
PJA6
PJA7
PLA1
ECIRequest workload Batch client workload
PLA2
ECIRequest workload
IP port sharing
IIOP
IIOP
Browser Web application WebSphere Application Server V4.0
Figure 8-2 on page 167 shows how our AORs accessed VSAM and DB2 data.
166
SC66
PAA1
VSAM PLS
PAA4
VSAM LSR
PJA6
PJA7
DB2
Figure 8-2 VSAM and DB2 data access
Table 8-1 sums up the functions of various CICS regions in our setup.
Table 8-1 CICS region summary
CICS APPLID
SCSCCMAS SCSCPTA1 SCSCPTA2 SCSCPLA1 SCSCPLA2 SCSCPAA1 SCSCPAA4 SCSCPAA6 SCSCPJA6 SCSCPJA7
CICS SYSID
CMAS PTA1 PTA2 PLA1 PLA2 PAA1 PAA4 PAA6 PJA6 PJA7
8.2 Scenarios
In this section, we provide a brief description of scenarios that we used to demonstrate how CICS PM can be used to identify and rectify performance problems in a complex CICS system. Note: The scenarios were used to provide situations that would allow us to demonstrate the use CICS Performance Analyzer reports and extracts. The CICS regions were not necessarily tuned for peak performance and in some cases had a high level of tracing active. Therefore, these scenarios and the results provided should be seen as demonstrations only and do not provide definitive results for a customer environment.
167
168
JCICS application on resettable JVM EJB application with Xresettable=YES EJB application on resettable JVM with a resettable event or JVM phaseout TCP/IP connection status TCP/IP Maxsockets
169
170
Chapter 9.
171
TPNS
SC66
PTA1
PTA2
HTTP
CMAS PAA1
PJA2 WUI
CICS PM CLIENT
CAS
172
We give it a Description of ENQ Delays, an Event severity of HW (High Warning) and an operator of GE to a value of 00:00:10. In other words, if we get a task delayed by an ENQ for more than 10 seconds, this event will be triggered. General task response delays seems another reasonable field to look at, so we set that to 10 seconds as well. The resource group level below Tasks is Taskfile, which looks suitable, but on closer inspection the resource types in there seem to contain more file activity type information, which we dont want because we dont know our applications likely counts. Note in each case we set the Resource identifier to * to allow it to default. Descriptions of the resource types can be found in the CICSPlex System Manager Resource Tables Reference, SC34-6027. We found it easier to look in the View editor of the CICSPlex SM Web user interface, which when selected to create a view within a view set, contains all the definitions of the resource types referred to in the resource groups. Searching through the resource groups for appropriate definitions, we find some in the resource group level Local Files. Selecting three of these and saving each threshold definition gives us a threshold definition as the one shown in Figure 9-3 on page 174.
173
The three local file thresholds weve gone for here are NUMSTRINGWT, which gives us the current number of string waits by file; WSTRCNT, which gives us the total requests queued because all the strings were in use by file; and WSTRCCURCNT, which gives us the peak number of tasks waiting for a string. The first two we set to trigger if GE to 8 and 12, respectively, and assigned as high severity. The third we set to trigger if GE 12 and assigned very high severity. We omitted LSR buffers, as it did not seem to contain anything useful to us for this scenario. From group LSR POOLS we specified thresholds for TOTWAITREQ, which is Total number of string waits, and MAXCCURSTR, which is the peak number of concurrent active strings, giving both values of High severity. We set TOTWAITREQ to 2000, and MAXCCURSTR to 30. Finally, from group ENQ Pools in Regions we select NQGCNQSW, which is the number of Enqueue requests waiting currently, to trigger if GE 2 at high severity. Then we selected Event Monitor, set our scope to just look at our AORs, and unleashed our workload.
174
The three High warnings shown have inadvertently triggered the Task response time threshold. They are in fact the CONL, COI0 and COIE long-running CICSPlex SM tasks, so are not of concern. Our first indication that we are overloading things comes with the following event picked up by the Event monitor, shown in Figure 9-5 on page 176.
175
Event TOTWAITREQ has been triggered showing that the total number of tasks delayed for strings in LSRPool 2 has risen above 2000, our first indicator of problems. In fact its above 5000, so maybe the threshold was too low. But it indicates a problem, and we can get more information by double-clicking in the Identifier column with the 02 in it. That then hyperlinks us through to the Web user interface to the screen shown in Figure 9-6 on page 177.
176
We can get some more detailed information here, and we can see that the total number of string waits is increasing rapidly, with a peak concurrent string wait of 49. We can also see that LSR pool 2 only has 2 strings, which seems to be a rather small number. Back to the Event monitor on CICS PM where we can see an escalation in the number of triggered events in Figure 9-7 on page 178.
177
Three more events have now been triggered showing that three filesITEMACTX, ITEMMASX, and LABOPSDXhave gone beyond our threshold for total requests queued because all available strings were in use. Double-clicking again on the identifier column for ITEMACTX brings us to the screen shown in Figure 9-8 on page 179.
178
This doesnt tell us much, other than that the file is in LSR pool 2. Selecting the 02 here, however, takes us to the detail display for LSR pool 2 shown in Figure 9-9 on page 180, which is more interesting.
179
We can see statistics for LSR pool 2 here, including the total number of string waits, the number of requests waiting, the maximum key length, etc. We also have a hyperlink to Files in this LSR Pool, which we select to bring us to the screen shown in Figure 9-10 on page 181.
180
There are quite a few files in LSR pool 2, which has only two strings defined to it, and many of them are having problems, as we can see if we return to the CICS PM Event monitor screen shown in Figure 9-11 on page 182. We will return to this Web user interface screen shortly as there are operations we can do against all files in LSR pool 2 using this display.
181
The number of files exceeding the string limit is increasing now, and there are a large number of tasks delayed, moving up into the 40 - 50 second mark. We can confirm that the VSAM problems are the cause of these task delays by looking at the tasks display screen in the Web user interface following the hyperlink on Tasks in the resource column. This generates a task display as shown in Figure 9-12 on page 183.
182
As we can see from this, there are a large number of PX3 transactions delayed for FCSRSUSP and FCPSWAIT reasons, coming from both TORs. If this were in a real production situation, we may well have a customer ringing us to find out what the problem is, so we decide to take drastic action. Not knowing the application mix, and reluctant to cause an outage, we decide to quickly define a new LSR pool, LSR pool 4, with the characteristics of LSR pool 2, but with considerably more strings, 100 instead of 2. Using the facilities of the CICS PM Web user interface3, we issue a close disable to all the files in LSR pool 2, by ticking the Select record All box in Figure 9-10 on page 181, which generates the confirmation panel shown in Figure 9-13 on page 184.
183
Figure 9-13 shows three options selected, Disabled, Closed and the LSR pool ID field. In fact, the practise we found was to select the disable and close options in the appropriate boxes, then select Yes to all, then check that the files have reached disabled and closed status by refreshing the files display for LSR pool 2 a few times until they were all in that state, before amending the LSR pool ID and again replying Yes to all. We got confirmations in both cases, SET REQUEST COMPLETED SUCCESSFULLY, and could subsequently see that all the files formally in LSR pool 2 were now in LSR pool 4. We then set everything to Open Enabled by again selecting Select record to All, changing the two status indicators, and again replying Yes to all. When we now return to the CICS PM status monitor, everything has returned to normal, as shown in Figure 9-14 on page 185.
184
So perhaps amending the LSR pool for the problem files was the right thing to do, but there will doubtless be questions asked about the number of transaction abends that occurred while we were doing it, which, although we did not have a threshold set for it, we can see from the Web user interface display shown in Figure 9-15 on page 186.
185
At this point, however, the problem once again may be discovered by a phone call from an irate customer complaining of hung systems and slow responses. Looking again at the CICS PM Event monitor display shown in Figure 9-16 on page 187, we can see that a new problem has arisen.
186
Tasks are again hanging, and our Enqueue threshold has been triggered, both for a task and for the ENQ pool FCDSRECD. We can again hyperlink from this display to the Enq pools display by double-clicking the link ENQ Pools in the Resource column. This takes us to a display for the Enqueue manager, shown in Figure 9-17 on page 188.
187
This confirms the Enqueue pool ID, and shows the total number of enqueues issued in that pool, and the total number that have waited. Selecting the Enqueue pool ID FCDSRECD takes us to a detail display for the pool that shows us information such as the total number of enqueues that have been issued, the number that have been rejected, the number that have waited, and so on. At the bottom of the Enqueue manager detail display, we have another link to Units of work enqueue, which takes us to a display shown in Figure 9-18 on page 189.
188
This shows a number of current enqueues, some tasks that are owners of enqueues, some that are waiting on enqueues. Refreshing the screen a few times shows that one particular enqueue-owning task is staying around and not moving, the SC6 at the top of the display, task number 98933. Selecting the UOW ID hyperlink takes you to the Unit of work enqueue detailed display, shown in Figure 9-19 on page 190.
189
We can get more information about the enqueue and the task holding it here, including a hyperlink to the Resource name, which is obviously a very useful field to get to in debugging the situation, and we can link to the task and transaction details, which may be useful for getting rid of it. The Resource name panel generated is shown in Figure 9-20 on page 191.
190
This is actually a DSName level lock, indicating that the UOW is engaged in doing some sort of file-wide operation on the file. Going back to the Unit of work enqueue display in Figure 9-19 on page 190 and selecting the task number associated with that unit of work takes you to the task detail display screen shown in Figure 9-21 on page 192.
191
One of the things we can see from Figure 9-21 is that the task is purgeable, so thats what we do, having noted the name of the transaction and enqueued resource to give to the programmer responsible.
192
With the offending task removed, things start to come back to normal.
193
TPNS
3270 VSAM/RLS workload
SC66
PTA1
VSAM RLS
CICS PM Client
194
We set RLSWAITTIME to .05 seconds, which is higher than the observed response time component in the CICS PA VSAM RLS scenario, but we are, of course, potentially overloading the system. In the resource group level Taskfile, we set MNRRLSIOWT (RLS wait time and I/O count) to .05. Also (and we cheated slightly here by doing a couple of dry runs), we set the Connectivity root node, resource group level ISC & MRO, to trigger an alert if there were more than six concurrent queued allocates for sessions, and more than 20 concurrent outstanding allocate requests. We retained the enqueue, file, and LSR pool settings to the values we had previously set, and added some settings for the Logging root node, resource group MVS Log streams, as shown in Figure 9-25 on page 196.
195
The settings in Figure 9-25 are fairly arbitrary. They are RWRITES, the logger write rate which we set to trigger at 20; TFCWAIT, the total waits for buffer flushes, which we set to trigger if exceeding 20000; and LGGLGDEFER, the logger deferral interval, which we set to 4.
196
We can hyperlink from this wait event, double-clicking the Identifier attribute, but the RLS wait time component, although a major factor in the overall transaction response time, is in fact very small, and the transactions are running quickly, certainly too quickly for our hyperlink to catch them. By selecting the Activity -> Tasks link from the navigation panel in the Web user interface screen that we hyperlinked to, we can see the cause of threshold triggering, as shown in Figure 9-27 on page 198.
197
The level of accesses by concurrent IX8 transactions to file ITEMACTX is enough to cause the RLS delays were seeing. Obviously, the frequency setting in the TPNS script has allowed quite a few IX8 transactions to kick off initially, and these are impacting on each otherindicating perhaps that our thresholds are too low for the throughput were allowing. Another event is triggered, which is an indication of logger delays, shown in Figure 9-28 on page 199.
198
A logstream event has occurred, the threshold we set for TFCWAIT, the total wait for buffer flushes. Double-clicking on this takes us to a screen with the MVS log stream names highlighted. Selecting the DFHLOG, one of these takes us to a screen where we can get some more information about this, shown in Figure 9-29 on page 200.
199
The total number of tasks that have waited for buffer flushes is high, but this figure will always increase unless statistics are reset, and this is the threshold we ve triggered. Possibly not, therefore, a good threshold to have active. However, it may indicate a need to perform some logstream tuning, that is, the avgbufsize and maxbufsize parameters in the log structure definitions may warrant attention. The peak number at any one time has been 12, which may of course reflect the high concurrent workload. Returning to the CICS PM monitor, we can see that another log stream threshold has been triggered, shown in Figure 9-30 on page 201.
200
Along with our TFCWAIT and RLSWAITTIME thresholds, the threshold RWRITES has been triggered, the rate of writes to the logger. Double-clicking this hyperlink takes us back to the logstream details screen shown in Figure 9-29 on page 200. This threshold appears to directly reflect our workload, but we can reduce it by increasing our AKPFREQ setting. The performance guide recommends using the default setting of 4000 for AKPFREQ. Looking at our region setting display shown in Figure 9-31 on page 202, we can see it has been defined as 1000 on the two AORs, SCSCPAA1 and SCSCPAA4.
201
AKP settings can be amended through the Web user interface, so we increase both values from 1000 to 4000, action the change with Apply changes and reset the statistics. Shortly afterwards the event disappears from the monitor. The next indication of an event arrives. This is one we havent set, as shown in Figure 9-32 on page 203.
202
An event !!SAMMAX has appeared on the CICS PM Event monitor. The resource is shown as unknown, but the Description column to the far right reads Max tasks, and the region is shown as SCSCPTA1, one of the TORs. SAMMAX is a CICSPlex SM system availability monitoring function for maximum tasks reached, a predefined condition that weve picked up on. Furthermore, looking at the Region histories display as in Figure 9-32, we can see that where both our AORs are triggering events, and one of our TORs has shown problems, one is not registering any triggered events, which seems a bit suspicious, and may be the root of the max tasks situation. Double-clicking the Region column for SCSCPTA1 hyperlinks to a region display where we can see that max tasks have indeed been reached, as shown in Figure 9-33 on page 204.
203
The max task setting is 200, and weve reached it 73 times, so we double it to 400 (which you might not do in a production situation). We also discover a problem in our virtual terminal designations on our TPNS script that is directing traffic to only one TOR, so we correct that and the terminal workload starts to spread more evenly. Another potential problem indicator appears, again due to our throughput ratio, shown in Figure 9-34 on page 205.
204
We have two OUTSALLOC events triggered from TOR SCSCPTA1 to both the AORs, indicating that the peak number of outstanding allocate requests for sessions on these connections has passed 20, and we can confirm this by hyperlinking on the identifier for the sessions, and also by looking at the task delay factors on the Web user interface as shown in Figure 9-35 on page 206.
205
The task display shows activity on the TOR SCSCPTA1 with delays for files, logger delays, and delays for sessions. The delays are all fairly minor. You have to refresh the screen once or twice to catch them, but they do reflect an anomaly in that the outgoing session numbers from SCSCPTA1 to the AORs are specified as 200, whereas on the AORs going back the sessions are limited to 99. Given our throughput, 99 is not adequate and will cause the queued sessions as shown. However, we cant amend this without stopping the IRC link, so we leave it for the next outage. Another indicator that our throughput is perhaps too high and is being impacted by bottlenecks can be seen in the next task display, Figure 9-36 on page 207, where we see a large number of dispatcher waits.
206
9.4 Conclusion
The VSAM scenarios in this chapter are intended to show some of the functions of CICS PM and the Web user interface. We have shown how thresholds can be defined to be triggered when specific events occur, and that the event displays can be used to hyperlink to Web user interface panels that contain more data about the problems causing the events to occur. The CICSPlex SM commands that can be performed through the Web user interface can allow corrective CEMT Perform and Set actions to be issued to correct problems, for instance tasks may be purged, or settings may be increased or decreased. None of the delays in the VSAM RLS scenario actually caused our arbitrary task response time threshold of 2 seconds to be triggered, other than the long-running CICSPlex SM tasks. Apart from showing some of the functions of CICS PM, therefore, they also indicate that it is necessary to exercise constraint when defining thresholds. Thresholds defined to be triggered when peak events have occurred, such as the TFCWAIT threshold we set, may reach the threshold inevitably throughout a run of CICS between statistics reset intervals, and may not, therefore, be meaningful. Similarly, thresholds that are set too low will be triggered constantly, and will flood the event monitor while not actually being indicative of a problem. Unstructured tuning to remove
Chapter 9. VSAM system problems scenarios
207
arbitrary thresholds that are indicating bottlenecks will often simply relocate the bottleneck, as with our example, where increasing max tasks allowed the number of sessions to become exhausted. Increasing both might have further overloaded our RLS manager, or impacted our systems further for dispatcher availability. The key to the use of CICS PM to manage CICS online problems and to provide alerts for relevant personnel, is quite clearly to define meaningful and accurate thresholds. The information that can be pinpointed and reached by CICS PM hyperlinks to the Web user interface can be supplemented by setting site-specific Web user interface screens through the Web user interface View editor, to provide the kind of information pertinent to the application mix and likely problem situations that the experienced systems programmer can anticipate.
208
10
Chapter 10.
209
210
TCBs
OPENAPI - DB2 6 and above SUBTASK - DB2 5 and below
Threads
Command Unprotected Protected Unprotected Protected Unprotected
Entry Pool
Figure 10-2 on page 212 shows how TCBs and threads are related to your DB2 connection between your CICS transactions and DB2 data.
211
DB2 Connection
CICS
Tran A1 Tran A2 Tran B Tran Y Tran C Tran DSNC Tran D
TCBs
Entry Thread for A
DB2
Plan
Pool Thread
PROTECTed
Pool Thread
Pool Thread
Command Thread
Unused TCB
Threads
Figure 10-2 CICS DB2 thread overview
The thread overview picture in Figure 10-2 shows a system that is using all available threads and has a transaction D that is waiting on a pool thread. Transaction A has dedicated threads and all other transactions are forced to use the pool. There is one command thread in use via the DSNC transaction. The key object of this thread picture is to show that you need to have enough TCBs to cover all active threads, otherwise youll cause transactions to wait on TCBs. In this example we could have increased the pool threads to 4 to eliminate transaction D from waiting. The following SIT parameter is required to create the total number of TCBs in Figure 10-2:
MAXOPENTCBS = 7
Remember that your total number of TCBs has to be equal to all your command, pool, and entry threads added together, or larger. The CSD DB2Conn definition in Example 10-1 on page 213 was used to create the three pool threads in Figure 10-2. It is also allowing the CICS region to use all available TCBs by having the TCBlimit parameter match the MAXOPENTCBS parameter in the SIT. Finally, it defines the one command thread we need so we can issue DB2 commands.
212
The CSD DB2Entry definition in Example 10-2 was used to create the two dedicated entry threads for transaction A setting 1 of them as a protected thread. It is also forcing all excess thread requests to wait for an available thread.
Example 10-2 CEDA DB2Entry definition parms for transaction A
DB2Entry PROtectnum = 1 THREADLimit = 2 THREADWait = Twait
213
What you'll probably end up doing is a little of the above methods all at once, making some educated guesses to start off, then using CICS PM to monitor your settings and making some adjustments to get things working the way you want. Now for some suggestions on when to make adjustments and why.
Pool Threads
Dedicated Threads
The big question is which is longer, the amount of time you waited or the amount of time you saved by reusing a thread.
DB2 for z/OS and OS/390 Version 7 Selected Performance Topics, SG24-6894 CICS TS for z/OS: CICS DB2 Guide, SC34-6014
215
Threshold Resources
Event Resources
Threads and subsys resources are CICSPlex SM generated and thus do not have a one-to-one link with CICS resources.
Figure 10-4 DB2 thresholds and events in CICS PM GUI
In the next few sections we briefly go over each DB2 Threshold group, then we set up some sample thresholds and go over a couple of DB2 workload scenarios to show how they can be used at a typical CICS site.
216
Information about each Attribute selection can be found in the DB2CONN Resource Table section of CICSPlex System Manager Resource Tables Reference, SC34-6027. Additionally, you can find information for most of the parameters in the CEMT INQUIRE DB2CONN section of CICS Supplied Transactions, SC34-5992. The CICSPlex SM manual will give you the syntax and short description, and the Supplied Transactions manual, if it includes the parm you are looking for, will give a detailed description. Obviously you can use the DB2CONN thresholds to monitor your CICS DB2 Interface, displaying an event alert when your DB2 connection has been disconnected. But not as obvious is the fact that your pool threads are defined under the DB2 Connection. Thus you use the DB2CONN thresholds to monitor your POOL threads and connection TCBs as well. For our sample testing we chose to set the DB2CONN thresholds shown in Figure 10-6 on page 218.
217
218
Information about each Attribute selection can be found in the DB2ENTRY Resource Table section of CICSPlex System Manager Resource Tables Reference, SC34-6027. Additionally, you can find information for most of the parameters in the CEMT INQUIRE DB2ENTRY section of CICS Supplied Transactions, SC34-5992. The CICSPlex SM manual will give you the syntax and short description, and the Supplied Transactions manual, if it includes the parm you are looking for, will give a detailed description. The DB2ENTRY thresholds can be used to monitor your DB2 Entry non-pool threads. There are thresholds for both protected and non-protected threads. You can also find thresholds to monitor how many transactions are waiting for a DB2 entry thread. For our sample testing we chose to set the DB2ENTRY thresholds shown in Figure 10-8 on page 220.
219
220
Information about each Attribute selection can be found in the DB2TRN Resource Table section of CICSPlex System Manager Resource Tables Reference, SC34-6027. Additionally, you can find information for most of the parameters in the CEMT INQUIRE DB2TRAN section of CICS Supplied Transactions, SC34-5992. The CICSPlex SM manual will give you the syntax and short description, and the Supplied Transactions manual, if it includes the parm you are looking for, will give a detailed description. The DB2TRN thresholds are rather uneventful. Most of your event alert action will take place using the DB2CONN and DB2ENTRY thresholds, but if you really want to get creative, the CICS PM developers provide them for your use. We did not set any DB2TRN thresholds for our tests.
221
Information about each Attribute selection can be found in the DB2THRD Resource Table section of CICSPlex System Manager Resource Tables Reference, SC34-6027. Since these values are sourced internally from within CICSPlex SM. They cannot be found outside of the CICSPlex SM manuals. The DB2THRD thresholds partially duplicate the DB2CONN thresholds and are CICSPlex SM-generated data. Because we were using DB2 7.1, CICS TS 1.3, and CICS TS 2.2, we did not receive any data for these thresholds. These thresholds are for use with releases before the CICS SPI was available. None of the DB2THRD thresholds ever popped during our tests, even though they were set very low.
Note: When the DB2 attach adapter became part of CICS, they introduced the SPI on CICS. CICSPlex SM exploited this new feature with DB2CONN and DB2ENTRY. Prior to this, when no SPI was available in CICS, CICSPlex SM peeked in the RCT and got the information from the DB2SS and DB2THRD views.
For our sample testing we chose to set the DB2THRD thresholds shown in Figure 10-11 on page 223.
222
223
Information about each Attribute selection can be found in the DB2SS Resource Table section of CICSPlex System Manager Resource Tables Reference, SC34-6027. Since these values are sourced internally from within CICSPlex SM. They cannot be found outside of the CICSPlex SM manuals. The DB2SS thresholds partially duplicate the DB2CONN thresholds and are CICSPlex SM-generated data. Because we were using DB2 7.1, CICS TS 1.3, and CICS TS 2.2, we did not receive any data for these thresholds, which are for use with releases before the CICS SPI was available. None of the DB2SS thresholds ever popped during our tests, even though they were set very low.
Note: When the DB2 attach adapter became part of CICS, they introduced the SPI on CICS. CICSPlex SM exploited this new feature with DB2CONN and DB2ENTRY. Prior to this, when no SPI was available in CICS, CICSPlex SM peeked in the RCT and got the information from the DB2SS and DB2THRD views.
For our sample testing we chose to set the DB2SS thresholds shown in Figure 10-13 on page 225.
224
225
Information about each Attribute selection can be found in the XTASK Resource Table section in CICSPlex System Manager Resource Tables Reference, SC34-6027. Additional information on the four fields of interest, DB2RDYQW, DB2CONWT, DB2WAIT & DB2REQCT, can be found in the CICS TS for z/OS: CICS DB2 Guide, SC34-6014 and the CICS Performance Guide, SC34-6009. The CICSPlex SM manual will give you the syntax and short description, and the DB2 Guide will give a detailed description.
Note: For DB2 Version 6 and above, the DB2WAIT value will always be zero.
The XTASK thresholds are system clock times and can be used to monitor your DB2 tasks for conditions that take too long. In particular, if a task waits longer than .05 seconds for a thread, you could have an event alert pop up. For our sample testing we chose to set the XTASK thresholds shown in Figure 10-15.
226
Note: These scenarios were used to provide situations that would allow us to demonstrate the use of CICS PM monitoring when running with a CICS DB2 workload. The CICS regions were not necessarily tuned for peak performance and in some cases had a high level of tracing active. Therefore, these scenarios and the results provided should be seen as demonstrations only and do not provide definitive results for a customer environment.
Figure 10-16 gives you a high level overview of the test system used for our test scenarios.
SC47 TPNS
3270 DB2 workload
SC66
PTA1
PTA2
DB2
Figure 10-16 Subset of our test system architecture used for this chapters testing
Our basic test system consists of two TORs connected to two AORs connected to a single DB2 V7.1 subsystem. A VTAM generic is used to balance work between the two TORs. The actual eDB2 transactions are hard coded to go to specific AOR regions. TPNS is used to run a varying workload running one of three CICS/DB2 transactions, one of which performs an update.
227
To start off, open the CICS PM GUI and go to the Threshold Definitions tab, select a group that you want to use or create a new one if necessary, then select the DB2 Connection resource under the Files & DB2 CICS Resources section, as shown in Figure 10-17.
Using the Thresholds pull-down menu, select New to create a new threshold in your threshold group as shown in Figure 10-18.
228
We then select the CONNECTST attribute to monitor the connections status, as shown in Figure 10-19.
Figure 10-19 Choose the CONNECTST attribute to monitor the status of your connection
Click NE and then select CONNECTED from the pull-down box to make a conditional statement that says your DB2 connection will trigger an alert if its CONNECTST attribute is not equal to CONNECTED, as shown in Figure 10-20.
In Figure 10-21 on page 230 we fill in the rest of the fields. Either enter a * (asterisk) in the resource identifier field, or use some level of qualification if you have various connection
229
group names. We chose DB2* to pick up all DB2CON connections, yet omit or ignore our other connections called RCT21. Add a 30-character description. Use something short but sweet, because this is one of the key fields that will be used when your threshold event triggers. In fact, if you can tell what action to perform that would be even better. We chose to say Make sure DB2 is connected, but we could have said something like DB2 needs to be reconnected Next fill in the Sample time interval with a time in seconds of how often you want this object to be checked, and then give it a Severity. We chose VHS because this failure would stop everything dead in its tracks and could then trigger other alerts or could start generating AEY9 and/or AZI6 abends.
To finish up your new threshold, click Save -> Install, which will generate a prompt. See Figure 10-22 on page 231, as to where you want your threshold installed. In our environment we had a group called CSGAOR set up that contains both the PJA6 and PJA7 CICS regions, which are the two CICS regions that process all DB2 transactions. You could either use the Shift or Ctrl keys and select each region directly, or you can choose the group, CSGAOR, as we did.
230
Once we defined and installed our DB2 connection threshold, we then clicked the DB2 Connections tab in the CICS PM GUI events tab Resource Summary section in Figure 10-23 on page 232, which opened a DB2 connections Web user interface window, shown in Figure 10-24 on page 232.
231
Figure 10-23 Using the CICS PM GUI to start a DB2 Connection WUI window
We then selected the check box on the left of the PJA6 region and clicked Disconnect to drop the connection setting status to Notconnected.
Just to verify that we are doing the right thing, we are presented with a verification screen. We just clicked Yes to perform the actual disconnect.
232
The CICS PM Events screen started showing DB2CON connection alerts for the PJA6 region, identifying that it is Notconnected, as shown in Figure 10-26.
The excerpt from the CICSPlex System Manager Resource Tables Reference, SC34-6027 in the table below shows you the source for the DB2ENTRY threshold data and what it means.
Name
CONNECTST
Source
INQ
Description
Connection status
233
The CONNECTST threshold points out that our DB2CON connection has changed its status to anything but CONNECTED. At this point the solution seems very obvious and the fix is also very simple: Just use the Connect button to reconnect the DB2 interface. In our simple test environment thats what we will do, but in a large complex shop, other issues may come into play. Most notably, you may be flooded with other alerts. This is why our alert was given the highest severity possible, to hopefully bring it to the top of the list. This is a good time to point out that you should choose the severity of your alerts wiselydont make everything Very High Severity. For instance, if you make max task a High Severity alert, then your connection alert would sort to the top and allow you to detect why you have gone max task. Another issue to cover at this junction is why the connection went away. Perhaps DB2 is being recycled or is being brought down for scheduled maintenance. You will need to carefully educate your CICS PM users as to what actions to take. Well go through a couple of screen shots to show you how we used CICS PM to correct the situation.
Figure 10-27 Clicking the Identifier from the event alert brings you here
Figure 10-27 shows you that the status of your DB2 connection is Notconnected, but we already knew that from the alert. But in reality, if DB2 was just being recycled, it could actually have automatically reconnected by the time you get to this screen. Since thats not the case with our scenario, we need to reconnect the connection, which we do by selecting the DB2CON connection by checking the check box on the left in Figure 10-28 on page 235.
234
Then click the Connect box, which takes you to a verification screen to give you a chance to verify your actions and, if necessary, cancel without taking any action. In our case lets go ahead and click Yes in Figure 10-29.
You will then get a confirmation message and see in Figure 10-30 on page 236 that the connection status is back to Connected.
235
Thats it, everything is back to normal. But remember that this is only a test. In the event of a real emergency, your phones would have rung off the hook and several managers may have walked into your office.
We started a TPNS workload running on 200 terminals that were workload-balanced between the two TORs, which then route the transactions down to the AORs for the actual processing of the DB2 requests.
236
Alerts started to show up on the CICS PM Events screen as seen in Figure 10-31. The DB2U transaction has reached its max thread limit of 5 and has eight tasks waiting for threads.
The excerpt from CICSPlex System Manager Resource Tables Reference, SC34-6027 in Table 10-1 shows the source for the DB2ENTRY threshold data and what they mean.
Table 10-1 DB2ENTRY threshold data
Name
THREADS RQCUR
Source
INQ STAT
Description
Number of active threads Ready Queue Current
The THREADS threshold points out that we have reached our pr-set threshold limit of 5 active threads, and the RQCUR threshold shows that we have more than 5 tasks on the Ready Queue waiting for a thread. We also set up two versions of the THREADS threshold. One was set at THREADS GE 4 to give us a warning and the other at the max of 5 with a higher severity to indicate we are out of threads. Since we lowered our threads to a very low level to cause a system problem on purpose, we know exactly what actions we need to take to resolve the problem: just raise the available thread limits. If this was not a simulation, we would then also be looking for other event thresholds that may indicate what other types of problems may be involved. Well go through a couple of screen shots to show you how we used CICS PM to diagnose and correct the situation.
Chapter 10. DB2 system problems
237
By clicking the DB2U Identifier field from the Event screen shown in Figure 10-31 on page 237, CICS PM opens up a Web browser window taking you to the DB2 Entry status screen seen in Figure 10-32.
Figure 10-32 Selecting the Thread Ready Q alert or the Thread limit alert takes you here
It doesnt matter which alert you choose to click on in Figure 10-31 on page 237, they both take you to the same place, Figure 10-32. We then click the DB2U field in the DB2 entry name column, which take us to the DB2 entry general information detail screen shown in Figure 10-33 on page 239. There you can see the actual Maximum number of threads setting for tran DB2U, currently set to 5.
238
Figure 10-33 DB2U entry details screen where we make adjustments to its dedicated threads
After determining a new value, Figure 10-34 on page 240 shows how we can overtype the current value of 5, changing it to 8 for our example (dont forget, you must also select the check box to the left of it).
239
Now that we have our new values set, select the Apply changes box to install our new limit. Figure 10-35 on page 241 shows the results of selecting Apply changes. You should receive a successful completion message at the top of the screen, and the new value of 8 is now set for the Maximum number of threads field.
240
We didnt include it in this example, but depending on the type of threshold you set, you may need to readjust your thresholds. In this case, raising the value for the DB2U Entry threads from 5 to 8 will still cause a true condition. The initial THREADS alerts in Figure 10-31 on page 237 will never go away until the threshold itself is adjusted and reinstalled. Unlike the THREADS alert, the RQCURR threshold is based on a queue count. Once the threads have been increased, the tasks waiting should clear out and the RQCUR alert should go away on its own.
Example 10-4 DB2 Entry definitions with 0000 dedicated threads and Pool overflow
I DB2E STATUS: RESULTS - OVERTYPE TO MODIFY Db2e(DB2N ) Tas Use Ena Poo Hig Pro( 0000 ) Pth(0000) Threadl( 0000 ) Threads(0000) Tpo Plan( PROGDB2N ) Db2e(DB2R ) Tas Use Ena Poo Hig Pro( 0000 ) Pth(0000) Threadl( 0000 ) Threads(0000) Tpo Plan( PROGDB2R ) Db2e(DB2U ) Tas Use Ena Poo Hig Pro( 0000 ) Pth(0000) Threadl( 0000 ) Threads(0000) Tpo Plan( PROGDB2U )
Example 10-5 shows the DB2 Connection settings. Were keeping the pool settings pretty small so we can trigger some alerts. The test environment is using a DB2 V7 subsystem, which has the newer L8 TCBs dedicated for DB2 threads. The Tcblimit parameter limits us to using only 4 of the 15 available MAXOPENTCBS, and the Threadlimit restricts us to using 3 active thread connections.
Example 10-5 DB2 Connection with 4 TCBs and 3 pool threads
I DB2C STATUS: RESULTS - OVERTYPE TO MODIFY Accountrec( None ) Planexitname( ) Authid( ) Priority( High ) Authtype( Sign ) Purgecyclem( 00 ) Comauthid( ) Purgecycles( 30 ) Comauthtype( Cuserid ) Resyncmember( Comthreadlim( 0001 ) Signid( CICSTS ) Comthreads(0000) Security( ) Connecterror( Sqlcode ) Standbymode( Reconnect ) Connectst( Connected ) Statsqueue( CDB2 ) Db2groupid( ) Tcblimit( 0004 ) Db2id( D7Q2 ) Tcbs(0000) Db2release(0710) Threaderror(N906d) Drollback(Norollback) Threadlimit( 0003 ) Msgqueue1( CDB2 ) Threads(0000) Msgqueue2( ) Threadwait( Twait ) Msgqueue3( ) Nontermrel( Release ) Plan( TRADERP )
We started a TPNS workload running on 200 terminals that are workload-balanced between the two TORs, which then route one of the 3 DB2x transactions down to the AOR for the actual processing and DB2 requests. Alerts popped up almost immediately. The CICS PM events screen in Figure 10-36 shows that the DB2 Connection THREADS object has reached 3 and the DB2 Connection TCURR object has also reached 3.
242
The excerpt from CICSPlex System Manager Resource Tables Reference, SC34-6027 in Table 10-2 shows the sources for the DB2CONN threshold data and what they mean.
Table 10-2 Sources for the DB2CONN threshold data
Name
THREADS TCURR
Source
INQ STAT
Description
Number of active pool threads TCB Current
The THREADS threshold points out that we have reached our preset threshold limit of 3 active threads and the TCURR threshold shows that we also have used up 3 L8 TCBs. Because we lowered our TCBs and Threads to a very low level in order to cause a system problem on purpose, we know exactly what actions we need to take to resolve the problem: just raise the available TCB and Thread limits. If this was not a simulation, we would also be looking for other event thresholds that may indicate what other types of problems may be involved. Well go through a couple of screen shots to show how we used CICS PM to diagnose and correct the situation. By clicking the DB2CON Identifier field on the Event screen shown in Figure 10-36, CICS PM opens up a Web browser window, taking you to the DB2 Connection status screen seen in Figure 10-37 on page 244.
243
Figure 10-37 CICS PM Connection display link from the Event screen
We then click the DB2CON field in the DB2 connection name column, which will take us to the DB2 connection - general information detail screen for our PJA7 regions DB2 connection, shown in Figure 10-38 on page 245. We need to scroll down and find the Maximum number of subtask TCBs field and overtype the value, changing our current limit of 4 to a new value of 8. You also select the check box to the left of it. Next we scroll down to find the Maximum number of pool threads field and overtype the value, changing our current limit of 3 to a new value of 6, and again you must also select the check box to the left of it. Now that we have our new values selected, just scroll down to the bottom of the screen and select Apply changes to set our new values.
244
Figure 10-38 DB2 Connection detail screen showing new TCB and Thread values
After clicking Apply changes, you should get a message at the top of your screen saying your changes have been applied successfully, as seen in Figure 10-39 on page 246.
245
Figure 10-39 Successful update of the TCBlimit and Threadlimit connection parameters
Now that we have modified our system values, we need to also consider updating the same limits in our other AOR, the PJA6 region, along with readjusting the threshold limits that identified the problem in the first place. Figure 10-40 shows using the CICS PM client Threshold Definitions screen. The two alerts that identified our problem are highlighted and need to have new values set to accurately signify when we have hit our new maximums.
Once youve adjusted the threshold values, click Save -> Install and install them in the appropriate CICS regions, and you are done.
246
Figure 10-41 Custom menu for our tailored View displays of DB2 resources
Clicking on the List all Connections with Details link in Figure 10-41 takes you to our custom DB2 Connection Status screen in Figure 10-42 on page 248, and clicking on the List all Entry Definitions with Detail link in Figure 10-41 takes you to our custom DB2 Entry status screen in Figure 10-44 on page 250.
247
Figure 10-42 Custom DB2 Connection screen showing max active threads and threads waiting
After the problem was resolved, the screen shown in Figure 10-42 was refreshed and you can now see the results of increasing the TCBlimit and Threadlimit parameters in Figure 10-43 on page 249.
248
Figure 10-43 Custom DB2 Connection screen showing the SCSCPJA76 region has been corrected
249
Figure 10-44 Custom DB2 Entry screen which shows all tran activity currently using the pool
250
TRANSACTION: DB2U PROGRAM: PROGDB2U TASK: 0085671 APPLID: SCSCPJA6 DISPLAY: 00 STATUS: COMMAND EXECUTION COMPLETE CALL TO RESOURCE MANAGER DSNCSQL EXEC SQL UPDATE P.AUTH=CICSUSER, S.AUTH= PLAN=PROGDB2U, DBRM=PROGDB2U, STMT=00387, SECT=00002 SQL COMMUNICATION AREA: SQLCABC = 136 AT X'115AA4C8' SQLCODE = 000 AT X'115AA4CC' SQLERRML = 000 AT X'115AA4D0' SQLERRMC = '' AT X'115AA4D2' SQLERRP = 'DSN' AT X'115AA518' SQLERRD(1-6) = 000, 000, 00001, -1, 00000, 000 AT X'115AA520' SQLWARN(0-A) = '_ _ _ _ _ _ _ _ _ _ _' AT X'115AA538' SQLSTATE = 00000 AT X'115AA543'
OFFSET:X'00186C'
LINE:UNKNOWN
EIBFN=X'1802'
ENTER: CONTINUE PF1 : UNDEFINED PF4 : SUPPRESS DISPLAYS PF7 : SCROLL BACK PF10: PREVIOUS DISPLAY
END EDF SESSION USER DISPLAY STOP CONDITIONS ABEND USER TASK
By using CEDF to run the DB2U transaction, which does updates, we were able to lock up a record for an indefinite period of time. This generated a ton of alerts, shown in Figure 10-47.
251
To help figure out what was going wrong we sorted by Attribute and noticed that most of the alerts were for DB2RDYQW. CICS TS for z/OS: CICS DB2 Guide, SC34-6014 describes this object attribute as:
The elapsed time in which the user task waited for a DB2 thread to become available.
Thus, at this point we know there are thread problems.
252
Figure 10-47 Check for Units of work, get an idea of what is out there
Lets look at the Units of work link, which is found under the Activity tab on the left side of the screen in Figure 10-47. Since there are many screens of data and the initial screen above only shows a few lines, we sort the data by Transaction that started UOW by clicking the sort icon.
253
The sorted display, which is much better than a CEMT Inquire display, lets us quickly see that there are many DB2U transactions out there, and having a little bit of knowledge of the application, we know they are Update transactions, so we can suspect a lockout.
254
Figure 10-49 Custom-made WUI screen that includes valuable data all on one screen
Using our custom-made DB2 Entry display, made with the CICSPlex SM View editor, we notice that the DB2U transaction has all 5 threads in use and has some waiting.
255
To make identifying problems like this easier, you should use the PRINT preview icon button on the top right corner of any screen to convert the display into a print ready version. When you have several hundred lines of output to look through it could take forever. No other tool or CEMT Inquire displays can match this feature. You now have a scroll bar on the right side of the screen and you can scroll up and down very quickly, looking for trends.
256
The UOW link screen, taken from the left tool bar menu under the Activity topic, shows you what kinds of work is out there waiting. You can see that there is RMI DSNCSQL work related to the DB2U work, so we can now suspect something on the DB2 side.
257
Again the print preview icon helps us look through 403 lines of data in just an instant.
258
Figure 10-53 Task screen from the menu on the left side of the screen
Sorting the TASK display by Task ID using the descending button lets us look for tasks with older task numbers, and of course at this point we are looking to purge a DB2U transaction.
259
Selecting the record box next to the old DB2U tran and then clicking Forcepurge quickly dispatches the offender and things start to clear up.
260
Going back to our custom display, remember that with the Web user interface windows, you can have every one you ever looked at still open. A quick refresh shows that no one is waiting on threads anymore and we are well below max thread usage.
TRANSACTION: DB2U PROGRAM: PROGDB2U TASK: 0072722 APPLID: SCSCPJA6 DISPLAY: 00 STATUS: COMMAND EXECUTION COMPLETE CALL TO RESOURCE MANAGER DSNCSQL EXEC SQL UPDATE P.AUTH=CICSUSER, S.AUTH= PLAN=PROGDB2U, DBRM=PROGDB2U, STMT=00387, SECT=00002 SQL COMMUNICATION AREA: SQLCABC = 136 AT X'115AA4C8' SQLCODE = 000 AT X'115AA4CC' SQLERRML = 000 AT X'115AA4D0' SQLERRMC = '' AT X'115AA4D2' SQLERRP = 'DSN' AT X'115AA518' SQLERRD(1-6) = 000, 000, 00001, -1, 00000, 000 AT X'115AA520' SQLWARN(0-A) = '_ _ _ _ _ _ _ _ _ _ _' AT X'115AA538' SQLSTATE = 00000 AT X'115AA543'
OFFSET:X'00186C'
LINE:UNKNOWN
EIBFN=X'1802'
ENTER: CONTINUE PF1 : UNDEFINED PF2 : UNDEFINED PF3 : END EDF SESSION PF4 : SUPPRESS DISPLAYS PF5 : WORKING STORAGE PF6 : USER DISPLAY DFHAC2206 17:37:14 SCSCPJA6 Transaction DB2U failed with abend AED3. Updates to local recoverable resources backed out.
261
262
Example 10-7 shows the DB2 Connection settings that will be used for the pool threads in the PJA6 region. Were keeping the pool settings pretty small so we can trigger some alerts. The test environment is using a DB2 V7 subsystem, which has the newer L8 TCBs dedicated for DB2 threads. The Tcblimit parameter limits us to using only 4 of the 15 available MAXOPENTCBs and the Threadlimit restricts us to using only 3 active thread connections at a time. What youll see in this test is that the PJA7 region needs at least 6 TCBs to process all the defined dedicated threads and we only have 4 available, thus we are guaranteed to have TCB waits.
Example 10-7 DB2 Connection with 4 TCBs and 3 pool threads
I DB2C STATUS: RESULTS - OVERTYPE TO MODIFY Accountrec( None ) Planexitname( ) Authid( ) Priority( High ) Authtype( Sign ) Purgecyclem( 00 ) Comauthid( ) Purgecycles( 30 ) Comauthtype( Cuserid ) Resyncmember( Comthreadlim( 0001 ) Signid( CICSTS ) Comthreads(0000) Security( ) Connecterror( Sqlcode ) Standbymode( Reconnect ) Connectst( Connected ) Statsqueue( CDB2 ) Db2groupid( ) Tcblimit( 0004 ) Db2id( D7Q2 ) Tcbs(0000) Db2release(0710) Threaderror(N906d) Drollback(Norollback) Threadlimit( 0003 ) Msgqueue1( CDB2 ) Threads(0000) Msgqueue2( ) Threadwait( Twait ) Msgqueue3( ) Nontermrel( Release ) Plan( TRADERP )
We initially started with one TPNS workload running 200 terminals that are workload-balanced between the two TORs, which then route our DB2 transactions down to the AOR for the actual processing and DB2 requests.
263
Figure 10-58 We start seeing alerts on the PJA7 region running the dedicated threads right away
The CICS PM Events screen starts showing a bunch of alerts for the PJA7 region. One alert is from the DB2CONN group and all the others are from the XTASK objects. The excerpt from CICSPlex System Manager Resource Tables Reference, SC34-6027 in Table 10-3 shows the source for the DB2CONN threshold data and what it means.
Table 10-3 Source for DB2CONN threshold data
Name
TCBRDYQHWM
Source
STAT
Description
Peak number of tasks on TCB Readyq
The TCBRDYQHWM threshold means that we have more tasks waiting for TCBs than our threshold limit of 814 tasks are waiting. This indicates that we need to increase our TCBLIMIT system parameter to get more throughput. Since we have lowered our TCBs and Threads to a very low level to cause a system problem on purpose, we know exactly what actions we need to take to resolve the problem: just raise the available TCB and Thread limits. If this was not a simulation, we would also be looking for other event thresholds that may indicate what other types of problems may be involved.
264
The excerpt from CICSPlex System Manager Resource Tables Reference, SC34-6027 in the table below shows you the source for the XTASK threshold data and what it means.
Table 10-4 Source for the XTASK threshold data
Name
DB2CONWT
Source
CMF
Description
DB2 Connection (TCB) wait time
The DB2CONWT threshold means that each alert is a task waiting on the ready queue for an OPEN TCB to process its thread. Notice that there are 14 of these alerts, so these are the tasks on the TCBRDYQ mentioned in the first alert. What this means is that all the alerts are telling us the same thing: that we need more TCBs to process the threads we have waiting. But if we only increase the TCBs, then we will just change over to THREAD wait alerts, so we need to raise both our TCBlimit and Threadlimit parameters. Well go through a couple of screen shots to show how we used CICS PM to diagnose and correct the situations:
Clicking DB2CON Identifier in Figure 10-59 takes you to the DB2 Connection screen in Figure 10-59. Region PJA7 is the region running with the dedicated protected threads. So at this point the PJA6 region is running fine with 4 TCBs and 3 pool threads, but the PJA7 region is unable to keep up with 4 TCBs and 3 transactions each using 2 dedicated threads. That definition setup seems to be a guaranteed problem. We can quickly see this with 3 transaction definitions, but what about systems that may have hundreds of DB2Entry definitions? You could easily have more thread requirements than your TCBs could handle. We can attempt to increase the number of threads that each transaction uses, but it wont do much good because there will still only be 4 TCBs for an even larger number of dedicated threads to fight over. So well start out by increasing the PJA7 regions Connection and TCBlimit to see if we can provide more resources for the 6 dedicated threads.
265
To help make it easier to see what is going on with 2 different regions both running different thread setups, were going to use 2 custom screens we made that include the necessary fields we need to see exactly what is going onall on one screen.
Figure 10-60 Custom DB2 Connection Status screen showing PJA6 using the pool
The screen in Figure 10-60 shows the DB2 Connection information for both PJA6 and PJA7 at the same time. The data on the first line shows activity in the pool threads, and the lack of data on the second line shows that the pool threads are not in use. Were getting a lot of pool thread waits in the PJA6 region, but due to the waits we are also picking up a lot of thread reuse, so you get a little goodness with the bad. Lets look at our custom DB2 Entry display screen. In Figure 10-61 on page 267 you can see that the activity is swapped. The PJA7 region is now getting statistical numbers, thus showing that PJA7 is using dedicated threads. The lack of information in the PJA6 lines shows that its activity is all taking place under the DB2 Connection area using the pool.
266
Figure 10-61 Custom DB2 Entry Status screen showing PJA7 using dedicated threads
From these numbers we can see that we are getting tons of thread reuse but also quite a lot of waiting, so if we can fix the waiting we can speed things up. We successfully increased the DB2 Connection TCBlimit and Threadlimit parameters until we started to get some results, that is, less waits and less alerts. By leaving the PJA7 dedicated thread settings alone we focused on the PJA7 DB2 Connection settings, changing TCBlimit/Threadlimit from 4/3 to 5/4. Nothing really seemed to improve, but at 8/6 some of the alerts started to clear up.
Example 10-8 .DB2 Connection with 8 TCBs and 6 pool threads
I DB2C STATUS: RESULTS - OVERTYPE TO MODIFY Accountrec( None ) Planexitname( ) Authid( ) Priority( High ) Authtype( Sign ) Purgecyclem( 00 ) Comauthid( ) Purgecycles( 30 ) Comauthtype( Cuserid ) Resyncmember( Comthreadlim( 0001 ) Signid( CICSTS ) Comthreads(0000) Security( ) Connecterror( Sqlcode ) Standbymode( Reconnect ) Connectst( Connected ) Statsqueue( CDB2 ) Db2groupid( ) Tcblimit( 0008 ) Db2id( D7Q2 ) Tcbs(0000) Db2release(0710) Threaderror(N906d) Drollback(Norollback) Threadlimit( 0006 ) Msgqueue1( CDB2 ) Threads(0000) Msgqueue2( ) Threadwait( Twait ) Msgqueue3( ) Nontermrel( Release ) Plan( TRADERP )
267
With these new TCBlimit/Threadlimit settings some of the DB2CONWT alerts finally started to go away, which means that we have 3 transactions, each with 2 dedicated threads, so we need at least 6 TCBs to process those dedicated threads without incurring TCB waits.
Figure 10-62 Alerts are going away with 8 TCBs servicing 6 total dedicated threads
We then kept increasing the TCBlimit/Threadlimit until all the DB2CONWT alerts went away. We ended up with a TCBlimit of 12 and a Threadlimit of 10, at which time we were left with one alert, which is a High Water Mark and would not go away until we reset our statistics.
268
Figure 10-63 Final remaining alert that required a CEMT Perform Stat Res to clear out
In conclusion, have you noticed that we used TPNS to shove as much workload into each region as it could handle and we never saw an alert for the PJA6 region running its transactions all through the pool? So for our very simply workload a pool of 4 TCBs and 3 threads easily outperforms 8 TCBs and 6 dedicated threads, even with some waiting going on. So now we want to try and generate some more alerts. So we decided to start an additional 1200 terminals of workload through TPNS. At this point we have our 200 terminals running and start 1200 more. Running like this we started to get some threshold alerts on the PJA7 region running with the dedicated threads, so we again increased its settings up to a TCBlimit of 14 and a Threadlimit of 10, leaving the dedicated threads at 2 per transaction. Alerts kept coming out, so we adjusted the Threadwait option to Tpool to allow the excess thread requests for the PJA7 dedicated threads to roll over to the pool. Again we could have easily increased the number of threads for either or both regions, but what we were trying to show is the effectiveness of using the pool. The PJA6 region never received any alerts. It did have waits, but it also had thread reuse due to the waits. On the flip side, we ran another test later on changing the MAXOPENTCBS parameter for both regions up to 75, then bumping the PJA6 regions pool up to a TCBLimit of 75 and a THREADLimit of 70. We then adjusted the dedicated thread region, PJA7, to use a TCBLimit
269
of 75 and a THREADLimit of 70 but set its dedicated threads to 20 threads for each transaction, plus setting them all to protected threads. This scenario easily showed the dedicated protected threads outperforming the pool. Everything was now switched, no alerts for the dedicated thread region, and a never ending battle to clear up alerts for the region with the pool.
DB2R0001 Printed at 11:42:22 10/11/2002 CICS PA CICS TS V2.2 and DB2 V7.1
APPLID SCSCPJA6
Page
Tran/ Program/ #Tasks/ SSID Planname #Threads DB2N D7Q2 PROGDB2N PROGDB2N 11178 11165
Avg Max Avg Max DB2ConWt DB2ConWt DB2ThdWt DB2ThdWt Time Time Time Time .0020 .2020 .0000 .0000
Avg DB2Rqst Count 17.0 Pool= .0255 .2294 .0021 .1113 3.0 3 .1 1
Avg Max Avg Max UserCPU UserCPU Response Response Time Time Time Time .002269 .004864 .0410 .3034
#Abends
Thread Utilization Class1: Thread Time Class2: In-DB2 Time Buffer Manager Summary Locking Summary
Entry= 10212 Avg: Elapsed= Max: Elapsed= Avg: Elapsed= Max: Elapsed= Avg: GtPgRq= Max: GtPgRq= Avg: Suspnd= Max: Suspnd=
953 Command= 0 CPU= .001457 CPU= .002361 CPU= .000804 CPU= .001617 SyPgUp= .0 SyPgUp= 0 DeadLk= .0 TmeOut= DeadLk= 0 TmeOut=
.0 0
MxPgLk= MxPgLk=
1.0 1
Figure 10-64 on page 271 shows a sample XTASK DB2CONWT threshold that is set using a Max DB2ConWt value pulled from the CICS PA report in Example 10-9.
270
What were able to do with the two products, CICS PA and CICS PM, is to run through a tuning and monitoring cycle where we use the CICS PA reports to fine tune our systems, then we set CICS PM thresholds based on worst case results (or perhaps just a little bit worse) so that when we get event alerts on these particular thresholds we are now aware that our systems may need a retune, and thus we start the cycle over again. In our sample scenario, we set DB2CONWT GE 00:00:00.2020 so that if we see a threshold alert we know we need to adjust some settings, and if we can run without any threshold alerts, then we can assume that we are running within our tuning specs.
10.5 Conclusion
Using various CICS PM thresholds we have been able to show several different ways you can use the CICS PM product to keep track of how your system is running and when a problem occurs. Weve shown how you can quickly research what is going on in your system using the information available to you in the alerts and the Web user interface screens. Additionally, you can set up custom screens that are extremely effective at displaying the exact fields you want to look at and at sorting through large amounts of data, especially if you take advantage of the Print preview icon to reformat the multitude of screens that may have been presented to you into one single report with a scroll bar that you can quickly browse. We found searching a 4018-line report extremely fast using this method as opposed to paging down forever on a CEMT Inquire display. CICS PM provides you with an overwhelming number of threshold settings. In this chapter weve tried to sort through them and provide you with a manageable number of thresholds you can take advantage of. The key to generating successful alerts is to start out with just a
Chapter 10. DB2 system problems
271
few thresholds. Be careful that you dont go wild and define thresholds for everything you find. The problem is that when something goes wrong, youll get flooded with alerts and wont be able to figure anything out since youll be sorting through threshold alerts instead of solving your problems. Start out slowly with just a few thresholds and then build on them as you analyze the problems they are showing you by adding more thresholds that will ultimately help you identify the real problem. Use the Severity field to make thresholds that are closest to your resources the highest severity and use lower severities for resources that are more generic. For example, if you go Max Task, its usually because of a resource that is constrained, so your max task threshold should be a lower severity than any actual resource that is constrained, such as a shortage of DB2 threads.
272
11
Chapter 11.
Note: The scenarios are designed to provide situations that would allow us to demonstrate how CICS PM can be used for daily monitoring. The CICS regions were not necessarily tuned and in some cases regions were constrained in order to cause certain events to trigger. Therefore, these scenarios and the results provided should be seen as demonstrations only.
273
The user-key storage area for all user-key task-lifetime storage below the 16 MB boundary.
Read-only DSA (RDSA)
The key-0 storage area for all reentrant programs and tables below the 16 MB boundary.
Shared DSA (SDSA)
The user-key storage area for any non-reentrant user-key RMODE(24) programs, and also for any storage obtained by programs issuing CICS GETMAIN commands for storage below the 16 MB boundary with the SHARED option.
CICS DSA (CDSA)
The CICS-key storage area for all non-reentrant CICS-key RMODE(24) programs, all CICS-key task-lifetime storage, and for CICS control blocks that reside below the 16 MB boundary.
Extended user DSA (EUDSA)
The user-key storage area for all user-key task-lifetime storage above the 16 MB boundary.
Extended read-only DSA (ERDSA)
The key-0 storage area for all reentrant programs and tables above the 16 MB boundary.
274
The user-key storage area for any non-reentrant user-key RMODE(ANY) programs, and also for any storage obtained by programs issuing CICS GETMAIN commands for storage above the 16 MB boundary with the SHARED option.
Extended CICS DSA (ECDSA)
The CICS-key storage area for all non-reentrant CICS-key RMODE(ANY) programs, all CICS-key task-lifetime storage, and CICS control blocks that reside above the 1 MB boundary.
The use of CICS facilities such as program LINK, SHARED storage GETMAINs, the type of file requests, use of temporary storage, application program attributes, or the number of concurrent DBCTL, or DB2, requests affect storage requirements.
Number of file definitions
These subpools may only be tuned by reducing the number of file definitions, or by using MRO.
The use of DBCTL or DB2
These subpools are only present if DBCTL or DB2 are used. The subpools may be tuned by reducing the number of threads, or by using maximum tasks (MXT) or transaction classes.
Nucleus and macro table storage
It may be possible to reduce the macro table storage by reducing the number of macro definitions or by migrating selected macro-defined tables to RDO.
Number and type of terminal definitions
The OPNDLIM system initialization parameter may also be tuned to limit storage use. See the CICS Performance Guide, SC34-6009 for more information about CICS subpools.
275
The resource type descriptive names within CICS PM correspond to 8-byte names by which the resources are know in CICSPlex SM resource tables and described in CICSPlex System Manager Resource Tables Reference, SC34-6027. Also, when defining threshold definitions, the attributes displayed by CICS PM are those defined in the CICSPlex SM resource tables and described in the same manual. The following table cross-references the CICS storage-related CICS PM resource type names with their corresponding CICSPlex SM resource table names.
Which attribute of a resource type to monitor requires some research. In some instances, the same or similar attributes can be found in multiple resource types. For example, the short-on-storage attribute is defined in the CICSRGN resource table as SOSSTATUS, and in the CICSSTOR resource table as SMSSOSTATUS. They share the same values (SOS, NOTSOS, SOSABOVE, and SOSBELOW) and the same source, which is CICS statistics. CICSPlex SM system availability monitor also has an !!SAMSOS attribute that represents the
276
same condition. In this instance, either resource would prove sufficient for triggering a short-on-storage CICS PM event. Threshold attributes for the normal monitoring of CICS DSA, however, present a different opportunity. For example, if the free space low water mark for dynamic storage areas is to be monitored, then there are several methods available to accomplish this goal. Figure 11-2 is an example of several events being triggered for free space low water marks.
Figure 11-3 shows that events were triggered for CICS Storage, DSAs, and CICS DSA Storage resource types.
Of the seven events triggered, three are for CDSA free storage low water mark, CMZA6602, CMZ52A82, and CMZ82600. Figure 11-4 through Figure 11-7 on page 279 show the CICS PM threshold definitions used to trigger each event. The CMZA6602 definition in Figure 11-4 uses the SMSCLWMFSTG attribute associated with the CICS PM CICS Storage resource type. It is found in the CICSSTOR resource table. The attribute description is CDSA LWM free storage. Also in the threshold definition, the Resource Identifier field is set to * for a generic wild card. In the event list the identifier resolves to a hypertext link with the CICS region name as text. In this threshold definition, the attribute designates a specific dynamic storage area to which it applies.
The CMZ52A82 definition in Figure 11-5 on page 278 uses the SMSLWMFSTG attribute and is associated with the CICS PM DSAs resource type and it is found in the CICSPAGP resource table. The attribute description is LWM free storage. In this definition, the Resource Identifier is set to * for a generic wild card. In the event list, three events have
277
been triggered with the identifier field resolving to CDSA, RDSA, and EUDSA. This points out the fact that the SMSLWMFSTG attribute is not specific to any dynamic storage area. This threshold is tested against all dynamic storage areas and events are triggered for each area in which the test evaluates to true. In each of the events, the identifiers are hypertext links with the name of the dynamic storage area as text.
The CMZ82600 definition in Figure 11-6 uses the LWMFREE attribute, which is associated with the CICS PM CICS DSA Storage resource type and is found in the CICSDSA resource table. The attribute description is LWM of free storage. In this definition, the Resource Identifier is set to CDSA. This will limit the testing of this definition to the CDSA free low-water-mark. This attribute is not limited to a specific dynamic storage area. This threshold is normally tested against all dynamic storage areas, and events are triggered for each area in which the test evaluates to true. The identifier in this event is a hypertext link with CDSA as the text. This definition outlines a method of limiting the scope of testing a threshold to a specific dynamic storage area.
The CMZ7AB42 definition in Figure 11-7 on page 279 uses the PCTFREE attribute which is associated with the CICS PM DSAs resource type and is found in the CICSPAGP resource table. The attribute description is Percent available total space. In this definition, the Resource Identifier field is CD*, which is a wild card specification that limits testing to dynamic storage areas whose names begin with CD. In this instance the CDSA dynamic storage area meets this criteria and the test evaluated to true. With these two facts being true, the event is triggered.
278
As these examples show, CICS PM provides great flexibility in deciding how to set up threshold definitions. The sample definitions outline several alternative approaches to setting up for monitoring CICS dynamic storage areas. Tests can be made to specific values for storage limits or a percentage of allocated storage. The use of wild cards in the identifier field allows testing of some or all dynamic storage areas. Conversely, tests can be limited to specific dynamic storage areas by using the name of the dynamic storage area in the identifier field or selecting an attribute that refers to a specific dynamic storage area. The applicability of any of these methods depends upon the goal to be achieved by monitoring. Before setting any storage thresholds for CICS PM, some time should be spent obtaining statistics about the storage profile of all regions to be monitored. Since CICS PM is more of an exception monitor, some notion of the characteristics of a normal system versus a peak processing system would be valuable in determining what thresholds to set and which dynamic storage areas need monitoring.
279
The first threshold, CMZA6602, uses the PCTCFREE attribute of the CICS Storage resource type. PCTCFREE is defined as the free storage percentage of the CDSA. The threshold will trigger a warning event if the percentage of free storage in CDSA falls to twenty percent or less. The AORStorage group was installed against the WLSAORS CICS system group, which contains the SCSCPAA1 and SCSCPAA4 AOR regions. Figure 11-9 on page 281 shows the CICS PM event monitor screen with two events triggered. Both events are for the CMZA6602 threshold.
280
The events show that in both of the regions, SCSCPAA1 and SCSCPAA4, the total free space for CDSA is below twenty percent. SCSCPAA1 is at 14.453 percent and SCSCPAA4 is at 14.844 percent. To begin the investigation into this event, the entry for SCSCPAA1 is selected and the Identifier hypertext link is clicked. See Figure 11-10 on page 282. In this instance, the Identifier field holds the region name.
281
Of the four hypertext links on the highlighted entry, the Identifier link promises to navigate to the Web browser window closest to the issue. Figure 11-11 on page 283 shows the Web browser window that appears in response to the identifier link being clicked.
282
Resettable fields
Hypertext links
The browser screen shows an overview of CICS dynamic storage allocations for the SCSCPAA1 region. The screen has four hypertext links and two resettable fields. The resettable fields allow the limits of DSA and EDSA to be reset in the event the values were not adequate. The hypertext link fields facilitate drilling down to finer detail in an effort to gain a complete picture of what is occurring within the dynamic storage areas. In this scenario, the Total storage currently allocated to DSAs link is clicked, which causes the Dynamic Storage Areas (DSA) browser screen to appear. See Figure 11-12 on page 284.
283
Hypertext links
Figure 11-12 Dynamic Storage Areas (DSA) screen
This screen shows an overview of the four dynamic storage areas located below the 16 MB boundary. For CDSA, the size is 1048576 and free storage is 143360. This screen has a hypertext link to the EDSA version of the same screen. It also has hypertext links to the four dynamic storage areas and to current subpool details. Clicking the CDSA hypertext link on this screen will cause the Dynamic Storage Area - CDSA screen to appear. See Figure 11-13 on page 285 and Figure 11-14 on page 286 for a complete view of this screen.
284
285
After reviewing the information presented in the various Web browser screens, we can conclude that for this scenario there is no real issue with CDSA being less than twenty percent free. The basis for this conclusion is found on the first Web browser screen navigated to via the Identifier hypertext link in Figure 11-11 on page 283. On that screen the value for the current DSA limit is 5242880 while storage currently allocated to DSAs is 2097152. This indicates that only approximately fifty percent of the total available DSA space has been utilized. The SCSCPAA1 region has the opportunity to grow its allocated DSA close to 100 percent before lack of storage becomes a real problem. Other than linking to the EDSA version of these screens, the CICS Dynamic Storage Areas screen has two other hypertext links that could prove useful in tracking down a storage issue. They are Domain subpools and Task subpools.
Domain subpools
The hypertext link to the domain subpools screen in Figure 11-15 on page 287 presents summary information concerning subpools in all of the dynamic storage areas. Clicking on a subpool name, a hypertext link, drills down to a screen with detailed information about the subpool (Figure 11-16 on page 288). On the Dynamic Storage Areas (DSA) screen in Figure 11-12 on page 284 the values on the Current Subpools (domain and task) row are also hypertext links that present the domain subpools screen showing only the domains in the selected dynamic storage are.
286
Hypertext links
Figure 11-15 Domain subpools summary screen
287
Task subpools
The task subpools hypertext link presents summary information concerning CDSA, ECDSA UDSA, and EUDSA for the selected CICS region on the Task subpool screen in, Figure 11-17 on page 289. The DSA names in the Name of dynamic storage area (DSA) column are also hypertext links that drill down to task subpool detail information; see Figure 11-18 on page 290.
288
Hypertext links
289
290
Note: CPSM provides a facility called Stall detection. In the MAS you have a number of EYUPARMS that may be specified, called STALLxxxTSK and STALLxxxCNT. One of the values for xxx is PGM, for Program stall detection, and another is STG for storage stall detection. STALLPGMTSK(2), STALLPGMCNT(4), STALLSTGTSK(1), and STALLSTGCNT(3) are the default values.
When an MAS starts up, one of the things that is automatically started is Stall detection. CICSPlex SM looks for suspended transactions with a particular suspend type and puts those transactions in different buckets. For example, there is a program stall bucket for transactions that have a CICS suspend type of PROGRAM. Likewise, there is a storage stall bucket for transactions that have a CICS suspend type of xDSA or ExDSA. When a transaction is put into a stall bucket, and there are at least STALLxxxTSK number of transactions for STALLxxxCNT number of checks, then you will get the SAMSTALL with the specified reason. For a list of STALLxxxTSK/CNT parameters and their default values and the CICS suspend types associated with each value, see the section CICSPlex SM System parameters in CICS Transaction Server Installation Guide, GC34-5985.
CICSPlex SM short-on-storage events Figure 11-19 Event monitor window with short-on-storage events
Upon seeing the short-on-storage event, the first action was to click the region field hypertext link associated with the short-on-storage event. The SCSCPAA1 region was identified as the region with the problem.
291
Clicking the region hypertext link caused the CICS region - Overview browser screen to appear. Figure 11-20 shows the screen presenting general information about the SCSCPAA1 region.
The appearance of a scroll bar on the right of the window indicated more information was available on the window. Therefore, the window was scrolled down to the area that had information about storage in the region; see Figure 11-21 on page 293.
292
This view of the CICS region - Overview screen showed the SCSCPAA1 Short on storage status to be Sosabove. This indicates that EDSA is the area that has exhausted its available storage. This view also showed a hypertext link to CICS storage browser screens. Clicking the CICS storage hypertext link caused the CICS Dynamic Storage Areas browser screen to show; see Figure 11-22 on page 294.
293
EDSA information
Investigation of the EDSA information on this screen revealed that currently allocated EDSA was equal to the current EDSA limit. Also on the screen, the Current EDSA limit field is a list box with a check box to the left. There is also an Apply changes tab at the bottom of the screen. The current EDSA limit can be modified by entering the new desired limit in the list box, clicking the check box, and clicking the Apply changes tab. In this scenario, the Current EDSA limit was changed from 57671680 to 62914560; see Figure 11-23 on page 295.
294
With the change to current EDSA limit in place, some processing occurred in the region, but it soon exhausted EDSA to the new limit; see Figure 11-24 on page 296.
295
EDSA information
We repeated the steps used to reset the current EDSA limit and the EDSA limit was raised from 62914560 to 73400320. With this change, the system availability monitor events for the SCSCPAA1 region were removed from the event monitor screen. A refresh of the CICS Dynamic Storage Areas screen in Figure 11-25 on page 297 confirmed the region Short on storage status field had changed to Notsos and the SCSCPAA1 region was processing normally again. The screen also showed that the amount of storage allocated to EDSA had peaked at 63963136.
296
EDSA information
At this point the short-on-storage condition had been relieved and the SCSCPAA1 region was back in good health. Now the issue was trying to find what caused the condition to occur. On the CICS Dynamic Storage Areas screen, there is a hypertext link to the Domain subpools browser window; see Figure 11-26 on page 298.
297
Problem subpool
Figure 11-26 Domain subpool screen
When the Domain subpool screen appeared, the descending sort icon on the Current page storage column was clicked. This caused the subpools to be sorted by size, largest first. The TSMAIN subpools current page storage size was 21430272, which indicated temporary storage had grown to 21 MB. In this scenario, this was a clear indication of what caused the SCSCPAA1 region to go short-on-storage.
11.3.1 Conclusion
CICS allocates dynamic storage areas automatically. This eliminates the requirement to specify the size of each individual dynamic storage area. It appears this also means that the collective sizes of the dynamic storage areas at the time of observation may not add up to the maximum as specified in DSALIM or EDSALIM. CICS will manipulate the individual dynamic storage areas with the goal of sustaining workload at MXT without causing a short-on-storage condition. CICS will take storage from one dynamic area that has some excess and assign it to another dynamic area that has a need to expand when allocated DSA has expanded to its specified limit. By doing this, CICS complicates the task of setting thresholds for monitoring the condition of the various CICS dynamic areas. Because of the dynamic nature of storage manipulation in the CICS dynamic storage areas, thresholds such as high water marks, low water marks, and percentage of free space become
298
more relevant only as the current storage allocations for DSA and EDSA approach their respective DSALIM and EDSALIM. Two thresholds that could be useful when used in conjunction with thresholds mentioned above are associated with the CICS Storage resource type, which is under the Regions limb of the resource tree. The attributes are: SMSHWMDSATOT - High water mark DSA total SMSHWMEDSATO - High water mark EDSA total Figure 11-27 shows an example of the two thresholds being defined to trigger an event if the high water mark of either total allocated DSA or total allocated EDSA grows to be equal to or greater than ninety percent of DSALIM or EDSALIM.
Figure 11-27 Threshold definitions for DSA and EDSA high water marks
With these two thresholds installed, in conjunction with other thresholds that more specifically target important dynamic storage areas, storage events would only become critical when one of the total high water mark events triggers, along with an event that highlights a specific dynamic area that is growing close to a problem status.
299
300
12
Chapter 12.
Note: The scenarios were used to provide situations that would allow us to demonstrate the use of CICS Performance Analyzer reports. The CICS regions were not necessarily tuned for peak performance and in some cases had a high level of tracing active. Therefore, these scenarios and the results provided should be seen as demonstrations only and do not provide definitive results for a customer environment.
301
302
bean objects can handle multiple requests from multiple clients so sessions can be pooled. An OTS transaction cannot span through method invocations. Stateful session beans that support multiple consecutive method invocations originating from the same client. The state data is maintained between method invocations. The bean object exists for the duration of a single client/server session and an OTS transaction can span multiple method invocations. CICS fully supports both types of session beans. It is important to understand that a method invocation for both types of session beans maps to one CICS transaction, so the system behavior in both cases conforms to the CICS pseudo-conversational programming model. Entity beans that usually encapsulate access to relational data. Entity beans are not supported by CICS TS. Relational data that would usually reside in a local DB2 database can be accessed through JDBC or SQLJ interfaces.
Data type
SCLOCK SCLOCK SCLOCK SCLOCK SCLOCK SCLOCK
Source
CMF CMF CMF STAT STAT STAT
Len
8 8 8 8 8 8
Sum
MAX MAX MAX AVG AVG AVG
Description
J8 TCB Mode CPU time Total JVM elapsed time Total JVM suspend time Maximum JVM TCB delay time JVM elapsed time - initialize JVM elapsed time - resetting
Data type
BIN
Source
STAT
Len
4
Sum
AVG
Description
Number of Current JVMs
303
Alternatively, taking historical analysis from the CICS Performance Analyzer (CICS PA) product will give a good indication of how the system runs. We looked at the values produced during the writing of IBM CICS Performance Analyzer V1.2, SG24-6882 for JVMsqand although the scenarios are not exactly identicalthe average values found in those reports should give us a ballpark figure to start with. Details of the CICS PA summaries are shown in 12.2.4, Threshold group JVM_reset_NO on page 304 and 12.2.5, Threshold group JVM_reset_YES on page 306.
Page
Tran JTS1
Avg Avg Avg #Tasks Response Dispatch User CPU Time Time Time 95 187.091 18.3475 1.8681
Avg Avg Avg Avg Avg Avg J8 CPU JVM Elap JVMITime JVM Meth JVMRTime JVM Susp Time Time Time Time Time Time 1.8667 17.0962 16.4318 .6062 .0582 .0205
Example 12-2 CICS PA Dispatcher wait summary for JVM with Xresettable=NO
V1R2M0 CICS Performance Analyzer Performance Summary __________________________________________________ Data from 15:48:00 10/14/2002 to 15:57:01 10/14/2002
Page
Tran JTS1
Avg Avg Avg Avg Avg Avg Avg Avg Response Dispatch User CPU Suspend Disp1Dly DispWait MaxJTDly MaxOTDly Time Time Time Time Time Time Time Time 187.091 18.3475 1.8681 168.743 .0001 .0037 168.723 .0000
304
Using this historical data, threshold definitions were created. For category Activity, CICS resource Tasks, they are illustrated in Figure 12-2; for category Enterprise Java, CICS resource JVMPools, they are illustrated in Figure 12-3 on page 306.
305
Page
Tran JTS1
Avg Avg Avg #Tasks Response Dispatch User CPU Time Time Time 13376 .3839 .0530 .0130
Avg Avg Avg Avg Avg Avg J8 CPU JVM Elap JVMITime JVM Meth JVMRTime JVM Susp Time Time Time Time Time Time .0117 .1115 .0000 .1084 .0031 .0589
Example 12-4 CICS PA Dispatcher wait summary for JVM with Xresettable=YES
V1R2M0 CICS Performance Analyzer Performance Summary __________________________________________________ Data from 18:01:59 10/14/2002 to 18:07:00 10/14/2002
Page
Tran JTS1
Avg Avg Avg Avg Avg Avg Avg Avg Response Dispatch User CPU Suspend Disp1Dly DispWait MaxJTDly MaxOTDly Time Time Time Time Time Time Time Time .3839 .0530 .0130 .3309 .0001 .0110 .2715 .0000
Using this historical data, threshold definitions were created. For category Activity, CICS resource Tasks, they are illustrated in Figure 12-3; for category Enterprise Java, CICS resource JVMPools, they are illustrated in Figure 12-4 on page 307.
306
307
12.3.1 Topology
The TPNS script runs on a separate z/OS region, SC47, and logs 25 terminals onto TOR PTA1, and another 25 terminals onto TOR PTA2. For each of the fifty simulated terminals, one and the same transaction is started. When transaction output is received, a new (identical) transaction run is immediately requested. The transaction is statically routed to AOR PJA6.
SC47
TPNS Workload
z/OS
PTA1
TOR Static Routing
PTA2
TOR
PJA6
AOR Xresettable=NO|YES
z/OS
The transaction being started on each TPNS terminal invokes a CICS-supplied sample Java program, examples.TSQ.ClassOne. This Java program simply accesses a temporary storage queue, and is provided to illustrate use of the JCICS TSQ class. The transaction is JTS1 and the CICS program name is DFJ$JTS1. The resources are defined in the CICS-supplied CSD group DFH$JVM.
Java Applications in CICS, SC34-6000 fully describes how to generate, set up and run this sample TSQ application.
To run a Java program in a JVM environment, the PROGRAM definition must: Specify JVM=YES.
308
IBM Tools: CICS Performance Monitor V1.1
Specify the fully (package) qualified name of the Java class in the JVMCLASS attribute. Specify the appropriate member of the DFHJVM DD data set on the JVMPROFILE attribute. Figure 12-7 shows the relevant parameters from the DFH$JTS1 PROGRAM definition.
OBJECT CHARACTERISTICS CICS RELEASE = 0620 CEDA View PROGram( DFJ$JTS1 ) + REMOTEName : Transid : EXECUtionset : Fullapi Fullapi | Dplsubset JVM ATTRIBUTES JVM : Yes No | Yes JVMClass : examples.TSQ.ClassOne : : : : JVMProfile : DFHJVMPR JAVA PROGRAM OBJECT ATTRIBUTES Hotpool : No No | Yes
SYSID=PJA6 APPLID=SCSCPJA6 PF 1 HELP 2 COM 3 END 6 CRSR 7 SBH 8 SFH 9 MSG 10 SB 11 SF 12 CNCL
The JVMPROFILE represents a set of JVM characteristics, which may be different on an application-by-application basis. The profile is located in a data set with DD-name DFHJVM. The settings of most significance in this case are:
JVMPROPS
Specifies the full path of the system properties file that resides in HFS and that CICS is to use when creating a JVM. On our system we have: JVMPROPS=/u/cicsts22/props/scscpj##/dfjjvmpr.props Specifies the path to user applications. On our system we have: CLASSPATH=/u/cicsts22/lib/eablib.jar:/u/cicsts22/lib/recjava.jar:/usr/lpp/cicst s/cicsts22/samples/dfjcics Specifies whether the JVM is eligible to be reused again for execution of other suitable JVM programs. In our test, we run separate tests with both Xresettable=NO and Xresettable=YES.
CLASSPATH
Xresettable
Note: We did not make use of the shareable classes path optimization when running the JCICS scenario. It was used, however, for the EJB scenario.
Finally, there is one CICS system initialization parameter to be considered: MAXJVMTCBS specifies the maximum number of open TCBs CICS can create for use by JVM programs. We decided to run with the default value of 5.
309
Note: These scenarios were used to provide situations that would allow us to demonstrate the use of the CICS Performance Monitor when running with a Java workload. The CICS regions were not necessarily tuned for peak performance and in some cases had a high level of tracing active. Therefore, these scenarios and the results provided should be seen as demonstrations only and do not provide definitive results for a customer environment.
Linking through to the Web user interface view for JVMPools shown in Figure 12-9 on page 311, we can see that our Peak number of JVMs matches our maximum available from
310
SIT parm MAXJVMTCBSso we know our workload is pushing our system at or beyond its maximum capabilities.
Figure 12-9 WUI screen showing JVMPOOL status for Xresettable=NO case
To see a further indication of the workload backlog, we can link through to the Web user interface Activity Tasks view shown in Figure 12-10 on page 312.
311
We can now clearly see the number of tasks backed up waiting for a JVM, as the dispatcher shows that the task is waiting on resource JVM_POOL. Notice that the Event Monitor did not show our SJGCURRJVMS >= 5 condition (threshold definition CMZ00941) being hit. This is due to the timing involved with Xresettable=NO, as JVMs are coming and going in such a way that the sample interval rarely catches site of them. This threshold definition does function much better with Xresettable=YES, as we will see later.
312
Note: The definitions installed for the previous scenario were discarded before installing the JVM_reset_YES group.
The JVM Profile was modified to specify Xresettable=YES, and CEMT SET JVM PHASEOUT was issued on the AOR PJA6 to ensure that we were starting with a scratch JVM environment. No region recycle was required for this change. The TPNS workload was restarted, and at the first collection sweep we can see that again we have a stack of JVM tasks waiting due to a dispatcher delay on the JVMPOOL. This is because the first 5 JTS1 transactions must create and initialize a JVM. Figure 12-11 again shows the effects of the expensive initial cost of creating and initializing the 5 JVMs. This caused our JVMTIME and (more specifically) TMRJVMIT event to fire. This screen shot is from the first Event Monitor auto-refresh after the TPNS workload was started.
This is hardly surprising, as our CICS PA-based thresholds are based upon a sample period that excludes the initialization stage. We expect the initialization step to be expensive, and to delay the following transactions for a while until the backlog clears.
313
Figure 12-12 shows a further snapshot, a few minutes later. Although the number of transactions still Dispatcher-delayed is quite high, we can see that the delay time is significantly less. As the workload continues, the backlog effect of the initialization diminishes.
Note: Now the initialization stage is passed, we are no longer seeing the TMRJVMIT Event firing (while still seeing JVMTIME, due to dispatch delay). If we were to see the TMRJVMIT event firing later in the workload run, it would give us an indication that an application may have caused a resettable event to occur. This is simulated in the last EJB scenario.
The tasks continued to be Dispatcher-delayed for JVM availability, albeit for much shorter times as the workload ran. This was due to our intentionally low MAXJVMTCBs setting.
Linking from the SJGCURRJVMS Event to the JVM Pool Web user interface view, shown in Figure 12-13 on page 315, we can see that this time, with Xresettable=YES, the 5 JVMs are concurrently active, as they are not torn down at the end of each transaction.
314
Figure 12-13 WUI View of JVM pool when running at MAXJVMTCBs and Xresettable=YES
315
12.4.1 Topology
Figure 12-14 shows the topology of the EJB setup. Two IIOP listener regions, PLA1 and PLA2, receive a balance workload via TCP/IP port sharing on z/OS. Each client IIOP request is ultimately passed over to one of the AORs, PJA6 or PJA7, where the EJB methods are executed. Refer to Chapter 8, System setup and scenario overview on page 163 for further background on the overall system setup.
PJA6
AOR
PJA7
AOR
PLA1
AOR Xresettable=NO
PLA2
AOR Xresettable=YES
IP port sharing
z/OS
316
Selecting the last company as our current company... The The The The The The The The The The The The The last company is named IBM, here are its details: price per share in dollars = $164.0 total of outstanding shares = 1000 shares total outstanding share value = $164000.0 price one day ago = $163.0 price two days ago = $162.0 price three days ago = $160.0 price four days ago = $161.0 price five days ago = $159.0 price six days ago = $156.0 price seven days ago = $157.0 cost to sell a share = $15.0 cost to buy a share = $10.0
The portfolio of FRED2 contains 834 shares in IBM worth $136776.0. Selling 23 shares.... Buying 16 shares... The portfolio of FRED2 now contains 841 shares in IBM worth $137924.0. *tsbc.cmd*: Finished. *tsbc.cmd*: Finished. C:\itsocts22\WLM_Batch_CMDs>
317
318
Figure 12-16 WUI view for SQLX CorbaServer - JNDI and shelf
319
Figure 12-17 WUI view for SQLX CorbaServer - Enterprise bean in CorbaServer
Figure 12-18 on page 321 shows JVM-related events for AORs PJA6 and PJA7 on the GUI Event Monitor. The scope is set to group CSGALL, so we can also see other miscellaneous events outstanding on the CICSplex. This illustrates CICS PM in action when viewing multiple CICS regions. Subsequent screen shots in this scenario will focus the scope onto PJA6 and PJA7 only. This screen shot was taken following the first Event Monitor auto-refresh, after the EJB workload was started. During the JVM create and initialization period, we can again see that JVM initialization time is high, causing the TMRJVMIT Event to fire on both systems for each of the ten available JVMs.
320
The next screen shot, Figure 12-19 on page 322, was taken after a few minutes have elapsed. The workload on both systems is settling down and the backlog has fallen significantly. We can see that due to the constraint of the number of JVMs, we are getting closer to the CICS PA-based threshold values. When the EJB workload was left to run significantly longer, the number of events firing due to initialization, suspend, dispatch, and delay became minimal.
321
With the EJB workload running well, we hyperlinked over to the JVM pool Web user interface view shown in Figure 12-20 on page 323. This shows that all five available JVMs are up and running, trying to satisfy the workload. The JVM statistics show that we have a high rate of reuse. The 14 mismatched requests are residual from earlier test runs.
322
323
The most expensive event that can occur for an established reusable JVM is a resettable event. This means that a user application has executed code that may have altered a state that the JVM cannot guarantee to reset. An example of a resettable event would be the calling of a native method from the user application. A native method is an unknown quantity to the JVM; it could do practically anything that may affect a subsequent transaction executing of the same Java program. As the JVM cannot then guarantee a subsequent clean run, it marks the JVM for termination. The resettable event takes that JVM TCB out of action while it is terminated. A new JVM is created and initialized. Meanwhile, a healthy workload may well be stalled by the temporary loss of a JVM from the pool, causing a backlog. The resettable event can cause much more of a problem than simply delaying the next transaction unlucky enough to be waiting for that JVM TCB. Therefore, it is very useful to be able to identify JVMs that are being hit with resettable events. The first signature to look for is that of tasks suddenly hitting the Excessive JVM Initialization event, TMRJVMIT. Depending on the workload, this may well be followed by Excessive Elapsed JVM time events, JVMTIME and possibly JVM Excessive Dispatch Delay events, TMRJTDLY. Also, if workload is unduly affected by resettable events, then this will show up in the statistics given by the Web user interface JVMPools view. The number of reused JVMs will be significantly lower than the number of JVM requests. In this example, we show what would happen if all JVMs in one region suddenly hit a Resettable Event, by marking all of the JVMs for termination. This is a rather heavy-handed approach, but time did not permit us to engineer the EJB source code. Figure 12-21 on page 325 shows the Event Monitor when an established EJB workload is running well across AORs PJA6 and PJA7. We do still have some events showing, but we have not attempted to fine tune our threshold definitions since setting them to the average values found by CICS PM. Therefore, this is to be expected and is useful for the demonstrative purposes of the redbook.
324
Figure 12-21 Event Monitor showing AORs PJA6 and PJA7 running normally under EJB workload
We now take the extremely expensive action of terminating all of the JVMs on PJA6 by issuing CEMT SET JVMPOOL PHASEOUT on that region. The process of recreating and initializing the JVMs must be completed before the workload can continue to run. The event monitor screen shot in Figure 12-22 on page 326 shows that PJA6 is badly affected while PJA7 continues to run reasonably well. We took out the JVMs from PJA6 by issuing CEMT SET JVMPOOL PHASEOUT. This is obviously an exaggerated demonstration of the effect a bad JVM application can have on a running system.
325
Figure 12-22 Event Monitor showing the effect of a JVM PHASEOUT on PJA6 under the EJB workload
Attribute
ACTSOCKETS OPENSTATUS
Datatype
BIN CVDAS
Source
INQ INQ
Len
4 4
Sum
AVG LIKE
Description
Active sockets TCP/IP Service Status
Figure 12-23 on page 327 shows the threshold definition we created for the ACTSCOKETS attribute under the TCP/IP Resource of the Connectivity category.
326
Figure 12-24 shows the threshold definition we created for the OPENSTATUS attribute under the TCP/IP Services Resource of the Connectivity category.
Figure 12-24 Connectivity/TCPIP Services threshold definition for the OPENSTATUS attribute
327
Figure 12-25 Event monitor showing that the TCPIPSERVICE has closed
Linking through to the Web user interface view for TCP/IP Service P8083, we can see in Figure 12-26 on page 329 that the event was raised because the TCP/IP Service has a state of CLOSED.
328
Figure 12-26 WUI view of the TCP/IP service showing a status of CLOSED
Clearly, if a crucial business function is serviced over a TCP/IP connection, it is essential that the service remains OPEN, and early notification of such a problem is vital.
329
CICS PM flagged the event showing that we were running at our maximum socket capability for the listener region. Figure 12-27 shows the event displayed on the Event Monitor.
Figure 12-27 Event monitor showing that both PLA1 and PLA2 listener regions have hit MAXSOCKETS
Looking at the Web user interface view for the TCPIPService, we can see from the Peak number of connections field that we have been running at our limit of six sockets. This is shown below in Figure 12-28 on page 331.
330
Figure 12-28 WIU view showing that the Peak number of client connections has hit 5
Remember that one socket is accounted for by the IIOP socket listener transaction. In order to relieve this symptom, we tuned the Maxsockets limit on each region TCP/IP resource, and updated the MAXSOCKETS SIT parameter.
331
12.6 Conclusion
In this chapter we considered different Java-based workloads running in CICS TS and looked at some monitoring options for JVM performance and TCP/IP connectivity. There is much to consider and appreciate about the workload itself when choosing suitable threshold definitions, and even more so when setting threshold values. The JVM performance thresholds we considered are of most use when reliable historical data for the average timings of specific applications is available (such as that provided by the CICS Performance Analyzer reports) for the threshold values, and when the threshold groups can be organized in such a way that they can match the scope on the specific applications in the CICSplex. In the TCP/IP scenario, we managed to set thresholds that could provide a high level notification of when potentially critical inbound connections were compromised, and when the TCP/IP configuration was inadequate for the user demand. The former could be very useful as an indication of current EJB traffic, but can also be applied to other TCPIPSERVICE-based connections such as CICS Web Services, CTG (connected using ECI over TCP/IP). This exercise has been intended to simply be a starting point for JVM and TCP/IP monitoring. There are more attributes on offer in CICS PM to provide more sophisticated event monitoring for both JVM- and TCP/IP-related workloads.
332
13
Chapter 13.
333
You will then be prompted to enter a threshold group name. Type your selected name. There is no uppercase translation, so decide if you wish your group name to remain lower or upper case. Your new group name will appear in the left column. We chose to call ours ISCGROUP as shown in Figure 13-2 on page 335.
334
335
From this expanded selection we need to highlight the ISC & MRO resource. This will not show anything at this time as we have yet to create a definition. Select the Thresholds drop-down menu and then click New, as shown in Figure 13-4 on page 337.
336
By clicking New we get a selection panel to set up our threshold, as shown in Figure 13-5 on page 338. As you can see, we have seven fields to fill. The first thing to do is to select the ATTRIBUTE because anything entered in any field will be cleared once the attribute has been selected. For a list of the attributes and their meaning, we need to find the CPSM resource table name associated with the resource name. Refer to the CICS Performance Monitor Users Guide, GC34-6194 to find this. In our case the CICSPlex SM resource table name for ISC & MRO is CONNECT. Once we have this we need to look in the CICSPlex System Manager Resource Tables Reference, SC34-6027 to find the connect resource table.
Note: Select the attribute before typing anything else as all fields are cleared when the attribute is selected.
337
338
339
Click Save located at the bottom right of the screen. The bold line around the threshold details will change to a thin line once the threshold has been saved. We then need to install it to the CICS region. Once Install has been clicked, a menu appears in the left top corner enabling you to expand down to the CICS region. The install is completed by clicking on the CICS region. Figure 13-8 shows the panel once Install has been clicked and the levels broken down to CICS regions. Highlight the desired CICS region and click OK .
340
13.2 Problems
13.2.1 Dealing with the first problem
The first thing we see is a red alert on the status panel as seen in Figure 13-10 on page 342.
341
So we need to resolve the problem. We do this by double-clicking the identifier, which starts a CICSPlex SM Web user interface session as shown in Figure 13-11 on page 343. We can see from this display that the connection is released. If we now select the record and then click Acquire, we receive a confirmation screen asking if we wish to proceed, to which we obviously reply yes, and the connection is reestablished and the CICS/PM warning is resolved at the next interval.
342
343
We would normally deal with the highest severity problem first, so if we double-click the identifier for the queued allocates, a session with the CICSPlex SM Web user interface is started as shown in Figure 13-13 on page 345.
344
As can be seen on this screen, we have an allocation queue depth of 1, which is how we set our threshold up in 13.1.6, Setting up other thresholds on page 340. If we now click the connection link, it expands our view further and we can scroll down to the relevant statistics; see Figure 13-14 on page 346.
345
So this confirms that we have a problem with the number of allocations requested. We could now look at the transactions running in our CICS region. To do this we select Activity -> Tasks. We are then presented with the tasks currently running in the system, shown in Figure 13-15 on page 347.
346
We can see that we have a terminal sitting on a ZCIOWAIT. We could investigate this further by selecting one of the blue underlined links, but as this is a fabricated problem it will not show us very much. We were given permission from the user and support team to purge the tasks, so we then click the Select record box for both tasks. Having selected the tasks, we can click Force Purge and reply Yes to all. If we then select Refresh, we should find that our tasks have gone, and when we go back to our CICS/PM client, we see that the warnings have also gone; see Figure 13-16 on page 348.
347
This may be a unique problem due to the sudden usage of transaction TRN1, or it could be a recurring problem that normally gets resolved by other means. To investigate further we would use CICS/PA to find the answer.
13.3 Conclusion
We have created problems and shown how, through CICS/PM and the link to the Web user interface, we can investigate and correct them. We attempted to show how easy it would be to set up a few simple thresholds to monitor your CICS connections through CICS/PM. The thresholds we chose are fairly basic but these can be expanded upon, once you have experience with the product. A number of thresholds are available, but some will be static information, that is, things that will not change unless the connection attributes are changed, and therefore perhaps not what you would set thresholds on. You would use CICS/PA statistical data to initially establish what your thresholds should be. Ideally, once a problem has been highlighted, you would run CICS/PA reports to determine if there is a history of the problem, a trend building towards a problem, or if it is just a unique problem.
348
Related publications
The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook.
IBM Redbooks
For information on ordering these publications, see How to get IBM Redbooks on page 349.
IBM CICS Performance Analyzer V1.2, SG24-6882 DB2 for z/OS and OS/390 Version 7 Selected Performance Topics, SG24-6894 Enterprise JavaBeans for z/OS and OS/390 CICS Transaction Server V2.2, SG24-6284 CICS Transaction Gateway V5 The WebSphere Connector for CICS, SG24-6133
Other resources
These publications are also relevant as further information sources:
CICS TS for z/OS: CICS DB2 Guide, SC34-6014 CICS System Definition Guide, SC33-1682 CICS Transaction Server Installation Guide, GC34-5985 CICS Performance Monitor Users Guide, GC34-6194 z/OS Security Server RACF Security Administrators Guide, SA22-7683 CICSPlex System Manager Resource Tables Reference, SC34-6027 CICSPlex SM Administration, SC34-6019 CICS RACF Security Guide, SC34-6011 CICS Supplied Transactions, SC34-5992 CICS TS for z/OS: CICS DB2 Guide, SC34-6014 Java Applications in CICS, SC34-6000 CICSPlex SM Web User Interface Guide, SC34-6018
You can also download additional materials (code samples or diskette/CD-ROM images) from that site.
349
350
Index
A
Abend codes AEY9 230 AZI6 230 abends 185 Adding libraries to your regions 55 AIDS 340 AOR 171 APAR OW44393 22 PQ34321 302 PQ63141 6 PQ63143 6 PQ63527 5 attribute field 338 CICS PA 7 Analyze CICS application performance 7 Evaluate the effects of CICS system tuning efforts 7 Improve CICS resource usage 7 Improve transaction response time 7 Provide awareness of usage trends, assisting with future growth estimates 7 Provide ongoing system management and measurement reports 7 CICS PA threshold monitoring 226 CICS Performance Analyzer benefits 7 comparison with CICS PM 8 CICS Performance Minitor Threshold definition component Resources pane 102 CICS Performance Monitor 6 benefits 7 CICS PM Architecture 12 CICS PM DB2 thresholds 215 CICS PM GUI application 15 CICS PM initial screen 76 CICS PM window navigation 78 CMZSRVIP TCPIPSERVICE 22 comparison with CICS PA 8 DB2 Connection thresholds and alerts 216 DB2 Entries thresholds and alerts 218 DB2 Trans thresholds and alerts 220 Download the CICS PM workstation client 68 Event log 18 Event monitor 16 Install the CICS PM workstation client 69 Installed Threshold Definitions 20 Resource summary 17 server component 6 TCP/IP-related thresholds 326 Threshold definitions 18 Threshold Groups 19 Topology 16 workstation client 6 Workstation client components 78 XTASK thresholds and alerts 225 CICS PM Architecture 12 CICS PM Client Installation 67 CICS PM DB2 thresholds 209 CICS PM Event monitor fields 80 Attribute 94 Collapse All 83 Current Value 94 Default Selection 84 Event 93 Expand All 84 Expand to Events 84 Filter by Region 96 Filter by Resource 96
B
Batch update Utility CMZBATCH 41 BBACTDEF 36 BBVDEF 36 Business Transaction Services (BTS) 5
C
CEDF 251 CEDF transaction 141, 250 CEMT 113 CEMT DISCARD 113 CEMT SET 132 INQUIRE DB2CONN 217 INQUIRE DB2ENTRY 219, 221 Perform Stat Res 269 SET JVM PHASEOUT 313 SET JVMPOOL PHASEOUT 325 CICS and DB2 210 CICS DB2 attachment tuning overview. 213 DB2Entry verses running in the pool 214 Overview of the CICS DB2 Attachment Facility 210 Protected verses unprotected threads 214 Thread re-use and thread waits 214 CICS and Java 302 CICS Connection threshold exceeded 168 CICS DB2 Attachment Facility 210 CICS DB2 attachment tuning 209 CICS dispatcher 6 CICS Dynamic Storage Area (CDSA 274 CICS Dynamic Storage Areas 134 CICS GTF data 9 CICS Interdependency Analyzer 8 CICS Monitoring Facility transaction resource class data 9 CICS Online Transmission Time Optimizer 8 CICS OTTO 8
351
Identifier 93 Launch WUI 84 Operator 94 Priority 94 Reference Value 94 Refresh 83 Region 93 Resource 93 Show OK Icon 84 Show Regions 84 Time 94 CICS PM Event monitor window event list pane 92 Regions histories view 88 Resources summary view 85 status pane 85 topology pane 82 CICS PM Event monitoring component 80 CICS PM GUI application 15 CICS PM installation Post SMP/E install and configuration 24 Server software pre-reqs 22 Workstation pre-reqs 23 CICS PM server 22 CICS PM Threshold definition component 98 Threshold definition groups 98 Threshold Groups pane 99 CICS PM Threshold fields 98 Copy 101 Cut 101 Discard Changes 102 Expand all 104 Expand To Definitions 104 Install 102 Name 98 Paste 101 Resource ID 107 Select All 104 Sort by 106 CICS PM Web user interface main menu 122 CICS pseudo-conversational programming 303 CICS Region low in free space in a dynamic storgage area 168 CICS Regions Views 126 CICS Resources 19 CICS SPI Inquire 216 CICS storage (DSA and subpool usage) 6 CICS subpools 275 CICS SVC 39, 50 CICS Tools 4 CICS Interdependency Analyzer 4, 8 CICS Online Transmission Time Optimizer 4, 8 CICS Performance Analyzer 4 CICS Performance Monitor 4 CICS Transaction Gateway 165 CICS TS 274 CICS DSA (CDSA) 274 CICS dynamic storage areas 274 CICS subpools 275 extended CICS DSA (ECDSA) 275
extended read-only DSA (ERDSA) 274 extended shared DSA (ESDSA) 275 extended user DSA (EUDSA) 274 read-only DSA (RDSA) 274 shared DSA (SDSA) 274 user DSA (UDSA) 274 CICS Web Support (CWS) 5 CICS/TS 1.3 CSD upgrade program 32 CICS/TS 2.2 CSD upgrade programs 32 CICSplex creation 41 CICSPlex SM 35 address space (CMAS) 13, 164 Agent code 14 CMAS SIT SYSIN parameters 38 CMAS startup 40 CMZBATCH 41 CMZCMAS 36 CMZCSDUG 31 concepts for CICS PM 13 CPSM components 13 CPSM coordinating address space (CAS) 13 CPSM CSD definitions 31 CPSM datasets 29 CPSM for CICS PM 14 CPSM initialization parameters for the WUI 51 CPSM installation overview 27 CPSM Web User Interface address space (WUI) 13 DFHCNV Code page conversion 48 EYUDREP 14 EYUPARM 38 Local MAS 14 MVS environment set up 29 Remote MAS 14 SCMZINST 29 Starting the CAS 34 Web user interface usage 122 Web user interface View Editor 147 WUI region datasets creation 46 WUI region startup 53 client download 25 CMAS SIT SYSIN parameters 39 CMAS start up procedure 36 CMZ00941 312 CMZ13CSD 32 CMZ62CSD 32 CMZBase threshold definitions 114 CMZBATCH 41 JCL 41 job 45 program 41 sample 43 CMZBTCH input statements 63 CMZCMAST 36 CMZCSI13 38 CMZCSI22 38 CMZCVW$ 48 CMZDEFDS 29 CMZPMD15 146 CMZPROC 33 CMZUIPM 51
352
CMZWUIT 46 COBOL 164 Code - Time zone table 30 Code page conversion 48 COI0 175 COIE 175 CONL 175 Coordinating address space (CAS) creation 33 Corba Server SQLX 317 COSD transaction 40 COVC transaction 53, 64, 155 COVI 64 CPSM WUI initialization parameters for EYUPARM 51 CSD definitions 56
DFHTERMC 39 DFJ$JTS1 308 Domain Subpools in CDSA 136 DSNC transaction 212 Dynamic Storage Area Limit (DSALIM) 274 Dynamic transaction routing 164
E
ECI interface 165 ECI over TCP/IP 165, 332 ECIRequest 166 EJB application on resettable JVM with a resettable event or JVM phaseout 169 EJB application with Xresettable=YES 169 EJB Client 316 ENQ Pools 187 Enqueue file problems 168 Enqueue pools 6 Enterprise JavaBean definition 302 entity bean 303 session beans 302 Environment Services System Services (ESSS) 13 EPI interface 165 ESM 158 ESSS 15 Event log 18 Event Monitor auto-refresh 320 Event Monitor log 18 Resource Identifier 18 Resource Type 18 Event Monitor window 16 Event Severity field 339 Event views panel 142 EXCI interface 165 EXEC CICS LINK 302 START TRANSACTION 302 EXEC CICS SET command 274 EXEC CICS SET MONITOR STATUS(ON) 9 Extended CICS Dynamic Storage Areas (ECDSA) 274 Extended Dynamic Storage Area Limit (EDSALIM) 274 EYU953G0 32 EYU953G1 32 EYU953GW 32 EYU962G0 32 EYU962G1 32 EYU9XDUT 30 EYU9XECS 36 EYU9XLOP 165 EYUDREP 36 EYULOG 36 EYUPARM 36 EYUTSODS 41
D
DB2 Connection drop 168 DB2 entries 7 DB2 Entry non-pool threads 219 DB2 Threshold definitions 215 DB2 transaction workload Entry vs Pool threads 262 DB2 transaction workload lockout 168 DB2 transaction workload using pool threads 168 DB2 V5 210 DB2 Version 7 210 DB2Conn 213 COMthreadlim 213 TCblimit 213 THREADLimit 213 DB2CONN definition 216 DB2CONWT field 226 DB2ENTRY 218 DB2Entry 213 PROtectnum 213 THREADLimit 213 THREADWait 213 DB2Entry definition 213 DB2RDYQW field 226 DB2REQCT field 226 DB2SS definition 223 DB2THRD definition 221 DB2TRN defintion 220 DB2U transaction 237 DEFINE language support 50 DFH$JVM 308 DFH$MOLS 5 DFH0STAT 5 DFHAPPL 5 DFHAUPLE 49 DFHAUXT 46 DFHCNV 48 DFHCSD 27 DFHCSDUP 31 DFHDEFDS 46 DFHDMP 27 DFHINTRA 27 DFHLOG 199 DFHRPL 14 DFHSIP 36 DFHSTUP 5
F
FCPSWAIT 183 FCSRSUSP 183 FEPI connections 139
Index
353
FTP download 68
G
gateways 165 Generalized Trace Facility 9 GUI Interface 75
L
Language support 50 LDAP server 164 Lightweight Directory Access Protocol (LDAP) 164 LMAS parameters 56 LMAS regions 32 LMAS restart 57 Logstreams 7 LSR pool string problems 168 LU62 connections 139
H
Hardware 23 HTML 3.2 24 HTTP 1.0 24 HTTP cookie support 24 HTTP proxy 157 HTTP-based protocol 22 hyperlinks 15, 18, 176 Hypertext link 121
M
Managed Application System (MAS) creation 55 Max Task problem 168 Maximum Task (MXT) 274 MAXOPENTCBS parameter 242 MAXSOCKETS condition 329 measurement criteria 4 Microsoft Windows 2000 24 Microsoft Windows NT 24 Microsoft Windows XP 24 Minimum WUI initialization parameters 52 Monitoring an IIOP Listener TCPIPSERVICE 326 MRO 164, 166 MRO links 139 MVS Log streams 195 MVS systems 24
I
IBM_CMZMENU primary menu 122 IEASYSxx 29 IEFSSNxx 34 IIOP 165 Import the CICS PM threshold 61 Import the CICS PM threshold definitions into your CMAS repository 63 IND$FILE GET 69 Independent Software Vendors (ISVs) 4 INITPARM 50 Installation of CICSPlex SM 27 Installation Worksheet 27 Installed threshold definition Topology pane 112 Installed threshold definition window 112 Installed threshold definitions pane 113 Installed threshold defintion Resources pane 112 Interactive Problem Control System (IPCS) 9 IPL 34
N
Navigation Frame 124
O
Object Transaction Service (OTS) 302 Operating System 23
J
Java 1.0.2 classes 24 Javascript 1.1 scripts 24 JCICS 302 Enterprise JavaBeans 302 Java language programs in CICS 302 JCICS workload scenario 308 JCICS application on non-resettable JVM 168 JCICS application on resettable JVM 169 JCICS workload 308 JDBC support in CICS 302 JNDI API 164 Job userid 30 Journals 7 JVM 303 JVM performance 303 Threshold Values 303 JVM pool 6
P
Parallel Sysplex 164 PassTicket support 59, 61, 65 performance objectives 4 pool thread 212 print preview 128 PROGRAM definition 309 PTF UQ68396 6 UQ68398 6 UQ68400 6 UQ68731 5 UQ70905 6 UQ70908 6 UQ70913 6 UW72529 22 PTFs 5
354
R
RACF 22 RAM 23 RCT 222 Recommended Maintenance 23 Recovery manager, including UOW analysis 6 Redbooks Web site 349 Contact us xii Region Shutdown options 131 Required Maintenance 22 Resettable events and JVM pool phaseout 324 Resource filter 17 Resource Management Facility 9 Resource pane 112 Resource summary 17 Resources pane 102 REXX 43 REXX/370 runtime 43
stateless session beans 302 Statistics interfaces 216 SYSID 29 sysplex 14 System initialization parameters 56 System Logger 5 System Logger problems 168 System Management Facility 9
T
Task subpools 288 TCB 210 TCP/IP connection status 169 TCP/IP Maxsockets 169 TCP/IP related thresholds 326 TCPIPPORT 32 TCPIPSERVICE status scenario 327 Teleprocessing Network Simulator 164 Temporary storage 6 Temporary storage queues 7 Terminal Emulation 68 Terminal emulation download 69 Terminals 7 THREADLimit parameter 214 THREADWAIT paremeter 214 Threshold definition form field descriptions 107 Threshold Definitions 20 Threshold Definitions window CICS Resources 99 Threshold Groups 99 Thresholds 99 Threshold definitions window 99 Threshold groups 18 DB2ENTRY 218 DB2SS 223 DB2THRD 221 DB2TRN 220 Name 98 XTASK 225 Threshold Groups menu selection 101 Threshold Groups pane 99 System-supplied default group 100 User-defined threshold group list 100 Thresholds 174 CONNECTST 229 CONNSTATUS 338 JVMTIME 313 MAXCCURSTR 174 MAXJVMTCBS 311 NQGCNQSW 174 NUMSTRINGWT 174 QUEDALLOCATE 340 RWRITES 201 SJGCURRJVMS 312, 314 TFCWAIT 207 TMRJVMIT 320 TOTWAITREQ 174 WSTRCCURCNT 174 WSTRCNT 174 Thresholds pane 105 Index
S
Sample code page table 48 sample parameters web user interface initialization 52 sampling interval 108 SCMZINST 31 SCMZINST library 41 SCMZPROC 36 SCMZSAMP 38 SDFHAUTH 36 Securing CICSPlex SM 58 SEYUADEF 36 SEYUAUTH 29, 36 SEYULINK 29 SEYULOAD 31 SEYUVDEF library 36 short on storage 168 single system image (SSI) 12 SIT parameters 32 AKPFREQ 201 APPLID 56 CPSMCONN 56 DSALIM 135 EDSALIM 274 MAXJVMTCBS 309, 311 MAXOPENTCBS 212 MAXSOCKETS 331 MXT 274 SEC 50 SMF records CICS Monitoring Facility (110) 5 DB2 Accounting (101) 5 System Logger (88) 5 SMP/E 11 SMS 31 SQLJ support in CICS 302 SRM 9 STAT 215 stateful session beans 303 stateless CORBA objects 302
355
Copy 106 Cut 106 Delete 106 Deselect All 106 New... 106 Paste 106 Select All 106 Show All 106 Sort by 106 Tivoli Decision Support for OS/390 Accounting 10 Availability 10 Capacity planning 10 Change and problem management 10 Performance and tuning 10 Service levels 10 Topology pane 112 TOR 164 TPNS 227 Transaction classes 7 Transaction manager 6 Transactions 7 Transient data 6 Transient data queues 7
X
XTASK definition 225
Z
z/OS 8 z/OS V2.2 CICSplex 209 z/Series Tools Offerings 4 ZCIOWAIT 347
U
unit of work 189 upgrade your Web user interface region 61 upgrade your WUI region CSD with the CICS PM resources 62
V
View Editor 147 View Menus 142 view sets 147 creating your own 147 VSAM Local Shared Resource 164 VSAM Record Level Sharing 164 VSAM RLS 165, 193 VTAM APPLID 29 VTAM buffers 9 VTAM Generic Resource 164
W
Web user interface procedure 46 Windows 2000 164 Workstation client startup 76 Workstation Web Browse 12 WUI iniitialization parameters DEFAULTMENU 52 WUI initialization parameters 52 AUTOIMPORTTDQ 53 DEFAULTCMASCTXT 52 DEFAULTCONTEXT 52 DEFAULTNAVIGATE 52 DEFAULTSCOPE 53 TCPIPHOSTNAME 53 TCPIPPORT 53
356
Back cover
SG24-6922-00
ISBN 0738429295